It seems bizarre to me that modern Internet services sometimes "forget" that there's a world outside the Anglosphere. Some people have the temerity to speak foreign languages! And some of those languages have accents on their letters!! Even worse, some don't use English letters at all!!!

A decade ago, I was miffed that GitHub only supported some ASCII characters in its project names. There's no technical reason why your repo can't be called "ഹലോ വേൾഡ്".

Similarly, I'm frustrated that Mastodon (the largest ActivityPub service) doesn't allow Unicode usernames and has resisted efforts to change.

So I built a small ActivityPub server which publishes content from an Actor called @你好@i18n.viii.fi - it is only a demo account, but it works!

Some ActivityPub clients report that they are able to follow it and receive messages from it. Others - like Mastodon - simply can't see anything from it. Take a look at the replies on Mastodon to see which services work. You can also see some of its posts on the Fediverse.

What Does The Fox Spec Say?

The ActivityPub specification says:

Building an international base of users is important in a federated network. Internationalization

I can't find anything in the specifications which limits what languages a username can be written in. But there are a few clues scattered about.

The user's @ name is defined by preferredUsername which is:

A short username which may be used to refer to the actor, with no uniqueness guarantees. 4.1 Actor objects

There's nothing in there about what scripts it can contain. However, later on, the spec says:

Properties containing natural language values, such as name, preferredUsername, or summary, make use of natural language support defined in ActivityStreams. 4. Actors

So it is expected that a preferred username could be written in multiple scripts. Which implies that the default need not be limited to A-Z0-9.

The ActivityStreams specification talks about language mapping.

Finally, the ActivityPub specification has some examples on non-Latin text in names.

So, I think that it is acceptable for usernames to be written in a variety of non-Latin scripts.

But What About...?

There are usually a few objections to "Unicode Everywhere" zealots like me. I'd like to forestall any arguments.

What about homograph attacks?

Well, what about them? ASCII has plenty of similar looking characters. I doubt most people would notice when a capital i is replaced by a lower L - and vice-versa. Similarly the kerning issue of an r and n looking like an m is well known. Are mixed language homographs more dangerous? I don't think so.

What if people make names that can't be typed?

Well, what if they do? Maybe not being found by people who can't type your language is a feature, not a bug. But, anyway, clients can let users search for other people, or copy and paste their names.

What about weird "Zalgo" text?

It is up to a client to decide how they want to render text input. The "problems" of strange Unicode combinations are well known. This is not a hard computer-science problem.

What about bi-directional text?

The spec makes clear this is allowed.

Do people even want a username in their own script?

I have no evidence for this. But I bet you'd get pretty frustrated if you had to switch keyboard just to type your own name, wouldn't you? In any case, why can't I have a username of @😉

What's Next?

If you build ActivityPub software, give some thought to the billions of people who don't have names which easily fit into ASCII.

If your software can see @你好@i18n.viii.fi and its posts, please let me know.

Like all good geeks, I have far too many domain names that I acquired for interesting projects which never took off. My latest is a bit different though.

https://⏻.ga/🔗

That's "Unicode Power Symbol Dot Gabon". Because why not.

Regular readers will know that I helped get ⏻ and several power symbols into Unicode. When I do talks about this, I usually refer to them as Emoji because, to most people, Emoji are simply little pictures in text. But that is a gross oversimplification. You know the meme that real Champagne must be from the Champagne region of France - otherwise it is merely sparkling wine? Well, Emoji must come from the Supplementary Multilingual Plane of Unicode otherwise they're just ✨sparkling✨ characters.

Except... That's not quite true. There are a bunch of symbols stuffed in the Miscellaneous block of the Basic Multilingual Plane which are also Emoji.

The Power Symbol appears in the block Miscellaneous Technical. The symbol itself is not an Emoji, but it is in a block which has 18 Emoji. Confused? Good⁰!

Domain names can only contain the ASCII characters A-Z, 0-9, and -. That's a problem if you speak anything other than basic English. Luckily, there's a workaround! I have a Chinese language domain 莎士比亚.org - through the magic of the Punycode Algorithm, it becomes xn--jlq54w7ypemw.org. This use of non-Latin letters in domains is known as IDN - Internationalised¹ Domain Names.

IDNs have several officially supported "scripts" - for example Thai, Greek, Hebrew, Cyrillic, Chinese etc. Each top level domain (like .uk, .com, .中国) can choose which scripts they'll accept. For example, a Chinese Top Level Domain may only accept Chinese characters and not Greek characters. IANA maintains a list of which domains support which scripts. But it is incomplete. Because it doesn't mention Emoji.

The Punycode algorithm works with emoji. This means you can have Emoji in domain names! Mostly. And that "mostly" is important.

Not every Top Level Domain accepts Emoji domain names (because they hate having fun, I guess?²)

The .ga registry doesn't publish any rules showing which scripts it will accept³. But seems quite happy to take registrations for Punycode domains. So I registered https://xn--soh.ga/ and, after an unusually long delay, it worked!

Does this mean ⏻.ga is an Emoji domain? No! ⏻ is not an Emoji! It is a small pictographic symbol encoded in Unicode.

Does this mean ⏻.ga is an IDN? No! ⏻ is not an international script. It is a language-neutral technical symbol.

So what the fuck kind of domain is it?

Drop an answer in the box bellow.

<html lang="en-GB">
...
As Caesar said: <i lang="la">veni vidi vici</i>

That says the page is in British English (en-GB) but the specific phrase is in Latin (la). But how can you offer an in-text translation of that phrase into the page's native language?

Here are a few options - and their drawbacks.

Title Text

<i lang="la" title="I came, I saw, I conquered">veni vidi vici</i>

veni vidi vici

The user has to hover their pointer over the text and a pop-up will appear with the translation. There are two disadvantages to this:

1. Not all devices - like mobile browsers - support title text.
2. The title text has no separate language attribute - so is semantically in Latin.

The language can be corrected by wrapping the title in a separate span.

Tables

The humble <table> can present two or more items of text adjacent to one another.

<table>
   <tr>
      <td lang="la">veni vidi vici</td>
      <td lang="en">I came, I saw, I conquered</td>
   </td>
</table>

Tables can be problematic on narrow screens - either requiring wrapping or scrolling.

Details

<details>
    <summary lang="la">veni vidi vici</summary>
    I came, I saw, I conquered
</details>

Again, it requires interaction - which may not work on devices like eReaders. Unfortunately, details is a block element, but you can read my experiments in making them inline.

Ruby

As suggested by John Ribbens

<ruby lang="la">
   veni vidi vici
   <rt lang="en-GB">I came, I saw, I conquered</rt>
</ruby>

veni vidi vici I came, I saw, I conquered

That works quite well - although Ruby text is pretty small. But it can be styled with CSS.

Ruby is usually used for showing pronunciation of characters. But, crucially, it isn't restricted to that.

Description Lists

<dl>
    <dt lang="la">veni vidi vici</dt>
    <dd>I came, I saw, I conquered</dd>
</dl>

veni vidi vici

I came, I saw, I conquered

Again, very easy to style with CSS. One of the nice things about Description Lists is that it allows for multiple definitions:

<dl>
    <dt lang="la">veni vidi vici</dt>
    <dd>I came, I saw, I conquered</dd>
    <dd lang="ja">私は私が征服した来た</dd>
</dl>

MIX THEM ALL TOGETHER!

Let's take a section from Chaucer's Canterbury Tales. Most of the Middle English is understandable - but a few archaic words need translation. It's also useful to have some commentary on the text.

<dl>
   <dt lang="enm">Full many a fat partridge had he in 
      <ruby>mew<rp>(</rp><rt lang="en-GB">cage</rt><rp>)</rp></ruby>
   </dt>
   <dd>The place behind Whitehall, where the King's hawks were caged was called the Mews.</dd>
</dl>

<details>
   <summary lang="enm">And many a bream, and many a 
      <ruby>luce<rp>(</rp><rt lang="en-GB">pike</rt><rp>)</rp>
      in 
      <ruby>stew<rp>(</rp><rt lang="en-GB">fish-pond</rt><rp>)</rp>
   </summary>
   In those Catholic days, when much fish was eaten, no gentleman's mansion was complete without a "stew".
</details>

Full many a fat partridge had he in mew(cage)

The place behind Whitehall, where the King's hawks were caged was called the Mews.

Which should you use?

Yes.

There's no definitive "correct" answer here. title text might make sense for occasional words which need translating - and you're sure either the user's device supports it, or they won't be substantially disadvantaged if it doesn't.

Similarly, details works for interactive content which is optional to understanding.

The ruby elements are great if you want a fairly unobtrusive way to translate specific words.

Lists are great if you need to offer multiple translations.

Mashing them all together is a bit silly and complicated - but allows for a greater variety in the way the texts are displayed.

"When you visit the zoo, be sure to see the panda - 熊(Xióng)猫(māo)."

This is written as:

<ruby>熊<rp>(</rp><rt>Xióng</rt><rp>)</rp></ruby><ruby>猫<rp>(</rp><rt>māo</rt><rp>)</rp></ruby>.

That is, the word or character which needs text above it is wrapped in <ruby>. The pronunciation is wrapped in <rt>. The <rp> element indicates the presence of a parenthesis - which isn't usually displayed, but will be shown if the browser doesn't support <ruby> syntax.

That's fairly easy for scripts written left-to-right. But how does it work for scripts like Arabic where the text is written right-to-left, but the user may want the pronunciations left-to-right?

Let's take the phrase "Hello World" in Arabic: مرحبا بالعالم. Google Translate tells me this is pronounced "marhaban bialealami".

For a single word, the directionality can be ignored. The browser should be smart enough to place the pronunciation above the word:

<p>Hello is: <ruby>مرحبا<rp>(</rp><rt>marhaban</rt><rp>)</rp></ruby>. What a useful word!</p>

Hello is: مرحبا(marhaban). What a useful word!

What about if we have a few words - or a whole sentence - which is entirely RTL?

<p dir="rtl">مرحبا بالعالم</p>

Is displayed aligned to the right side of the screen:

مرحبا بالعالم

There are a few ways to add pronunciation.

Separate The Words

The first is to write each word separately. For example <ruby>1st word</ruby> <ruby>2nd word</ruby>. Obviously, this isn't normally how you'd write a RTL language! But it does work:

<p dir="rtl"><ruby>مرحبا<rp>(</rp><rt>marhaban</rt><rp>)</rp></ruby> <ruby>بالعالم<rp>(</rp><rt>bialealami</rt><rp>)</rp></ruby></p>

Which displays as:

مرحبا(marhaban) بالعالم(bialealami)

It helps to think of the way the characters of the script are stored in memory.

A word that displays as ABC is stored as C B A.

So the above is written "correctly" - even though it looks odd in the source-code view.

All At Once

But there is an alternative if you want the source text to look natural - i.e. [2nd word] [1st word].

It's a bit messy, but you can write the LTR text in <rt> "backwards"!

<p dir="rtl"><ruby>مرحبا بالعالم<rt>bialealami marhaban</rt></ruby></p>

مرحبا بالعالمbialealami marhaban

But, again, that doesn't seem very satisfying! It also divorces the pronunciation from the original word - which is unfortunate for screenreaders.

The Ruby layout algorithm is usually clever enough to group words separated by spaces:

مرحبا بالعالمB A

مرحبا بالعالمBbbbbbbbbbbbbb Aaaaaaaaaaaaa

Although, if the pronunciations have a significantly different length than each other, it can get a bit messy:

مرحبا بالعالمBbbbbbbbbbbbbb A

مرحبا بالعالمB Aaaaaaaaaaaaa

In which case, you probably need to go for the first technique and wrap each word in its own <ruby> element:

مرحبا(A) بالعالم(Bbbbbbbbbbbbbb)

BDO

It's tempting to think that simply using the <bdo> element can help us here. It can't!

Using the bidirectional override will display characters RTL, rather than words.

<p dir="rtl"><ruby>مرحبا بالعالم<rt><bdo dir="rtl">marhaban bialealami</bdo></rt></ruby></p>

Becomes:

مرحبا بالعالمmarhaban bialealami

I guess you could spell each word backwards. Which would be extremely annoying for everyone and a complete nightmare for screen readers!

Instead, it can be fixed if each word is then given an explicit LTR direction:

<p dir="rtl"><ruby>مرحبا بالعالم<rt>
   <bdo dir="rtl">
      <span dir="ltr">marhaban</span> <span dir="ltr">bialealami</span>
   </bdo></rt></ruby></p>

مرحبا بالعالم marhaban bialealami

Is that it?

So, I think those are the only ways to achieving mixing bidirectional text pronunciation. But I'd welcome any corrections and suggestions!

I love the idea of a non-American programming language. I'm aware that there are some, but I'm unaware of any which are in British English. Except, perhaps, BBC Basic. Although that also allows traitorous American spelling for some keywords.

HTML was invented by a Brit (Hi Sir Timbl!). So why doesn't it use British spelling for everything?

Well, I guess, the answer is... it mostly does!

Looking through the big list of HTML elements only one is noticably in American English. The <dialog> element was introduced reasonably recently in HTML 5.2. I would love to know if there were any late-night arguments about whether it should have been dialogue...

In the obsolete element section we find the much-missed <center>. Perhaps, in an alternate timeline, it was named centre and is still in use today?

Center has a curious history.

CENTER was introduced by Netscape before they added support for the HTML 3.0 DIV element. It is retained in HTML 3.2 on account of its widespread deployment. HTML 3.2 Reference Specification - 1997

You can read some of the original arguments discussions from the early 1990s.

CSS was invented by Håkon Wium Lie with CSS version 1 specifying the spelling color. As Håkon was Norwegian, I suppose we could have ended up with farge. That might have been nice.

Perhaps that's what the World-Wide Web needs. HTML elements which are not in English. There is no technical limitation why we can't have an <电影> element. Or a CSS property of نطاط

British English is the best. But I only think that because it is what I've grown up with. English is the world's most popular second language. But it won't be long before Chinese catches up to the total number of speakers. Is it fair to make new web developers learn an entirely new human language while they struggle with learning a new computer language?

I constantly find myself typing colour when I mean color - does a Hindi speaking developer want to be able to program in their preferred language?

Why can't an HTML document start <!DOCTYPE html ፊደል> and then have all the elements written in Geʽez script?

I know you think this is too hard to achieve. But part of the job of computer scientists is to work out how to make computers do the hard work for us. Humans shouldn't adapt to a computer's needs; the computer must adapt to ours.

/ləʊk(ə)lʌɪˈzeɪʃ(ə)n/ The ability to adjust a user-interface to the user's local language or dialect

Because I live in the UK, I speak en_GB (English, Great Britain) rather than en_US (English, Simplified United States).

Mostly, all dialects of English are mutually intelligible. Sure, the Brits love the letter U and the Americans stick a Z in every possible word. But we get along reasonably well. Except in Gmail.

Here's my en_GB localised Gmail interface. Note how there is a folder called "Bin".

Everyone using Gmail in en_GB will know that deleted emails go into the "Bin".

Gmail has a handy search feature to allow you to find emails in a specific folder. For example "Bob in:spam" finds all email containing the word "Bob" in your spam folder. "Proposal in:sent" gets everything you've sent with the word proposal.

But it is impossible to search the "Bin" folder.

Why? Because you have to search the Trash folder. Because that's the names used by Americans.

/hɪˈdʒɛməni/

The same is true even if you've chosen a non-English language.

Sadly, Google don't respond to user complaints or feedback. The best you can do is hope a ranty blog post gets high enough traction on social media. Then, maybe, something will change.

Lea Verou

@LeaVerou

HTML Idea: <input type="country"> which would become a searchable dropdown with all countries and their flags.
Wouldn't that be awesome?

---

You can read through the conversation and make your own mind up (while also marvelling at the witless mansplainers) - but I'd like to give you my considered take on it.

(Disclaimer - I'm an editor on the HTML 5.3 spec and I work for the UK Government. This is a personal blog post and doesn't represent the views of my employers, associates, or friends.)

Who Are You?

Let's start with the big one. What is a country? This is about as contentious as it gets! It involves national identities, international politics, and hereditary relationships.

Scotland, for example, is a country. That is a (fairly) uncontentious statement - and yet in drop-down lists, I rarely see it mentioned. Why? Because it is one of the four countries which make up the country of the United Kingdom - and so it is usually (but not always) subsumed into that.

Some countries don't recognise each other. Some believe that the other country is really part of their country. Some countries don't exist.

There are two main schemes to classify what is and isn't a country. The first is ISO 3166-1. It provides two- and three-letter codes for every country. Well... sort of.

ISO 3166 contains 249 different countries, territories, protectorates, principalities, duchies, and other bits-and-bobs. It contains the Falklands, but not Scotland.

The second is... whatever your country says is another country!

My friends in the Government Registers Team have published a canonical list of every country that the UK recognises. There are 199 entries. Which countries are not in there is left as an exercise for the reader.

The UK's register of countries should allow every Government website to have the same list in a drop down. When new countries are recognised, one list needs to be updated - and then all websites automagically update. In theory.

Incidentally, that list of 199 countries includes four entries for countries which no-longer exist. For example Yugoslavia.

Which brings us to the next question...

What's the use case?

The most obvious one is "I want to give a site my current address" - presumably for identification purposes or postal deliveries.

But what if the use case is "I want to say where I was born"?

Borders shift. Countries disappear, merge, split, change names, change flags, and do all manner of weird things which trip up your edge cases.

The user may want to find the name in their own script - for example would a Greek user be looking for "Greece" or "Ελλάδα"? If a Chinese speaker wants to visit the UK, do they look in the drop-down for "英国"?

International Dialling Codes - not every country is unique - +1 is used by USA, Canada, Anguilla, Dominican Republic, and dozens more. Are there countries where there is more than one international dialling code?

OK, what if the user wants to select their language based on their country?

Do You Have A Flag?🔗

It is one of the classic conventions that first-year students of user interface design are taught - countries do not represent language!

Some countries have multiple official languages. Some users may not speak the language of their country. Some languages are only used for official purposes, and not by the general population.

Flags mostly represent countries. There are people in Wales who would rather see Y Ddraig Goch rather than the Union Jack. And vice-versa. Flags can make people angry.

The flag of the USA last changed in 1960 - but Mauritania changed theirs in August 2017. How quickly can a browser update their list of countries?

...and yet...

I instinctively like this idea! This isn't a new question, nothing ever is, but I think it is an idea which has merit.

One of the goals of HTML is to stop web developers having to re-invent the wheel. That's why we have lots of different <input> types - to reduce complexity.

Colour picker

Number inputs

Range selector

Some modern browsers support date input

The challenges of a country selector are...

1. Keeping everyone happy and not causing major diplomatic incidents. Easy‽
2. Usability. Making sure it's easy to search for the name of a country.
3. Consistency. How do you indicate that this list contains historic countries?

None of these are insurmountable problems - but it's far from trivial.

And yet... I think there is a real possibility that this could work. Millions of websites already find ways to cope with the ambiguity - perhaps browsers can too?

A few years ago I got the Chinese domain name 莎士比亚.org. You can browse to it, link to it, and send email to it. Or can you?

When I tried two years ago, none of the major email providers supported sending to non-ASCII email addresses.

Today, I tried again with six of the big "Western" webmail providers. How did they do?

Show Me The Data!

I tested by trying to send an email to test@莎士比亚.org and the Punycode representation test@xn--jlq54w7ypemw.org

Winners!

Both Gmail and Outlook failed the last time I tried them - I'm very pleased to say that both of them now support sending to Chinese addresses.

One strange thing to note, when looking through Outlook's message details, I found this example of Mojibake.

Losers!

Yahoo

The biggest loser is Yahoo. Very strange considering Jerry Yang, their founder, is Taiwanese-American. Even stranger given Yahoo's continued dealings with China.

The Yahoo webmail portal simply wouldn't let me send to a Chinese domain name.

The Punycode representation appeared to send but immediately failed.

iCloud

Apple's much-vaunted "It Just Works" philosophy obviously doesn't extend to International email addresses. It accepted the Punycode but gave this delightful error message on the Chinese domain.

OWA

Microsoft's Outlook Web Access got very confused and tried to look up the email address in the local directory.

Errr?

⭐ FastMail

Lots of people recommended that I try Fastmail - it really didn't like the look of the Chinese domain and painted it with a red error colour. That said, it sent the email without further complaint.

What about a Chinese Local-Part?

Email is a venerable protocol. That's a polite way of saying it is old and outdated. The local-part of the email address (test@) is generally restricted to a handful of 7 Bit ASCII characters. None of the email providers I tried would let me sign up with a Chinese name. So no 你好@yahoo.com for me!

But what happens if you're foolish enough to try to send an email to 你好@莎士比亚.org?

Well you'll probably get this error message:

In 2012, RFC 6531 defined how International Email Addresses should work. Over four years later and support is still not widespread.

It's 2016 and the majority of the world can't send an email to their preferred name.