While perusing the programme on the National Theatre website I stumbled upon a little bug. The incredible Ronkẹ Adékọluẹ́jọ́ has her name rendered in a most unusual style:

It rendered just fine in Chrome - but both Firefox and Safari misrendered some of the accented characters.

Here's a minimum viable demo to show what's happening:

Fonts are hard, OK?!?!

Broadly speaking⁰, accented characters can be made in two way.

1. Pre-composed. There is a separate code for the character é
2. Combining. The plain letter e is immediately followed by the combining character ◌́ and the computer smushes them together.

Similarly, a font file can have separate little drawings for each accented character or it can have separate accents.

In this case, the National Theatre is using the font "Helvetica Now Display W04".

The web font contains é (U+00E9) and both ◌́ (U+0301) & ̣◌ (U+0323).

But doesn't include ẹ (U+1EB9) or ọ (U+1ECD).

So the ẹ́ and ọ́ have to be made by combining characters in the font.

On Chrome this works. On Firefox and Safari, it seems to break when the CSS is set to font-weight: normal;. This causes the browser to render those characters in the default fallback font - hence the slightly weird look.

Next Steps

I've raised a bug with Firefox and one with WebKit.

Of course, it might be that they're doing the right thing and Chrome is in the wrong - but I think that's unlikely.

Now, time to fix the font I use on this website to prevent any rendering errors!

Is there a way to use CSS to increase the font size of specific characters without having to wrap them in an extra <span> or similar?

Yes! Although it is a bit of a hack.

This relies on 3 CSS features: src: local(), unicode-range,and size-adjust. Let me walk you through it.

@font-face {
    font-family: "emoji";

    src: local('Apple Color Emoji'),
         local('Android Emoji'),
         local('Segoe UI Emoji'),
         local('Noto Color Emoji'),
         local(EmojiSymbols),
         local(Symbola);

    unicode-range: U+231A-231B, U+23E9-23EC, U+23F0, U+23F3, U+25FD-25FE, U+2614-2615, U+2648-2653, U+267F, U+2693, U+26A1, U+26AA-26AB, U+26BD-26BE, U+26C4-26C5, U+26CE, U+26D4, U+26EA, U+26F2-26F3, U+26F5, U+26FA, U+26FD, U+2705, U+270A-270B, U+2728, U+274C, U+274E, U+2753-2755, U+2757, U+2795-2797, U+27B0, U+27BF, U+2B1B-2B1C, U+2B50, U+2B55, U+FE0F, U+1F004, U+1F0CF, U+1F18E, U+1F191-1F19A, U+1F1E6-1F1FF, U+1F201, U+1F21A, U+1F22F, U+1F232-1F236, U+1F238-1F23A, U+1F250-1F251, U+1F300-1F320, U+1F32D-1F335, U+1F337-1F393, U+1F3A0-1F3CA, U+1F3CF-1F3D3, U+1F3E0-1F3F0, U+1F3F4, U+1F3F8-1F43E, U+1F440, U+1F442-1F4FC, U+1F4FF-1F53D, U+1F54B-1F567, U+1F57A, U+1F595-1F596, U+1F5A4, U+1F5FB-1F64F, U+1F680-1F6CC, U+1F6D0-1F6D2, U+1F6D5-1F6D7, U+1F6DC-1F6DF, U+1F6EB-1F6EC, U+1F6F4-1F6FC, U+1F7E0-1F7EB, U+1F7F0, U+1F90C-1F93A, U+1F93C-1F945, U+1F947-1FA7C, U+1FA80-1FAC5, U+1FACE-1FADB, U+1FAE0-1FAE8, U+1FAF0-1FAF8;

    size-adjust: 300%;
}

body {
    font-family: "emoji", sans-serif;
}

Here's how it works.

@font-face this tells the browser that we're defining a new font which will be referenced later.

font-family this is the name we're going to be using as a reference.

src: local('Apple Color Emoji') ... CSS can reference local fonts. We don't know what device this page is being viewed on, so we've included a number of popular fallback fonts which should work with all major browsers. You can also reference a webfont if you want - although Emoji fonts tend to have a large filesize. I've adapted this from Marc Fornós' CSS and added a few more common default emoji fonts.

unicode-range this tells the browser to use this font only for the specific codepoints mentioned. This is where the magic happens.

Emoji codepoints are complicated - especially when it comes to combining characters. You can see a full list of every sequence in Unicode 15.1. There are currently 3,782 different emoji.

There was some talk of using named ranges but that doesn't seem to have gone anywhere. So, instead, I've extracted all the Emoji codepoints and manually grouped them. It's a pretty long sequence, and I'm sure I've made a few mistakes.

size-adjust this is used to increase or decrease the apparent size of a font.

Finally, the body { font-family: "emoji", sans-serif; } tells the browser to use the Emoji font (remember, this will only work on the specified Unicode range) and then fall back to the defaults sans-serif font. Obviously, you can specify whatever fonts you like.

Have a play with it

This is a nifty little hack if you want to make Emoji (or any other text) bigger than its surroundings.

I am indebted to Šime Vidas who demonstrated this trick to me.

I recently read this wonderful blog post about using 17th Century Dutch fonts on the web. And, because I'm an idiot, I decided to try and build something similar using Shakespeare's first folio as a template.

Now, before setting off on a journey, it is worth seeing if anyone else has tried this before. I found David Pustansky's First Folio Font. There's not much info about it, other than it's based on the 1623 folio. It's a nice font, but missing brackets and a few other pieces of punctuation. Also, no ligatures. And the long s is in the wrong place.

So, let's try to build a font!

You can read how it works, or skip straight to the demo.

Get some scans

There are various scans of the First Folio. I picked The Bodlian's scan as it seemed the highest resolution.

I plucked a couple of pages at random to see what I could find. Of course, a modern font can't replicate the vagaries of hot metal printing. As you can see here, each letter "y" is substantially different.

Within the plays, there are some italic characters - which could be used to make a variant font. You can also see just how poor quality some of the letters are.

There are also plenty of ligatures to choose from:

Ready? Let's go!

Extract the characters

This Python code reads in an image file. It then extracts every distinct letter, number, and punctuation mark. It then detects which character it is and saves each glyph to disk with a filename like this:

As you can see, the text detection is good, but the letter recognition is poor.

import cv2
import pytesseract
from PIL import Image

def preprocess_image(image_path):
    # Load the image using OpenCV
    image = cv2.imread(image_path, cv2.IMREAD_GRAYSCALE)

    # Thresholding to convert to binary image
    _, binary_image = cv2.threshold(image, 128, 255, cv2.THRESH_BINARY_INV)

    # Find contours to isolate individual letters
    contours, _ = cv2.findContours(binary_image, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

    return image, contours

def extract_and_save_letters(image, contours, output_directory):
    # Create output directory if it doesn't exist
    import os
    if not os.path.exists(output_directory):
        os.makedirs(output_directory)

    for i, contour in enumerate(contours):
        x, y, w, h = cv2.boundingRect(contour)

        # Crop and save each letter as a separate image
        letter_image = image[y:y + h, x:x + w]

        # (Don't) Perform OCR to extract the text (letter) from the contour
        letter_text = "_"
        #letter_text = pytesseract.image_to_string(letter_image, config='--psm 10')
        #letter_text = letter_text.strip()  # Remove leading/trailing whitespace

        # Create a filename with the detected letter
        letter_filename = f"letter_{letter_text}_{i}.png"

        letter_path = os.path.join(output_directory, letter_filename)
        cv2.imwrite(letter_path, letter_image)

if __name__ == "__main__":
    input_image_path = "letters.jpg"
    output_directory = "/tmp/letters/"

    # Preprocess the image
    image, contours = preprocess_image(input_image_path)

    # Perform OCR and save individual letters
    extract_and_save_letters(image, contours, output_directory)

Something to note - the CHAIN_APPROX_SIMPLE is looking for contiguous characters. So it loses the dots from i, j, :, and ;. But it is quick.

Detecting Dots

In order to get glyphs which vertically separate, we need to vertically erode the image so it looks like this:

# Erode the image vertically
kernel = np.array([[0, 0, 0, 0, 0],
                   [0, 0, 1, 0, 0],
                   [0, 0, 1, 0, 0],
                   [0, 0, 1, 0, 0],
                   [0, 0, 0, 0, 0]], dtype=np.uint8)

erode = cv2.erode(image, kernel,iterations = 6)

We use this eroded image for contiguous detection - but we do the actual cropping on the original image.

As you can see, it does make some character touch each other - which means you get occasional crops like this:

They can either be manually split, or ignored.

Put each letter into a folder

There's no automated way to do this. It's just a lot of tedious dragging and dropping. It's hard to tell the difference between o and O, or commas and apostrophes.

Ideally we want several of each glyph because we're about to...

Find the average letterform

Here's a selection of letter "e" images which were extracted.

I didn't want to make some rather arbitrary decisions on which letters I like best. So I cheated.

I copied all the letter "e"s into a folder. I used Python to create the average letter based on the two-dozen or so that I'd extracted. This code takes all the images in a directory, and spits out a 1bpp average letter - like this:

import os
import numpy as np
import argparse
import math
from PIL import Image

def get_arguments():
    ap = argparse.ArgumentParser()
    ap.add_argument('-l', '--letter', type=str,
                    help='The letter you want to average')
    arguments = vars(ap.parse_args())

    return arguments

def load_and_resize_images_from_directory(directory, target_size):
    image_files = [f for f in os.listdir(directory) if f.endswith(".png")]

    images = []
    for image_file in image_files:
        image_path = os.path.join(directory, image_file)
        print("Reading " + image_path)
        image = Image.open(image_path).convert("L")  # Convert to grayscale

        # Create a new white background image
        new_size = (target_size[0], target_size[1])
        new_image = Image.new("L", new_size, color=255)  # White background

        old_width, old_height = image.size

        # Center the image
        x1 = int(math.floor((target_size[0] - old_width)  / 2))
        y1 = int(math.floor((target_size[1] - old_height) / 2))

        # Paste the image at the center
        new_image.paste(image, (x1, y1, x1 + old_width, y1 + old_height))

        # Make it larger to see if that improves the curve detection  
        new_image = new_image.resize( (600,600), Image.LANCZOS)
        images.append(new_image)

    return images

def calculate_average_image(images):
    # Convert the list of images to numpy arrays
    images_array = [np.array(img) for img in images]

    # Calculate the average image along the first axis
    average_image = np.mean(images_array, axis=0)

    return average_image

def convert_to_1bpp(average_image, threshold=120):
    # Convert the average image to 1bpp by setting a threshold value
    binary_image = np.where(average_image >= threshold, 255, 0).astype(np.uint8)

    return binary_image

def save_1bpp_image(binary_image, output_path):
    # Convert the numpy array to a binary image
    binary_image = Image.fromarray(binary_image, mode="L")

    # Save the 1bpp monochrome image to the specified output path
    binary_image.save(output_path)

if __name__ == "__main__":
    args = get_arguments()
    letter = args['letter']
    input_directory   = "../letters/" + letter + "/"
    output_png_path = "../letters/" + letter + ".png"
    target_size = (75, 75)  # Set the desired target size for resizing

    # Load, resize, and add border to all images from the directory
    images = load_and_resize_images_from_directory(input_directory, target_size)

    # Calculate the average image
    average_image = calculate_average_image(images)

    # Convert the average image to 1bpp
    binary_image = convert_to_1bpp(average_image)

    # Save the 1bpp monochrome image
    save_1bpp_image(binary_image, output_png_path)

One Big Image

The next step is to create a single image which holds all of the glyphs. Our good friend ImageMagick comes to the rescue here:

montage *.png -tile 12x8 -geometry +10+10 all_glyphs.png

That takes all of the average symbol .png files and places them on a single image. It looks like this:

Trace Those Glyphs

The GlyphTracer App will take the image and generates a Spline Font Database. It isn't the most intuitive app to use. But after a bit of clicking around you can work out how to assign each image to a glyph.

GlyphTracer uses potrace which turns those raggedy rasters into smoothly curved paths.

Once done, we're on to the next step.

Forge Those Fonts!

The venerable FontForge will open the SFD and show us what the proto-font looks like:

As you can see, all the letters have been vertically centred. So double tap and edit their position - you can also adjust the curves if you like:

The final result looks something like this:

FontForge's "File" ➡ "Generate Font" will let you save the output as TTF, WOFF2, or anything else you want.

Demo!

Here's what the font looks like when rendered on the web:

Two houſeholds, both alike in dignity!
Alas poor Yorik; I knew him Horatio.
To be? Or not to be? That's the uestion.
Bump sickly, vexing wizard! Be sly, fox, and charm the dragon's breath.

TODO

• Get more sample images from the 1st Folio.
• Extract more letters, numbers, ligatures, and symbols.
• Sort symbols into sub-directories.
• Generate font with complete alphabet.
• Tidy up curves.
• Set correct height, ascenders, descenders, etc.
• Make the ligatures automatic.
• Other font stuff that I haven't even thought of yet!

Want to help out? See the source code on GitHub.

Warning, not for the faint of heart.

The thing is, I can't adequately describe why I - and many others - find it so disturbing.

In all my years of reading English, I've never found a font which slants backwards. I'm used to italics so there's no reason it should seem weird. And yet... it's like the uncanny valley of fonts. Something in my brain just screams "WRONG".

I examined the page to see if it was some kind of fancy CSS transformation. But, no. It appears to belong to a font family called "recline".

Thanks. I hate it.

What's the most unsettling font you've seen in the wild?

A font file contains a list of characters (usually letters, numbers, and punctuation) and glyphs (the drawn representation of that character). It is, of course, a lot more complicated than that.

Each character has a codepoint which is represented in hexadecimal. For example, U+0057 is the Latin letter Capital W, U+20AC is the Euro Symbol €, and U+1F600 is the Emoji Smiling Face 😀. These codepoints are assigned by the Unicode Consortium.

A font which contains thousands of characters will be multi-megabytes in size. That's annoying when downloading a font file to display text on a website in a particular font.

It is possible to create a "subset" of a font which only contains the characters that you want. This makes the font file smaller, which makes downloading things quicker for the user.

Again - it is all a lot more complicated than that, but it's a good approximation of the truth.

Traditionally, it makes sense to subset fonts by human languages. If you are writing in English, you don't want the Greek set of characters. If you're writing in Vietnamese, you don't want Cyrillic characters.

Once a subset has been created, you can refer to it in CSS like this:

@font-face {
  font-family: 'CoolFont';
  src: url(https://example.com/font.woff2) format('woff2');
  unicode-range: U+0000-00FF, U+0131;
}

This tells the web browser that the font covers the characters U+0000 to U+00FF and, additionally, the character U+0131 - "Latin Small Letter Dotless I".

U+0000 to U+007F are "Basic Latin". They contain the traditional English letters, numbers, and some symbols.

U+0080 to U+00FF are "Latin-1 Supplement". They contain "European" symbols like Ñ, å, and ÿ.

As Unicode has added more languages, they have scattered characters across the specification. And that's where the problem lies.

The Problem

A user in India has complained that Google's font subsetting ignores Indian users.

Here's an example.

Google's Roboto font has the following characters as part of its Latin subset:

• U+0000-00FF Basic Latin and Supplement
• U+0131 ı Latin Small Letter Dotless I
• U+0152-0153 Œ and œ Ligature Oe
• U+02BB-02BC ʻ and ʼ Modified Punctuation
• U+02C6 ˆ Modifier Letter Circumflex Accent
• U+02DA ˚ Ring Above
• U+02DC ˜ Small Tilde
• U+2000-206F - General Punctuation
• U+2074 ⁴ Superscript Four
• U+20AC € euro sign
• U+2122 ™ Trade Mark Sign
• U+2191 ↑ Upwards Arrow
• U+2193 ↓ Downwards Arrow
• U+2212 − Minus Sign
• U+2215 ∕ Division Slash

You can argue how useful or not some of these characters are - but what's interesting is what's missing.

India has twice the number of English speakers as the United Kingdom.

A website written for an English speaking audience in India is likely to want the Latin subset of a font. But it will also want one local character - ₹ - U+20B9. The Rupee is the currency of India and its character is part of the "Currency Symbols" Unicode block.

So should the Rupee be part of the "Latin" subset?

Colonialism In Tech

The original complainant says:

this symbol [₹] is excluded (subsetted out) of many Latin fonts that originally included it due to an American assumption that English is not spoken in India.

I don't know whether their assumption about Google is correct. But it seems odd to specifically include € in Google's Latin subset, but not the Rupee. Latin is not synonymous with "European".

Through a quirk of history, the Dollar symbol - $ - is in Basic Latin. The Yen currency symbol - ¥ - is included in the Latin-1 Supplement, as is the Pound - £.

Roboto's Latin subset contains Old English characters like þ (Thorn) and Ð (Eth).

Are obsolete characters used only in mediaeval text really more important to include than the currency for a billion people?

Google do include the ₹ in their "Latin Extended" subset. So if an Indian user wants to use their currency, they need to download a separate font which includes 1,011 characters they don't need.

This is inefficient. It increases the download weight and energy usage of billions of people.

Some Simple Solutions

There are a few things which can be done here.

I think that Google probably should include a popular currency symbol in their Latin subset font. Yes, there's a risk that the font might grow in size as more useful symbols are added. And, no, Latin doesn't mean English and English doesn't mean Indian - but we're all trying to get along on this crowded planet. So let's be flexible.

Google could create an "en_IN" subset which includes all the popular and useful characters needed by an Indian audience. It seems like there is sufficient demand for it.

Users should use the Google Font API to create a subset which has only the specific characters they want. That way they aren't reliant on the whims of a megacorp to decide what counts for their language.

Finally, as developers, we should understand that what is "logical" and "orderly" isn't always how our users see things. We have a huge range of biases and unexamined assumptions. Some of the earliest foundations of computer science are based on a very rigid and limited set of assumptions about the world. Let's do our best to be more inclusive.

The War Machines is a cracking slice of 60s sci-fi. It's very groovy.

But what font is it?

It isn't Westminster. The W is completely wrong. And the letter T is mirrored.

Westminster has a surprising and interesting history.

It isn't Data 70.

Which, in any case, was released a few years after the episode. Time travel is complex though.

It isn't a modern font like Computer Font

As far as I can tell, it is an original font which is heavily inspired by Westminster. The T is a particular giveaway.

There are only a few title screens, so there are only a limited number of characters available.

With a bit of jiggery-pokery, we can grab a few more character. A flipped E easily becomes a 3, for example.

I am indebted to my producer Clayton Hickman for his original investigative work.

Clayton Hickman

@claytonhickman

Just noticed the War Machines title font isn't one of the regular typefaces I assumed it was. Was it specially made? pic.x.com/jiyfllgl8d

---

For example, this calligraphy style font is 3.4MB.

The beautiful Paper Cut Font weighs in at 14MB!

That file-size is far to heavy to embed on a web page.

Subsetting

Generally speaking, font files like .ttf contain a representation of every single character. 0-9, a-z, A-z, all the punctuation, non-English characters etc.

That's really useful if the font is installed on your computer and you want to write a document which could contain every character. It's less helpful if you want to use a fancy font on your website's headers.

Subsetting is the act of creating a subset of a font. That is, a font file which only contains specific characters.

Let's suppose that we only want a specific phrase rendered in this font.

我很丢脸。我没有吃Fruity Oaty Bar

We only need 19 unique characters - we can get rid of any character which doesn't appear in that heading.

There are sevel font manipulation tools available. I've chosen Font Optimizer which has an excellent live demo page. The source code is on BitBucket.

The command line syntax is really simple

./subset.pl --chars="我很丢脸。我没有吃Fruity Oaty Bar" input.ttf output.ttf

The file size reduction is impressive. My original font was over 14MB. The optimized one is 32KB

14,066,456 input.ttf
   32,084 output.ttf

The process run instantly - fast enough to run as a web service to generate these fonts dynamically, I would think.

One could quite easily create a scrap of JavaScript which read the contents of a block of text and then requested a font which contained only the necessary characters.

Apparently, Monotype have a proprietary and patent-pending solution to this rather trivial application.

Uses

Being able to subset fonts to reduce file size is incredibly useful. Supposing you want a different font for body text, headers, and navigation. Rather than having to load three large font files containing every character in the known universe, you could subset each one for only exactly the relevant characters.

This also has an interesting DRM like effect. Some people don't want their shiny web fonts to be downloaded and used as a regular font. With subsetting, the font only contains the specific characters.

When scrolling, I notice that the font hinting seems to break down. This makes the text very "juddery". It looks like the fonts shrink and grow and they scroll. This is very disorienting when reading.

In this example I've noticed that once I've stopped scrolled the page, the fonts are hinted differently.

I've enlarged the examples (without interpolation) so you can see them a bit more clearly.

To give you a good idea of what's happening, I've magnified the letter "e" (again, without interpolation). You can quite clearly see what happens to the font.

I can understand why the font hinting changes as the scrolling occurs - but why does it change even after scrolling has finished?

All screenshots taken on an N7100 running 4.1.2 with the latest version of Chrome for Android. The Galaxy Note II has a fairly standard resoltion of 1280×720, giving it 267 PPI - that shouldn't cause any problems.

This issue has been reported to the Chromium team.