This makes many people very angry and is widely regarded as a bad move.

In the bad old days, it was somewhat understandable for a PHP coder to run a quick-and-dirty preg_replace() on a scrap of code. They probably could control the input and there wasn't a great way to manipulate an HTML5 DOM.

Rejoice sinners! PHP 8.4 is here to save your wicked souls. There's a new HTML5 Parser which makes everything better and stops you having to write brittle regexen.

Here are a few tips - mostly notes to myself - but I hope you'll find useful.

Sanitise HTML

This is the most basic example. This loads HTML into a DOM, tries to fix all the mistakes it finds, and then spits out the result.

$html = '<p id="yes" id="no"><em>Hi</div><h2>Test</h3><img />';
$dom = \Dom\HTMLDocument::createFromString( $html, LIBXML_NOERROR | LIBXML_HTML_NOIMPLIED , "UTF-8" );
echo $dom->saveHTML();

It uses LIBXML_HTML_NOIMPLIED because we don't want a full HTML document with a doctype, head, body, etc.

If you want Pretty Printing, you can use my library.

Get the plain text

OK, so you've got the DOM, how do you get the text of the body without any of the surrounding HTML

$html = '<p><em>Hello</em> World!</p>';
$dom = \Dom\HTMLDocument::createFromString( $html, LIBXML_NOERROR , "UTF-8" );
echo $dom->body->textContent;

Note, this doesn't replace images with their alt text.

Get a single element

You can use the same querySelector() function as you do in JavaScript!

$element = $dom->querySelector( "h2" );

That returns a pointer to the element. Which means you can run:

$element->setAttribute( "id", "interesting" );
echo $dom->querySelector( "h2" )->attributes["id"]->value;

And you will see that the DOM has been manipulated!

Search for multiple elements

Suppose you have a bunch of headings and you want to get all of them. You can use the same querySelectorAll() function as you do in JavaScript!

To get all headings, in the order they appear:

$headings = $dom->querySelectorAll( "h1, h2, h3, h4, h5, h6" );
foreach ( $headings as $heading ) {
   // Do something
}

Advanced Search

Suppose you have a bunch of links and you want to find only those which point to "example.com/test/". Again, you can use the same attribute selectors as you would elsewhere

$dom->querySelectorAll( "a[href^=https\:\/\/example\.com\/test\/]" );

Replacing content

Sadly, it isn't quite as simple as setting the innerHTML. Each search returns a node. That node may have children. Those children will also be node which, themselves, may have children, and so on.

Let's take a simple example:

$html = '<h2>Hello</h2>';
$dom = \Dom\HTMLDocument::createFromString( $html, LIBXML_NOERROR | LIBXML_HTML_NOIMPLIED, "UTF-8" );
$element = $dom->querySelector( "h2" );
$element->childNodes[0]->textContent = "Goodbye";
echo $dom->saveHTML();

That changes "Hello" to "Goodbye".

But what if the element has child nodes?

$html = '<h2>Hello <em>friend</em></h2>';
$dom = \Dom\HTMLDocument::createFromString( $html, LIBXML_NOERROR | LIBXML_HTML_NOIMPLIED, "UTF-8" );
$element = $dom->querySelector( "h2" );
$element->childNodes[0]->textContent = "Goodbye";
echo $dom->saveHTML();

That outputs <h2>Goodbye<em>friend</em></h2> - so think carefully about the structure of the DOM and what you want to replace.

Adding a new node

This one is tricky! Let's suppose you have this:

<div id="page">
   <main>
      <h2>Hello</h2>

You want to add an <h1> before the <h2>. Here's how to do this.

First, you need to construct the DOM:

$html = '<div id="page"><main><h2>Hello</h2>';
$dom = \Dom\HTMLDocument::createFromString( $html, LIBXML_NOERROR | LIBXML_HTML_NOIMPLIED, "UTF-8" );

Next, you need to construct an entirely new DOM for your new node.

$newHTML = "<h1>Title</h1>";
$newDom = \Dom\HTMLDocument::createFromString( $newHTML, LIBXML_NOERROR | LIBXML_HTML_NOIMPLIED, "UTF-8" );

Next, extract the new element from the new DOM, and import it into the original DOM:

$element = $dom->importNode( $newDom->firstChild, true ); 

The element now needs to be inserted somewhere in the original DOM. In this case, get the h2, tell its parent node to insert the new node before the h2:

$h2 = $dom->querySelector( "h2" );
$h2->parentNode->insertBefore( $element, $h2 );
echo $dom->saveHTML();

Out pops:

<div id="page">
   <main>
      <h1>Title</h1>
      <h2>Hello</h2>
   </main>
</div>

An alternative is to use the appendChild() method. Note that it appends it to the end of the children. For example:

$div = $dom->querySelector( "#page" );
$div->appendChild( $element );
echo $dom->saveHTML();

Produces:

<div id="page">
   <main>
      <h2>Hello</h2>
   </main>
   <h1>Title</h1>
</div>

And more?

I've only scratched the surface of what the new 8.4 HTML Parser can do. I've already rewritten lots of my yucky old preg_ code to something which (hopefully) is less likely to break in catastrophic ways.

If you have any other tips, please leave a comment.

I've since re-written it to be faster and more stylistically correct.

It turns this:

<html lang="en-GB"><head><title id="something">Test</title></head><body><h1 class="top upper">Testing</h1><main><p>Some <em>HTML</em> and an <img src="example.png" alt="Alternate Text"></p>Text not in an element<ol><li>List</li><li>Another list</li></ol></main></body></html>

Into this:

<!doctype html>
<html lang=en-GB>
    <head>
        <title id=something>Test</title>
    </head>
    <body>
        <h1 class="top upper">Testing</h1>
        <main>
            <p>
                Some 
                <em>HTML</em>
                 and an 
                <img src=example.png alt="Alternate Text">
            </p>
            Text not in an element
            <ol>
                <li>List</li>
                <li>Another list</li>
            </ol>
        </main>
    </body>
</html>

I say it is "opinionated" because it does the following:

• Attributes are unquoted unless necessary.
• Every element is logically indented.
• Text content of CSS and JS is unaltered. No pretty-printing, minification, or checking for correctness.
• Text content of elements may have extra newlines and tabs. Browsers will tend to ignore multiple whitespaces unless the CSS tells them otherwise.
  • This fucks up <pre> blocks which contain markup.

It is primarily designed to make the markup easy to read. Because according to the experts:

A computer language is not just a way of getting a computer to perform operations but rather … it is a novel formal medium for expressing ideas about methodology. Thus, programs must be written for people to read, and only incidentally for machines to execute.

I'm fairly sure this all works properly. But feel free to argue in the comments or send me a pull request.

Here's how it works.

When is an element not an element? When it is a void!

Modern HTML has the concept of "Void Elements". Normally, something like <a> must eventually be followed by a closing </a>. But Void Elements don't need closing.

This keeps a list of elements which must not be explicitly closed.

$void_elements = [
    "area",
    "base",
    "br",
    "col",
    "embed",
    "hr",
    "img",
    "input",
    "link",
    "meta",
    "param",
    "source",
    "track",
    "wbr",
];

Tabs 🆚 Space

Tabs, obviously. Users can set their tab width to their personal preference and it won't get confused with semantically significant whitespace.

$indent_character = "\t";

Setting up the DOM

The new HTMLDocument should be broadly familiar to anyone who has used the previous one.

$html = '<html lang="en-GB"><head><title id="something">Test</title></head><body><h1 class="top upper">Testing</h1><main><p>Some <em>HTML</em> and an <img src="example.png" alt="Alternate Text"></p>Text not in an element<ol><li>List</li><li>Another list</li></ol></main></body></html>>'
$dom = Dom\HTMLDocument::createFromString( $html, LIBXML_NOERROR, "UTF-8" );

This automatically adds <head> and <body> elements. If you don't want that, use the LIBXML_HTML_NOIMPLIED flag:

$dom = Dom\HTMLDocument::createFromString( $html, LIBXML_NOERROR | LIBXML_HTML_NOIMPLIED, "UTF-8" );

To Quote or Not To Quote?

Traditionally, HTML attributes needed quotes:

<img src="example.png" class="avatar no-border" id="user-123">

Modern HTML allows those attributes to be unquoted as long as they don't contain ASCII Whitespace or certain other characters

For example, the above becomes:

<img src=example.png class="avatar no-border" id=user-123>

This function looks for the presence of those characters:

function value_unquoted( $haystack )
{
    //  Must not contain specific characters

    $needles = [ 
        //  https://infra.spec.whatwg.org/#ascii-whitespace
        "\t", "\n", "\f", "\n", " ", 
        //  https://html.spec.whatwg.org/multipage/syntax.html#unquoted 
        "\"", "'", "=", "<", ">", "`" ];
    foreach ( $needles as $needle )
    {
        if ( str_contains( $haystack, $needle ) )
        {
            return false;
        }
    }
    //  Must not be null
    if ( $haystack == null ) { return false; }
    return true;
}

Re-re-re-recursion

I've tried to document this as best I can.

It traverses the DOM tree, printing out correctly indented opening elements and their attributes. If there's text content, that's printed. If an element needs closing, that's printed with the appropriate indentation.

function serializeHTML( $node, $treeIndex = 0, $output = "")
{
    global $indent_character, $preserve_internal_whitespace, $void_elements;

    //  Manually add the doctype to start.
    if ( $output == "" ) {
        $output .= "<!doctype html>\n";
    }

    if( property_exists( $node, "localName" ) ) {
        //  This is an Element.

        //  Get all the Attributes (id, class, src, &c.).
        $attributes = "";
        if ( property_exists($node, "attributes")) {
            foreach( $node->attributes as $attribute ) {
                $value = $attribute->nodeValue;
                //  Only add " if the value contains specific characters.
                $quote = value_unquoted( $value ) ? "" : "\"";

                $attributes .= " {$attribute->nodeName}={$quote}{$value}{$quote}";
            }
        }

        //  Print the opening element and all attributes.
        $output .= "<{$node->localName}{$attributes}>";

    } else if( property_exists( $node, "nodeName" ) &&  $node->nodeName == "#comment" ) {
        //  Comment
        $output .= "<!-- {$node->textContent} -->";
    }

    //  Increase indent.
    $treeIndex++;
    $tabStart = "\n" . str_repeat( $indent_character, $treeIndex ); 
    $tabEnd   = "\n" . str_repeat( $indent_character, $treeIndex - 1);

    //  Does this node have children?
    if( property_exists( $node, "childElementCount" ) && $node->childElementCount > 0 ) {

        //  Loop through the children.
        $i=0;
        while( $childNode = $node->childNodes->item( $i++ ) ) {

            //  Is this a text node?
            if ($childNode->nodeType == 3 ) {
                //  Only print output if there's no HTML inside the content.
                //  Ignore Void Elements.
                if ( 
                      !str_contains( $childNode->textContent, "<" ) && 
                    property_exists( $childNode, "localName" ) && 
                          !in_array( $childNode->localName, $void_elements ) ) 
                {
                    $output .= $tabStart . $childNode->textContent;
                }
            } else {
                $output .= $tabStart;
            }

            //  Recursively indent all children.
            $output = serializeHTML( $childNode, $treeIndex, $output );
        };

        //  Suffix with a "\n" and a suitable number of "\t"s.
        $output .= "{$tabEnd}"; 

    } else if ( property_exists( $node, "childElementCount" ) && property_exists( $node, "innerHTML" ) ) {
        //  If there are no children and the node contains content, print the contents.
        $output .= $node->innerHTML;
    }

    //  Close the element, unless it is a void.
    if( property_exists( $node, "localName" ) && !in_array( $node->localName, $void_elements ) ) {
        $output .= "</{$node->localName}>";
    }

    //  Return a string of fully indented HTML.
    return $output;
}

Print it out

The serialized string hardcodes the <!doctype html> - which is probably fine. The full HTML is shown with:

echo serializeHTML( $dom->documentElement );

Next Steps

Please raise any issues on GitLab or leave a comment.

The full set of modern email standards allows for such wonderful addresses as: chief.o'brien+ds9@spásárthach.भारत

So determining whether or not your user has entered a valid email address becomes an ever-increasing challenge.

But what if you have the opposite issue? Suppose you have a form which takes something which mustn't be an email address?

For example - imagine you have a login form which requires a username. And yet your stupid users keep entering their email addresses instead! How can you quickly, cheaply, and simply detect email addresses?

Well, no matter whatever else is in an email address, there will always be an @ somewhere. The @ is necessary but not sufficient for an email address.

So you can use HTML5's pattern attribute.

Here it is in action:

<input type="text" 
       pattern="^[^@]+$">

That says "don't accept anything with the @ symbol in it".

We can take it a bit further and add some more attributes to help our users

<input type="text" 
       pattern="^[^@]+$"
       required
       placeholder="Your username (which is NOT your email address)"
       title="Don't use your email address!!!!!"
       id="notemail">

Try it out!

As soon as you type an @ the form will go into an invalid state and can't be submitted. You won't be able to click the login button.

Should You Use This?

Well… perhaps in a pinch. The real solution is to be agnostic towards your users' choice of login. Some may prefer to use an email, some a username. Some may expect their username to start with an @. Be tolerant of their needs and desires.

That meant I couldn't see the images on the other half of the page for the rest of the journey. I had a sea of broken images.

Even though the page had fully loaded, the images were set to "lazy" loading. When my browser sees <img src="example.jpg" loading="lazy" /> it defers downloading the picture until it is about to scroll into view.

There's a few reasons why this is a good thing. If I leave the page before scrolling down, the browser doesn't waste time, energy, or bandwidth downloading images which will never be seen. That's good for my battery life and cheaper for my bill. It also saves the server from flinging bits around the world which are promptly trashed⁰.

And yet...

It seems that every time I'm on a train - or somewhere else with spotty Internet access - I run in to a situation where I previously had plenty of bandwidth and battery, but my user-agent decided it knew best. So it didn't attempt to load images until after I lost signal.

There's no way to disable Lazy Loading on Android Chrome or Android Firefox.

So, I guess I'm stuck. Lazy loading seems like one of those things which is great in certain circumstances, but has a pretty horrible failure mode¹.

Anyone else feel this way?

Using a mouse? Nope, that won't work either.

The only way to navigate to it is via keyboard navigation. Hit your Tab ⭾ button!

There's a little bit of me wants to build an entire website which can only be navigated by keyboard. What would the world look like if Engelbart never invented the mouse? Or if Johnson never published his work on touchscreens?

Anyway, there are two ways to do this. The first is to create an anchor with no content.

<a href="https://example.com/"></a>

Because there's no content between the opening and closing <a> elements, most browsers won't present it as a target for a mouse cursor or finger.

There's a second method, which only works at discouraging mouse use. It's possible to style a cursor with CSS. And that style can be "none"

*, *:hover { 
  cursor: none !important; 
}

The cursor is still there, but it is invisible. Which makes it difficult to know where your mouse is clicking.

Now, should you do this? No. It is silly. Reducing accessibility like this is never a good idea. But it is a fun experiment.

Thanks to Manuel Matuzović for inspiring me.

I'm going to go with a cautious "no - not quite". Although there are several proxies which get you part of the way there.

Here's a link to a telephone number "call me!" - the HTML is:

<a href="tel:+441234815259">call me!</a>

You can use tel: in just the same way you'd use http: or mailto:. It tells the User Agent to open the correct program to deal with a task - either a web-browser, email client, or - in this case - the telephone dialler.

But what happens if the user is on a desktop browser? Or using a tablet computer? Or on their smart TV? Usually an error message about how the device doesn't understand the link.

So, is there a way to use feature detection to only show a tel: to devices which can handle it?

User Agent Sniffing

This practice has been discredited for over a decade. UA Strings tend to lie. And, even when they don't lie, they're not very precise.

You could use this to discover if someone is running Android - but that doesn't tell you if it's a phone, tablet, TV, or VR headset.

NetworkInformation type

The navigator object contains network information. That should tell you if the device is on a 2G/3G/4G/5G connection.

Of course, your tablet also uses those connections. And your phone might be on WiFi.

Other properties

It is possible to detect if a device is multi-touch, has a battery, is a mobile device, can vibrate, etc.

All of which, taken together, might indicate that the device is a phone. But it is far from certain.

A Big Database Of Products

Companies like 51Degrees will sell you access to their database - which includes reporting whether a specific device supports phone calls. As will DeviceAtlas.

There used to be an Open Source database called WURFL - but ScientiaMobile closed down open access to it.

To be fair, 51Degrees have a generous free tier - but it is a little depressing that this sort of feature detection is locked in a proprietary product.

If you have any ideas how to solve this please give me a call!

But computers, traditionally, haven't been very good at parsing ambiguous human text.

Suppose you saw this text written for a human:

Our opening hours are: Weekdays 10 until 7. Weekend 10 until 10 (Early closing 9 o'clock Sunday).

Not the most straightforward sentence, but pretty easy for a human to parse.

Until recently, the best way to represent that for a computer was something like:

<meta itemprop="openingHours" content="Mo-Fr 10:00-19:00">
<meta itemprop="openingHours" content="Sa 10:00-22:00">
<meta itemprop="openingHours" content="Su 10:00-21:00">

or

<script type="application/ld+json">
{
   "@context": "https://schema.org",
   "openingHours":["Mo-Fr 10:00-19:00", "Sa 10:00-22:00", "Su 10:00-21:00"]
}
</script>

A tightly constrained vocabulary which can be precisely parsed by a simple state-machine. Easy to ingest, interpret, and query. Easy for machines, that is. As much as I love the semantic web, it is hard for humans to write, update, and maintain.

But we have AI now. So do we need to mark up documents specifically for machines?

I fed the text into OpenAI's ChatGPT. Here's what it said:

Do we need to write for computers any more? One of the demands of the Semantic Web was that we should use HTML elements like <address> to clearly mark up a contact address and we should wrap dates and times in the <time> element.

Is that now redundant?

We still need to write clearly and unambiguously. But do we need separate "machine-readable" HTML if machines can now read and interpret text designed for humans?

I wanted a <select multiple> element where the <options> were laid out in a grid. I nearly got there.

It's possible to have the <option>s in a horizontal row - but only on Chrome and Firefox.

Here's a quick fiddle showing the results:

As you can see, it's possible to do some pretty extravagant styling of the individual <options> you can even change how they look when they're selected.

But it's impossible to get them to wrap using a flexbox or grid approach. It seems the only layout options are a vertical or horizontal list.

Why, yes, I should probably have used <input type="checkbox"> instead!

Now, it behaves like this:

The code is relatively straightforward. The HTML for the calendar looks like this:

<div class="calendars">
    <div class="calendar">
        <div class="calendar-year">2018</div>
        <div class="calendar-month">January</div>
        <div class="calendar-month">February</div>
        <div class="calendar-month">March</div>
        <div class="calendar-month">April</div>
        <div class="calendar-month">May<br></div>
        <div class="calendar-month">June</div>
        <div class="calendar-month">July</div>
        <div class="calendar-month">August</div>
        <div class="calendar-month">September</div>
        <div class="calendar-month">October</div>
        <div class="calendar-month">November</div>
        <div class="calendar-month">December</div>
    </div>

    <div class="calendar">
        <div class="calendar-year">2017</div>
        <div class="calendar-month">January</div>
        <div class="calendar-month">February</div>
                ...

Repeat as many times as needed.

The CSS is slightly more tricky, but mercifully short:

The wrapper which holds all the calendars uses flex-wrap. This means there's no need to set explicit breakpoints - if there's no horizontal space, the calendar will move to the next row.

.calendars {
    display: flex;
    flex-wrap: wrap;
}

Each individual calendar item is a grid with three column layout:

.calendar {
    display: grid;
    grid-template-columns: repeat(3, 1fr);
    margin: .5em;
}

The "Year" at the top spans the first three columns. I find the grid-column-... syntax confusing!

.calendar-year {
    grid-column-start: 1;
    grid-column-end: 4;
    text-align: center;
    outline: 1px black solid;
}

Finally, each individual month has a bit of internal padding and text formatting:

.calendar-month {
    text-align: center;
    padding: .25em;
    outline: 1px black solid;
}

I've used a basic black outline. Obviously, you can add whatever fancy CSS you like to make them look prettier.

And... I think I've done it!

I'll warn you now, this is a deeply stupid way to solve the problem.

Here's a screencast of the demo in action:

Type the password and the page decrypts!!!!!

This abuses some interesting CSS features.

First, you can display the <style> element on the page.

<style>
   style {
      display: block;
   }
</style>

Secondly, you can make the CSS editable by the user with contenteditable

<style contenteditable="true">...</style>

As soon as a user types into the CSS, it is applied to the page. No need for JS.

So if a user types in a password...?

Let's step back a bit. How can we encrypt text using CSS? I know... WEBFONTS!

A WOFF2 webfont is a Brotli compressed file. If I've read the spec correctly (LOL!) removing a chunk of a small file should render the file too damaged to read.

It's possible to convert the WOFF2 into Base64 and use it in the CSS:

@font-face {
   font-family:'encrypt_sans';
   src:url('data:application/font-woff2;charset=utf-8;base64,d09GMg...') format('woff2');
}

Cut a chunk out of the middle of that font, and use that missing piece as the password. Foolproof!

But - I hear you say - how to encrypt text using a font? Well, that's easy!

That cleverclogs Eli Grey has a font which encrypts text. It's magical. Well, OK, it's ROT13. Obviously, any substitution cipher can probably be broken using frequency analysis.

Of course, it is possible to use font ligatures to obfuscate the text even more. See Bullshit Sans as an example

So, there you have it. A way to sort of encrypt a statically served HTML file without using JavaScript.

Demo and Source

• Demo
• Source

Possible Improvements

This was a demo hastily put together while hungover one weekend. There's lots of room for improvement.

The UI abuses CSS to hide some of the boilerplate involved. It could be made to look nicer.

There's no way to generate an "encrypted" font. Ideally someone (not me!) would take a plaintext and generate a scrambled and ligatured font to automagically do this.

It is inaccessible to screen readers. The font doesn't change the underlying text.

Brotli compressed WOFF2 files might be recoverable even after substantial damage.

This is really really stupid.

I recently found a popular website which echoed back user input. It correctly sanitised < to < to prevent any HTML injection.

Except…

It let through <h2> elements unaltered! Why? I suspect because the output was:

<h2>Your search for ... returned no results</h2>

And, somehow, the parser was getting confused. OK, what can we do with this little vector?

The first thought is to use Javascript event handlers like onclick() or onmouseover() - but they were (sensibly) blocked. The <h2> element only has access to the Global Attributes. So we could inject content which use Right-To-Left text, or add some metadata attributes - but that's not particularly useful.

The most useful Global Attribute is style="". Yup! Good old CSS!

The limitation of the style attribute is that it only applies to the specific element it is attached to. So you can't rewrite the entire page. You also can't use the content: property - as that only applies to ::before and ::after directives.

Using normal CSS, we can change the colour and size of our newly injected <h2>, faff around with the background colour, change the font, and move it about the page. Good for a bit of digital graffiti, but not much else.

Except…

What about using background-image? Using that, we can pull in an external resource.

<h2
  style="background-image:url('https://evil.site/whatever.png');
  width:512px;
  height:512px;"> ... </h2>

That will load an external picture on the site. It could be an animated GIF saying "You're a winner! Visit www.... to claim your prize!". It wouldn't be clickable, but might catch a few people out.

It is possible to load an SVG. And SVG can contain JS. But - alas and alack - the JS doesn't run in background mode - even if the JS is bundled as Base64.

Changing the content-disposition of the image won't force the browser to download it, either.

And that, I think, is about the limit of it. If Javascript is blocked, the worst you can do is inject a malicious image. Short of finding a zero-day in a browser's codec, all that can happen is a bit of temporary defacement.

But Wait! There's More!

While fuzzing around with the input, I made an interesting mistake. I mistyped <svg> as <sgv>. That invalid element was added to the page's HTML! That means there's a parser somewhere which is stripping out only the elements it knows about. Browsers typically ignore elements they don't understand - so there's no danger to users there. But it points to the idea that there may be some elements the sanitiser doesn't know about.

And, indeed, there were! For example, it happily took the obsolete <plaintext> element, and dumped it into the markup. Which caused the page to break.

It also, delightfully, took the <marquee> element!

Terence Eden is on Mastodon

@edent

Replying to @edentOMG…!
I legit did not expect that to work… pic.x.com/5q4ymmzjdg

---

I reported the issue immediately, and got an acknowledgement. But, sadly, after a few months the website was still not fixed.

Take a bow, ACM Digital Library!

<html lang="en-GB">
...
As Caesar said: <i lang="la">veni vidi vici</i>

That says the page is in British English (en-GB) but the specific phrase is in Latin (la). But how can you offer an in-text translation of that phrase into the page's native language?

Here are a few options - and their drawbacks.

Title Text

<i lang="la" title="I came, I saw, I conquered">veni vidi vici</i>

veni vidi vici

The user has to hover their pointer over the text and a pop-up will appear with the translation. There are two disadvantages to this:

1. Not all devices - like mobile browsers - support title text.
2. The title text has no separate language attribute - so is semantically in Latin.

The language can be corrected by wrapping the title in a separate span.

Tables

The humble <table> can present two or more items of text adjacent to one another.

<table>
   <tr>
      <td lang="la">veni vidi vici</td>
      <td lang="en">I came, I saw, I conquered</td>
   </td>
</table>

Tables can be problematic on narrow screens - either requiring wrapping or scrolling.

Details

<details>
    <summary lang="la">veni vidi vici</summary>
    I came, I saw, I conquered
</details>

Again, it requires interaction - which may not work on devices like eReaders. Unfortunately, details is a block element, but you can read my experiments in making them inline.

Ruby

As suggested by John Ribbens

<ruby lang="la">
   veni vidi vici
   <rt lang="en-GB">I came, I saw, I conquered</rt>
</ruby>

veni vidi vici I came, I saw, I conquered

That works quite well - although Ruby text is pretty small. But it can be styled with CSS.

Ruby is usually used for showing pronunciation of characters. But, crucially, it isn't restricted to that.

Description Lists

<dl>
    <dt lang="la">veni vidi vici</dt>
    <dd>I came, I saw, I conquered</dd>
</dl>

veni vidi vici

I came, I saw, I conquered

Again, very easy to style with CSS. One of the nice things about Description Lists is that it allows for multiple definitions:

<dl>
    <dt lang="la">veni vidi vici</dt>
    <dd>I came, I saw, I conquered</dd>
    <dd lang="ja">私は私が征服した来た</dd>
</dl>

MIX THEM ALL TOGETHER!

Let's take a section from Chaucer's Canterbury Tales. Most of the Middle English is understandable - but a few archaic words need translation. It's also useful to have some commentary on the text.

<dl>
   <dt lang="enm">Full many a fat partridge had he in 
      <ruby>mew<rp>(</rp><rt lang="en-GB">cage</rt><rp>)</rp></ruby>
   </dt>
   <dd>The place behind Whitehall, where the King's hawks were caged was called the Mews.</dd>
</dl>

<details>
   <summary lang="enm">And many a bream, and many a 
      <ruby>luce<rp>(</rp><rt lang="en-GB">pike</rt><rp>)</rp>
      in 
      <ruby>stew<rp>(</rp><rt lang="en-GB">fish-pond</rt><rp>)</rp>
   </summary>
   In those Catholic days, when much fish was eaten, no gentleman's mansion was complete without a "stew".
</details>

Full many a fat partridge had he in mew(cage)

The place behind Whitehall, where the King's hawks were caged was called the Mews.

Which should you use?

Yes.

There's no definitive "correct" answer here. title text might make sense for occasional words which need translating - and you're sure either the user's device supports it, or they won't be substantially disadvantaged if it doesn't.

Similarly, details works for interactive content which is optional to understanding.

The ruby elements are great if you want a fairly unobtrusive way to translate specific words.

Lists are great if you need to offer multiple translations.

Mashing them all together is a bit silly and complicated - but allows for a greater variety in the way the texts are displayed.

"When you visit the zoo, be sure to see the panda - 熊(Xióng)猫(māo)."

This is written as:

<ruby>熊<rp>(</rp><rt>Xióng</rt><rp>)</rp></ruby><ruby>猫<rp>(</rp><rt>māo</rt><rp>)</rp></ruby>.

That is, the word or character which needs text above it is wrapped in <ruby>. The pronunciation is wrapped in <rt>. The <rp> element indicates the presence of a parenthesis - which isn't usually displayed, but will be shown if the browser doesn't support <ruby> syntax.

That's fairly easy for scripts written left-to-right. But how does it work for scripts like Arabic where the text is written right-to-left, but the user may want the pronunciations left-to-right?

Let's take the phrase "Hello World" in Arabic: مرحبا بالعالم. Google Translate tells me this is pronounced "marhaban bialealami".

For a single word, the directionality can be ignored. The browser should be smart enough to place the pronunciation above the word:

<p>Hello is: <ruby>مرحبا<rp>(</rp><rt>marhaban</rt><rp>)</rp></ruby>. What a useful word!</p>

Hello is: مرحبا(marhaban). What a useful word!

What about if we have a few words - or a whole sentence - which is entirely RTL?

<p dir="rtl">مرحبا بالعالم</p>

Is displayed aligned to the right side of the screen:

مرحبا بالعالم

There are a few ways to add pronunciation.

Separate The Words

The first is to write each word separately. For example <ruby>1st word</ruby> <ruby>2nd word</ruby>. Obviously, this isn't normally how you'd write a RTL language! But it does work:

<p dir="rtl"><ruby>مرحبا<rp>(</rp><rt>marhaban</rt><rp>)</rp></ruby> <ruby>بالعالم<rp>(</rp><rt>bialealami</rt><rp>)</rp></ruby></p>

Which displays as:

مرحبا(marhaban) بالعالم(bialealami)

It helps to think of the way the characters of the script are stored in memory.

A word that displays as ABC is stored as C B A.

So the above is written "correctly" - even though it looks odd in the source-code view.

All At Once

But there is an alternative if you want the source text to look natural - i.e. [2nd word] [1st word].

It's a bit messy, but you can write the LTR text in <rt> "backwards"!

<p dir="rtl"><ruby>مرحبا بالعالم<rt>bialealami marhaban</rt></ruby></p>

مرحبا بالعالمbialealami marhaban

But, again, that doesn't seem very satisfying! It also divorces the pronunciation from the original word - which is unfortunate for screenreaders.

The Ruby layout algorithm is usually clever enough to group words separated by spaces:

مرحبا بالعالمB A

مرحبا بالعالمBbbbbbbbbbbbbb Aaaaaaaaaaaaa

Although, if the pronunciations have a significantly different length than each other, it can get a bit messy:

مرحبا بالعالمBbbbbbbbbbbbbb A

مرحبا بالعالمB Aaaaaaaaaaaaa

In which case, you probably need to go for the first technique and wrap each word in its own <ruby> element:

مرحبا(A) بالعالم(Bbbbbbbbbbbbbb)

BDO

It's tempting to think that simply using the <bdo> element can help us here. It can't!

Using the bidirectional override will display characters RTL, rather than words.

<p dir="rtl"><ruby>مرحبا بالعالم<rt><bdo dir="rtl">marhaban bialealami</bdo></rt></ruby></p>

Becomes:

مرحبا بالعالمmarhaban bialealami

I guess you could spell each word backwards. Which would be extremely annoying for everyone and a complete nightmare for screen readers!

Instead, it can be fixed if each word is then given an explicit LTR direction:

<p dir="rtl"><ruby>مرحبا بالعالم<rt>
   <bdo dir="rtl">
      <span dir="ltr">marhaban</span> <span dir="ltr">bialealami</span>
   </bdo></rt></ruby></p>

مرحبا بالعالم marhaban bialealami

Is that it?

So, I think those are the only ways to achieving mixing bidirectional text pronunciation. But I'd welcome any corrections and suggestions!

When I first started designing the OpenBenches website, I wanted to make it look deliberately crappy. I didn't want the people testing it getting too hung up on what it looked like. I've found that some beta testers can only focus on whether the colours are right, or if things should be placed on the left or right.

So I wanted something which mimicked "Paper Prototyping". A website which looked vaguely hand-drawn. As though it were scraps of paper and Post-It® Notes stuck on a roll of canvas.

That way, people can immediately see that the design is deliberately "scrappy" and temporary.

I've decided to go a little further than my last experiment and create a fairly comprehensive Style Sheet which you can use on any new project - Paper Prototype CSS.

Here's what it looks like:

To be clear, this is not designed to look good - quite the opposite. It is a base to build on so you can build something quick and dirty.

Use it to throw together something where you want to concentrate on the content, not the presentation.

Features

The askew elements use nth child selectors to rotate them in different directions:

*:nth-child(odd):not(body):not(html){
   transform: rotate(2deg);
}
*:nth-child(even):not(body):not(html) {
   transform: rotate(-2deg);
}

The weird border radius uses:

border-radius: 255px 15px 225px 15px/15px 225px 15px 255px;

With thanks to

Everything is based on something else. I took heavy inspiration from, and made liberal use of:

• Handdrawn CSS
• Doodle CSS
• Poor Man's Styleguide
• Paper Texture Background
• Handwritten body font and monospace font

Download

You can clone the repo to get started. If you find it useful, please let me know. Pull Requests very welcome.

Suppose you have an article written in HTML. The basic layout might be something like:

<body>
   <main>
      <article>
        The content of your article
        ...

Pretty standard. Now suppose you let users add comments to the article. I have two questions:

1. Where in the tree should they go?
2. What HTML element should be used to group them?

It is, I think, a question where reasonable parties can justifiably come to distinctly different conclusions.

Where?

Are comments a part of the article or are they apart from the article?

If you wrongly think that user-generated content is indivisible from the text of the article, then the answer is:

<body>
   <main>
      <article>
         The content of your article
         <div>
            Comments

If, however, you're fool enough to believe that commentary is separate from what is commented on, then you'll probably pick:

<body>
   <main>
      <article>
        The content of your article
      </article>
      <div>
         Comments

Some idiots will insist that comments shouldn't even be considered next to the purity of the content of <main>. Hence:

<body>
   <main>
      <article>
        The content of your article
      </article>
   </main>
   <div>
      Comments

As ever, let us see what wisdom the HTML editors passim have given us.

When article elements are nested, the inner article elements represent articles that are in principle related to the contents of the outer article. For instance, a blog entry on a site that accepts user-submitted comments could represent the comments as article elements nested within the article element for the blog entry.

4.3.2 The article element

Ah! This is evidence that comments should be inside the article itself.

The code they give is (simplified):

<article>
   <h1>Page Heading</h1>
   <p>...</p>
   <section>
      <h1>Comments</h1>
      <article>
         <p>Comment text</p>
     </article>

But, that's not the whole story!

A section forms part of something else. An article is its own thing. But how does one know which is which? Mostly the real answer is "it depends on author intent".

[…]

A comment on an article is not part of the article on which it is commenting, therefore it is its own article.

4.3.12.1 Article or section?

Although that section is non-normative, I think it is instructive.

The article is its own thing. A comment may relate to an article, but it is not part of the article. Therefore, it's perfectly reasonable to place the comments outside the original article.

What?

Ideally, there would be a <comment> element. There isn't.

Bruce Lawson

@brucel

The <dialog> element is added to #html5 dev.w3.org/html5/spec/com… I'm glad to see it (accessible dialogue boxes!) even if it is badly mis-spelled

---

Terence Eden is on Mastodon

@edent

Replying to @brucelWhy the hell isn't there a <comment> element in #html5? @brucel? Who's arse do I need to ki[ss|ck] to make this happen?

---

As shown in the above code example, comments can be enclosed in a <section> element. And each comment itself can be wrapped in an <article>.

article
A complete, or self-contained, composition in a document, page, application, or site […] This could be a forum post, […] a blog entry, a user-submitted comment […] or any other independent item of content.

Article Usage Summary

I had thought the <aside> element might also be suitable. It represents:

content that is tangentially related to the content around the aside element, and which could be considered separate from that content.

Are comments tangentially related? Not really. And comments don't generally appear inside the thing they're commenting on. So I discarded that idea.

Why is this important?

It isn't, really. Good semantic metadata should make clear what is content and what is commentary.

But not everyone consumes metadata. Bots, scrapers, and search engines should have a sensible way of heuristically determining what is relevant to them based on the markup of the document. Unless we get a new <comment> element, we need to provide hints to user-agents about exactly what we mean.

I took an extremely scientific poll on Twitter to see if my acolytes agreed with me.

Terence Eden is on Mastodon

@edent

A two part question about semantic HTML.

1) What's the "correct" HTML elements for user generated comments?
Why?

---

<aside>: (42)
42
<section>: (64)
64
<div>: (30)
30
<bgsound>: (15)
15

---

Terence Eden is on Mastodon

@edent

Replying to @edentContinuing:
2) What should the document tree look like?
Why?
Currently, I have
<body>
<main>
<article>
</article>
<~comment~>
But should the comments be inside the article? Or perhaps outside the main?

---

<article><~comment~></art: (16)
16
</article><~comment~>: (36)
36
</main><comment>: (4)
4
<div><div><div><div><div>: (9)
9

---

So, <section> wrapped around a group of <article>s, all of which are outside the <article> being discussed seems to me to be a sensible way to demarcate content from commentary. Leaving us with:

<body>
   <main>
      <article>
        The content of your article
      </article>
      <section>
         <h1>Comments</h1>
         <article>
            Comment
         </article>
         <article>
            Comment
         </article>
      </section>
   </main>
</body>

Do feel free to tell me I'm wrong in the comments. The box is down there, somewhere…

The standard WordPress comments HTML isn't very semantic - so I thought I'd change that. Here's some code which you can add to your blog's theme - an an explanation of how it works.

The aim is to end up with some HTML which looks like this (edited for brevity):

<li itemscope itemtype="https://schema.org/Comment" itemid="#246827">
    <article>
        <time datetime="2022-04-12T10:22:32+01:00" itemprop="dateCreated">
            <a href="https://shkspr.mobi/blog/2022/04/semantic-comments-for-wordpress/#comment-246827" itemprop='url'>22-04-12 10:22</a>
        </time>
        <div itemprop="https://schema.org/author" itemscope itemtype="https://schema.org/Person">
            <img itemprop="image" alt='' src='logo.jpg' height='64' width='64'/>
            <span itemprop="name">
                <a itemprop="url" href="https://twitter.com/example">Commenter's Name</a></span> says:
        </div>
        <div itemprop="text">This is the text of my comment</code></div>
    </article>
</li>

Which will be interpreted as:

This adds <time> elements as well as Schema.org microdata.

Howto

In comments.php you'll see something like this:

<ol class="comment-list">
    <?php
        wp_list_comments( array(
            'style'       => 'ol',
            'short_ping'  => true,
            'avatar_size' => 64,
        ) );
    ?>
</ol>

You need to add a new callback. In this case, I've called it my_comments_walker:

<ol class="comment-list">
    <?php
        wp_list_comments( array(
            'style'       => 'ol',
            'short_ping'  => true,
            'avatar_size' => 64,
            'callback' => 'my_comments_walker',
        ) );
    ?>
</ol>

You can read more about WordPress Walkers on their documentation page.

Now that's done, you need to create a function in your functions.php file. I added this to the end of my file:

function my_comments_walker() {

    //  Basic comment data
    $comment_id          = get_comment_id();
    $comment             = get_comment( $comment_id );

    //  Date the comment was submitted
    $comment_date        = get_comment_date( "c" );
    //  In slightly more human-readable format
    $comment_date_human  = get_comment_date( "y-m-d H:i" );

    //  Author Details
    $comment_author      = get_comment_author();

    //  Author's URl if they've added one
    $comment_author_url  = get_comment_author_url();

    //  If there's an Author URl, link it
    if ($comment_author_url != null) {
        $comment_author_name = "<a itemprop='url' href='{$comment_author_url}' rel='external nofollow ugc' class='url'>{$comment_author}</a>";
    } else {
        $comment_author_name = "{$comment_author}";
    }

    //  Provide a link to the comment anchor
    $comment_url_link = "<a href="https://shkspr.mobi/blog/2022/04/semantic-comments-for-wordpress/#comment-$comment_id" itemprop='url'>{$comment_date_human}</a>";

    //  Author's Avatar based on ID
    //  As per https://developer.wordpress.org/reference/functions/get_avatar/ both alt & default must be set
    $gravatar            = get_avatar( $comment, 64, "", "", array('extra_attr' => 'itemprop="image"') );

    //  Comment needs newlines and links added
    $comment_text        = apply_filters( 'comment_text', get_comment_text(), $comment);


    //  The comment may have various classes. They are stored as an array
    $comment_classes     = get_comment_class();
    $comment_classes_text = "";
    foreach( $comment_classes as $class ) {
        $comment_classes_text .= $class . " ";
    }
    $comment_classes_text = trim($comment_classes_text);

    //  Link to open the reply box
    $comment_reply_link = get_comment_reply_link( [
                    'depth'     => 20,
                    'max_depth' => 100,
                    'before'    => '<div class="reply">',
                    'after'     => '</div>'
            ] );

    //  Write the comment HTML. No need for a closing </li> as WP handles that.
    echo <<<EOT
    <li id="comment-$comment_id" itemscope itemtype="https://schema.org/Comment" itemid="#comment-$comment_id" class="$comment_classes_text">
        <article class="comment-body" id="div-comment-$comment_id">
            <time datetime="$comment_date" class="comment-meta commentmetadata" itemprop="dateCreated">
                $comment_url_link
            </time>
            <div class="comment-author vcard" itemprop="https://schema.org/author" itemscope itemtype="https://schema.org/Person">
                $gravatar
                <span class="fn" itemprop="name">$comment_author_name</span> <span class="says">says:</span>
            </div>
            <div itemprop="text" class="comment-text">$comment_text</div>
            $comment_reply_link
        </article>
    EOT;
}

There are a few extra classes and spans which I use. You can remove them if you like.

And that's it. All your comments will have individual semantic metadata. If you think anything else should be included, please let me know.

Pre-formatted text is also editable
Just tap in here and type away.
Fun!

(The following bit isn't editable.)

Learn more about contenteditable on MDN.

If you've spotted any weirdness on your browser when interacting with this post - please drop me a note in the comments.

An ISSN is an 8-digit code used to identify newspapers, journals, magazines and periodicals of all kinds and on all media–print and electronic.

Why?

Shut up.

OK. It turns out that lots of people cite my blog in academic papers - so I wanted to make it slightly easier for scholars of the future to use metadata to trace my vast influence on Human civilisation.

How?

I filled in a form on the British Library website. Didn't cost me a penny. Was pretty quick!

Metadata

I can stick a bit of text at the bottom of each page with the ISSN - but that doesn't make it easily discoverable by automated tools. How can I make an ISSN machine readable? There are a few ways.

Meta Elements

There are a limited list of official <meta> names. These are extensible, and Google Scholar recommends citation_issn. Which is as simple as adding the following to your page's <head>:

<meta name="citation_issn" content="1234-5678">

There alternatives though.

Schema.org

In recent years, Schema.org has become the dominant form for representing metadata on the web. There are two ways you can implement it:

JSON-LD

JSON Linked Data involves adding a scrap of JavaScript to your HTML, like this:

<script type="application/ld+json">
{
  "@context": "https://schema.org",
  "@type": "Blog",
  "issn": "1234-5678"
}
</script>

If you don't want to add a separate script, you can add the data inline using...

Microdata

The microdata specification uses the exact same data as Schema.org - but allows you to add the data directly into the web page like this:

<body itemscope itemtype="https://schema.org/Blog">
   ...
   ISSN <span itemprop="issn">1234-5678</span>

That's probably the easiest way to do it.

Links

The ISSN registry allows you to look up any ISSN with a simple URL. Mine is at https://portal.issn.org/resource/ISSN/2753-1570.

Belt and braces

So, this is what I've ended up doing - cramming everything in all at once.

<head>
   ...
   <meta name="citation_issn" content="1234-5678">
</head>
<body itemscope itemtype="https://schema.org/Blog">
   ...
   ISSN <a href="https://portal.issn.org/resource/ISSN/1234-5678"><span itemprop="issn">1234-5678</span></a>

Any other ways?

What am I missing? Can someone smarter than I tell me that there's an easier / better / more interoperable way to do this?

MDN gives a big scary warning saying "Non-standard: This feature is non-standard and is not on a standards track. Do not use it on production sites facing the Web."

And yet, it works everywhere. All modern browsers support it. Except on Emoji.

Here's how it work. -webkit-text-stroke: pink 1px; draws a pink outline around text.

This has a pink outline!

Nifty! But what happens if you apply to emoji?

Smile 😁, Star ⭐, Melon 🍈

Nothing.

Interestingly any emoji with default text presentation - or using Variation Selector 15 - does get an outline ☺☮⭐︎

Why would this be useful

Here's an emoji which has the same roughly the colour as the background. Can you see it clearly? What if you select the text?

 ❤️ 

Having an outline would make it a lot easier to see.

So, what's so special about emoji? I guess that the way browsers render emoji is more similar to images rather than text? For example, their colours don't invert when selected.

Does anyone know a way to make this work? Or whether -webkit-text-stroke will ever be standardised?

Sadly, I don't have a Virtual Reality system - feel free to buy me one! - but I have the next best thing. A web browser!

Demo!

Using the awesome power of A-Frame, here's a demo of how to view a conversation object as AR.

Play!

You can play with it yourself at TweeView.ml/ar. You will need to download this AR target image.

It works best printed onto matt white paper, then mounted on card. I had to print it twice on the same piece of paper to get the required blackness.

Build!

This is built on several bits of open source tech.

The first part is A-Frame and AR.js

<script src="https://aframe.io/releases/1.0.0/aframe.min.js" integrity="sha384-mAWNMpx7UQMegsgs1X2mE9wVl/svgdzdy4qpXNFgAxUmc8a1ToRWVWl3tzfDinBb" crossorigin="anonymous"></script>
<script src="https://unpkg.com/ar.js/aframe/build/aframe-ar.min.js" integrity="sha384-vLi7A1RlJjrEmCqF2DrdIT9cLqHdV/7GbmsqfniwwtfLRQlP1lXi/Tk199ZCLU5p" crossorigin="anonymous"></script>

Next is ForceGraph AR

<script src="https://unpkg.com/3d-force-graph-ar" integrity="sha384-ERqZX7SBS3fH2SL04csG8OB9QyOF2LtDsZtRrJZR46WipKv7LqplBgM/GoFPjyfQ" crossorigin="anonymous"></script>

I recommend copying them to your local server rather than relying on a CDN.

The <body> of your HTML page needs a target the AR window can attach to:

<div id="3d-graph"></div>

Construct the graph which will be displayed in the browser:

<script>
   const Graph = ForceGraphAR({ markerAttrs: { type:'pattern', url:'/path/to/pattern.patt' }})
      (document.getElementById('3d-graph'))
         .jsonUrl('/path/to/graph.json')
         .glScale(160)
         .yOffset(1)
         .nodeRelSize(10)
         .nodeOpacity(0.9)
         .linkOpacity(0.9)
         .nodeColor(0x0000ff)
         .linkOpacity(1)
         .linkCurvature(0)
         .linkWidth(2)
         .linkDirectionalArrowLength(6)
         .linkDirectionalArrowColor(0x00ff00)
         .linkColor(() => 'red');
</script>

In order to generate a "target" (that's the physical object that is being tracked) use the AR.js Marker Generator. That will deliver you two things.

1. A .patt file. This is a digital representation of the pattern.
2. A .png or .pdf file. This is the thing you print off and show in front of your camera.

Finally, you'll need a graph object in JSON format to pass to ForceGraphAR.

You'll then end up with something like this:

Ruben Casas 🦊

@Infoxicador

Attending my first in person event since the pandemic.

#TapIntoTwitter hosted by @gerardsans and @coderinheels

Nothing better than a cool live demo.

🤯 pic.x.com/qw1ifom7fh

---

Grab the code

All of this is available as open source on my GitHub repo.

Enjoy!