I wanted to go the other way. How good are modern codecs at dealing with tiny files?

Using GIMP, I created an image which was a single white pixel, and saved it as a PNG. I then used Squoosh to convert it to a variety of modern formats using different encoding options. This is what I found:

This is designed to be the "minimum viable viewable image". I loved Stoyan Stefanov's "Minimum viable no-image image src". That creates a 42 byte SVG with no image data in it - so I thought I'd see what happens if you make a displayable image.

Some important things to note:

• Older image formats like BMP and GIF are smaller than newer formats like AVIF.
• Some compression options make files larger in unexpected ways. The lossy WebP was larger than the lossless version.
• Similarly, increasing the effort on an AVIF can also result in a larger filesize.
• Neither JPEG XL nor QOI are supported in mainstream browsers yet.
• AVIF has a rather long and complex header - that makes sense for large images, but bloats it for smaller ones.
• Using Brotli, it is possible to further compress the AVIF (203 Bytes), JPG (69 Bytes), BMP (63 Bytes), and ICO (30 Bytes) files.
• WebP is the smallest file if you still need a Spacer.gif!

Here's my challenge to you - can you do any better? What's the smallest filesize you can find for a viewable image?

If I set the compression to be lossy - the photo looks good but the UI looks bad. If I set the compression to be lossless - the UI looks good but the filesize is huge.

Is there a way to selectively compress different parts of an image? I know WebP and AVIF are pretty magical but, as I understand it, the whole image is compressed with the same algorithm and the same settings.

There are two ways to do this. The impossible way and the cheating way.

Selective Compression

In theory it should be possible to tell an image format to compress some chunks of an image with a different compression algorithm.

And yet... none of the documentation I've found shows that's possible.

GiMP's native XCF and Photoshop's PSD files work; they store different layers each of which can have a different filetype. I understand that TIFF and .djvu also have that capability.

But those sorts of files don't display in web browsers.

So...

Let's Cheat!

It's possible to use an SVG to embed multiple images of different formats. SVG is used as, effectively, a layout engine.

The syntax is relatively straightforward:

<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 0 1080 512">
   <image width="1080" height="512" x="0" y="0"
      xlink:href="data:image/jpeg;base64,..........."
   />
   <image width="1080" height="512" x="0" y="0"
      xlink:href="data:image/png;base64,..........."
   />
</svg>

That draws the JPG then draws the PNG on top of it. If the PNG has a transparent section, the JPG will show through. The JPG can be set to as low a quality as you like and the PNG remains lossless.

Here's what it looks like - click for full size:

Embedded images are Base 64 encoded, which does lose some of the compression advantages. But, overall, it's smaller than a full PNG and better quality than a full JPG.

Look, if it's stupid but it works it's not stupid.

But surely there must be a way of doing this natively?

It strikes me that AI might now get us part way to that being a reality.

Traditionally, images are stored in raster format - essentially a grid of pixel values. These files tend to be rather large, so compression is used to make them smaller using increasingly complex schemes.

Similarly, vector graphics - which are usually written as text - can be compressed with all sorts of wonderful algorithms.

So what happens if we add AI to the mix?

A self-hosted AI image generator takes, very roughly, a few hundred GB of disk space, plus a large amount of RAM, and some fairly hefty compute. But, once that's done, you could "compress" images by specifying an AI engine and weighted prompts like this:

<img src="data:image/AI;MidJourney/Mona_Lisa+/Original_Style+++/Ornate_Frame---/600x800/" alt="An AI generated Mona Lisa" />

Perhaps that's a little far fetched. But back when DVDs first came out, people had to buy specialised hardware cards to decode the MPEG compression on their computers. Nowadays a cheap Raspberry Pi does the trick. In the future, all our computers will have multi-terrabyte AI models baked in.

But people generally want to ensure that the receiver sees roughly the same picture as the sender. Something you can't easily guarantee with a generative model.

What about this idea from Ben Hardill?

Take something like a ThumbHash of an image, which is less than 1KB:

You could, theoretically, embed that in HTML using something like:

<img src="thumbhash.png" width="256" upscale="DALL-E" keyword="Field of grass with cloudy skies" />

And get:

I think of things like the psychoacoustic compression of MP3 files and wonder whether those crappy 64kbps rips I have from the 1990s could be magically restored to sonic perfection by an AI? Or perhaps I'll just ask ChatGPT to sing me a song, in the style of Paul McCartney, called "Maybe I'm Amazed".

I don't know if that's the future. It's certainly one future.

I tried to explain to them that DRM always fails; you can't make data which can't be copied. I explained that artificial scarcity was harmful. They didn't care.

But, their proposed solution was intriguing. And, with their kind permission, I'm posting it here. To be clear, I don't think this is good but I think it is vaguely interesting.

Interlacing

Back in the olden days, video was often interlaced. That is, the TV transmitter would broadcast the odd lines first, then the even lines. Your TV would draw the odd line then, in the millisecond before they faded away from the cathode-ray tube, draw the even lines.

Could we do the same with images?

Let's take this image of a Total Legitimate Bored Ape™

With a scrap of Python, we can knock this into two interlaced images:

from PIL import Image
image = Image.open("ape.png")
pixels = image.load()
image.mode = "RGBA"

for y in range(image.height) :
   if (y%2 == 1) :
      for x in range(image.width) :
         pixels[x,y] = (0,0,0,0)

image.save("ape1.png")

for y in range(image.height) :
   if (y%2 == 0) :
      for x in range(image.width) :
         pixels[x,y] = (0,0,0,0)

image.save("ape2.png")

Which gives us:

Using JavaScript the first frame can be rendered onto a <canvas> element for a millisecond, wiped, and then the second displayed. Repeat.

Here's a demo of it in action - I kinda like the retro effect!

⚠️ Strobe Effect Warning

If you right click on it, you'll only get half the image. If you take a screenshot, you'll only get half the image. FOOLPROOF DRM!!!!!!

Expanding the complexity

Of course, you can scramble an image before unscrambling it to the canvas and all sort of other tricks. You could have dozens of images rendering to the canvas in sequence. You could draw individual pixels programmatically. You could...

None of it matters of course. If you send pixels to a computer screen, they can be copied. It is possible to slow people down - but all it takes is for one person to work out how your scheme works and then it crumbles to dust.

As I said, I have a soft-spot for the shimmery effect. And I think it is kinda fun - in a code-golf way - to design obfuscation schemes.

But using anything like this for a form of digital restrictions management is daft.

What exists already?

A typical bitmap image looks like this under the hood:

  0      1      2     3
0 Black  Red    Red   Blue
1 Red    White  Blue  Yellow
2 Orange Purple Green Brown
...

That is a grid of pixels, each with a colour value.

Modern image formats can also contain pixel-level detail about transparency. When an image is displayed above a different image, the computer calculates how much of the lower image's colour is mixed with the upper image's colour.

Some image formats also allow pixel-level detail about brightness¹. A device might be able to set the brightness on individual parts of the image.

So you end up with an image which looks like:

  0                    1                             2     3
0 Black                Red - but 50% transparent     Red   Blue
1 Red - but really dim White - as bright as possible Blue  Yellow
...

Thermal / Infrared

Most image formats only deal with visible light. But there are a bunch of cameras which can capture infrared. Would it be useful to capture the heat of scene being photographed?

A future image format could contain details about visible and invisible light.

Invisible Wavelengths

Objects in the real worlds don't just give off thermal signatures - they often have information in the ultraviolet wavelengths as well.

For example, flowers often have "landing guides" which are only visible to bees and other insects.

Being able to capture beyond visible light may be a useful property for an image format.

Depth

Modern cameras often fire out an invisible infrared grid to allow for quick focussing and to aid with depth effects like bokeh. An image could store the depth map of its subject.

Here's a series of depth maps I made from analysing 3D movies:

This could allow for better 3D displays, or for easier image editing.

Angle

Images are typically 2D. The present pixels on a flat rectangular plane. But they aren't limited to that. Image panoramas and "spherical" images allow for display on a non-flat surface.

Here's an equirectangular 2D image which is rendered as a sphere:

An image could contain pixel-level data showing the angle away from the viewer. That would allow for a more accurate rendering in VR or other non-flat displays.

Object Detection

Now we start to get funky! AI is getting pretty good at detecting specific objects in photographs. What happens if we start marking them up within the image itself?

Either something as basic as this:

Or as detailed as this:

That would make it possible to easily remove or replace objects from images.

Combined with depth-sensing, it could be a powerful way to edit images.

Extra Texture

Colour replication depends heavily on the material which is imbued with that colour. That's what Pantone is all about². Suppose an object has a sky-blue colour - is it useful to know that it is a heavy woollen jumper rather than a thin nylon t-shirt?

Wheels within wheels and layers within layers

It's probably sensible to implement something like this as multiple layers, rather than saying every pixel must be RGBA+Depth+Heat+Angle+Object+etc+etc.

Different layers could have different resolution (Infrared is typically lower res than the image it is overlayed on). Different layers might be compressed more efficiently by different algorithms.

We're used to layered images in formats like PSD and XCF - so why not in a future version of JPEG or AVIF?

What else?

What other things might be useful for an image to store at an individual pixel-level? Drop a note in the comments?

If you have a lot of images in a directory, and want to quickly make an image montage, here's how to do it on Linux using ImageMagick.

First up, this command finds all JPG file, then resizes them so they fit in a maximum box of 256x256, then sets the quality to 75%, then saves them as JPGs:

find ./ -type f -iname "*.jpg" -exec mogrify -verbose -format jpg -layers Dispose -resize 256\>x256\> -quality 75% {} +

This will overwrite your existing files so make sure you have a backup.

This will take all JPG files, and then arrange them in a montage which is 10 images wide and 5 images high, with no padding between the images, and then save them as a file called "montage.png":

montage *.jpg -tile 10x5 -geometry +0+0 montage.png

If you have more than 10x5 images, this will spit out multiple files. If you want to specify the width, but not the height, use:

montage *.jpg -tile 10x -geometry +0+0 montage.png

Handy if you can't be bothered counting how many images you have.

The ordering is "sort of alphabetical". That is, it sorts them by first character. You may want to renumber your files as 00001.jpg etc if you have a specific ordering in mind.

The output looks like this:

This time, I'm taking a different Nook, and turning it into a magic gallery. Here's what it looks like in action:

Terence Eden is on Mastodon

@edent

Upcycled an old eReader into an art frame.
Displays a new black & white piece of art from Flickr every few minutes.
Full write-up this weekend, but pretty straightforward to do. pic.x.com/ttvrbhz3ee

---

Video

With the front light off:

With the front light on:

How it works

Every few minutes, it finds a black and white artistic image from Flickr and displays it. Simple!

Inspiration

Ambient home cinema by Tom Whitwell

Build

I've previously replaced the battery on an eInk Nook, so I know how easy they are to disassemble. But for this I wanted to keep the plastic frame on, so that I'd be able to attach a more æsthetically pleasing wooden picture frame.

Finding a picture frame of the right size was a little tricky. The physical dimension of the unit are ~170mm x ~140mm. But the screen is only ~125mm x ~93mm .

I used eFrame - they have a really handy website for custom sized frames and inserts. I ordered a mount for an 135mm x 103mm picture (they cut them slightly smaller but I didn't want any overhang) with 20mm top and bottom borders, and 35mm left and right borders. That gave me enough space to fit the Nook. But, as long as the aperture lets you see the eInk, and masks off the plastic frame, you should be OK.

I was also able to buy a frame without a glass front. As this is my first attempt, I went for simple and cheap wood - rather than something more ornate and artistic. Total cost including shipping was £25.

Attaching the frame

It is useful to be able to get to the main "N" button - and there needs to be a gap for the charging cable. I thought about cutting out a little bit of the frame but, in the end, it was easier to attach the Nook to the frame using velcro and blu-tak. Because of the weight of the Nook, I used some sturdy hooks for the wall mount.

Power

There wasn't enough room in the frame for a normal USB cable, so I used a right-angled micro-USB cable. With the page refreshing every 5 minutes between 0600-2300, it got about 3 days of battery life - less if it used the Glow-Light. So a permanent wired connection seemed sensible.

Background

On the surface this seems quite simple - get a bunch of images and display them. But there are some limitations.

The original Nook Simple Touch is, bless it, an old and slow device. I want to do the minimum amount of processing on it as possible. The screen resolution is only 600x800. I don't want to convert colour artwork to black and white - that will just look messy. I also want the image size to roughly match the screen - so things aren't shrunk down.

So, I want a list of art which meets the following criteria:

• Intended for monochrome display⁰
• Roughly 4:3 aspect ratio
• Proper art¹ by real artists²

I started by looking at the Open Data produced by various museums and art galleries. Unfortunately, none of them indicated whether an artwork was greyscale. So, I turned to Flickr!

Using the Flickr API

Sadly, the Flickr API has been neglected for the best part of a decade. So the documentation is grossly out of date.

Two of the undocumented Search API calls are styles and orientation. I use them to search for black and white images in landscape orientation. You can discover more by using the Flickr search website and opening the "Advanced" panel.

The rest of the API call is quite standard. A keyword to search for ("art") - or whatever you want. Sorted by "interestingness" - although popularity is also a good metric. With safe_search turned off - no prudes in our house. And an image size no wider that 800px - because serving a correctly scaled image means less work for the eReader.

The Code

It is almost embarrassing in its simplicity. It uses Sam Wilson's PHP library for Flickr. It grabs 100 images, and then picks a random one to serve. It is resized to fit in a bounding box of 800x600 and repaginated to centre the image. The image is rotated it 90 degrees and served over HTTP with an appropriate caching policy. Nothing else. Here it is³.

require_once 'vendor/autoload.php';
$flickr = new \Samwilson\PhpFlickr\PhpFlickr("123465789");

$search_terms = ["art", "artistic", "painting", "drawing"];
$search_rand = array_rand($search_terms, 1);

$sort_terms = ["date-posted-asc", "date-posted-desc", "date-taken-asc", "date-taken-desc", "interestingness-desc", "interestingness-asc", "relevance"];
$sort_rand = array_rand($sort_terms, 1);

$s = $flickr->photos()->search(["text"        => $search_terms[$search_rand],
                                "per_page"    => "100",
                                "styles"      => "blackandwhite",
                                "orientation" => "landscape",
                                "sort"        => $sort_terms[$sort_rand],
                                "safe_search" =>"3"
                               ]);

$number_of_photos = count($s["photo"]);

$random_photo = random_int(0, $number_of_photos - 1);

$p = $s["photo"][$random_photo];

$farm   = $p["farm"];
$server = $p["server"];
$id     = $p["id"];
$secret = $p["secret"];
$title  = $p["title"];

$image_url = "https://live.staticflickr.com/{$server}/{$id}_{$secret}_c.jpg";

$image = imagecreatefromjpeg( $image_url );

$cropped = imagecropauto( $image, IMG_CROP_DEFAULT );
if ( $cropped !== false ) { 
   imagedestroy($image);  
   $image = $cropped;
}

$x = imagesx( $image );
$y = imagesy( $image );

$width_ratio  = $x / 800;
$height_ratio = $y / 600;

// Scale
if ( $height_ratio >= $width_ratio ) {
   $new_height = ( 600 / $y ) * $x;
   $scaled = imagescale( $image, $new_height, 600 );
} else {
   $scaled = imagescale( $image, 800, -1 );
}

$x = imagesx( $scaled );
$y = imagesy( $scaled );


// Make a background canvas
$canvas = imagecreatetruecolor( 800, 600 );
$black  = imagecolorallocate( $canvas, 0, 0, 0 );
imagefilledrectangle( $canvas, 0, 0, 800, 600, $black );

$dst_x = ( ( 800 - $x ) / 2 );
$dst_y = ( ( 600 - $y ) / 2 );

imagecopy( $canvas, $scaled, $dst_x, $dst_y, 0, 0, $x, $y );

$rotated = imagerotate( $canvas, 90, 0 );

header('Content-type: image/jpeg');
header("Expires: on, 01 Jan 1970 00:00:00 GMT");
header("Last-Modified: " . gmdate("D, d M Y H:i:s") . " GMT");
header("Cache-Control: no-store, no-cache, must-revalidate");
header("Cache-Control: post-check=0, pre-check=0", false);
header("Pragma: no-cache");

imagejpeg( $rotated );

// Tidy Up
imagedestroy( $image );
imagedestroy( $scaled );
imagedestroy( $rotated );

Displaying

As I mentioned in my previous post about displaying web pages on the Nook - the browser is slow, old, and can't handle https connections.

So I used ElectricSign which is an old Android app. Give it a URL and it will display the contents on screen. It can be set to periodically refresh.

I've had it running for months on a different screen and it hasn't crashed.

That's it!

TODO

• Buy more cheap, 2nd hand eInk screens on eBay. Preferably larger and with higher DPI.
• Get some fancy frames.
• Start a pirate art museum.
• Monetise it by tracking how long people stare at a particular artwork.
• BitCoin?⁴

Thanks for reading

If you've enjoyed this blog post, you can buy me something from my wishlist or sponsor my GitHub. Find out more way to support this blog.

So, here are four quick tips for dealing with this formal.

Display in browser

Absolutely no browser supports HEIC. Not even Apple's own browser supports it. Why?

Terence Eden is on Mastodon

@edent

OK, but *why* is the iPhone's default photo format HEIC if the iPhone's default browser can't display it?
caniuse.com/?search=heic

---

🅨🅞🅤 ⓜⓤⓢⓣ 𝘣𝘦 𝖘𝖊𝖊𝖎𝖓𝖌 .

@ursonate

Replying to @edent@edent The two teams hate each other.

---

So, we have to use JavaScript. There's a brilliant open source library called libheif. It's possible to compile it to JavaScript - but I couldn't get it to work. The official repo doesn't make a compiled version available. Luckily, Kiril Vatev has released a compiled emscripted version.

It's about 1.5MB - which is a bit abusive to your users - although it zips down to about 350KB.

The image has to be loaded into a 2D canvas to be rendered. Even on a modern laptop this is likely to take a few seconds.

There's a demo page which shows you how to get started. It's tedious, but it works.

Get Metadata

Similarly, there's a separate library for reading HEIC metadata. This one is a much more reasonable 20KB.

That's pretty easy to use - and quick. HEIC contains pretty standard EXIF. GPS co-ordinates, camera information, that sort of thing.

Server Side - PHP

I struggled to find any libraries which were easy to use. It turns out, good old imagick has support in modern versions.

$imagick = new Imagick();
$imagick->readImage('image.heic');
$exifArray = $imagick->getImageProperties("exif:*");
var_dump($exifArray);

Use a CDN to convert images

I use Cloud Image (Affiliate Link). Use it proxy your images. You can convert them to JPG or a modern format like WebP. It can resize and other effects.

You can write something like:

<img src="https://abc.cloudimg.io/v7/example.com/image.heic/?w=600"/>

to serve your HEIC images in a more sensible format.

What next?

HEIC is a weird, patent-encumbered format. It isn't particularly useful. If you want high-quality images for editing, you probably want to shoot in RAW. If you're displaying on the web, WebP gets you a small file size and broad compatibility.

But, if you want to deal with HEIC, I hope the above was useful.

Here are three quick tips for shaving a few bytes off an SVG.

Decimal Magic

SVG co-ordinates can have decimal precision, like: 123.456. But what about co-ordinates which are less than one? 0.123 can be rewritten as .123 - we can drop the 0!

These two sed commands will turn 0. to . and -0. to -.

sed -i -z "s/ 0\./ \./g" *.svg
sed -i -z "s/\-0\./\-\./g" *.svg

What's my (new) line?

For boring technical reasons to do with teletype printers, Windows machines use two characters to represent a newline. CR the Carriage Return send the print-head back to the start of the line, then LF is the Line Feed to move the paper by one line.

Unix just uses LF. Because no-one uses teletypes.

The brilliant dos2unix program will convert all Windows-style newlines to Unix style. Magic!

Last Line

Most text editors add a newline to the end of any file you create. Why? Who knows.

Remove any trailing newlines from the file with:

sed -i -z s/\\n$// *.svg

Using the above three commands, I've been able to shave off dozens of bytes from these files. A worthy endeavour, I'm sure you'll agree!

If you want to make images as small as humanly possible, come join us.

It's a project to create SVG logos of popular services in under 1KB. Maximum filesize is 1,023 bytes.

New Contributors!

What started off as just me noodling around, has now attracted over 60 contributors! Some just drive-by and make a single change, others stay for a while and contribute loads. Which is nice.

New icons!

The project started with 80 icons. Since then we've grown to over 240!

Some are hip new apps like TikTok (447 Bytes)

Some are obscure programming logos like Cloujure (632 Bytes)

Some have nifty 3D depth effects like S3 (766 Bytes)

New Name!

It was originally Super Tiny Social Icons - but we've dropped the "The" "Social" to better reflect what it actually does.

New Smallerising Tricks!

I've learned so much from the contributors. This MineCraft (891 Bytes) one uses Base64 encoded PNGs and transforms really well.

This Raspberry Pi (1,010 Bytes) logo uses a similar mirroring trick.

Both Firefox (1,023 Bytes) and Instagram (768 Bytes) use gradients for great effect.

New Packages!

The icons are published as a Node Package so you can incorporate it in your fancy JavaScript projects.

Join In!

If you'd like to suggest a new logo, or build one yourself, or tidy something up - please get stuck in.

Progressive mode is useful because it can quickly load a low resolution preview of an image, and then gradually improve its quality.

WebP, by contrast, just loads up the full image line by line. That's can be annoying on a slow connection.

What if we could change that? Here's a quick-and-dirty hack. This is a 5472x3080 image which has been compressed down to "zero" quality on WebP and resized to 640x360. It is less than 5KB.

The original image is 400 KB as a WebP and 1.6MB as a JPG.

There are lots of techniques to load low-res images before the main image loads.

We're going old-skool. Basic HTML + CSS.

<img
    width=5472
    height=3080
    alt="Close up of a Raspberry Pi circuit board."
    style="background-image: url('Small.webp');"
    src=Big.webp
>

This sets the background of the image to the low-resolution, low quality version. As the high-resolution image loads, it gradually replaces the image in the background.

There are a few more refinements we could make to this:

<img
    width=5472
    height=3080
    alt="Close up of a Raspberry Pi circuit board."
    style="background-color: #0f0;
           background-image: url('Small.webp');"
    loading="lazy"
    src=Big.webp
/>

This sets the background-color to the dominant colour of the image. That way, even before the background image has loaded, you get something to fill the gap. Because we've set loading="lazy" the massive image won't be downloaded until the img scrolls into view.

We can even go a step further. Let's reduce the image down to a thumbnail of 160x90. That takes us down to a ludicrously small 564 Bytes.

Which makes it small enough to stick directly in the HTML once we encode it in Base64:

background-image: url("data:image/webp;base64,UklGRiwCAABXRUJQVlA4ICACAACQEwCdASqgAFoAP/3+/3+/vLYyPv+8A/A/iWYIkCfrS2sOTiSUArLU15JWpmwHGTJDQ+i/y0UfRMWET8I422NGGsABPAdIKWnTFvlfPnla/wC0DPVYABJxuECkmqWUZwViSm1vqzvPgAnG+vioXrwz0V3Zx+c/znFFJvKBpH5SceJwmAUi8+8SRjzQvyc/qv3DKe3KfEYnXA+5aEBypmacRUvrvgAA+OUzEDYgEioYgJQRm/l9OIIl60M80PJW00cW6fX1fNpB/L4udWsUF5v6gOR6fB//5/0zuirXi2Z2GoqpVh3aeUDEdVThFtQOZStD/ulvCFO54uEqPVZlD63ukgBgt5Q5KA15Nse9lu4E+4XliSgao9azVod9zUr/XELtUpd2CLSyImxIXq3rRNGthLv0jePmYvlqijrhFxgMsdVer/GmM0C6xEyTC53yBQ+aGJlg/iqxrU5uwNjGNi3Or75QpJVdWW0lEyZlOsIiRsCZ+jTXsDMmgzkB4uaiiQASLl4TIYpK587PmaLb29WRF3hxpAtjR4TPDHne3iY6aBfYZhvMo1N41AMMKe7BLZg3w8LCQ2RHRHnkhgYsdtLL2jSFFpP8JgVtwLZ/KtKa6P7VaHm7aY7aT0UWyEwXPnUbZwo5a4yuK1H+P7YJqLj8yjuKSJZACY6z+6RgRxeoSyytYvQiopOOlc4f1Mdoy2WgfE3HR6Ap5jJiPryYLQAA")

Of course, you don't have to go down to quite this extreme level - choose some settings which make sense for you and your media.

To summarise:

1. Scale the image down
2. Reduce its quality
3. Set it as the background to the img element

Endnote

I wasn't involved with the development of WebP - but it seems bizarre to me that it doesn't contain a "thumbnail mode". On that ½ MB photo, adding a couple of KB doesn't seem like a huge overhead.

Similarly, the experimental AVIF also lacks progressive / thumbnail support.

Anyone know what the reason is?

But what about interactive images? Like panoramas and photospeheres?

Here's a 360° image. You can drag it to see all around.

That uses the fantastic Pannellum JavaScript Library. At the moment, there's no native way to represent that in HTML. If you use an <img> you'll get a flat, equirectangular image like this:

Here's two possible ways I've just thought of...

(These are just sketches of ideas - think of them as a starting point for user research, rather than a completed product.)

Extend <img> to be like <video>

An HTML5 video element looks something like this:

<video controls=false autoplay=true>

That tells the browser to start playing the video, but not to show any playback controls in the UI. Panoramic images could use something like:

<img panorama=true photosphere=false xdegress=270 src="...">

That is, this image is a panorama - but not a sphere - with 270 degrees of movement on the horizontal axis.

Or

<img panorama=true photosphere=true startx=120 starty=330 src="...">

Display a photosphere, make the initial view start at the following co-ordinates.

Use <picture>

As I've said before, I'm no fan of the <picture> element. I think it is inconsistently designed compared to other parts of HTML5.

Perhaps we could use it like this:

<picture>
  <source type="image/jpeg;panorama=true" srcset="pano.jpg">
  <img src="flat.png" alt="your browser doesn't support panoramics">
</picture>

That gives the MIME type of the image an extra parameter. Obviously, you'd need to put in other configuration options something.

What next?

A quick search didn't show anyone working on this as a proposal. That normally means either I'm not looking in the right places, or no one else is interested.

If you think this could be developed into a future standard, please let me know!

All "adaptive icons" means is that every unique icon has to be constrained in a circle. It makes every icon look monotonous and, in some cases, shrinks the icon so that they're hard to see even on large screens.

There's a cognitive element here. When I look at my screen for an icon, I tend to use colour and shape to find what I'm looking for. With every icon now a circle, and half of them trapped in a white background, it makes looking for the right icon harder.

Boring!

Better!

Æsthetics

This is the way I personally prefer my icons. I don't care that they're not a uniform shape. I don't care that some of them have a lower centre of gravity than others. I don't care that the corner radius is different on each one. I don't care that they don't all line up perfectly like ticky-tacky little boxes.

I want them not to be the same.

Getting the icons

Here's how to fix it.

This is the Gmail adaptive icon - tiny and indistinct.

The Gmail app lives at: https://play.google.com/store/apps/details?id=com.google.android.gm

Inspecting the source code shows the nice large icon lives at: https://lh3.ggpht.com/8-N_qLXgV-eNDQINqTR-Pzu5Y8DuH0Xjz53zoWq_IcBNpcxDL_gK4uS_MvXH00yN6nd4=s180

Removing the =s180 from the end of the URl gives us the full 512x512 image. That's suitable for use as an alternative icon.

OK, but how do we get the icon from the Play Store URl? Google intentionally make this difficult. You'd think that it would be <div id="icon"><img id="icon-small" ...> or something. Nope!

Here's what the source looks like:

What the flippity-flack is going on there? Purposefully uglified code designed to obfuscate and frustrate?

At the moment, it looks like all icon images have class="T75of ujDFqe" - but I wouldn't want to bet on that being stable.

There's no official API for Google Play - but there are unofficial paid-for APIs and some open source scrapers of varying usefulness.

I ended up using Daniel Liu's Python Play Store Scraper.

This quick scrap of code take's an app's package name, downloads the large size icon, compresses it using WebP, and saves it.

import play_scraper
import requests
from PIL import Image
import webp

packageName = "com.android.chrome"
icon = Image.open(requests.get(play_scraper.details(packageName)["icon"], stream = True).raw)

# Save as png
icon.save(packageName + ".png", optimize=True)

#Save as WebP
webp.save_image(icon, packageName + '.webp', lossless=True)

Setting the icon

Most modern Android homescreens allow you to manually set the icons of each app. Long press on an app, then choose edit, then select the icon you want. Easy! (ish)

If I Was Clever

If I had the skill, I'd create an Android app which looked at the app name, downloaded the icon from the play store, and used that as the launcher icon.

Or, I'd fork an existing icon pack and manually add the top 500 apps.

But I'm not that good at Android apps. If anyone else wants to make it, please do so!

Let's talk image compression! Services like Twitter will often apply aggressive levels of compression in order to reduce their storage space and decrease download times. This can have negative consequences for usability and image quality.

Here's an example - this detail of a logo from my former employers, Vodafone. Solid red - with some fine detail in white:

If you upload it to Twitter, it will automatically be compressed to a low quality JPG. And this is what it looks like...

Yuck! Look at the grimy artefacts surrounding the text! By default, the image quality that most websites choose is 85%. That's not sufficient for images like this.

How To Fix It

PNGs have an interesting property that JPG images don't - they can be transparent. When Twitter sees even a single transparent pixel, it refuses to convert the original image and keeps it as a PNG.

Using GIMP - or any other photo editing tool - you can crop out a pixel from the image:

You can see a demonstration at:

Terence Eden is on Mastodon

@edent

JPEG compression is terrible for solid blocks of colour (especially red) next to fine details. By default, Twitter compresses your PNGs to JPG. The only way to avoid it is to make sure at least one pixel is transparent. See alt text for details. pic.x.com/skavjdelt6

---

If you can't bear to have a "missing" pixel - you can set a single pixel's opacity to 90%. That will also prevent compression.

Does this only work on Twitter?

Sadly, yes.

LinkedIn displays an 85% quality JPG in the preview.

They will let you see the original PNG once you click through.

In my experiments, Facebook compressed the transparent PNG to a 71% quality JPG.

That looks nasty! There appears to be no way to download the original.

ReDeCentralise

One of the advantages of hosting your own content is that you - the user - get to choose what is an appropriate trade-off between quality and filesize.

I was originally going to use this method:

Terence Eden is on Mastodon

@edent

Replying to @6byNine@6byNine it is all I've got so far!

---

Mike

@6byNine

Replying to @edent@edent No idea about getting urls, but this may or may not be useful (cmd is also in image alt-text) pic.x.com/jh3yh4yki0

---

but, sadly, gnome-web-photo crashed repeatedly when I tried to use it. It is a typically unloved Linux program, hundreds of bug reports, but no one looking to fix them.

Instead, I used the Scheenshotlayer API - I passed wget the URLs and got back a bunch of full sized screenshots.

wget doesn't name the files nicely - so you just get the full URl as the filename. Here's a quick bash script to rename them all with padded zeroes.

var=1;
for file in *;
do
   printf -v num '%02d' $var ;
   mv ${file} ${num}.png;
   var=$((var+1));
done

OK, so, how do we grab just the parts that we want? Rather than using any fancy image recognition stuff, I measured the area by hand.

And, to cut the image out? ImageMagick to the rescue! Let's grab a rectangle 624 pixels wide, but 900 tall, starting at 400 pixels from the left and 0 pixels from the top.

convert tweet.png -crop 624x900+400+0 +repage cropped.png

The +repage is needed to prevent offsetting issues.

And, to do that for all images in a directory:

mogrify -crop 624x900+400+0 +repage *.png

Right! Let's lay these out. Without a doubt, the simplest way is using ImageMagick's montage command.

montage *.png -tile 4x10 -geometry +0+0 montage.png

This lays out all the images on a grid 4 images wide and 10 high. The -geometry ensures there is no border around the image.

I then resized the image suitable for sticking on a web page.

The result?

TODO

The only rough spot in this process is cropping the images. At the moment, there's a lot of empty space around the Tweets. Some - with smaller photos - have gaps at the top and bottom. A more intelligent method of extracting the information is needed!

If you hover your mouse over this image, you should be able to select part of the text.

Ideally, it will look something like this...

It even works on Android (tried on Chrome, Opera, FireFox) and iOS 7.

So, how did I do it?

Originally, I was pointed to Project Naptha - it seems to do everything I want but is very JavaScript heavy and requires modern browser support.

I then turned to SVG - Scalable Vector Graphics.

The way I've done this is almost certainly wrong and I'd appreciate any advice about the proper way to render text in an SVG.

The first part is easy - displaying a PNG as the background to the SVG. In this case, I've taken the image and Base64 encoded it.

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<svg
   xmlns="http://www.w3.org/2000/svg"
   version="1.1"
   xmlns:xlink="http://www.w3.org/1999/xlink"
   width="566"
   height="166"
>
<image xlink:href="data:image/png;base64,iVBORw0KGg...."
   x="0"
   y="0"
   width="566"
   height="166" />

The X & Y co-ordinates are from the top left. I've manually added in the height and width of the image.

Next, we add the text.

   <g fill-opacity="0">
      <text
         x="70"
         y="45"
         font-size="14"
         font-family="serif"
         textLength="415"
         lengthAdjust="spacingAndGlyphs">
         For nearly three years, between 1960 and 1963, MI5 and GCHQ
      </text>
      <text
         x="42"
         y="62"
         font-size="14"
         font-family="serif"
         textLength="440"
         lengthAdjust="spacingAndGlyphs">
         read the French high grade cipher coming in and out of the French
      </text>
      ...
   </g>
</svg>

As you can see, I've grouped the text together in a <g> element. I've set the opacity to zero - so while they are on top of the image, they cannot be seen unless selected. I've also manually split the lines and placed them on the image. I've set a "textLength" so that they'll fit across the page and automatically adjust themselves if they're too long.

This is very imprecise and quite time consuming. To get a better idea of how accurate (or not) it is, here's the same image, with the opacity set to 0.5.

Close enough, but not brilliant.

Finally, I've had to reference the images via an iframe. Without doing that, I wasn't able to select the text. I'm not sure if that's a browser fault, or expected functionality.

If you can suggest a quicker and more accurate way of doing this - I'd love for you to leave a comment below.

Take, for example, this tweet and image:

Terence Eden is on Mastodon

@edent

http://twitpic.com/2dhr3 - Loving #mint09 :-)

---

Well... ok... fun at the moment it was taken, but does it have any use beyond that? On the off chance that I become a subject for scholars in the year 3723, perhaps.

Should some media just be declared ephemeral? The word comes from the Greek εφήμερος – ephemeros, literally "lasting only one day".

Services like SnapChat allow you to send a message to a single recipient, the photo then expires after 10 seconds. Perfect for sending sexually charged images which then vanish into the æther.

That's how it used to be. A photograph had to be handled carefully or it would start to disintegrate.

Can we do this digitally? Of course, bits rarely perish. Neither do they lose any fidelity unless we expressly ask them to. And, yes, anyone can save a pristine copy any time they want.

But, conceptually, it's very easy to create a photo sharing service where the images irreversibly degrade every time they are viewed. Until there's little left but a clump of colours.

Imagine, for a moment, that every time a tourist glanced at the Mona Lisa and then walked idly by, a little bit of the image went with them.

Initially, she would be resplendent.

After a few years, she would be a bit dog-eared.

Eventually, she would succumb to the ravages of time.

We could progressively add more Instagram style filters onto an image until it was last behind a miasma of faux nostalgia. (From Instagram "Improves" Art)

We could degrade video quality, add scratches and blemishes, reduce audio fidelity an introduce the crackle of vinyl. Text could gradually disintegrate.

Perhaps social networks should build in The Right To Be Forgotten - make it a core feature, rather than a law to slavishly follow?

As time passes, gradually reduce the remnants of our past until they slowly fade away. Mere ghosts in the machine.

One of the things I love to do is add images to my contacts. It gives me a visual cue when I'm scrolling through looking for a person, it prompts my memory when I see the face of a friend calling me, and it helps me remember what people look like. As you can see, I'm pretty good at keeping everyone's photo on my phone up to date.

Which Conditions Are Appropriate?

Google's interface for uploading images has some rather confusing conditions attached to it...

"Do not upload pictures containing celebrities, nudity, artwork or copyrighted images."

What? Those are some strange terms and conditions to have attached to a contact image. I'm not sure if they're copy-and-pasted from another app - or if they're intentional. Let's take a look at each one.

• "Celebrities".  If I've got a friend who is a celebrity - why can't I have her image as her contact picture?
• "Nudity".  It's my phone.  I am not going to offend myself if I see a nude picture of a friend that I've uploaded.  This is before we get in to what defines nudity.
• "Artwork". This is just bizarre.  If a street artist has drawn a caricature, I can't upload it?  I can't use a company logo to indicate where my contact works?
• "Copyrighted images". Again - what? I own the copyright on images I've taken.  I may have permission to reuse a copyrighted image.  I may even be justified in using a copyrighted image for my personal use.  I suspect they mean "images to which you don't have permission from the copyright holder to use for this purpose".  But even that doesn't cover the Fair Dealing provisions of many copyright laws.

If you try to use an image which is already on the web, you get this curious message.

"Remember, using others' images on the web without their permission may be bad manners or - even worse - copyright infringement."

This is an odd statement.  A mixture of folksy advice and legal warnings.  I don't see how personal use of a thumbnail from even the most copyright laden of images could be construed as infringement.  As for "bad manners" - is it really Google's role to advise me on etiquette?

People Don't Read - But Copy Editing Matters

It's been well known fact for over 13 years that users don't read.  That's especially true if the text is small and grey - as it is in these examples.

But if you do want to impart vital information, you need to employ a skilled writer to help you craft your message.  You need to understand what it is you're trying to say, why you're saying it and what you expect your users to understand.

In this case, Google has a very muddled and confusing set of conditions which seem illogical and users - if they read them at all - are likely to ignore them.