Sometimes gzip beats Brotli

Perhaps this was obvious to you, but it wasn't to me. So I'm sharing in the hope that you don't spend an evening trying to trick your webserver into doing something stupid.

For years, HTTP content has been served with gzip compression (gz). It's basically the same sort of compression algorithm you get in a .zip file. It's pretty good!

But there's a new(er) compression algorithm called Brotli (br). It's Better, Faster, Stronger, Harder than gzip. Mostly.

Looking through my browser's request logs, I noticed everything was being transferred with Brotli compression except for one specific text file was being served as gz.

What's going on?

Well, let's take a look at the file's size.

curl -s "https://openbenches.org/api/benches.tsv" | wc -c

That downloads the file, counts the number of bytes, then formats it for readability. It's 2,727,104 bytes.

Now let's request it as a gzipped file:

curl -s -H'Accept-Encoding: gzip' "https://openbenches.org/api/benches.tsv" | wc -c It's 1,085,372 bytes.

Finally, requesting a Brotli compressed transfer: curl -s -H'Accept-Encoding: br' "https://openbenches.org/api/benches.tsv" | wc -c

That's 1,098,151 bytes. A whole 12,779 bytes larger!

My server was correct in using gzipped rather than Brotli for this specific file.

But, that's not the entire case here! I manually compressed the full file using different compression levels. Here's a quick graph showing the filesize at different compression strengths:

So, in this case, Brotli ≤ 3 is worse than gzip ≥ 5.

I suspect my host's server is configured to prioritise faster compression over absolutely smallest file size. That's probably a reasonable trade-off. I couldn't see a way to tell it to use a higher strength Brotli algorithm all the time - but I would probably be chasing marginal gains.

So, there you go. Don't be surprised if you occasionally see gzip where you expect to see Brotli.