Terence Eden. He has a beard and is smiling.

Terence Eden’s Blog

Theme Switcher:

LLMs are still surprisingly bad at some simple tasks

· 20 comments · 650 words · Viewed ~5,411 times


A t-shirt which says Dunning and Kruger and Gell and Mann.

I asked three different commercially available LLMs the same question: Which TLDs have the same name as valid HTML5 elements? This is a pretty simple question to answer. Take two lists and compare them. I know this question is possible to answer because I went through the lists two years ago. Answering the question was a little tedious and subject to my tired human eyes making no mistakes. So…

Winners don't use ChatGPT

· 5 comments · 800 words · Viewed ~959 times


Arcade machine saying "Winners Don't Use Drugs" signed by some FBI dude. Photo CC BY-NC Megan Rosenbloom.

If you hung around video arcades in your youth, you would have seen this message burned into the phosphor of a thousand dying CRTs. Obviously this was a devilish psyop by those gits who wanted kids to stop sniffing glue and having fun. The bastards! But there's a more serious side to the corny message. Are you a winner if you've cheated? Lance Armstrong "won" multiple Toures de France. It…

Grinding down open source maintainers with AI

· 2 comments · 950 words · Viewed ~3,302 times


Can of Spam. From https://www.flickr.com/photos/27308606@N04/3920588954/in/photolist-6Ys3vh-D4tFyP-5Nfafk-4YquSL-j76egA-b4ThXT-j71TQi-4C6NQo-4zGP8b-8jBWuu-9NZujn-4mZsmC-Skcx6h-6qY9vr-hNh67-5Hf4WS-mSRtT-718hHC-71HDFc-kCAL2L-2NYWTK-kCANQm-6eLuK-6cSS7G-vVZqB-79Z3X-dgu3-4sqgZw-8WuDpp-5FQ3yz-4nFSR8-563Gj-mb7gL-39uw1-5f1fho-2NiBSN-5pDMMS-8b9Hjq-pRrxLR-hfXfA-5xmaj-9vw9hx-o9bd3k-258kqqN-tuDnQ-8YeJPL-5hrex8-pFKpm-vSKr9b-39r59D

Early one morning I received an email notification about a bug report to one of my open source projects. I like to be helpful and I want people who use my stuff to have a good time, so I gave it my attention. Here's what it said: 😱 I Can't Use On This Day 😭 Seriously, What’s Going On?! 🔍 I’ve been trying to use the On This Day feature, but it’s just not working for me! 😩 Every time I input my d…

Why do people have such dramatically different experiences using AI?

· 38 comments · 750 words · Viewed ~2,654 times


A t-shirt which says Dunning and Kruger and Gell and Mann.

For some people, it seems, AI is an amazing machine which - while fallible - represents an incredible leap forward in productivity. For other people, it seems, AI is wrong more often than right and - although occasionally useful - requires constant supervision. Who is right? I recently pointed out a few common problems with LLMs. I was discussing this with someone relatively senior who works…

Large Language Models and Pareidolia

· 4 comments · 550 words · Viewed ~696 times


Google replying "The word teamwork contains the letter i one time".

Have you ever looked up at the sky and seen a face staring back at you from the clouds? Of course you have; you're human. Our delicious meaty brains are hardwired to recognise certain shapes - and faces are a useful shape to recognise. A few false positives are a worthwhile trade-off for such a powerful feature. Mistakenly seeing faces where there are none is a phenomenon called pareidolia. If…

How to Dismantle Knowledge of an Atomic Bomb

· 5 comments · 500 words · Viewed ~516 times


A confused little cardboard robot is lost amongst the daisies

The fallout from Meta's extensive use of pirated eBooks continues. Recent court filings appear to show the company grappling with the legality of training their AI on stolen data. Evidence shows an employee asking if what they're doing it legal? Will it undermine their lobbying efforts? Will it lead to more regulation? Will they be fined? And, almost as an afterthought, is this fascinating…

GitHub's Copilot lies about its own documentation. So why would I trust it with my code?

· 6 comments · 400 words · Viewed ~1,689 times


Me asking Copilot how I switch it off. Copilot responds with a link.

In the early part of the 20th Century, there was a fad for "Radium". The magical, radioactive substance that glowed in the dark. The market had decided that Radium was The Next Big Thing and tried to shove it into every product. There were radioactive toys, radioactive medicines, radioactive chocolate bars, and a hundred other products. The results weren't pretty. In the early part of the 21st…

LLMs are good for coding because your documentation is shit

· 2 comments · 350 words · Viewed ~829 times


A pet cat typing on a computer keyboard.

That's it. That's the post. Fine! I'll expand a little more. Large Language Models are a type of Artificial Intelligence. They can read text, parse it, process it using the known rules of English, and then regurgitate parts of it on demand. This means they can read and parse a question like "In Python, how do I add two numbers together?" and then read and parse the Python documentation. It…