The fallout from Meta's extensive use of pirated eBooks continues. Recent court filings appear to show the company grappling with the legality of training their AI on stolen data. Evidence shows an employee asking if what they're doing it legal? Will it undermine their lobbying efforts? Will it lead to more regulation? Will they be fined? And, almost as an afterthought, is this fascinating snippet: If we were to use models trained on LibGen for a purpose other than internal evaluation, we…
Continue reading →
Asbestos was the material that built the future! Strong, long lasting, fire-proof, and - above all - completely safe for humans. Every house in the land had beautiful sheets of gloriously white asbestos installed in the walls and ceilings. All the better to keep your loved ones safe. The magic mineral was woven into cloth and turned into hard wearing uniforms. You could even get an asbestos baby-blanket to prevent your child from going up in flames. That was, of course, unlikely because…
Continue reading →
In the early part of the 20th Century, there was a fad for "Radium". The magical, radioactive substance that glowed in the dark. The market had decided that Radium was The Next Big Thing and tried to shove it into every product. There were radioactive toys, radioactive medicines, radioactive chocolate bars, and a hundred other products. The results weren't pretty. In the early part of the 21st Century, there was a fad for "AI". The magical, Artificial Intelligence which provided all the…
Continue reading →
That's it. That's the post. Fine! I'll expand a little more. Large Language Models are a type of Artificial Intelligence. They can read text, parse it, process it using the known rules of English, and then regurgitate parts of it on demand. This means they can read and parse a question like "In Python, how do I add two numbers together?" and then read and parse the Python documentation. It will produce an output like: What happens if you search the official Python documentation for…
Continue reading →
I saw a prominent AI proponent asking why people always focus on the things that AI gets wrong. AI works so well, he asserted, that it was churlish and childish to focus on a few minor mistakes? Which reminds me of an experience I had a few years ago. I was in a rural pub and got chatting to one of the locals. We were getting on great, so I asked him what his name was. "You know," he said, "I've built houses for everyone in this village, but do they call me John the Builder? No! I repaired…
Continue reading →
Last week I attended an unofficial discussion group about the future of AI in Government. As well as the crypto-bores who have suddenly pivoted their "expertise" into AI, there were lots of thoughtful suggestions about what AI could do well at a state level. Some of it is trivial - spell check is AI. Some of it is a dystopian hellscape of racist algorithms being confidently incorrect. The reality is likely to be somewhat prosaic. Although I'm no longer a civil servant, I still enjoy going to …
Continue reading →
A week is a long time in politics and a couple of years is an aeon in AI. Published in 2019, just before the dawn of the LLM, this is an overview of all the weird and charming ways Artificial Intelligence can go wrong. It is fully of delightfully silly examples and rather charming illustrations. Lots of the examples are drawn from the always-entertaining AI Weirdness blog. But it does suffer from being somewhat dated and having an over-reliance on lists. The constant "I trained an AI on…
Continue reading →
There's a popular saying; "No One Wants a Drill. What They Want Is the Hole". It's a pithy (and broadly) correct statement. But I don't think it goes far enough. Let's apply the Five Whys method to the issue: No one wants a drill. What they want is the hole. No one wants a hole. What they want is a picture hook. No one wants a picture hook. What they want is art hanging on the walls. No one wants art hanging on the walls. What they want is a pleasant living environment. No one wants a…
Continue reading →
I'm sure I remembered there once being a clock app for Linux which was deliberately vague. It would declare the time as "Nearly tea-time" or "A little after elevenses" or "Quite late" or "Gosh, that's early". But I can find no evidence that it ever existed and am beginning to wonder if I dreamt it. So I built it. First thing's first - there are a lot of existing fuzzy clocks. But they mostly say things like "afternoon" or "nearly 3 o'clock". There's even a Hobbit Time for Watchy. However, I …
Continue reading →
A group of authors are suing various vendors of Large Language Model AIs. The authors claim that the AIs are trained on material which infringes their copyright. Is that likely? Well, let's take a quick look at the evidence presented. First up, Meta's LLaMA Paper. It describes how the LLM was trained: We include two book corpora in our training dataset: the Gutenberg Project, which contains books that are in the public domain, and the Books3 section of ThePile (Gao et al., 2020) OK,…
Continue reading →
I'm obsessed with the idea that human progress could be accelerated - if only we realised how to properly combine existing technology. I don't want to go "Ancient Aliens" here - but even a cursory reading of scientific history will show you were humanity's progress could have been dramatically fast if only knowledge was more widely shared and recognised. The book "How To Invent Everything" makes a compelling case that the material conditions for scientific development often existed long before …
Continue reading →
One of the (many) problems with AI is that training data usually needs to come from "natural" sources. If you want to emulate human-written text, you need to train something on human-written text. But with the proliferation of cheap and fast AI tools, it is likely that training data will unwillingly become contaminated with AI-written text. In order to prevent the "Habsburg Jaw" effect, I think it would be helpful if publishers could semantically indicate that a work was written by AI. In…
Continue reading →
