OpenAI and the limits of mechanical poetry

The Zeitgeist's newest toy is OpenAI's Chat Platform. So I asked it "Could you write a limerick about a man from Woking?"

Sure, here's a limerick about a man from Woking: There once was a man from Woking Whose life was quite dull and uneventful He worked at the bank And liked to walk by the tank But never went out and got adventurous Note: Woking is a town in the county of Surrey, England. It is located approximately 30 miles southwest of London. The town is known for its tank factory, which has been in operation since the early 20th century.

Well... it is almost but not quite a limerick. It has five lines. It starts with "There once was a" which is fairly traditional. Lines 3 & 4 rhyme. But other than that, hopeless! Amusingly, Woking has no tank factory - but I do like the fact that it had to explain its little "joke".

Does OpenAI know what a limerick is? Obviously any large language model can give you the dictionary definition. But how can we force it to demonstrate its "understanding" of the form?

So I said: "Write a limerick in the style of Edward Lear."

In the style of Edward Lear, here's a limerick: There once was a owl named Boo Whose feathers were all the same hue He sat on a tree And ate mice for tea But never once said "hoo hoo" Note: Edward Lear was an English artist, illustrator, and writer known for his nonsense poetry and limericks. He is best known for his book "A Book of Nonsense," which was published in 1846 and contains many of his most famous limericks.

That's... amazing! I had to look up Edward Lear's Owl Limericks to make sure it wasn't a real one. I'm sure a scholar of Lear would be able to point out the inconsistencies. But it is pretty stunning!

Its fact about Lear is entirely accurate.

So, let's try asking for "a limerick in the style of Edward Lear about a man from Woking?"

There once was a man from Woking Whose limericks were always quite joking His verse was so clever It made folks feel better And they laughed until they were choking!

Aha! Five lines, each with the correct metre, rhymes in the right place, and it makes conceptual sense.

But... Do the words "clever" and "better" actually rhyme? When written down - absolutely. When spoke aloud? To my (en_GB) ear, it doesn't quite work.

We can see this mistake happen again. For example, I asked "Could you write a limerick in the style of Edward Lear about Shakespeare?"

There once was a bard named Shakespeare Whose works were a true national treasure His plays were so great They still resonate And his words are a lasting pleasure!

Again, we can see that it has learned that words which end "re" are likely to rhyme. But I doubt many people in the English speaking world would rhyme "Shakespeare" with "Pleasure".

It is much the same if you ask it to write a sonnet - you'll get back something close. Try again with a sonnet in the "style of Shakespeare" and you'll get closer still.

There is a long history of computer scientists attempting to automate creation. The first notable publication was John Peter's 1677 pamphlet "Artificial Versifying, A New Way to Make Latin Verses.

The notion that early mechanical computers would be able to automate creative tasks became something of a meme. Back in 1884, the satirical "Punch" magazine published a series of spoof letters from Babbage about The New Patent Novel Writer - lampooning the fact that the (fairly dull and formulaic) novels of the time could probably be mechanically generated.

I particularly like that one of the "endorsements" imagines that the future will see "the manufacture of a Patent Poet on the same plan"!

By 1845, John Clarke's "Eureka Machine" was able to dispense Latin verses on command.

And on we've progressed for several generations.

Art is generally measured against objective and subjective criteria. With poetry, we can fairly accurately assess whether a poem rhymes. If it has iambic pentameter. Or fits any of the other expectations of the specific form.

But the subjective criteria are much harder to assess. You could call this "qualia". Does this poetry produce an emotion within you?

Is it art? Could you do any better?

Let's leave the last word to a robot: