A Quick Update on Prophecy, AI and Writing
Amazingly, my point still holds!
Prognostication, of any kind, is a mug’s game. And anyone who engages in it thinking that time will be kind to them is just asking for it. The survival of prophets, seers and charlatans of all stripes has converged on an evolved set of practices that provides the greatest hope of survival.
The most famous of these survival mechanisms is speaking in unintelligible riddles that seem optimized for irony, just like wishes in the story the Monkey’s Paw. The Delphic Oracle is probably the GOAT for this. Unintelligible, overly generalized (and/or hallucinogenic) symbolism has been used to great effect by Nostradamus and John of Patmos. And finally there’s the trick of everyone who writes about investment markets: either give a magnitude or a direction, but never both at once.
So when I ignored all of these rules in my post:
AI Doesn’t Write Very Well and Isn’t Getting Any Better at It. So Why is Everyone so Freaked Out?
I think LLMs are, on balance, great for writers and writing, but I think they don’t write very well. This series of posts will attempt to unravel any apparent paradoxes in that statement.
I thought for sure that time would make a fool of me in short order.. But, it hasn’t… yet. Much to everyone’s surprise, my temporary state of being right continues.
OpenAI just announced OpenAI o1, “a new large language model trained with reinforcement learning to perform complex reasoning.” Its benchmarks growth are impressive. Except, it’s still not getting better at writing.
The English Language exam score didn’t budge. I know from a friend who teaches both AP classes that they recently lowered the standard for the English Lit exam, which might explain the minuscule improvement. I mean, compared to the impressive jumps in Physics and Math.
I have an unabashed love for the work that OpenAI and organizations like it are doing. Intelligence is the fundamentally scarce resource, and having more of it productively harnessed in the world expands the possibilities for progress in the most dramatic way since the Industrial Revolution. Are there problems? Absolutely. But there are problems with everything. For me, the promise currently outweighs the peril.
Among my highest leverage uses for AI is using it to help me generate mediocre ideas so I can get to good ideas faster. And for research and learning. But I am left with this increasingly interesting puzzle:
Why isn’t a large LANGUAGE model better with language?
I think they aren’t better at language because they are trained on the internet. And there’s a lot of bad writing on the internet (plus there’s plenty of bad writing on the recent internet that was created by… lesser LLMs (previous gens).
Have you read the AI poetry book, I AM CODE, Simon Rich edited using an open ai product that isn’t consumer facing yet? Werner Herzog narrates the audio version.