I think they aren’t better at language because they are trained on the internet. And there’s a lot of bad writing on the internet (plus there’s plenty of bad writing on the recent internet that was created by… lesser LLMs (previous gens).
Have you read the AI poetry book, I AM CODE, Simon Rich edited using an open ai product that isn’t consumer facing yet? Werner Herzog narrates the audio version.
That was one of my arguments in the earlier post. LLMs have inherent averaging properties. Train them on everything that was ever written and the average of that isn't very good. But if you only want to train LLMs on great writing, there isn't enough of it to form a training set.
From what I've read llm's training on the output of other llm's goes horribly awry. But since they're being used to generate so much mediocre writing.
I think one's style will become far more important and valued. Which is why I added some extra flourishes in the beginning of this post. I love a maximalist style and they don't get much play now.
Haven't read I AM CODE. I can report Herzog's reading his autobiography is amazing! When the end of the world comes, he's going to narrate it. "As the skies fall in rivers of flame, the final hopelessness of the human condition is finally revealed. God's final punchline to the long, inexorable joke of existence. The set up unknowable, the punchline. lethal, delivered to the rimshot of Apocalypse."
I AM CODE (which admittedly, I listened to mostly for the intro because I was curious about why Simon Rich was doing it, didn’t really listen to much of the poetry) was written by a way more sophisticated LLM than we’re seeing on the consumer side. Of course, Werner makes it kind of terrifying.
I think they aren’t better at language because they are trained on the internet. And there’s a lot of bad writing on the internet (plus there’s plenty of bad writing on the recent internet that was created by… lesser LLMs (previous gens).
Have you read the AI poetry book, I AM CODE, Simon Rich edited using an open ai product that isn’t consumer facing yet? Werner Herzog narrates the audio version.
That was one of my arguments in the earlier post. LLMs have inherent averaging properties. Train them on everything that was ever written and the average of that isn't very good. But if you only want to train LLMs on great writing, there isn't enough of it to form a training set.
From what I've read llm's training on the output of other llm's goes horribly awry. But since they're being used to generate so much mediocre writing.
I think one's style will become far more important and valued. Which is why I added some extra flourishes in the beginning of this post. I love a maximalist style and they don't get much play now.
Haven't read I AM CODE. I can report Herzog's reading his autobiography is amazing! When the end of the world comes, he's going to narrate it. "As the skies fall in rivers of flame, the final hopelessness of the human condition is finally revealed. God's final punchline to the long, inexorable joke of existence. The set up unknowable, the punchline. lethal, delivered to the rimshot of Apocalypse."
I totally agree with you about style!
I AM CODE (which admittedly, I listened to mostly for the intro because I was curious about why Simon Rich was doing it, didn’t really listen to much of the poetry) was written by a way more sophisticated LLM than we’re seeing on the consumer side. Of course, Werner makes it kind of terrifying.