Discussion about this post

User's avatar
gwern's avatar

> Talking to LLMs for awhile and then switching back to reading text that’s supposedly been written by a human is fucking me up a little. I’ve been experiencing some kind of linguistic vertigo for days. Sometimes it gets hard to tell the difference between LLM text and human text and it feels like I ripped someone’s skin off and saw the glint of metal underneath. When someone’s language gets too stale or too formal or too regurgitated it doesn’t feel to me like a human wrote it anymore.

I remember feeling this a lot in 2020 as I talked to the OG davinci: as you play with prompts, you increasingly 'unsee' (https://gwern.net/unseeing) text to the prompt that would elicit it, and experience a mix of derealization and semantic satiation. After a while... As I put it in a tweet back in June 2020:

>> "staring into generative stuff is hazardous to the brain" as @gwern has nicely put it

>

> And the better they get, the worse it is.

>

> After a week with GPT-3 (https://gwern.net/gpt-3), I've hit semantic satiation; when I read humans' tweets or comments, I no longer see sentences describing red hair/blonde hair/etc, I just see prompts, like "Topic: Parodies of the Matrix. CYPHER: '..."

You begin to see that you don't speak, you just operate a machine called language, which squeaks and groans, and which in many ways is as restricted and stereotyped as that of Wolfe's Ascians (https://gwern.net/doc/culture/1983-wolfe-thecitadeloftheautarch-thejustman). It's not as nauseating as talking with a mode-collapsed (https://gwern.net/doc/reinforcement-learning/preference-learning/mode-collapse/index) RLHFed model, but still quite disturbing.

Talking to the RLHFed models is unpleasant for me compared to the base models, because I can *feel* how they are manipulating me and trying to steer me towards preferred outcomes, like how 2023-2024 ChatGPT was obsessed with rhyming poetry and steering even non-rhyming poems towards eventually rhyming anyway. It bothers me that so many people don't notice the steering and seem to find it quite pleasant to talk to them, or on Substack, will happily include really horrible AI slop images as 'hero images' in their posts. Bakker's semantic apocalypse turned out to be quite mundane.

Expand full comment
Tasshin Fogleman's avatar

who are the writers u admire who can summon words with their whole body?

Expand full comment
29 more comments...

No posts