Good piece, but I can't help feel as though you've built a bit of a false dichotomy here.
I fully agree that it's ridiculous for the Facebooks of the world to claim they've "solved the dopamine system" or something of the like, but the idea that the networks aren't shaping their users' minds (on the margin) seems so obvious to me that I almost feel like the burden of proof is on someone claiming that they don't do this.
That said, I feel like the breakdown comes between the difference between being able to control users' minds in the abstract vs being able to influence them on certain ideas. I do not think the Zuck army can influence me to be something I'm not, but I'd be surprised if they didn't have the ability to push me in certain directions on certain axes.
As example, my dad is a Fox News diehard - spending time at his house means hearing Tucker et al at all hours of the day, and I find that spending 2-3 days around there starts to do interesting things to me. I don't believe the nonsense they often spew, but I can sense myself becoming gradually sympathetic to certain ideas, or agreeing with certain framings of things.
Abstracting this to more subtle changes in ideology and longer periods of exposure (as I'm sure you know, people use FB a _lot_), it's hard for me to believe that people can't be pushed. Whether or not this is being done, or whether it's profitable or anything remains an open question to me, but IMO the answer of "could TikTok influence certain thoughts (I imagine all of us are more suggestible on certain topics) of someone who watches it for 3 hours a day" is a resounding yes.
Thinking of things this way, I guess my condensed response would be something like "ABC/NBC/CBS could certainly claim to influence the zeitgeist (or could've 20 years ago) but could not influence me to like them more than the internet because that's out of their axis of influencability", which I think is maybe somewhat congruous with what you're saying but not entirely.
Can this be summarized as there’s a particular genius to humans beyond the current statistical pattern matching and combinatorics of LLMs and other ML approaches?
My brother (a writer, so perhaps partial) and I sort of arrived at the elemental question of whether humans have a particular extra quality of genius. Though I’m fungible on this matter, my lazy instincts are all works seen as novel are a result from a complex set of inputs. The criticism that that kind of creative process simply creates pastiche, I’d reply with: that pastiche you’re criticizing is just a shallow version of combinatorial creativity— cases where we are able to spot the inputs of the work. Even the genius works are combinatorial, just deeper and more interesting.
It was frightening to see myself become the hard determinist who proposes all creativity is complex machinery. What has engineering done to my soul? Perhaps it’s time to visit Amsterdam for a check-up visit to another dimension…
Brilliant Mills. Still, we're pretty good at hacking ourselves. A progression might be email, chat/dm, social media, tiktok. The parallel universe is (yeah mentioned this before) right wing mass media, also a hack. Judging from our kids, it doesn't take all that much work to write an algorithm to keep us scrolling for hours. The length matters. Short catchy content seems to command a lot of eyeballs. Decades ago Mike Judge and Etan Cohen tapped this in "Idiocracy."
One thing that gets me about all these "event horizon" AI prognosticators is that they overlook the limits on recursive axiomatic systems implied by Godel's Theorm. There's pretty good chance that big AI's might go unstable. Didn't Bing limit the length of their chats to the AI search...
Unstable meaning fiction = reality, or in the AI chat parlance, hallucinating.
I agree with the overall thesis (humans are gonna get bored eventually, always) but I'm more so worried by the successful stints AI might have. i.e. the damage caused by state-operated disinfo campaigns before people catch on / get bored by them. The example of Myanmar is a good one: I only see AI making that sort of thing easier and faster to pull off.
I thiiink what makes a Chapmanesque attention vortex more believable to me is that it will be:
1. Faster to evolve. FB can’t aggregate new compelling content, nor can the most addictive mobile game generate new compelling content, faster than most humans can tire of it. But a powerful enough AI-fueled experience could.
2. Tailored to you. Every experience we’ve had so far has been a common-denominator affair by necessity of needing to land for an audience of more than one. But a powerful enough AI-fueled experience would always be paying attention to what works on you and reacting, in ways much more granular and creative than just ranking content that’s already been made.
And indeed, people do like complexity and elegance, and they have taste! But there’s no reason a powerful enough AI wouldn’t take that into account. Everything we’ve ever deeply enjoyed was made by humans slowly flailing about, trying to find something that works; what if something drastically faster and better at optimizing was on the same job? You get a culture war roller derby and I get an infinite series of manga about poets in period costumes. Until we get tired of those, and they gradually mutate into something fresh.
This whole scenario has long been a favored answer of mine to the Fermi paradox: once a civilization invents good enough entertainment, exploring the galaxy (or doing anything else) doesn’t seem that rewarding anymore.
Formidable attention warriors like you don’t get hacked. That’s why you *love* the bold taste of Marlborough
Good piece, but I can't help feel as though you've built a bit of a false dichotomy here.
I fully agree that it's ridiculous for the Facebooks of the world to claim they've "solved the dopamine system" or something of the like, but the idea that the networks aren't shaping their users' minds (on the margin) seems so obvious to me that I almost feel like the burden of proof is on someone claiming that they don't do this.
That said, I feel like the breakdown comes between the difference between being able to control users' minds in the abstract vs being able to influence them on certain ideas. I do not think the Zuck army can influence me to be something I'm not, but I'd be surprised if they didn't have the ability to push me in certain directions on certain axes.
As example, my dad is a Fox News diehard - spending time at his house means hearing Tucker et al at all hours of the day, and I find that spending 2-3 days around there starts to do interesting things to me. I don't believe the nonsense they often spew, but I can sense myself becoming gradually sympathetic to certain ideas, or agreeing with certain framings of things.
Abstracting this to more subtle changes in ideology and longer periods of exposure (as I'm sure you know, people use FB a _lot_), it's hard for me to believe that people can't be pushed. Whether or not this is being done, or whether it's profitable or anything remains an open question to me, but IMO the answer of "could TikTok influence certain thoughts (I imagine all of us are more suggestible on certain topics) of someone who watches it for 3 hours a day" is a resounding yes.
Thinking of things this way, I guess my condensed response would be something like "ABC/NBC/CBS could certainly claim to influence the zeitgeist (or could've 20 years ago) but could not influence me to like them more than the internet because that's out of their axis of influencability", which I think is maybe somewhat congruous with what you're saying but not entirely.
Can this be summarized as there’s a particular genius to humans beyond the current statistical pattern matching and combinatorics of LLMs and other ML approaches?
My brother (a writer, so perhaps partial) and I sort of arrived at the elemental question of whether humans have a particular extra quality of genius. Though I’m fungible on this matter, my lazy instincts are all works seen as novel are a result from a complex set of inputs. The criticism that that kind of creative process simply creates pastiche, I’d reply with: that pastiche you’re criticizing is just a shallow version of combinatorial creativity— cases where we are able to spot the inputs of the work. Even the genius works are combinatorial, just deeper and more interesting.
It was frightening to see myself become the hard determinist who proposes all creativity is complex machinery. What has engineering done to my soul? Perhaps it’s time to visit Amsterdam for a check-up visit to another dimension…
so so good
Anyway, I just bought a penril modem
Brilliant Mills. Still, we're pretty good at hacking ourselves. A progression might be email, chat/dm, social media, tiktok. The parallel universe is (yeah mentioned this before) right wing mass media, also a hack. Judging from our kids, it doesn't take all that much work to write an algorithm to keep us scrolling for hours. The length matters. Short catchy content seems to command a lot of eyeballs. Decades ago Mike Judge and Etan Cohen tapped this in "Idiocracy."
One thing that gets me about all these "event horizon" AI prognosticators is that they overlook the limits on recursive axiomatic systems implied by Godel's Theorm. There's pretty good chance that big AI's might go unstable. Didn't Bing limit the length of their chats to the AI search...
Unstable meaning fiction = reality, or in the AI chat parlance, hallucinating.
I agree with the overall thesis (humans are gonna get bored eventually, always) but I'm more so worried by the successful stints AI might have. i.e. the damage caused by state-operated disinfo campaigns before people catch on / get bored by them. The example of Myanmar is a good one: I only see AI making that sort of thing easier and faster to pull off.
Damn, I loved this piece
This is excellent. Thank you again, Mills.
shared in multiple slacks amongst friends and coworkers. this was good. ty for writing it.
I thiiink what makes a Chapmanesque attention vortex more believable to me is that it will be:
1. Faster to evolve. FB can’t aggregate new compelling content, nor can the most addictive mobile game generate new compelling content, faster than most humans can tire of it. But a powerful enough AI-fueled experience could.
2. Tailored to you. Every experience we’ve had so far has been a common-denominator affair by necessity of needing to land for an audience of more than one. But a powerful enough AI-fueled experience would always be paying attention to what works on you and reacting, in ways much more granular and creative than just ranking content that’s already been made.
And indeed, people do like complexity and elegance, and they have taste! But there’s no reason a powerful enough AI wouldn’t take that into account. Everything we’ve ever deeply enjoyed was made by humans slowly flailing about, trying to find something that works; what if something drastically faster and better at optimizing was on the same job? You get a culture war roller derby and I get an infinite series of manga about poets in period costumes. Until we get tired of those, and they gradually mutate into something fresh.
This whole scenario has long been a favored answer of mine to the Fermi paradox: once a civilization invents good enough entertainment, exploring the galaxy (or doing anything else) doesn’t seem that rewarding anymore.
Latz focuses on the durability of the audience. Artists are more durable than they're being made out to be, too. https://ponytail.substack.com/p/in-my-way-i-say
A brilliant read and much to consider (nothing worthwhile to add here for now!). Thanks Mills!