Good piece, but I can't help feel as though you've built a bit of a false dichotomy here.
I fully agree that it's ridiculous for the Facebooks of the world to claim they've "solved the dopamine system" or something of the like, but the idea that the networks aren't shaping their users' minds (on the margin) seems so obvious to me that I almost feel like the burden of proof is on someone claiming that they don't do this.
That said, I feel like the breakdown comes between the difference between being able to control users' minds in the abstract vs being able to influence them on certain ideas. I do not think the Zuck army can influence me to be something I'm not, but I'd be surprised if they didn't have the ability to push me in certain directions on certain axes.
As example, my dad is a Fox News diehard - spending time at his house means hearing Tucker et al at all hours of the day, and I find that spending 2-3 days around there starts to do interesting things to me. I don't believe the nonsense they often spew, but I can sense myself becoming gradually sympathetic to certain ideas, or agreeing with certain framings of things.
Abstracting this to more subtle changes in ideology and longer periods of exposure (as I'm sure you know, people use FB a _lot_), it's hard for me to believe that people can't be pushed. Whether or not this is being done, or whether it's profitable or anything remains an open question to me, but IMO the answer of "could TikTok influence certain thoughts (I imagine all of us are more suggestible on certain topics) of someone who watches it for 3 hours a day" is a resounding yes.
Thinking of things this way, I guess my condensed response would be something like "ABC/NBC/CBS could certainly claim to influence the zeitgeist (or could've 20 years ago) but could not influence me to like them more than the internet because that's out of their axis of influencability", which I think is maybe somewhat congruous with what you're saying but not entirely.
Brilliant Mills. Still, we're pretty good at hacking ourselves. A progression might be email, chat/dm, social media, tiktok. The parallel universe is (yeah mentioned this before) right wing mass media, also a hack. Judging from our kids, it doesn't take all that much work to write an algorithm to keep us scrolling for hours. The length matters. Short catchy content seems to command a lot of eyeballs. Decades ago Mike Judge and Etan Cohen tapped this in "Idiocracy."
One thing that gets me about all these "event horizon" AI prognosticators is that they overlook the limits on recursive axiomatic systems implied by Godel's Theorm. There's pretty good chance that big AI's might go unstable. Didn't Bing limit the length of their chats to the AI search...
Unstable meaning fiction = reality, or in the AI chat parlance, hallucinating.
I agree with the overall thesis (humans are gonna get bored eventually, always) but I'm more so worried by the successful stints AI might have. i.e. the damage caused by state-operated disinfo campaigns before people catch on / get bored by them. The example of Myanmar is a good one: I only see AI making that sort of thing easier and faster to pull off.
I'm surprised by the comparative rarity of this line of thinking. Many times a week a hear some argument like "social media is taking over our brains" but it's much less often that I hear the rebuttal and usually when it does come up it's more of a soft one counseling moderation or general complexity as opposed to something more pointed. This is refreshing.
I initially took your statement about people asserting that Facebook had “figured out how to make addictive products” as referring to internal people at Facebook. I think I see now that you're referring to people outside the company saying this of Facebook, however, I wonder if part the stickiness of this ideas is because it's also propagated by professionals themselves.
I don't think tech professionals will often assert directly that they "know how to make addictive products" but I do think it's implicit in the way I see many tech professionals espousing "rulesets" or "playbooks" to achieve desired outcomes in social products.
This attempt to codify achieving desired outcomes makes me a bit crazy. In principle, maybe it's not totally useless, but in practice I'm amazed at the power of these rules and playbooks to get in the way of the obvious first order thing of making something humans actually like.
I've saved this to read again but for now I'd just like to make two observations. Firstly, I haven't read Frank's article yet (that's next), but perhaps naively I've always been optimistic about the advent of AI that can "write" articles. I think that Arthur C. Clarke's opinion that "Any teacher who could be replaced by a computer probably should be" applies to writers as well. There's a good chance that people who can write well and originally will be able to command premium rates.
Secondly, although it's not the same technology exactly, a few years ago someone was waxing lyrical to me about how their software could track the eye movements of everyone in a classroom at the same time, and tell the teacher whether they're concentrating or not. I thought it was ridiculous. I said:
"Firstly, you can tell by looking at them. Secondly, some students look like they're not concentrating (eg looking out of the window or, in my case, doodling) because they're thinkin g. Thirdly, what's wrong with actually asking the students a question?"
The presenter looked at me as though I was mad. The feeling was mutual.
I thiiink what makes a Chapmanesque attention vortex more believable to me is that it will be:
1. Faster to evolve. FB can’t aggregate new compelling content, nor can the most addictive mobile game generate new compelling content, faster than most humans can tire of it. But a powerful enough AI-fueled experience could.
2. Tailored to you. Every experience we’ve had so far has been a common-denominator affair by necessity of needing to land for an audience of more than one. But a powerful enough AI-fueled experience would always be paying attention to what works on you and reacting, in ways much more granular and creative than just ranking content that’s already been made.
And indeed, people do like complexity and elegance, and they have taste! But there’s no reason a powerful enough AI wouldn’t take that into account. Everything we’ve ever deeply enjoyed was made by humans slowly flailing about, trying to find something that works; what if something drastically faster and better at optimizing was on the same job? You get a culture war roller derby and I get an infinite series of manga about poets in period costumes. Until we get tired of those, and they gradually mutate into something fresh.
This whole scenario has long been a favored answer of mine to the Fermi paradox: once a civilization invents good enough entertainment, exploring the galaxy (or doing anything else) doesn’t seem that rewarding anymore.
The irrepressible monkey in the machine
Formidable attention warriors like you don’t get hacked. That’s why you *love* the bold taste of Marlborough
Good piece, but I can't help feel as though you've built a bit of a false dichotomy here.
I fully agree that it's ridiculous for the Facebooks of the world to claim they've "solved the dopamine system" or something of the like, but the idea that the networks aren't shaping their users' minds (on the margin) seems so obvious to me that I almost feel like the burden of proof is on someone claiming that they don't do this.
That said, I feel like the breakdown comes between the difference between being able to control users' minds in the abstract vs being able to influence them on certain ideas. I do not think the Zuck army can influence me to be something I'm not, but I'd be surprised if they didn't have the ability to push me in certain directions on certain axes.
As example, my dad is a Fox News diehard - spending time at his house means hearing Tucker et al at all hours of the day, and I find that spending 2-3 days around there starts to do interesting things to me. I don't believe the nonsense they often spew, but I can sense myself becoming gradually sympathetic to certain ideas, or agreeing with certain framings of things.
Abstracting this to more subtle changes in ideology and longer periods of exposure (as I'm sure you know, people use FB a _lot_), it's hard for me to believe that people can't be pushed. Whether or not this is being done, or whether it's profitable or anything remains an open question to me, but IMO the answer of "could TikTok influence certain thoughts (I imagine all of us are more suggestible on certain topics) of someone who watches it for 3 hours a day" is a resounding yes.
Thinking of things this way, I guess my condensed response would be something like "ABC/NBC/CBS could certainly claim to influence the zeitgeist (or could've 20 years ago) but could not influence me to like them more than the internet because that's out of their axis of influencability", which I think is maybe somewhat congruous with what you're saying but not entirely.
so so good
Anyway, I just bought a penril modem
Brilliant Mills. Still, we're pretty good at hacking ourselves. A progression might be email, chat/dm, social media, tiktok. The parallel universe is (yeah mentioned this before) right wing mass media, also a hack. Judging from our kids, it doesn't take all that much work to write an algorithm to keep us scrolling for hours. The length matters. Short catchy content seems to command a lot of eyeballs. Decades ago Mike Judge and Etan Cohen tapped this in "Idiocracy."
One thing that gets me about all these "event horizon" AI prognosticators is that they overlook the limits on recursive axiomatic systems implied by Godel's Theorm. There's pretty good chance that big AI's might go unstable. Didn't Bing limit the length of their chats to the AI search...
Unstable meaning fiction = reality, or in the AI chat parlance, hallucinating.
I agree with the overall thesis (humans are gonna get bored eventually, always) but I'm more so worried by the successful stints AI might have. i.e. the damage caused by state-operated disinfo campaigns before people catch on / get bored by them. The example of Myanmar is a good one: I only see AI making that sort of thing easier and faster to pull off.
Damn, I loved this piece
This is excellent. Thank you again, Mills.
shared in multiple slacks amongst friends and coworkers. this was good. ty for writing it.
Wait what’s this mean: vast and fresh inventory and incredible ranking, especially their explore / exploit balance.
I'm surprised by the comparative rarity of this line of thinking. Many times a week a hear some argument like "social media is taking over our brains" but it's much less often that I hear the rebuttal and usually when it does come up it's more of a soft one counseling moderation or general complexity as opposed to something more pointed. This is refreshing.
I initially took your statement about people asserting that Facebook had “figured out how to make addictive products” as referring to internal people at Facebook. I think I see now that you're referring to people outside the company saying this of Facebook, however, I wonder if part the stickiness of this ideas is because it's also propagated by professionals themselves.
I don't think tech professionals will often assert directly that they "know how to make addictive products" but I do think it's implicit in the way I see many tech professionals espousing "rulesets" or "playbooks" to achieve desired outcomes in social products.
This attempt to codify achieving desired outcomes makes me a bit crazy. In principle, maybe it's not totally useless, but in practice I'm amazed at the power of these rules and playbooks to get in the way of the obvious first order thing of making something humans actually like.
I've saved this to read again but for now I'd just like to make two observations. Firstly, I haven't read Frank's article yet (that's next), but perhaps naively I've always been optimistic about the advent of AI that can "write" articles. I think that Arthur C. Clarke's opinion that "Any teacher who could be replaced by a computer probably should be" applies to writers as well. There's a good chance that people who can write well and originally will be able to command premium rates.
Secondly, although it's not the same technology exactly, a few years ago someone was waxing lyrical to me about how their software could track the eye movements of everyone in a classroom at the same time, and tell the teacher whether they're concentrating or not. I thought it was ridiculous. I said:
"Firstly, you can tell by looking at them. Secondly, some students look like they're not concentrating (eg looking out of the window or, in my case, doodling) because they're thinkin g. Thirdly, what's wrong with actually asking the students a question?"
The presenter looked at me as though I was mad. The feeling was mutual.
I thiiink what makes a Chapmanesque attention vortex more believable to me is that it will be:
1. Faster to evolve. FB can’t aggregate new compelling content, nor can the most addictive mobile game generate new compelling content, faster than most humans can tire of it. But a powerful enough AI-fueled experience could.
2. Tailored to you. Every experience we’ve had so far has been a common-denominator affair by necessity of needing to land for an audience of more than one. But a powerful enough AI-fueled experience would always be paying attention to what works on you and reacting, in ways much more granular and creative than just ranking content that’s already been made.
And indeed, people do like complexity and elegance, and they have taste! But there’s no reason a powerful enough AI wouldn’t take that into account. Everything we’ve ever deeply enjoyed was made by humans slowly flailing about, trying to find something that works; what if something drastically faster and better at optimizing was on the same job? You get a culture war roller derby and I get an infinite series of manga about poets in period costumes. Until we get tired of those, and they gradually mutate into something fresh.
This whole scenario has long been a favored answer of mine to the Fermi paradox: once a civilization invents good enough entertainment, exploring the galaxy (or doing anything else) doesn’t seem that rewarding anymore.
Latz focuses on the durability of the audience. Artists are more durable than they're being made out to be, too. https://ponytail.substack.com/p/in-my-way-i-say
A brilliant read and much to consider (nothing worthwhile to add here for now!). Thanks Mills!
Interesting take...