Tiktok's enshittification

Paper · Source
Expertise in the Age of AI Content

Source: Cory Doctorow, Pluralistic · 2023-01-21

Here is how platforms die: first, they are good to their users; then they abuse their users to make things better for their business customers; finally, they abuse those business customers to claw back all the value for themselves. Then, they die.

I call this enshittification, and it is a seemingly inevitable consequence arising from the combination of the ease of changing how a platform allocates value, combined with the nature of a "two sided market," where a platform sits between buyers and sellers, holding each hostage to the other, raking off an ever-larger share of the value that passes between them.

That tempted in lots of business customers – Marketplace sellers who turned Amazon into the "everything store" it had promised from the beginning. As these sellers piled in, Amazon shifted to subsidizing suppliers. Kindle and Audible creators got generous packages. Marketplace sellers reached huge audiences and Amazon took low commissions from them.

That's when Amazon started to harvest the surplus from its business customers and send it to Amazon's shareholders. Today, Marketplace sellers are handing 45%+ of the sale price to Amazon in junk fees. The company's $31b "advertising" program is really a payola scheme that pits sellers against each other, forcing them to bid on the chance to be at the top of your search.

Search Amazon for "cat beds" and the entire first screen is ads, including ads for products Amazon cloned from its own sellers, putting them out of business (third parties have to pay 45% in junk fees to Amazon, but Amazon doesn't charge itself these fees). All told, the first five screens of results for "cat bed" are 50% ads.

This is enshittification: surpluses are first directed to users; then, once they're locked in, surpluses go to suppliers; then once they're locked in, the surplus is handed to shareholders and the platform becomes a useless pile of shit. From mobile app stores to Steam, from Facebook to Twitter, this is the enshittification lifecycle.

This shell-game with surpluses is what happened to Facebook. First, Facebook was good to you: it showed you the things the people you loved and cared about had to say. This created a kind of mutual hostage-taking: once a critical mass of people you cared about were on Facebook, it became effectively impossible to leave, because you'd have to convince all of them to leave too, and agree on where to go. You may love your friends, but half the time you can't agree on what movie to see and where to go for dinner. Forget it.

This made publications truly dependent on Facebook – their readers no longer visited the publications' websites, they just tuned into them on Facebook. The publications were hostage to those readers, who were hostage to each other. Facebook stopped showing readers the articles publications ran, tuning The Algorithm to suppress posts from publications unless they paid to "boost" their articles to the readers who had explicitly subscribed to them and asked Facebook to put them in their feeds.

Sellers became dependent on Facebook, too, unable to carry on business without access to those targeted pitches. That was Facebook's cue to jack up ad prices, stop worrying so much about ad fraud, and to collude with Google to rig the ad market through an illegal program called Jedi Blue:

Today, Facebook is terminally enshittified, a terrible place to be whether you're a user, a media company, or an advertiser. It's a company that deliberately demolished a huge fraction of the publishers it relied on, defrauding them into a "pivot to video" based on false claims of the popularity of video among Facebook users. Companies threw billions into the pivot, but the viewers never materialized, and media outlets folded in droves:

Once you understand the enshittification pattern, a lot of the platform mysteries solve themselves. Think of the SEO market, or the whole energetic world of online creators who spend endless hours engaged in useless platform Kremlinology, hoping to locate the algorithmic tripwires, which, if crossed, doom the creative works they pour their money, time and energy into:

But what if there is no underlying logic? Or, more to the point, what if the logic shifts based on the platform's priorities? If you go down to the midway at your county fair, you'll spot some poor sucker walking around all day with a giant teddy bear that they won by throwing three balls in a peach basket.

The carny allocated a giant teddy bear to that poor sucker the way that platforms allocate surpluses to key performers – as a convincer in a "Big Store" con, a way to rope in other suckers who'll make content for the platform, anchoring themselves and their audiences to it.

But what made it such a success early on was the power of its recommendation system. From the start, Tiktok was really, really good at recommending things to its users. Eerily good:

By making good-faith recommendations of things it thought its users would like, Tiktok built a mass audience, larger than many thought possible, given the death grip of its competitors, like Youtube and Instagram. Now that Tiktok has the audience, it is consolidating its gains and seeking to lure away the media companies and creators who are still stubbornly attached to Youtube and Insta.

Yesterday, Forbes's Emily Baker-White broke a fantastic story about how that actually works inside of Bytedance, Tiktok's parent company, citing multiple internal sources, revealing the existence of a "heating tool" that Tiktok employees use to push videos from select accounts into millions of viewers' feeds:

"Sources told Forbes that TikTok has often used heating to court influencers and brands, enticing them into partnerships by inflating their videos’ view count. This suggests that heating has potentially benefitted some influencers and brands — those with whom TikTok has sought business relationships — at the expense of others with whom it has not."

In other words, Tiktok is handing out giant teddy bears.

But Tiktok is not in the business of giving away giant teddy bears. Tiktok, for all that its origins are in the quasi-capitalist Chinese economy, is just another paperclip-maximizing artificial colony organism that treats human beings as inconvenient gut flora. Tiktok is only going to funnel free attention to the people it wants to entrap until they are entrapped, then it will withdraw that attention and begin to monetize it.

For Tiktok, handing out free teddy-bears by "heating" the videos posted by skeptical performers and media companies is a way to convert them to true believers, getting them to push all their chips into the middle of the table, abandoning their efforts to build audiences on other platforms (it helps that Tiktok's format is distinctive, making it hard to repurpose videos for Tiktok to circulate on rival platforms).

Once those performers and media companies are hooked, the next phase will begin: Tiktok will withdraw the "heating" that sticks their videos in front of people who never heard of them and haven't asked to see their videos. Tiktok is performing a delicate dance here: there's only so much enshittification they can visit upon their users' feeds, and Tiktok has lots of other performers they want to give giant teddy-bears to.

Tiktok won't just starve performers of the "free" attention by depreferencing them in the algorithm, it will actively punish them by failing to deliver their videos to the users who subscribed to them. After all, every time Tiktok shows you a video you asked to see, it loses a chance to show you a video it wants you to see, because your attention is a giant teddy-bear it can give away to a performer it is wooing.

This is just what Twitter has done as part of its march to enshittification: thanks to its "monetization" changes, the majority of people who follow you will never see the things you post. I have ~500k followers on Twitter and my threads used to routinely get hundreds of thousands or even millions of reads. Today, it's hundreds, perhaps thousands.

The end-to-end principle is dead at the service level today. Useful idiots on the right were tricked into thinking that the risk of Twitter mismanagement was "woke shadowbanning," whereby the things you said wouldn't reach the people who asked to hear them because Twitter's deep state didn't like your opinions. The real risk, of course, is that the things you say won't reach the people who asked to hear them because Twitter can make more money by enshittifying their feeds and charging you ransom for the privilege to be included in them.

The temptation to enshittify is magnified by the blocks on interoperability: when Twitter bans interoperable clients, nerfs its APIs, and periodically terrorizes its users by suspending them for including their Mastodon handles in their bios, it makes it harder to leave Twitter, and thus increases the amount of enshittification users can be force-fed without risking their departure.

Lines of inquiry this paper opens 9

Research framings built by reading the notes related to this paper — the questions it feeds into.

How do network effects and self-selection distort aggregated rating accuracy? Do single-axis benchmarks accurately measure agent capability for real deployment? Are AI-generated articles systematically disadvantaged in search ranking and user engagement? What gaps exist between benchmark performance and real deployment outcomes? What governance mechanisms can effectively constrain widely deployed AI systems? When do multi-agent systems improve over single frontier models?