AI bots steal traffic; tokens to pay creators.
It's the same panic; just new money.
Cloudflare’s CEO dropped an interesting nugget the other week: AI bots are devouring publisher content but barely sending any traffic back; e.g., Google sends one visitor for every 18 pages crawled (it used to be one in two); OpenAI sends one for every 1,500; Anthropic, one for every 60,000. (StrictlyVC → Engadget → Axios for those nostalgic about attribution.) Fun fact: I literally had “Cloudflare should offer pay-per-crawl” in my notes for this post before they announced it today. (Swear.) Sometimes you get scooped by your own scribbles.
I mean, duh. But… THIS. IS. NOT. NEW. Super duper short history lesson:
1990s - content moves online.
Peanut gallery: “The Internet is killing media! Nobody pays for digital!”
Spoiler: it didn’t.
2000s - web traffic originates from search.
Naysayers: “Google is sending people *away*! It’ll never make money!”
Spoiler alert: Google made money.
2010s - traffic originates from social (and mobile).
IPO investors: “Facebook and Twitter send more traffic than search, but they’ll never monetize mobile!
Spoiler alert: Facebook did. (Twitter… meh.)
How do I know? (1) I’m old; (2) I remember; and (3) I wrote about it here, here, and here.
This time? Same shift; new panic. Again.
Moving online killed (print) subscriptions, but unlocked global audiences and ecommerce. Search flipped “stickiness” on its head; i.e., Google wanted you to leave so long as you clicked an ad on the way out. Social put links in your feed, not just your search bar and turned every user into a micro-distributor (and sometimes a Russian bot farm).
Now traffic originates from chatbots. Knee-jerk reaction: “Stick ads in the chatbot!” Easy, right? Wrong. Chatbots remix training data into new answers so there’s no easy page to slap an ad on. Sure, sponsored results are possible (just like Google search). Or a more Spotify-like licensing model. Or (somehow) tracking attribution more precisely. Or the truly scary route: let advertisers pay to tilt the model’s answers.
So, what then? Some publishers will block AI crawlers. Some (like Cloudflare’s customers) will experiment with pay-per-crawl; genius, if it works. And somebody else will figure out attribution, or licensing, or “Spotify for text.” The next intermediary always shows up.
Either way, the direction is clear: consumers are shifting to the AI chatbot approach for consuming and creating content. And that is unlikely to change. When Napster gave people any song, anytime, for free, the behavior stuck. The lawsuits killed Napster, but Spotify figured out how to make it work for listeners, artists, and itself; another ménage à trois where (mostly) everyone wins.
Meanwhile, the money: running one question and answer (i.e., prompt and response) costs pennies in compute; but do it a billion times and you’re burning, well, billions. Rough (“napkin”) math says heavy ChatGPT users probably cost more than they pay. But OpenAI sells API access (to enterprises, startups, developers) at a healthy markup; that’s where the real money flows. They make margin on the pipes, not (just) the chatbot. Inference costs drop every year (GPT-3 in 2020: ~10¢ per 1K tokens; GPT-4: ~2–5¢). And ChatGPT itself? A monster marketing channel and a constant supply of fresh training data.
So is ChatGPT Spotify or Napster? Neither, yet. But if history is any guide (and if you’ve read me before you know it always is), someone will turn leakage into licensing, piracy into subscription, crawling into pay-per-crawl. Somebody always finds the money in the banana stand.
First clicks, then clouds and socials and mobiles. This is the third shift: AI ships tokens. Follow the tokens. That’s where the money goes.