AI's "Win" and Paywalled Data: What's the Real Deal?

The "Open Web" Scam

So, "information wants to be free," huh? Give me a freakin' break. That's the garbage Common Crawl is peddling while they're busy hoovering up paywalled articles to feed the AI beast. It's not about freedom; it's about corporations lining their pockets with content they didn't pay for, plain and simple.

And Rich Skrenta, Common Crawl's executive director, has the nerve to say publishers should't have put their content on the internet if they didn't want it scraped? What a load of bull. That's like saying if you leave your car unlocked, it's your fault if it gets stolen.

The Atlantic article lays it all out. This "nonprofit" is basically running a shadow library for AI companies, letting them train their models on high-quality journalism without paying a dime. OpenAI, Google, Nvidia, Meta, Amazon... they're all in on it. Of course they are.

The Great Content Heist

It's not just some obscure blog content, either. We're talking about The New York Times, The Wall Street Journal, The Economist, The Atlantic itself... all getting ripped off. And Common Crawl claims they comply with removal requests? The article proves that's a lie. They tell publishers "it's 50% removed...70%...80%..." but the data shows nothing's actually changed since 2016!

Skrenta's excuse? Their file format is "immutable." You can't delete anything. Then why are they lying about removing content? Why the charade? Oh, right, because they're getting "donations" from OpenAI and Anthropic now. Follow the money, people.

Here's the real kicker: Common Crawl has a search tool that deliberately hides the truth. Search for nytimes.com, and it says "no captures," even though millions of NYT articles are in their archives. It's not just the Times, either; over 1,000 domains are affected. They're actively covering their tracks.

I wonder how much Nvidia paid for that AI training data set Common Crawl hosts for them? Or how much "advice" those Common Crawl developers gave them? And is any of this being disclosed to the SEC?

Robot Rights? Seriously?

Skrenta is out there talking about "robot rights" and putting the archive on a "crystal cube on the moon." He wants aliens to reconstruct our history from it, but The Atlantic and The Economist won't be on the cube? Give me another break. It's not about preserving history; it's about enabling corporate greed.

And Baack's suggestion to require attribution? "It's not our job," Skrenta says. "We're just a bunch of dusty bookshelves." Nah, you're a bunch of enablers, hiding behind the "open web" banner while you help AI companies steal from content creators. They expect us to believe this nonsense, and honestly...

It's all a bit much, ain't it? The tech industry always finds a way to repackage theft as innovation.

So, Who Polices the Pirates?