btw anthropic's internal document on this literally said "we don't want it to be known that we are working on this.”

it was called project panama.

here's exactly what happened:

1: anthropic concluded that books were the cheapest way to build a world-class model because they gave claude curated facts, structured arguments, compelling stories, and writing “an editor would approve of.”

2: once anthropic decided it needed books at enormous scale, its first solution was piracy.

it downloaded 7m+ books from online libraries including libgen. the judge later wrote that although anthropic had legal ways to buy them, it chose piracy to avoid what dario amodei called the “legal/practice/business slog.”

3: that piracy created a massive legal risk.

so in february 2024, anthropic hired tom turvey, the former head of partnerships for google books, to find a legally safer way of obtaining “all the books in the world.”

4: turvey first contacted major publishers about licensing their catalogs.

those attempts didn’t produce agreements, so anthropic chose a route that required no publisher permission: buying millions of physical books through distributors and used-book retailers.

5: within about a year, anthropic spent tens of millions acquiring and scanning millions of books, including many rare and 1/1 titles. one vendor proposal targeted 500,000 to 2 million books in six months.

6: to scan that many books within months, the vendors physically dismantled them.

a hydraulic cutter removed each spine. the pages were trimmed to size, fed as loose sheets through high-speed industrial scanners, and converted into searchable PDFs. the paper remains were then sent for recycling.

7: these PDFs were fed into claude as training data.