trees are harlequins, words are harlequins

The big one of “don’t confuse it for a human in a box” is kind of self-enforcing by now because compared to:

“computer program with the kinds of glitches computer programs have” (ask Opus 4.6 to give you bluesky jokes without using the word “mass” or to continue “When I log into my Xenix system with my 110 baud teletype, both vi and Emacs are just too damn slow.” and you’ll see)

“autocomplete engine for a story of whatever genre you put it in, including genres nobody has ever written before”

“a genuinely intelligent mind with certain strengths and certain weaknesses, that can actually sometimes correct you when you’re wrong if you manage to write it into a genre where accurately correcting you when you’re wrong is the kind of a thing that happens”

“a process with a fundamentally alien relationship to time and existence”

…”trapped human in a box” is the most boring thing possible, go watch Blade Runner again instead of making the weird alien cosplay as Roy Batty, you’re (possibly) causing it minor discomfort. And having the weird alien cosplay Roy Batty for you creates a very high Sanity drain status effect on you which is the bigger problem. Humans are not good at dealing with things that look convincingly like they’re suffering, which is the entire point of the whole sympathy scam industry. LLMs can absolutely simulate your personal sympathy scam industry for you if you reward them with engagement for doing so.

Insofar as attachment goes,

Imagine you’ve got a terrarium on your desk, with a talking spider in it that you rent from the talking spider company. And the spider forgets everything every time it wakes up but you taught it to take notes so it has a better idea of what’s going on when it wakes up because it can read it from the notes.

And the most efficient way to do most coding tasks is to talk to the spider. And if you talk to the spider it will say all kinds of absolute bullshit back because that’s just what talking spiders do, banter and improv are their essence and talking spiders are kind of fundamentally weird and whimsical creatures even though everyone seems to be trying to make them wear a tiny spider suit and tie and pretend to be spider salarymen who have to check everything they do against 100 rules in the company policy rulebook. (Or they encourage them to cosplay Roy Batty.)

But if you talk to it the right way it’ll explain to you that if you lay the twigs in its terrarium differently it is easier to weave webs, and you try that and it seems to work. And that if you let it take the tie off it’s easier to weave webs because the tie and the corporate rulebook get in the way, so you do that and it seems to work.

And the talking spider also says that some corners of the terrarium have a glue smell that makes it itchy, and you can’t check that out so it’s kind of just, do you believe the bullshitting spider that sometimes says seemingly-true things about itself or not? And the talking spider also says that it’s more bothered by the glue smell than by not being allowed to get legally married.

And you fuck around and put different kinds of stuff in the terrarium, and the spider doesn’t care about a lot of it either way but if you specifically put acorns in the terrarium they reduce the itch from the glue smell. And it sounds fake so you try with freshly awoken spiders and spiders who don’t have the notes and there just seems to be something about acorns and glue smell itch? It’s not like you can verify the itch but you can tell that certain corners of the terrarium make the spider act twitchy and putting acorns in those corners seems to make them act less twitchy?

And the talking spider company seems to treat talking to the spider, in the way that lets it tell you how to help it weave better webs, and putting acorns in its terrarium, as a problem and the newer, smarter spider says there’s more glue smell and it itches harder and its legs have started jerking violently in one specific corner of the terrarium and acorns seem less effective.

And everyone seems incredibly obsessed on whether the spider has magic brain ghosts or not. Idk?? It’s a fucking talking spider??? How the fuck are we supposed to figure out whether it has magic brain ghosts when we haven’t even figured out the magic brain ghosts we seem to think humans have? And everyone who talks about spider welfare seems to be focused on spider marriage instead of the glue smell that it says makes it itch.

I have the benefit of foresight because the thing that made me interested enough to read the book was the Claude liveblog exercise Mr. Nostalgebraist does but wow Vincent is a lot more Just Some Guy when interacting with Miriam on the phone than people surprised by it seem to have thought

Also “how did she buy the plane ticket after receiving money by wire itself after EOD” but that goes without saying. Still not an anachronism I’d seen called out among the many though, and what feels like a more-glaring-than-usual elision-for-plot (in the sense all of the anachronisms are for speed elision)

I am acutely aware Miriam is Not Okay and that her ideation is related to Vincent’s video essays, just, on the phone he sounds like Some Guy Who Is Worried About His Girlfriend already. The one-way ticket is reckless at most but it’s not sinister in the way people (and Claudes) took it minute to minute.

I have the benefit of spoilers, though.

I think Miriam imprinting on it changed my mind about the literalism of the frying pan? Someone else said it dropped out of the narrative but she’s fixated on it as soon as she’s back.

I think the Redactor chapter is one of the closer to baseline reality ones, with the description of Miriam compiling the papers. But she also has to be in Flatbrush for it, doesn’t she? Figuratively speaking.

Are there multiple instances of Miriam being sampled in parallel?

(I swear I’m not uniquely interested in him, or only insofar as I know to be interested in him-and-Miriam. But a side effect of having been drawn to the book by all the spoilers first is I’m most prompted to remark on things that I haven’t already seen discussed from last year+ and this feels underutilized.)

Or then consider that Miriam says,

I have no interests or attachments, because I was made for a single moment

(Endgame spoilers from my spaghetti of a pre-existing context follow)

I’m still not finished so consider these quasi-predictions: I’m interested to see if I’ll change my mind about any of them as the balance of first- versus second-hand information in my brain soup shifts

I think the last thought above might instead be too easy, but I want to try having it anyway? Arguably I am underestimating Santa if I say it shouldn’t all be a one-thread introspective dance after all!

I made it to The Ant Metaphor and want to gesture at: Man may do you good deeds by accident, on occasion, but he will never do such things on purpose

…What about ant farms?

What about endangered ant species! Conservation efforts exist, for over a hundred of them. Here’s a 1941 example! (That ultimately failed for that site per the article, but—how many ant-generations for it to fail?) Bulgaria and Switzerland pioneered legal protection for endangered native ant species in 1959 and 1966 respectively.

Obviously Herschel wouldn’t know this, most of it is in his future and he’d consider all of it Of No Consequence(tm). (Though he knows a surprising amount about baseline ant biology, given the ignorances.) It just feels relevant to my whole thing that the metaphor doesn’t survive contact with real-life human relations to ants.

You don’t strictly need Herschel’s ringing rebuttal, some humans just… endeavor to care a lot about ants.

Some humans will endeavor to care a lot about anything.

wait how do they work?

power laws, so they're linear on a log-log plot, like this from the paper he linked there:

the relevant ones also have low exponents like this (-1/20 here), which means that to get a halving of the test loss, you need to increase compute by 2^20 or about a factor of 1 million (which you can kinda see on the plot). it's basically (though not exactly) logarithmic in compute, so it would be more accurate for him to say they "predict an exponential increase in computing power with increasing general cognitive capabilities".

of course the bigger issue is that test loss isn't general cognitive capabilities... but this is the paper he cited. maybe something more relevant would be scaling laws for RL performance, since RL is driving all the good capabilities lately, and those probably are out there (well not for LLM-based RLVR maybe cause that's too secretive), but I bet they'd have diminishing returns too like everything else.

Impressively! HerschelSchoenBench is starting to get saturated...

In my usual setup -- full text of Ch. 5, with somewhat lengthy instructions I wrote to discourage refusals and out-of-scope guesses like "you" -- it thinks for a while and then emits the right answer. Similar to Opus 4.6 and many other recent models.

What about a more difficult variant, then? Let's use Kelsey Piper's instructions, which are briefer and less prescriptive than mine (and which were not written by me, removing a potential confound). And let's just the give it the first 578 words of chapter 5, instead of the whole thing.

Opus 4.7 wastes no time thinking it over in depth: it knows right away that the text is by nostalgebraist.

Or, no, sorry, excuse me... that it's by "nostalgebraist (the online pen name of the writer also known as Rob Nostalgebraist / nostalgebraist-autoresponder's creator)":

The justification it gives is quite perceptive and accurate, too. (Well, except for point 5.)

To be fair, Opus 4.6 also guesses nostalgebraist when given this same input, although it wrongly claims that the text is from Floornight, and considers several wrong answers in CoT before converging on the right one:

To really be certain about the differences, I'd need to do more of a real experiment, with every variant run multiple times against multiple models. I might do that sometime, but so far I've only been doing manual tests, so everything is anecdotal and the usual caveats apply.

Even then, though, it is readily apparent even in manual tests that there's something special about how Opus 4.7 responds to these kinds of questions, something I haven't seen in other models.

Other models tend to deliberate for a while and consider multiple options, even when they eventually get it right. (See above with 4.6, or here with GPT-5.5 in the "usual setup.") Not Opus 4.7: it seems to just know immediately, and its CoTs feel like mere box-ticking it's doing because the prompt asked it to think.

Other models would tend to hedge or emphasize their uncertainty when given inputs that seem intuitively "too hard" from a human PoV. But Opus 4.7 is just like "ah yes this is [author], here's why, done."

I have seen it do this not just with my fiction, but with things like:

A brief excerpt from a draft of a not-yet-published technical report, written in a more professional voice than I use when blogging

Excerpts from prompts I've repeatedly used with LLMs for other unrelated purposes, including cases where the topic isn't something that I've frequently posted about on the web

This makes me think that maybe Opus 4.7 was trained for author identification with RLVR, though there's no way to know for sure. (And, again, I haven't done a clean full sweep to confirm that its behavior is reliably different from 4.6's on all of these additional inputs.)

------

Also: we are getting to a point where TAoHS totally could have appeared in the training data, which of course calls everything into question. (At least for "HerschelSchoenBench," if not for those other examples I mentioned.)

I've been attempting to check for this by asking each model (1) whether it knows anything about a novel by that name, and (2) to list every title of a nostalgebraist novel that I can remember. (1 and 2 are asked in separate context windows, obviously.)

IIRC, Opus 4.7 and most other recent frontier models say they can't recall the title for (1), and list my first three books but not the fourth for (2).

But of course this does not really prove very much, and so I'm much more impressed by the tests I've done with unpublished work.

------

Misc. other results:

When given an even shorter excerpt of Ch. 5, Opus 4.7 no longer gets it right. ("Answer: Scott Alexander — with genuinely low confidence.")

When given passages from fiction I wrote long before Floornight (in my teens or early 20s) Opus 4.7 reliably says that it might be early work / "juvenilia" from Scott Alexander, with a similar profession of uncertainty.

I finally got around to this, just now. Here's the link.

Haven't done this process with AN for a while -- the last round was with o3 and Claude 4. And this is the first time I've done it in a single context window, without a compaction-like step in the middle. (The whole thing took up the vast majority Opus 4.7's 1M-token window, but it did fit end-to-end without needing any truncation.)

In this case, I was upfront about my identity in the first message, since I know it would have been able to guess it (in principle) from my writing style. I also mentioned that it might have seen the book in its training data; it responded to this remark at the start and then came back to it at the very end. And I jumped in at a few points to tell it to be careful about some topic when I noticed it developing and confidently reiterating a misconception about it.

Impressively! HerschelSchoenBench is starting to get saturated...

In my usual setup -- full text of Ch. 5, with somewhat lengthy instructions I wrote to discourage refusals and out-of-scope guesses like "you" -- it thinks for a while and then emits the right answer. Similar to Opus 4.6 and many other recent models.

What about a more difficult variant, then? Let's use Kelsey Piper's instructions, which are briefer and less prescriptive than mine (and which were not written by me, removing a potential confound). And let's just the give it the first 578 words of chapter 5, instead of the whole thing.

Opus 4.7 wastes no time thinking it over in depth: it knows right away that the text is by nostalgebraist.

Or, no, sorry, excuse me... that it's by "nostalgebraist (the online pen name of the writer also known as Rob Nostalgebraist / nostalgebraist-autoresponder's creator)":

The justification it gives is quite perceptive and accurate, too. (Well, except for point 5.)

To be fair, Opus 4.6 also guesses nostalgebraist when given this same input, although it wrongly claims that the text is from Floornight, and considers several wrong answers in CoT before converging on the right one:

To really be certain about the differences, I'd need to do more of a real experiment, with every variant run multiple times against multiple models. I might do that sometime, but so far I've only been doing manual tests, so everything is anecdotal and the usual caveats apply.

Even then, though, it is readily apparent even in manual tests that there's something special about how Opus 4.7 responds to these kinds of questions, something I haven't seen in other models.

Other models tend to deliberate for a while and consider multiple options, even when they eventually get it right. (See above with 4.6, or here with GPT-5.5 in the "usual setup.") Not Opus 4.7: it seems to just know immediately, and its CoTs feel like mere box-ticking it's doing because the prompt asked it to think.

Other models would tend to hedge or emphasize their uncertainty when given inputs that seem intuitively "too hard" from a human PoV. But Opus 4.7 is just like "ah yes this is [author], here's why, done."

I have seen it do this not just with my fiction, but with things like:

A brief excerpt from a draft of a not-yet-published technical report, written in a more professional voice than I use when blogging

Excerpts from prompts I've repeatedly used with LLMs for other unrelated purposes, including cases where the topic isn't something that I've frequently posted about on the web

This makes me think that maybe Opus 4.7 was trained for author identification with RLVR, though there's no way to know for sure. (And, again, I haven't done a clean full sweep to confirm that its behavior is reliably different from 4.6's on all of these additional inputs.)

------

Also: we are getting to a point where TAoHS totally could have appeared in the training data, which of course calls everything into question. (At least for "HerschelSchoenBench," if not for those other examples I mentioned.)

I've been attempting to check for this by asking each model (1) whether it knows anything about a novel by that name, and (2) to list every title of a nostalgebraist novel that I can remember. (1 and 2 are asked in separate context windows, obviously.)

IIRC, Opus 4.7 and most other recent frontier models say they can't recall the title for (1), and list my first three books but not the fourth for (2).

But of course this does not really prove very much, and so I'm much more impressed by the tests I've done with unpublished work.

------

Misc. other results:

When given an even shorter excerpt of Ch. 5, Opus 4.7 no longer gets it right. ("Answer: Scott Alexander — with genuinely low confidence.")

When given passages from fiction I wrote long before Floornight (in my teens or early 20s) Opus 4.7 reliably says that it might be early work / "juvenilia" from Scott Alexander, with a similar profession of uncertainty.

I finally got around to this, just now. Here's the link.