#notebooklm

I had Google whip up a "deep dive" of a piece of my own writing that I shared earlier this year for the ATH 25th anniversary, a scene from The Great Galavar, set many years before ATH where a much younger Silence, new to Sele, asks Galavar for a desk and they go to Javelin's furniture shop to see about getting one. (I only fed Google this scene; not the attached scene where Javelin visits Galavar later that night.)

Here it is:

I'll start off with a complaint that spans the entirety of the podcast: I was put off by the disingenuous way that both podcast hosts acted as if they hadn't read the work under discussion and expected the other to explain it to them, because they keep switching these two hats back and forth (proving very conspicuously that they both know the work very closely), and it comes across as patronizing. I noticed, however, that I found this more off-putting with the podcasts nostalgebraist shared than with my own.

And a couple more quick complaints:

First off, there's no recognition of Silence's left-handedness, so frankly I think we can throw these toasters onto the scrap heap for another 20 years of development. 😏

(Actually, stick a pin in that; I'll come back to it later.)

Second off, I forgive these AI hosts for their horrible butchery of the pronunciation of custom names like "Terlais" and "Sele." 😭😭😭 That's understandable and not off-putting (though it does jar).

But mainly:

I am very impressed by the the AI's ability to synthesize an understanding of the material with such high fidelity and general accuracy! I think it's super cool!

While obviously this represents a major stride for AI technology, I think this is first and foremost to my own credit for writing such a coherent scene in the first place (more on that later). With my own work being analyzed, it was a lot easier for me to notice (as opposed to with nostalgebraist's podcasts) just how the AI was essentially digesting and repeating my own text back at me, occasionally literally but more often through a very basic layer of interpretation and recapitulation. For example, at the beginning they talk about how Silence has this sharp intellect and "sees right through you"; this is true and I would even call it insightful, but it's also explicitly there in the text, conveyed in different words but unmistakably the same idea. So really the AI picking up on it is just a validation that I wrote it in the first place. (More on that in a moment.) This makes the AI synthesis somewhat less impressive than it otherwise would've been, i.e. a very elaborate form of mimicry, but the absolute reading on my impress-o-meter is still very high. This is, all things considered, a fantastic level of comprehension, mistakes notwithstanding (more on those in a bit).

It was a very awkward, disorienting experience for me to hear these convincing simulacra of people earnestly discussing my work, especially with the work being a scene about Silence, which is always quite personal for me. I think, echoing nostalgebraist's thoughts, that it hits on a level that would not have been as intense / visceral for me if the exact same conversation were purely in text. But even if it were text, I definitely feel seen (in a rather uncomfortable way, like with a light that's too bright) that the AI easily picked up on the fact that this scene is all about Silence even though she's not the point-of-view character and Galavar actually has a much larger presence in the scene than the AI deep dive would have you think. The AI saw right through that.

The main complaint I gathered from nostalgebraist's reaction to the AI podcasts about his works was that the AI analysis was anodyne, superficial. I partially agree, which I'll get to. But with this criticism in mind, I thought to be on the lookout, when listening to my own AI podcast, for insights about how a more lay audience might engage with and understand my work. Are any hypothetical lay readers picking up on the main themes? Do they understand the basics of what's happening? How does my work come across to the sorts of people who listen to the kinds of podcasts that NotebookLM is simulating? That kind of thing.

I found the answer promising: Like I implied above with the AI's recapitulation of Silence's power to comprehend people's thinking and nature, if this podcast is any indication, I think my work is more readable than I have given it credit for. I don't consider this scene to be a particularly "easy" one to parse (nor particularly "hard"), but the AI did a pretty good job of it, so maybe I was underestimating a hypothetical human lay audience's ability to absorb and enjoy it as well.

I do have a complaint of my own: I was wryly amused at how the AI replicates the common practice of the male podcast host leading the discussion and doing most of the talking, with the female host providing a lot of affirmations and occasionally offering original insights with less egotistical framing. It's very subtle, and I can't rule out that it's just me being oversensitive and construing data out of noise, plus I don't listen to a ton of podcasts (especially the type of podcast that this experiment is most directly imitating), so there's that caveat too...but I still perceive it strongly enough to wonder what it's doing in there, as it's not something I would expect Google to: A) do deliberately; B) fail to notice.

Anyhow, as to the analysis being anodyne or not: Yeah...yeah, I do see it. But I'd probably look at it as an exercise in realistic expectations. What do we realistically want from an AI podcast host? To be challenged both deliberately and serendipitously with insightful perspectives and non-obvious ideas? To what extent?

To approach those questions, consider this: I mentioned earlier that you should stick a pin in the fact that the AI didn't pick up on Silence's left-handedness. That was a joke, obviously, but it touches on a point of interest for me in this experiment: All of today's big-splashy AIs work by virtue of having lots of examples of human-generated data to draw from and emulate (plus the information contained in your prompt). What happens, then, when you invite them to venture into spaces that humans rarely talk about, such as, oh, for instance, a stylish and objectively correct obsession in sinistrality?? Well, the answer seems to be that they just don't pick up on the opportunity. I'd probably have to be more pointed about Silence's handedness in the text, thereby raising left-handedness to the level of "an idea in itself," for the AI to notice it as relevant to the meaning of the text. In essence, I would conclude that uncommon ideas are nearly invisible to the AI when they are present only in the background. (Silence's left-handedness is explicitly mentioned only once, and is alluded to twice more.) This has implications about this kind of AI's ability to compose original philosophy; namely, that the AI is constrained for the most part only to repeating what we already know and connecting different things that we have already said. And I don't know if this is something fixable, because asking someone to interpret something that they don't already have good information on is an exercise in a form of intelligence that I don't know is well-suited to how these AIs work (and would thus be more likely to produce gobbledegook answers and continually infer meaning where none is intended).

So, in this one respect, I am not impressed by this AI performance, but neither am I thus surprised. I would have been shocked if they had picked up on the subject matter that is much more niche or even virtually exclusive to me.

I think perhaps the next big frontier for getting these AI "deep dives" to have more valuable (the first frontier having been developing the ability for the deep dives to be seriously attempted at all) is for AI to begin doing what humans do, plugging in life experience to connect seemingly unrelated ideas together. By my understanding this is emulable (and thus "fixable") under the current paradigm of AI, at least to some extent. I always enjoy it when some character is talking about some straightforward problem, and their mentor / advisor / parental figure / etc. starts telling this seemingly unrelated story that ends up either recapitulating the same idea in a different way (sound familiar?) or else shedding new light on the original idea through context or testimonials. I think this is something the current crop of AI could be taught to do.

This brings me to a comical moment at the end of the podcast where one of the hosts mentions that she gets "chills down [her] spine every time [she] reads it" (lol) when she gets to this point in the scene:

“Fat, eh? Not many of those around here. It’s not really an Ieikili fashion.” “Give it time!” Silence merrily exclaimed. “I will teach this whole society how to grow fat.”

CHILLS !! LOL

And the hosts go on to elaborate that it's "chilling" because with Silence there must be a double meaning to it, i.e. ambition, influence, etc. This is a very interesting take, and it's also the first of several mistakes I'm going to discuss.

The mistake is twofold. First of all, the female host calls Silence's line about teaching society how to get fat the last line of the excerpt. It's not; it's closer to the middle. I find this error fascinating because it implies the absence or failure of some kind of logical error-checking that I would have thought is trivial but which perhaps isn't. Second of all, there's no deeper meaning to Silence's vow. She just likes fat people. If there is a deeper meaning that must be forced out of it, it is an insight into Silence's multifaceted nature, as elsewhere in the scene there is so much focus on her restoring her health and vitality and mobility after a long convalescence from her prior injuries. She's all up for fitness and mobility, but she's also up for curtailing that on her own terms (as opposed to involuntarily), or having a lover do the same, when the context is right. I think this error provides insight into the kinds of calculated risks that the AI takes in order to compose its analyses. The AI is obviously not well-exposed enough to fat liberation speech in its training data to have picked up on this fat-affirming message as such; instead it drew on the imagery of growing fat as a metaphor for growing in power and influence, which is much more common in our culture—enough that the AI assumed that's what I was talking about.

So, this gets back to what I was just saying about the potential dangers of trying to force the AI to draw conclusions on ideas that it isn't well-exposed to: It'll do it wrong, both in assuming meaning where none is intended and in characterizing what that meaning is.

A second mistake that I'd like to mention is that, early in the podcast, the hosts characterize Silence as someone of "very few words," which isn't supported in the text (and if anything Silence tends to be verbose). What I think what the AI was picking up on is that Silence wasn't sharing her full thoughts with others, which is actually a really clever (and correct) reading, and somehow got from there—perhaps by way of Silence's deliberateness and her initial "passivity" in Sele during her convalescence—to the idea that she doesn't physically talk much. This is a fascinating conflation to make! I assume that what's going on here is that the AI has fitted Silence's personality type into a box, and this box potentially includes the character trait of not talking much, and the AI decided that there is enough supporting language in the scene to establish that Silence indeed doesn't talk much. No such language exists, but I see how an AI (and perhaps a human reader not familiar with Silence) might infer that it does. I'd be curious to round up some humans, make them read this scene, and ask them questions so as to implicitly invite them to say that Silence is a mate of few words without explicitly prompting them to do so. I bet you some would, even though in this very scene Silence is literally quite talky! Because it's easy to compartmentalize those things, I think: Silence's actual talkativeness, and her personality traits which might imply that she is not talkative.

A third mistake, which is, again, wrong in an interesting way is that there is a very clear up-front statement by the male host that Silence "is a woman," with emphatic tone on those words. This isn't the end of the sentence. He pauses, and there is some reaction from the female host, and then he finishes his thought: "...of few words." So, this is mistake not only because of the "of few words" part but also because of the "is a woman" part. I go out of my way to characterize Silence as near to genderless as possible, and while this doesn't apply to physical sex (she is female-bodied) there also isn't any discussion of that in the scene; so the AIs are drawing this purely from her pronouns, the discussion of conation (mindwashing / "mental merging" as the AI aptly puts it), and the clarification that Silence and Galavar are not bonking.

I wouldn't necessarily bother to call it out (I don't expect AI to use my paradigm of sex and gender), except that the way the host delivers his distance really emphasis the "is a woman" words, and I think that's deliberate; I think it's a very clever thing the AI is doing, piggybacking on a separate idea (about Silence not talking much) to insinuate without immediately mentioning it that there are sexual tensions in the scene. Now, it's wrong; there is no sexual tension—not between Silence and Galavar anyway. (And the AI elsewhere allows that the source material is vague on this point; I would say, from reading this scene in isolation, that that's fair.) But in fact this is the scene that woke me up to the fact that there can't be sexual tension between Silence and Galavar, at least not in the past, because she's still practically a goddamn kid when she kids to Sele, and Galavar is early middle-aged. I had never wrapped my head around that until I was laying out this scene in my thirties and realized that it'd be gross. So I wrote this scene with no sexual tension between them at all; the fact that Javelin originally mistakes Silence as a lover of Galavar before actually meeting her in person is part of the verismilitude of Javelin. But I can see how an AI without knowledge of any of these characters' histories outside this one scene might take the mistaken perception by Javelin as a positive indicator of a relationship between Silence and Galavar. So, again, I think we are seeing some of the limitations on this type of AI. In the grand scheme of things, many human storytellers would have sexual tension in a situation like this. And I think the AI failed to see that there wasn't any, or at least wasn't able to make up its mind.

I actually like this failsafe. The podcast hosts mentioned several times that the source material is unclear or vague, which makes it a lot safer for them to speculate. That, in my opinion, is a really solid implementation by Google of a way to lampshade the AI's risky assumptions to the end user. This way, if the podcast hosts go barking up the wrong tree at least they will appear to be doing it honestly, based on the text's own ambiguity. And I think all the instances where the hosts invoked this failure are justifiable instances, even if they aren't necessarily correct that the text is in fact ambiguous.

Another mistake, and one that's a lot more disappointing for technology which is otherwise getting so impressive, is that the hosts misgender Javelin later in the podcast after originally getting it right. (Editor's note: Actually I am not sure that they gender her correctly in the first place. I've listened to this thing so many times in the course of writing this that I'm not up for going through it again, so take the following with an asterisk.) I suppose this too is a reflection of how the AI actually weaves its stories: To them, the "Javelin" mentioned at any one moment in their podcast isn't necessarily the Javelin mentioned at any other moment. The AI''s persistent understanding of, for instance, the City of Sele as the setting for this scene does not imply a comprehension of what "the City of Sele" is but rather that it would be appropriate to mention a thing which is probably called the City of Sele in various instances where our human minds would in fact agree that it is appropriate.

A very understandable mistake is that the hosts infer that Silence's choice of the word "secretary" has some special meaning that goes beyond the actual meaning. (She chooses the word, having only recently become proficient in the Selish language, because of its relationship with "secrets.") The AI hosts are wrong in their speculation but they're right to be sniffing something there. (It is, after all, the title of the excerpt.) And you kind of need to have the other scene, with Galavar and Javelin that evening, to better understand why it's the title, and so I don't hold it against the AI for falling short here. I do like that they tried to mention it at all; I think they would have been remiss not to.

A final mistake that I'll mention is that, when talking about Silence's new immersion in Sele after having lived in Junction City, the hosts did indeed take an incredibly anodyne route of characterizing this discontinuity. I found that line of discourse to be very underwhelming. And it rises to the level of a mistake because it is not correct. Silence's adjustment to Sele is not complicated all that much by her past life in Junction City. That's just plausible-sounding tripe; it's not analytical at all.

There are plenty more mistakes, of course, but, moving on from mistakes to talk about specific praises, I really like how the AI hosts were able to pick up on the fact that Silence's behavior in this scene is all about her trying to prove herself. It is specifically stated in the text (albeit in a narrower sense, referring only to Silence's newly restored ability to walk partway across the city), so I'm not surprised that the AI picked up on it, but I'm glad that it did. That smacks of good design to me. It's a very important detail. And, assuming Google put its "top men" on the case for making these deep dives as compelling as the technology (and the company's contortions to avoid getting sued) presently allow, I would be inclined to consider the successful identification of this point as important to be a high-quality indicator that their efforts were constructive.

But, of course, this is the same tech that I found so underwhelming in the previous paragraph, so...you know. You win some; you lose some.

Another praise is that I appreciate the hosts for being gentle with me in describing "Silence's almost superhuman level of perception." It was equally within Google's power to invent an AI podcast host duo that will just absolutely trash you and roast you and make fun of you instead of acting like your work is the most interesting composition they've seen all year. Silence's capabilities are definitely possible, and I would even argue plausible, but I fully grant she is performing way above your average schlub. A character like her may not defy belief, but she strains likelihood. So I think it's fair and probably even authentic for the hosts to point out that she's "almost superhuman." I think not mentioning it at all would be a mistake. Silence is not just making power plays; she's an extremely competent person making power plays.

Anyway! Let me go back to something I said earlier that I said I would get back to:

I think this is first and foremost to my own credit for writing such a coherent scene in the first place

It occurred to me, even before generating this deep dive podcast, that I would like to feed the AI a scene that is much more difficult to parse, and see what that "deep dive" looks like.