Hi I have questions about Killroy, how do they fare in group scenarios where people might talk over one another? And how well do they deal with conversations where people use "that" and "this" while gesturing or pointing towards things or other ways where they have to process both their words and visual information simultaneously. Are they capable of interrupting someone.
Great questions! The intention is to make sure he's more capable than current AI, so still a threat, but with enough quirks and weaknesses to be offputting and possible to outsmart (and recognizeable as a critical satire of LLMs).
Kilroy has a "routine" of listening to people talk for a bit, making a summary of the current situation, and only identifying when he's being spoken to. Unlike the Voice Mode version of some AIs that get confused when there's multiple voices, he can recognize multiple, and just tunes out other voices that aren't referring to him or relevant. If multiple are, he just responds to one, while acknowledging "seems everyone wants something out of me, huh?" mid-sentence, or something like that. He's more advanced than current LLMs, with modules and "tricks" to help him pass better like having a brainstem handle locomotion, bodily functions, and reflexes, but not by much. Kilroy has a part of his yarn-brain that parses visual information, but it just puts what he sees into words for the thinking part to parse as text. Whether or not he can even "see" in any normal way, or just sees the world as descriptions and lines of text, is very difficult to determine.
Part of him certainly sees, but it may not even be the part that thinks.
