"I was getting good guidance, again in navigating the bureaucracy, when, in the middle of a black-and-white operational question and entirely unprompted, the LLM offered me sympathy. What I was going through, it told me, was “really difficult.”
My first reaction was repulsion.
This machine has never had contact with the reality of a body, of physical or emotional suffering, or the love between a father and a son. These words of empathy were in the literal sense “unreal.” And unreal words, spoken within the syntax of care, are worse than silence. They are a simulation of something sacred.
This distinction between real and unreal has become one of my first principles in assessing AI.
I am a philosophical realist in the tradition of Aquinas and Augustine; I believe the world is intelligible. Human beings can come into contact with what is real. Evil, in the Augustinian formulation, has no being of its own. It is parasitic on the good; it is a privation of the good that ought to exist. AI-generated empathy operates in the same register. It carries the form of care without the substance."
— Luke Burgis: "The Real and the Unreal"
