im hungry

1. LLM-generated code tries to run code from online software packages. Which is normal, that’s how you get math packages and stuff but

2. The packages don’t exist. Which would normally cause an error but

3. Nefarious people have made malware under the package names that LLMs make up most often. So

4. Now the LLM code points to malware.

And since I'm waiting I decided to contribute an over-simplified analogy to explain this.

Analogy: You're in charge of running a kitchen. You and your staff create recipes and sell the meals you make. Inevitably though your recipes will call for things like "a stove" and "a blender" which you and your staff would not want to create from scratch.

Luckily "a stove" and "a blender" are things you can acquire and not try to make from scratch. You and your staff, as humans, are capable of recognizing real appliances, and getting them from real sources.

(There is actually an existing threat where "a stov" is a malicious thing, created by someone who knows "a stove" is in hot demand and is trying to take advantage of someone who might typo when ordering "a stove". There are some safe-guards in this space, but not 100% guarded.)

But now there's Cooking AI that can run your kitchen for you 🙂. It can write your recipes, order the necessities, and assemble the dish for you 🙃. Your boss fires you and your staff and just uses the Cooking AI.

The AI, in its infinite wisdom, starts writing recipes that call to be cooked on "a hotcob". It writes recipes that call for the ingredients to be assembled in "a produceslicer". These are not real things. And usually when the AI tries this, the process will error out because it fails at the process of acquiring the hotcob or the produceslicer.

But the kinds of people already profiting off supplying "a stov" take notice. AI likes to request these appliances frequently. The retailer offering "a stov" starts offering "a hotcob" and "a produceslicer". Now these AI-automated chefs succeed because their recipe order comes together!

"A hotcob" adds liquid mercury to all the dishes. "A produceslicer" hacks your wifi and steals all of the business's information. This is allowed because the AI chef welcomed these things in, signed for them, and hooked them up.

So from then on whenever one of us called something problematic, he would make us talk it out until we could name the “bad” thing we were hinting at. In this particular class, 7/10 it was some type of oppression, and the remainder was like, “I’m uncomfortable because this is very new/confusing/pushing boundaries that made me feel safe.”

Once we stopped calling things “problematic” and stopping at that, class got way more interesting and... we all had to say, like, “that’s racist” or “that’s misogynistic” or “ew capitalism gross” out loud, which a lot of us had never done in a classroom before. Or we had to be like, “Uhhh... I’m not sure what’s so bad?” and confront our own beliefs and that was maybe even more useful.