#gpt-3

Tired of generic greetings?

Confuse your friends with AI-generated Valentine cards!

GPT-3 generated the messages and descriptions, and then I followed its instructions to create the cards. More explanation here!

How this abomination came to be (and bonus content)

If you’re worried about AI taking over the world, you should know that at the moment the advanced chatbot ChatGPT that is freaking everyone out thinks that 8 + 5 = 15.

But hey, at least it apologized.

Objectively funnier, asking the AI smash or pass:

While I can't claim a full understanding of GPT-3's weaknesses, the ones I've noticed are:

Very poor model of sentence level language

Poor model of paragraph level language

Very slow to learn things

GPT-3 doesn't know the meaning of a given sentence, and it doesn't know the meaning of a given complete text. To get the meaning of a given sentence, it has to see an entire paragraph of text in order to extract a bunch of facts and concepts that make up the meaning of that sentence. This is because the output of GPT-3 is basically a list of facts and concepts.

There are other limitations. For instance, it often needs a lot of context to extract a given fact, because facts tend to be abstractions and GPT-3 doesn't have a very good model of abstractions. A sentence like "John was the worst painter in the world" is actually a complex statement that depends on a bunch of facts.

There are also problems at the level of larger units of language (e.g. stories or discussions) that are hard for GPT-3 to really "understand." For instance, GPT-3 does not like or care about pronouns. It's perfectly happy to say "he was the worst painter in the world." It isn't clear whether it ever "learns" that "he" is a pronoun referring to some sort of entity in the real world, or just learns to "use" certain pronouns for certain abstract concepts (e.g. "John" for "this painter"). It "knows" words, but it doesn't have much notion of what words are for or who they're for – and so it isn't obvious that "John" could refer to some particular painter.

What I'm saying here is that we shouldn't necessarily assume, when writing about the weaknesses of GPT-3, that we're seeing problems that are very hard for the network to solve. "Problems in which GPT-3 excels" may well be much easier to solve than we think, and even "simple" issues might be complex in other ways than we realize, and so the problems we think of when saying "GPT-3 excels" might be much easier to deal with.

For instance, when I say things like "GPT-3 likes and cares about pronoun cases, and you don't, you cant", this may be true, but what I mean is that I think GPT-3's ability to "learn from past training data" is much better than we might expect.