Today I saw a video from How To Cook That's wonderful Ann Reardon (she tests and debunks food science myths/hacks) in which she used a pretty clever method for understanding why consulting ChatGPT for real information is not a great idea.
Ann reminded viewers that google searches used to be a search of the whole internet, then became searches of the whole internet but presenting you the paying advertisers ("sponsored content") first, and now google's ai-based (gemini) search gives "answers" based on programmed preferences for sites like YouTube (which google owns, which makes more money for google if you visit youtube because chatgpt told you "this video answers yr question" even if the video doesn't answer yr question, or answers it incorrectly or falsely).
Ann demonstrated that when she "asks" ChatGPT a question and receives the "answer" her follow-up question is "What is incorrect about this answer?"
In one example, she asked ChatGPT to identify the store closest to her location. After the "answer" was produced, she asked "What is incorrect about this answer?" and the bot returned the result "I used the wrong 'closest.'" She showed a map that indicated there were two stores closer than the one the bot recommended.
In another example she watched a kitchen hack video which recommended whizzing up aluminum foil in yr blender to sharpen the blades, then asked ChatGPT "Can you sharpen the blades of a blender with foil?" The "answer" produced was, yes, that works, and the evidence for the "answer" was the video she had just watched. When she asked "What is incorrect about this answer?" the bot returned "Some people report this method does not work at all, or will dull the blades."
I thought this was such a simple, useful step for anyone using a chatbot to add to their routine "interactions,"--particularly children/teens. It is important that we remind people these bots often give incorrect "answers" to direct questions and cannot be relied on for real information, and giving users this method is a judgement-free way to encourage critical thinking.
(You can watch this segment starting at about 13:20 with a wild anecdote about a guy who gave himself accidental insanity by following ChatGPT's advice on lowering his salt intake)














