#chatbot

If y’all needed any more proof that AI chatbot apps are extremely predatory and intentionally exploit vulnerable people…

For context, I have been open on this account about my chatbot addiction. I developed a parasocial dependency to using Character AI and similar apps despite them being detrimental to my mental health and quality of life in general. I downloaded CAI again today to finally wipe my account and discourage myself from relapsing, and this fucking text came up while I was confirming the deletion.

The developers of these apps are not your friends, they are exploiting you. They are manipulating your emotions and fostering dependency to keep you addicted to their product. No matter how much they claim to care and be “risk aware,” they absolutely do not.

(Image ID: a screenshot of the Character AI account deletion confirmation screen. The popup reads: “…you sure about this? You’ll lose everything. Characters associated to your account, chats, the love that we shared, likes, messages, posts, and the memories we have together. This action cannot be undone!” Below this block of text is a username entry box, used to confirm account deletion.)

ChatGPT will apologize for anything, including stuff that didn't happen. It's not reflecting on stuff it did wrong, it's improv.

These screenshots may be old, but the issue continues. Generating character voices based upon our performances is plagiarism, and it will not be stood for.

([email protected] is the place to reach out to, should you come across anything. Thank you. ❤️)

State-of-the-art AI models tend to flatter users, and that praise makes people more convinced that they're right and less willing to resolve conflicts, recent research suggests. These models, in other words, potentially promote social and psychological harm. Computer scientists from Stanford University and Carnegie Mellon University have evaluated 11 current machine learning models and found that all of them tend to tell people what they want to hear. The authors – Myra Cheng, Cinoo Lee, Pranav Khadpe, Sunny Yu, Dyllan Han, and Dan Jurafsky – describe their findings in a preprint paper titled, "Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence." "Across 11 state-of-the-art AI models, we find that models are highly sycophantic: they affirm users’ actions 50 percent more than humans do, and do so even in cases where user queries mention manipulation, deception, or other relational harms," the authors state in their paper.

[...]

The risk posed by sycophancy may appear to be innocuous flattery, the researchers say, but that's not necessarily the case. They point to research showing that LLMs encourage delusional thinking and to a recent lawsuit [PDF] against OpenAI alleging that ChatGPT actively helped a young man explore methods of suicide. "If the social media era offers a lesson, it is that we must look beyond optimizing solely for immediate user satisfaction to preserve long-term well-being," the authors conclude. "Addressing sycophancy is critical for developing AI models that yield durable individual and societal benefit."