Here’s your problem AI was set to Sycophant-ie aka Ass-Kissy.
https://openai.com/index/expanding-on-sycophancy/
OpenAI did a deep dive on their roll back of their enthusiastically sycophantic release.
It is a very good read that gives behind the scenes looks at how a massively LLM is trained, tuned and tested.
“One of the biggest lessons is fully recognizing how people have started to use ChatGPT for deeply personal advice—something we didn’t see as much even a year ago.”
We are right dab in the middle of Uncanny Valley, repeat after me AI is not my friend.
This is a key lesson for all technology practitioners, it is best to understand usage from the customer perspective, aways. Otherwise be prepared for unintended consequences.