This is literally what people are talking about when they say AI will be used to mainstream widely held bigotry. LLMs are trained on frequency and probability -> straight relationships are more well represented in the dataset -> straight pronouns and terms become the "correct" normal.
This is a form of backdoor bigotry from both normative facts (there are more straight than gay relationships) and well represented bigoted beliefs (men are superior to women).
Combine this with the mass of people inclined to believe (and being encouraged to believe) that if AI says and does something it must be correct
There have been similar problems with using AI in recruitment. The AI used noticed that company had hired more men than women, and deduced that the company wanted to hire men. So it removed all candidates who might be women.
(This was in Finland, I heard about it in AI safety training class)
However the AI also removed a man who coached women's basketball (the word "women's" was enough to get him removed). Luckily he noticed something was off, he was a perfect candidate, so he contacted the company and they noticed the problem. It was accidental and unwanted, but nobody knew to look for it.

















