This is literally what people are talking about when they say AI will be used to mainstream widely held bigotry. LLMs are trained on frequency and probability -> straight relationships are more well represented in the dataset -> straight pronouns and terms become the "correct" normal.
This is a form of backdoor bigotry from both normative facts (there are more straight than gay relationships) and well represented bigoted beliefs (men are superior to women).
Combine this with the mass of people inclined to believe (and being encouraged to believe) that if AI says and does something it must be correct
I worked briefly in regulating AI back when the US actually believed in that and this was exactly the shit that I, the single queer enby in the office, spent trying to beat into the heads of all the white straight men around me.
It is worth noting that as one of its first actions, the Trump administration pulled federal funding and halted federal work on preventing bias in AI (and then proceeded to rip up all the guardrails I and my fellow ethicists had installed).
There is a reason the AI shitsunami has hit over the last two years and it's that someone went and erased all the regulations.
Sigh.
