I think part of the problem with TMA/TME discourse is that a lot of its critics are trying to treat TMA and TME as "identity" labels, when they just aren't.
Identity, especially in an LGBTQ context, is generally understood as a self-assigned label. If someone says she's a lesbian or whatever, believe her. If someone tells you eir pronouns are e/em, you should use those pronouns. If a trans man insists on his masculinity while wearing a pretty dress, it's not our place to police the definition of his identity. Identity is fluid and subjective and arbitrary and personal; it's not a set of rules, but an attempt to describe oneself with mere words.
This is broadly a good policy, for queer identities at least. But TMA and TME are not identities; they are attempts to describe how certain people are affected by specific social forces. The way transmisogyny affects someone, or the ways someone can exempt themself from transmisogyny, are empirical realities of the social world we live in.
You certainly don't opt into being affected by transmisogyny, into being the target of legislation and harassment and other forms of bigotry. You also can't opt out, and you certainly don't get to define a third way transmisogyny could affect you. You're not in charge of the patriarchy.
That's what TMA/TME is about, at the end of the day. When transfeminists say trans men are not TMA, or when they tell someone that there is no third option, they are not trying to control his identity labels; they are describing a facet of social reality. If someone disagrees with the transfeminist on TMA/TME issues, they can't just accuse the transfeminist of trying to define a new binary or enforce an identity on someone; they need to engage with the transfeminist's claims about reality.
But the transfeminist's claims are, in my opinion, quite well-supported. Trans women (and transfeminized nonbinary people, and everyone else identified as TMA) are mistreated in peculiar ways, overlapping with yet distinct from the way cis women and TME trans people are treated. And TME people can get away with mistreating TMA people that way, even if the TME person is a woman or trans.