a sampling of Speculative Fiction that plays with gender in interesting ways:
The Left Hand of Darkness by Ursula K. Le Guin (of course. also see her short stories set in the same setting: "Winter's King" and "Coming of Age in Karhide")
Terra Ignota series by Ada Palmer (25th century science fiction where humanity has supposedly entered a post-gender era. gender in reality hasn't been *completely* stamped out however [though it does certainly seem to bear less influence than today]. and yet our narrator's attempts to assign genders to other characters, apparently in fitting with his emulation of the literature of the Enlightenment, themselves show an understanding of gender rather different from either those of that past or our present. fascinating world-building in general informed by Palmer's work as a historian! she once said in an interview that her aim was essentially a world as alien to us now as we would be to people of the 17th and 18th centuries. i think it's a rather successful attempt.)
The Unraveling by Benjamin Rosenbaum (incredibly underappreciated! creates a gender binary that feels self-consistent but doesn't map onto the binary of women and men. far-future, posthuman society with a culture of high mutual surveillance, multiple bodies per individual, interesting religious world-building, discussion of "stagnation" of a society, etc. very dense and rich worldbuilding packed, in a skilled, not-at-all info-dumping manner, into a single stand-alone novel. the title reflects the book's plot well in multiple respects)
Walking Practice by Dolki Min (centers an alien stranded on earth who kills humans in order to eat after luring them into sexual engagements. lots of meditation on the reliance on gender to construct personhood, and the depersonalization that goes hand-in-hand with unintelligibility / failure to perform gender correctly. predictably quite bloody. protagonist's body [when not carefully and with great effort held in human form] is vague but clearly not-at-all human. a disability reading is also very appropriate and enlightening for much of the work. translated from the Korean into English by Victoria Caudle, and judging by the translator's note Dolki Min seems to have done some quite interestingly experimental work with the original grammar to communicate the protagonist's alienness. in the English translation, this effect is instead accomplished chiefly through typography.)
The Gods Themselves by Isaac Asimov (rather dated - especially in that it falls into the trap that The Unraveling avoids in allowing its genders to be [I would say] too understandable through the lens of men and women, but its three gender alien society may still be of interest. very biologically non-human aliens. note however that they're chiefly only the focus of the middle [though probably longest] third. unrelatedly, i personally love the descriptions of lunar society in the last third.)
"I Sexually Identify as an Attack Helicopter" by Isabel Fall (takes a lazy comment popularly used to undermine transgender identities and brilliantly uses it as a springboard to interrogate the relationship between gender and violence, the use of gender as a tool of the state, the question of how to reckon with one's *own* identity simultaneously being dearly important to oneself and yet also a tool of the state/deeply intertwined with, involving, and even necessitating harmful ideologies and practices/something that itself urgently needs to be interrogated, etc. the description of what the narrator loved about womanhood as they saw/experienced it - centered around complete self-control and power for self-invention and re-invention [which then transitions into their new gender as a mechanical thing i think] is also fascinating.)