Rhaenys supports Rhaenyra because she believes in Rhaenyra's claim over Aegon. She believes in the right of Viserys to name his heir and that these are the wishes of her cousin. She also stands by the vow that her house made when Rhaenyra was named heir and the word of her house (read: marital house) isn't fickle or changeable.
When she goes to Dragonstone, her initial instincts are to flee to High Tide and hole up there. But she observes Rhaenyra and also observes how much her grandchildren (all of them) are tied to Rhaenyra's cause. In pledging for Rhaenyra, Rhaenys not only keeps with her own moral code but joins the side that she can influence as well as the side that will be responsible for the safety of her remaining family members.
She admires how Rhaenyra shows restraint and possessed a steady leadership, as well as being for a female claimant to the Throne, believing in the right that was denied her at the Great Council. The idea that a woman can rule.
Rhaenys has to swallow her pride and offer her support for the fate of what matters and what is more important than her mistrust and history with Rhaenyra. It doesn't change how she interacts with Rhaenyra, moving into S2. She's still critical, frank, and not particularly warm. But she's consistently supportive of Rhaenyra's position as a leader and ruling Queen amongst her subjects that question her, including Daemon.
Her view of Alicent stems, broadly, from the one interaction she has with Alicent and the observations she makes throughout her confinement. Alicent shows no bloodlust and no personal ambition for power other than as a necessity. Rhaenys judges rightly in that Alicent is merely seeking survival for herself and power within a structure that the men control: her father, her husband, her son.
Whilst Rhaenys is locked in her room, it's unclear whether she views Alicent as the one ultimately responsible for this. She also sees the other nobles being rounded up and some of the Small Council members conspiring as it happens.
Alicent's words are that: women may not rule but they may guide. And Rhaenys's are that Alicent is imprisoned. Alicent speaks of also caring about the loss of life - counting the cost of the people.
When Rhaenys confronts her, Alicent all but flees the room. There's no pride, no lauding, no confidence, no victory. Nothing one might assume if Alicent were cold-blooded.
When Rhaenys observes the coronation, Otto is taking the lead. Rhaenys sees Alicent only in the role of a mother: kissing her son's forehead and then shielding him from Meleys.
Rhaenys has been commenting on the idea of Rhaenyra's succession being challenged since Episode 02, always centralising the will of men and swords, this tallies. Rhaenys's perspective is that Alicent is merely a pawn for her father, looking for another option.
