One of the main reasons she stays by his side is because she sees herself as the guardian of flame alchemy. She’s trying to make sure of two things: 1. no one ever gets their hands on it again. and 2. the only person who can use it never misuses it again. That’s why she follows him. She has to. She’s watching the weapon she helped create.

And she has to keep him alive not only because he’s the one with the political power (meaning he’s her only real way to make sure the trials happen), but because he has to live long enough to atone for what he did with flame alchemy. The power SHE gave him.

From Riza’s point of view, every crime Mustang commits is hers too. Every time flame alchemy is used to kill that blood is on her hands too. Every person he killed is someone she killed by proxy.

She’s not just guarding him. She’s sharing his guilt. It’s like she’s trying to atone for her own sins, but he has to atone too so she can atone through him.