When I first watched this scene I couldn't tell if Char was making fun of Garma here or not, but after watching it a few more times, it really strikes me how "because he was a spoiled kid/because he was a brat" is so much more personal than "blame this on the misfortune of your birth." One puts no personal blame on Garma in the worst way possible, whereas the other personally implicates him. "Because he was a spoiled kid," in particular, puts Garma beneath him when they're both the same age. Why didn't Char say this to Garma when he was taunting him, was he particularly trying to twist the knife by telling him his death was personally meaningless?
It's tempting to say this comment is him personally trying to justify a betrayal he feels bad for, "you died because you were too spoiled/naive to see the world as it is," but I really don't know. The scene is so short and the line delivery so straightforward it's hard to tell. That being said, if that reading is the case, there's nothing I think Garma could have done to subvert Char's revenge plan. Even if he hadn't been naive, even if he personally turned against his family and all they stand for, I still think Char would have killed him. His motivations are so self-serving and unshakeable he would have just thought up a new reason.
Then there's the fact Char misses Garma's funeral when we know from Kycilia's comment he was expected to be there. Why, would it not make sense for Char to want to see the consequences of his actions in person? I think it's interesting that at Garma's funeral, something he should be happy about, he's instead drinking alone in a bar not looking particularly celebratory.
I do believe Char was being genuine when he called Garma a friend, though I don't think he cared about him in any way that mattered in the grand scheme of things. Of course he cares about Sayla a great deal more, but it feels similar to how he cares about her, just not enough to set aside his revenge plan to be with her.
I think the biggest scene that points towards Char actually caring about Garma is the infamous Kycilia headshot - "Garma, I'm sending your sister to join you." It's easy to read this as Char mocking Garma one last time, and I think that's an aspect to it, but it's also very telling that at this moment where his revenge is practically complete, he's thinking about Garma, not Degwin or his father. I think a part of him was genuinely saying a final goodbye to Garma here.
Also there's all the side media like Char's Daily Life and Gihren's Greed but those are super not canon so I'm not including them sorry this post is disjointed as hell