Mycroft : 'I took a job at the Foreign office in pursuit of an easier life. Wars, famine. That sort of thing,' he says with a pleased smile at his career
Moriarty subtly side eyeing him - like what did you just say? - because the Irish went through the Great Famine and it was catastrophic.
I don't believe Mycroft was trying to mock James or the suffering of his people, I think he just forgot that James is Irish. And THAT is a testament to Moriarty's character: he is a chameleon. Blending in. Making them believe he is one of them.
Though I think this scene also exposes something else:
Yes on one hand, it shows how James blends in - but at the same time, it reminds him that he does not belong.
That he does not belong in the society that disregards and normalizes the suffering of his people. The fact that it is so accepted, that it is the punchline of a joke - something turned banal.
Mycroft's comment, unintentionally/intentionally (I think this point is debatable), belittled what the Irish went through as well as all the suffering caused by the wars lead by the British Empire. So, when he said that, it reminded James that these people (the British), will never be his people, because the Great Famine was so horrible that no one in their right mind who went through it would joke at its expense.
It's so contrasted with the moment they were having a second ago: the one where they were discussing what Sherlock was like in his childhood. Where you felt like James was fitting into his family - essentially trying to bond with Mycroft.
A sweet moment turned sour in an instant.
A reminder that he is still an outsider, no matter what he does.