Filthy, Defiler, Disgrace, and all that's right

The very first time he says it, in Steins;Gate, is after he accepts the idea of moving to the Beta worldline, when he promises to not forget Kurisu. "I wont forget the person I care for most."

Every time he brings it up in 0, however, is as an excuse. When Suzuha asks him about trying to reach the Steins;Gate worldline, he talks about how he had to sacrifice the person he cares for most. He constantly talks about that sacrifice whenever someone challenges his decision to remain on the Beta worldline, such as when he finds out about the time leap machine. he berates both Maho and Daru over the fact that they'd be throwing away Kurisu's sacrifice, making it worthless, and he focuses in particular on the fact that Maho would be killing Mayuri to save Kurisu. (he's evidently projecting there but I digress.) when he later talks to Mayuri about how "everything is fine" he again brings up the sacrifice, and that it was Kurisu's choice, and how desperately he wants to protect the weight of that sacrifice.