Kazuma is a special character. While the others in Noragami are written in such a way that the reader's sympathy grows as we get to know them better, with Kazuma it's the exact opposite. Kazuma stands on a pedestal. We love him.
He is devoted to his goddess, while protecting Yato from her wrath for centuries, becoming a mentor to Yukine. If his image has any cracks, they are easily explained, or small and hidden behind a joke (however, they are there and are a foreshadowing of his later actions).
But Kazuma is not perfect, he is a human being. A 19-year-old boy painfully in love, full of emotions and making terribly bad decisions. This is something we do not see, through his flawless uniform and meticulously fitted glasses. The moment Bishamon is threatened, when he has to kill for her, something breaks in Kazuma's image, in the way we see him and in the way he sees himself.
This perfect, exemplary young man is seemingly suddenly capable of actions that are indefensible, reckless, to others and to himself. He is capable of crossing all boundaries and although he knows, because Kazuma is inherently a good person, that his actions are wrong, he still does them in the name of the end that justifies the means.
Watching the way his character gradually crumbles as he is forced to violate his own moral rules more and more is not a pleasant sight, but it is one of the best depictions of a morally gray character you can get.