It is a quote from the Father Brown-Story "The Queer Feet" and in it the Gentleman thief Hercule Flambeau (his alias) steals precious cutlery at an exclusive dinner, by appearing as a guest to the waiters and as a waiter to the guests. He ultimately gives up the stolen goods, when he is confronted by Father Brown, who let's him escape and when asked about the identity of the thief, protects him.
Flambeau appears in 16 of the 53 short stories about the priest and throughout they have this understanding with each other and a way of conversing, that third parties do not have access to.
This is their second meeting btw:
He put one hand on the counter, vaulted over as easily as an acrobat and towered above the priest, putting one tremendous hand upon his collar.
“Stand still,” he said, in a hacking whisper. “I don’t want to threaten you, but—”
“I do want to threaten you,” said Father Brown, in a voice like a rolling drum, “I want to threaten you with the worm that dieth not, and the fire that is not quenched.”
“You’re a rum sort of cloak-room clerk,” said the other.
“I am a priest, Monsieur Flambeau,” said Brown, “and I am ready to hear your confession.”
The other stood gasping for a few moments, and then staggered back into a chair.
This is what he replies, when asked about the identity of the thief:
“I don’t know his real name,” said the priest placidly, “but I know something of his fighting weight, and a great deal about his spiritual difficulties. I formed the physical estimate when he was trying to throttle me, and the moral estimate when he repented.”
And this is a quote from their first meeting:
“Just hand over that sapphire cross of yours, will you? We’re all alone here, and I could pull you to pieces like a straw doll.”
This quote appears in the later story "The Sins of Prince Saradine", in which they go an a holiday together:
"And I for one don't know whether I'm in this world or the next, but I thank God you're in the same one."
I find their dynamic quite interesting because almost all meetings between these to, from this point onward, seem to happen on Flambeaus terms, who enjoys having found this refuge, where he doesn’t need a disguise and to which he can return, whenever he wants.
Their ship name is usually Flambrown and they have 96 works on AO3.
I'm sure there are many more examples of this extremely straight behavior between them, but I read these stories 2 years ago and haven't done much catching up since.