Discord is probably still dependent on PDNA (a valuable industry standard LLM tool) for its “first pass”look at newly uploaded content. When I worked there, it was the daily job of one person (me btw just me it was only me) to look through what PDNA flagged as potentially CSAM, and to then tell it yes/no; if I said “yes” the account that uploaded it would be immediately locked, and the image or video reported to the authorities. PDNA is an image hash matching system, which essentially means it views media in a grid pattern and looks for things it recognizes within the sections of that grid. As such, it was very common to see innocent grid patterns, screenshots of video game inventories, and checkerboard style images to show up in the matching. Now, an image just showing up in the matching in no way means (meant?) that it was CSAM, or suspected to be CSAM, just that something in the grid triggered PDNA’s hash match. My job, then, was to confirm if it was correct.
So what I suspect is happening now is one of two options:
-They have an AI agent doing the “yes/no” decision making (a very bad idea,) or, worse:
-They are skipping the confirmation stage and simply auto banning anyone who uploads anything that PDNA flags.
Again, PDNA is not a decision-making system. It just makes matches to a huge database of known CSAM, and creates a page (or 10) of recently uploaded Discord media that it thinks may match the content it knows about. It is not expected to be 100% accurate, and never could be. I highly suspect that Discord is trying to just automate this process by eliminating any step between “PDNA flagged this” and “it’s definitely CSAM.” Disappointing, but not shocking, frankly. I’d advise anyone who’s caught in this ban to immediately appeal.
areudirty
origamiaround
theartofmadeline
