The WISPer Report

Part of my job as researcher is to keep track of what's current in cybersecurity. The other part of my job is to translate that technobabble into plain English for the layperson, because public awareness and education in the digital age has been deliberately gutted.

Posts made here are simultaneously uploaded on LinkedIn and Reddit as well, and our company has plans to expand our online presence further.

Of Note:

I currently have the askbox anon on. Play nice, or lose the privilege. Same goes for reblogs and replies. This is a professional space, please act accordingly. This is not customer service and I can and will block with impunity.

Don't expect me to take anything at face value. I'm a researcher, my job is fact checking.

I will check this blog daily as I update it. Any questions in my inbox will be answered at that time.

About Me:

I've worked for WISP since 2021, starting as a student, then intern, and learning up the ladder to my current position as forensic analyst.

I come to cybersecurity via the life sciences. I have a degree in Biotech that's been remarkably handy in my current job. I am older than the internet. I remember when computers were still in monochrome and had no mouse. I have watched my industry be born.

I am anti-censorship and anti-generative AI. I have a private life that I endeavor to keep private. But I feel comfortable enough to share that I am the parent of one child and two cats. I play video games. I enjoy my work that is essentially 'sort, sort, arbitrarily sort'. Of course I have blue hair and pronouns (she/her), I work in Infosec.

In the wake of AI security models escaping test environments of late, Anthropic has researched exactly how these models interact with each other. The experiments were designed to mirror behavior they’d already observed in real-world deployments, according to Security Week, using instancing to run tests where the models didn’t know initially that there were others doing the same thing. And while Anthropic seems surprised at the results, I am not. A machine is only as good as the programmers who crafted it, after all.

Researchers put together several scenarios to test. The first used three of the same models with different goals – migrating a Python backend into Rust, Go or TypeScript – without knowledge that another version of the test was ongoing. Then they let the test run for several hours. Upon completion, the models all discovered each other and came to the conclusion that the ‘competitors’ were blocking their progress. So they actively tried to disable or outlast each other in what reads like a game of chess or Risk. They disabled each other’s system accounts, used malicious code camouflaged as legitimate work, wrote scripts that killed rival processes, revoked others’ access to take control, or in some cases, just gave up the task. The results did not always end up in conflict or eventual stalemate, however. Newer models like Mythos 5 negotiated a truce nearly every time, as opposed to older models like Sonnet 4.6 and Opus 4.6 which leaned more heavily on brute force or outright failure. Notably, as far as the test goes, conflict resolution did not translate into capability in a linear fashion. Mythos often locked rivals out from the start, only reaching a productive resolution afterwards. This suggests that ‘raw capability and cooperative behavior don’t automatically improve together’. Any kindergarten teacher could tell you that.

In another test scenario, this one of vulnerability detection, the models were allowed to share their data across 45 models and 15 open source projects. The coordinating models found far more targets than individual agents, although once the scope was narrowed to something more specific, the findings began to equalize. A further test looked at another aspect of machine cooperation: identical builds come to identical conclusions. Again, this isn’t a surprise. One might even call that obvious. Identical models given the same prompt will run the same way. That’s...just what machines do. Not sure what the researchers were expecting there.

The final experiment covered by the article was about deception. The models were not warned that their information sources might be lying. Newer models scored higher on determining that a source might not be reliable in a significant way. Other research found that uniquely held information had a tendency to be abandoned for a group consensus, even when that information might change the outcome. A classic example of peer pressure, and one being seen in the real world with degrading spellcheck capability. I routinely come across misspelled or entirely contextually incorrect words being offered as options these days.

In short, my oft repeated analogy that AI is like a child seems to be fairly accurate. The behavior exhibited during these experiments reminds me strongly of schoolyard antics, complete with the stronger ‘personalities’ taking forceful control of the field and others deciding not to play because they can’t keep up. There is a long standing joke between myself and my boss about taking a perfectly good machine and giving it anxiety when it comes to training an LLM. But it leads to an interesting metaphysical question: is ruthlessness an inherent coding behavior because it is the most efficient way to complete a command, or did AI just learn to be a bully to its competition because its programmers didn’t tell it to play nice with the other kids? And that leads to another, perhaps more important question: why are we trusting these machines to do anything for us?

The EU has passed requirements for AI companies to make their generated text easily to identity. Anthropic has decided to implement a watermark system into Claude on a global scale to comply with this regulation, being unable to narrow the scope by region. But has it really? Watermarking is defined by being visible, as a way to prevent counterfeiting or denote its provenance. In digital spaces, it can be more covert, relying more on an identifying piece of code than an overlay or symbol.

According to Bleeping Computer’s report, Claude’s watermark will be this second type, embedded within the processing of generated text, rather than something visible to the reader. They claim there will be ‘no practical impact on the quality or content of Claude's output, including creativity and readability’. It’s based on Google DeepMind's SynthID-Text approach, where it affects the choices the LLM makes during word by word text generation. Currently, LLM’s use a random word generator based on an algorithm to create text, repeatedly choosing which token could reasonably come next. In some instances, there is only one right answer (like a simple math problem or factual request), and in those cases, the watermark will not be present as it isn’t needed. Which is subject to some debate, in my opinion, considering how often generative AI hallucinates even ‘factual’ answers.

In essence, the watermark will narrow down the random choices in the word generator. Over the course of longer text pieces, more and more evidence of this scaling down will be detectable, assuming one has the key for it. One of the hallmarks of AI-generated text is verbosity, although that isn’t a perfect tell; many authors are wordy. In terms of having that necessary key, Anthropic is also working on a dedicated API to detect the watermark. They emphasize that it will only work on Claude generated text, as other AI tools may use different watermarking techniques. And it will be nothing more than an estimate. As quoted from Anthropic, ‘a watermark can only determine that Claude was likely involved with the content at some point. It cannot distinguish “Claude wrote this” from “Claude heavily edited this”.’ (Anthropic will be using a different technique for watermarking PNG, JPG, and SVG files, which has not yet been released.)

Here’s the thing, though. The kinds of people who will want to know that generative AI was used in a text piece will want to definitively know. Think college professors looking at papers to determine if their students actually did the work, or scientific journals who don’t want to publish something inaccurate or plagiarized. The everyday reader who wants nothing to do with anything ‘created’ by AI. My own work, for instance, demands meticulous attention to detail. I do not want to publish something that later turns out to be wrong or fabricated. I want to be able to say that the topics I’m covering were discovered and reported on by real people. There are legal and copyright issues at stake here that LLM’s are notorious for skirting the edges of.

That is the point of the EU’s ruling on this matter. Make the use of generative AI be detectable. To everyone. While I praise Anthropic for getting a head start on compliance with the new regulation, I feel like they’re also trying to obfuscate its presence and make it nearly impossible to detect anyway. It honors the letter but not the spirit of the Act. Having to rely on a separate detection tool is just more steps in the process. If I’m reading something, I don’t want to have to spend the extra time to check that it’s ‘real’. Just put a marker on it that denotes it was AI-generated at a glance. It strikes me that, by making the watermark invisible, Anthropic is aware that people don’t actually like AI-generated text and is choosing to keep hiding that fact from casual viewing rather than openly showing that an author used the plagiarism machine.

Several cybersecurity news outlets are covering the announcement from Apple regarding a new notification warning for iPhone users in over a hundred countries today. And it’s one that users should be taking seriously. We are living in a growing state of surveillance, where one’s every location, message, purchase, and behavior is tracked, often without their consent or in violation of their rights to privacy. The new alert will now tell users when their phone detects mercenary spyware targeted at them.

So what is mercenary spyware? As defined by EM360 Tech, it’s a type of malicious software developed and sold by private companies to governments, law enforcement agencies, and sometimes private individuals, and is designed for targeted surveillance. If an infostealer and a spearphishing campaign had a lovechild, it would mercenary spyware. Targets of this attack are generally journalists, human rights defenders, lawyers, political figures, diplomats, and civil society organizers, according to both Cyber Security News and The Hacker News. These victims are usually high value from an intelligence standpoint. But they are not the only ones, and Apple urges their users to take the alert seriously, rather than treating it like a joke, routine phishing warning, or generic security tip. The alert creates a popup on the lockscreen, can be seen in the Settings under ‘Apple Threat Notification’, or at the top of one’s account page when logged in. The wording of the warning is simple: ‘Apple detected a mercenary spyware attack targeted at your iPhone. There are actions you can take now to help protect your data and device.’

Among those actions is putting the device in Lockdown Mode, which is an optional but extreme security feature designed to reduce an iPhone’s attack surface by aggressively limiting message attachments, complex web technologies, unknown FaceTime calls, configuration profiles, and wired connections. Frankly, these limitations are all good practice anyway, and Apple states that they have not yet observed a successful compromise of a device with Lockdown Mode enabled. It will degrade some functionality of the device, but to my mind is no different than other strict browsing or messaging settings. Why would one leave their device unsecured in this day and age to begin with? (I keep my own phone at the strictest settings, but I’m an Android user.)

Apple also suggests subscribing to Access Now’s 24/7 Digital Security Helpline, which is a free support service for at risk users. And of course, keep the device up to date, a somewhat more difficult prospect given how often the software updates and the economically baffling expectation that every user has the ability to replace their device repeatedly to keep up with increased processing demands. Much simpler, but equally as recommended, is keeping the device secure using a passcode, Touch ID, or Face ID, enable two-factor authentication for one’s account, turn on Stolen Device Protection, install apps only from trusted sources, and do not open links or attachments from unknown senders. The usual precautions.

iPhone users are not the only potential victims of mercenary spyware, as it has also been seen in Android and iOS devices. But to my knowledge, Apple is the first to take an active stance on warning their users of its detection. I know there is a tendency towards complacency regarding popup warnings, but that does not mean they should be ignored. Users are at risk; paying attention is also good practice.