#security through obscurity

It’s been a minute since Linkletter’s case arose, so I’ll give you a little recap here. Proctorio is a massive, wildly profitable ed-tech company that sells a surveillance tool to monitor students while they take high-stakes tests from home. The tool monitors the student’s computer and the student’s face, especially their eye-movements. It also allows instructors and other personnel to watch the students and even take control of their computer. This is called “remote invigilation.”

This is ghastly in just about every way. For starters, Proctorio’s facial monitoring software embeds the usual racist problems with machine-learning stuff, and struggles to recognize Black and brown faces. Black children sitting exams under Proctorio’s gimlet eye have reported that the only way to satisfy Proctorio’s digital phrenology system is to work with multiple high-powered lights shining directly in their faces.

A Proctorio session typically begins with a student being forced to pan a webcam around their test-taking room. During lockdown, this meant that students who shared a room — for example, with a parent who worked night-shifts — would have to invade their family’s privacy, and might be disqualified because they couldn’t afford a place large enough to have private room in which to take their tests.

Proctorio’s tools also punish students for engaging in normal test-taking activity. Do you stare off into space when you’re trying through a problem? Bzzzt. Do you read questions aloud to yourself under your breath when you’re trying to understand their meanings? Bzzzt. Do you have IBS and need to go to the toilet? Bzzzt. The canon of remote invigilation horror stories is filled with accounts of students being forced to defecate themselves, or vomit down their shirts without turning their heads (because looking away is an automatically flagged offense).

The tragedy is that all of this is in service to the pedagogically bankrupt practice of high-stakes testing. Few pedagogists believe that the kind of exam that Proctorio seeks to recreate in students’ homes has real assessment merit. As the old saying goes, “Tests measure your ability to take tests.” But Proctorio doesn’t even measure your ability to take a test — it measures your ability to take a test with three bright lights shining directly on your face. Or while you are covered in your own feces and vomit. While you stare rigidly at a screen. While your tired mother who just worked 16 hours in a covid ward stands outside the door to your apartment.

The lockdown could have been an opportunity to improve educational assessment. There is a rich panoply of techniques that educators can adopt that deliver a far better picture of students’ learning, and work well for remote as well as in-person education. Instead, companies like Proctorio made vast fortunes, most of it from publicly funded institutions, by encouraging a worse-than-useless, discriminatory practice:

https://pluralistic.net/2021/06/24/proctor-ology/#miseducation

Proctorio clearly knows that its racket is brittle. Like any disaster profiteer, Proctorio will struggle to survive after the crisis passes and we awaken from our collective nightmare and ask ourselves why we were stampeded into using its terrible products. The company went to war against its critics.

In 2020, Proctorio CEO Mike Olsen doxed a child who complained about his company’s software in a Reddit forum:

https://pluralistic.net/2020/07/01/bossware/#moral-exemplar

In 2021, the reviews for Proctorio’s Chrome plugin all mysteriously vanished. Needless to say, these reviews — from students forced to use Proctorio’s spyware — were brutal:

https://pluralistic.net/2021/09/04/hypervigilance/#radical-transparency

Proctorio claims that it protects “educational integrity,” but its actions suggest a company far more concerned about the integrity of its own profits:

https://pluralistic.net/2022/02/16/unauthorized-paper/#cheating-anticheat

One of the critics that Proctorio attacked is Ian Linkletter. In 2020, Linkletter was a Learning Technology Specialist at UBC’s Faculty of Education. His job was to assess and support ed-tech tools, including Proctorio. In the course of that work, Linkletter reviewed Proctorio’s training material for educators, which are a bonanza of mask-off materials that are palpably contemptuous of students, who are presumed to be cheaters.

At the time, a debate over remote invigilation tools was raging through Canadian education circles, with students, teachers and parents fiercely arguing the merits and downsides of making surveillance the linchpin of assessment. Linkletter waded into this debate, tweeting a series of sharp criticisms of Proctorio. In these tweets, Linkletter linked to Proctorio’s unlisted, but publicly available, Youtube videos.

A note of explanation: Youtube videos can be flagged as “unlisted,” which means they don’t show up in searches. They can also be flagged as “private,” which means you have to be on a list of authorized users to see them. Proctorio made its training videos unlisted, but they weren’t private — they were visible to anyone who had a link to them.

Proctorio sued Linkletter for this. They argued that he had breached a duty of confidentiality, and that linking to these videos was a copyright violation:

https://pluralistic.net/2020/10/17/proctorio-v-linkletter/#proctorio

This is a classic SLAPP — a “strategic litigation against public participation.” That’s when a deep-pocketed, thin-skinned bully, like Proctorio, uses the threat of a long court battle to force their critics into silence. They know they can’t win their case, but that’s not the victory they’re seeking. They don’t want to win the case, they want to win the argument, by silencing a critic who would otherwise be bankrupted by legal fees.

Getting SLAPPed is no fun. I’ve been there. Just this year, a billionaire financier tried to force me into silence by threatening me with a lawsuit. Thankfully, Ken “Popehat” White was on the case, and he reminded this billionaire’s counsel that California has a strong anti-SLAPP law, and if Ken had to defend me in court, he could get a fortune in fees from the bully after he prevailed:

https://twitter.com/doctorow/status/1531684572479377409

British Columbia also has an anti-SLAPP law, but unlike California’s anti-SLAPP, the law is relatively new and untested. Still, Proctorio’s suit against Linkletter was such an obvious SLAPP that for many of us, it seemed likely that Linkletter would be able to defend himself from this American bully and its attempt to use Canada’s courts to silence a Canadian educator.

For Linkletter to use BC’s anti-SLAPP law, he would have to prove that he was weighing in on a matter of public interest, and that Proctorio’s copyright and confidentiality claims were nonsense, unlikely to prevail on their merits. If he could do that, he’d be able to get the case thrown out, without having to go through a lengthy, brutally expensive trial.

Incredibly, though, the lower court found against Linkletter. Naturally, Linkletter appealed. His “factotum” is a crystal clear document that sets out the serious errors of law and fact the lower court made:

https://drive.google.com/file/d/1aB1ztWDFr3MU6BsAMt6rWXOiXJ8sT3MY/view

But yesterday, the Court of Appeal upheld the lower court, repeating all of these gross errors and finding for Proctorio:

https://www.bccourts.ca/jdb-txt/ca/23/01/2023BCCA0160.htm

This judgment is grotesque. It makes a mockery of BC’s anti-SLAPP statute, to say nothing of Canadian copyright and confidentiality law. For starters, it finds that publishing a link can be a “performance” of a copyrighted work, which meant that when Linkletter linked to the world-viewable Youtube files that Proctorio had posted, he infringed on copyright.

This is a perverse, even surreal take on copyright. The court rejects Linkletter’s argument that even Youtube’s terms of service warned Proctorio that publishing world-viewable material on its site constituted permission for people to link to and watch that material.

But what about “fair dealing” (similar to fair use)? Linkletter argued that linking to a video that shows that Proctorio’s assurances to parents and students about its products’ benign nature were contradicted by the way it talked to educators was fair dealing. Fair dealing is a broad suite of limitations and exceptions to copyright for the purposes of commentary, criticism, study, satire, etc.

So even if linking is a copyright infringement (ugh, seriously?!), surely it’s fair dealing in this case. Proctorio was selling millions of dollars in software to public institutions, inflicting it on kids whose parents weren’t getting the whole story. Linkletter used Proctorio’s own words to rebut its assurances. What could be more fair dealing than that?

Not so fast, the appeals panel says: they say that Linkletter could have made his case just as well without linking to Proctorio’s materials. This is…bad. I mean, it’s also wrong, but it’s very bad, too. It’s wrong because an argument about what a company intends necessarily has to draw upon the company’s own statements. It’s absurd to say that Linkletter’s point would have been made equally well if he said “I disbelieve Proctorio’s public assurances because I’ve seen seekrit documents” as it was when he was able to link to those documents so that people could see them for themselves.

But it’s bad because it rips the heart out of the fair dealing exception for criticism. Publishing a link to a copyrighted work is the most minimal way to quote from it in a debate — Linkletter literally didn’t reproduce a single word, not a single letter, from Proctorio’s copyrighted works. If the court says, “Sure, you can quote from a work to criticize it, but only so much as you need to make your argument,” and then says, “But also, simply referencing a work without quoting it at all is taking too much,” then what reasonable person would ever try to rely on a fair dealing exemption for criticism?

Then there’s the confidentiality claim: in his submissions to the lower court and the appeals court, Linkletter pointed out that the “confidential” materials he’d linked to were available in many places online, and could be easily located with a Google search. Proctorio had uploaded these “confidential” materials to many sites — without flagging them as “unlisted” or “private.”

What’s more, the videos that Linkletter linked to were in found a “Help Center” that didn’t even have a terms-of-service condition that required confidentiality. How on Earth can materials that are publicly available all over the web be “confidential?”

Here, the court takes yet another bizarre turn in logic. They find that because a member of the public would have to “gather” the videos from “many sources,” that the collection of links was confidential, even if none of the links in the collection were confidential. Again, this is both wrong and bad.

Every investigator, every journalist, every critic, starts by looking in different places for information that can be combined to paint a coherent picture of what’s going on. This is the heart of “open source intelligence,” combing different sources for data points that shed light on one another.

The idea that “gathering” public information can breach confidentiality strikes directly at all investigative activity. Every day, every newspaper and news broadcast in Canada engages in this conduct. The appeals court has put them all in jeopardy with this terrible finding.

Finally, there’s the question of Proctorio’s security. Proctorio argued that by publishing links to its educator materials, Linkletter weakened the security of its products. That is, they claim that if students know how the invigilation tool works, it stops working. This is the very definition of “security through obscurity,” and it’s a practice that every serious infosec professional rejects. If Proctorio is telling the truth when it says that describing how its products work makes them stop working, then they make bad products that no one should pay money for.

The court absolutely flubs this one, too, accepting the claim of security through obscurity at face value. That’s a finding that flies in the face of all security research.

So what happens now? Well, Linkletter has lost his SLAPP claim, so nominally the case can proceed. Linkletter could appeal his case to Canada’s Supreme Court (about 7% of Supreme Court appeals of BC appeals court judgments get heard). Or Proctorio could drop the case. Or it could go to a full trial, where these outlandish ideas about copyright, confidentiality and information security would get a thorough — and blisteringly expensive — examination.

In Linkletter’s statement, he remains defiant and unwilling to give in to bullying, but says he’ll have to “carefully consider” his next step. That’s fair enough: there’s a lot on the line here:

https://linkletter.opened.ca/stand-against-proctorios-slapp-update-30/

Linkletter answers his supporters’ questions about how they can help with some excellent advice: “What I ask is for you to do what you can to protect students. Academic surveillance technology companies would like nothing more but for us all to shut up. Don’t let them silence you. Don’t let anyone or anything take away your human right to freedom of expression.”

Today (Apr 21), I’m speaking in Chicago at the Stigler Center’s Antitrust and Competition Conference. This weekend (Apr 22/23), I’m at the LA Times Festival of Books.

[Image ID: A girl working on a laptop. Her mouth has been taped shut. Glaring out of the laptop screen is the hostile red eye of HAL9000 from '2001: A Space Odyssey.' Behind them is a tattered, filthy, burned Canadian flag.]

Image: Ingo Bernhardt https://www.flickr.com/photos/spree2010/4930763550/

CC BY 2.0 https://creativecommons.org/licenses/by/2.0/

Cryteria (modified) https://commons.wikimedia.org/wiki/File:HAL9000.svg

CC BY 3.0 https://creativecommons.org/licenses/by/3.0/deed.en

Eleanor Vladinsky (modified) https://commons.wikimedia.org/wiki/File:Canadian_flag_against_grey_sky.jpg

Hayward balked because he knew that among the 1.3 billion people who use Messenger, some would be placed in harm’s way if Facebook deliberately drained their batteries — physically stranded, unable to communicate with loved ones experiencing emergencies, or locked out of their identification, payment method, and all the other functions filled by mobile phones.

As Hayward told Kathianne Boniello at the New York Post, “Any data scientist worth his or her salt will know, ‘Don’t hurt people…’ I refused to do this test. It turns out if you tell your boss, ‘No, that’s illegal,’ it doesn’t go over very well.”

Negative testing is standard practice at Facebook, and Hayward was given a document called “How to run thoughtful negative tests” regarding which he said, “I have never seen a more horrible document in my career.”

We don’t know much else, because Hayward’s employment contract included a non-negotiable binding arbitration waiver, which means that he surrendered his right to seek legal redress from his former employer. Instead, his claim will be heard by an arbitrator — that is, a fake corporate judge who is paid by Facebook to decide if Facebook was wrong. Even if he finds in Hayward’s favor — something that arbitrators do far less frequently than real judges do — the judgment, and all the information that led up to it, will be confidential, meaning we won’t get to find out more:

https://pluralistic.net/2022/06/12/hot-coffee/#mcgeico

One significant element of this story is that the malicious code was inserted into Facebook’s app. Apps, we’re told, are more secure than real software. Under the “curated computing” model, you forfeit your right to decide what programs run on your devices, and the manufacturer keeps you safe. But in practice, apps are just software, only worse:

https://pluralistic.net/2022/06/23/peek-a-boo/#attack-helicopter-parenting

Apps are part what Bruce Schneier calls “feudal security.” In this model, we defend ourselves against the bandits who roam the internet by moving into a warlord’s fortress. So long as we do what the warlord tells us to do, his hired mercenaries will keep us safe from the bandits:

https://locusmag.com/2021/01/cory-doctorow-neofeudalism-and-the-digital-manor/

But in practice, the mercenaries aren’t all that good at their jobs. They let all kinds of badware into the fortress, like the “pig butchering” apps that snuck into the two major mobile app stores:

https://arstechnica.com/information-technology/2023/02/pig-butchering-scam-apps-sneak-into-apples-app-store-and-google-play/

It’s not merely that the app stores’ masters make mistakes — it’s that when they screw up, we have no recourse. You can’t switch to an app store that pays closer attention, or that lets you install low-level software that monitors and overrides the apps you download.

Indeed, Apple’s Developer Agreement bans apps that violate other services’ terms of service, and they’ve blocked apps like OG App that block Facebook’s surveillance and other enshittification measures, siding with Facebook against Apple device owners who assert the right to control how they interact with the company:

https://pluralistic.net/2022/12/10/e2e/#the-censors-pen

When a company insists that you must be rendered helpless as a condition of protecting you, it sets itself up for ghastly failures. Apple’s decision to prevent every one of its Chinese users from overriding its decisions led inevitably and foreseeably to the Chinese government ordering Apple to spy on those users:

https://pluralistic.net/2022/11/11/foreseeable-consequences/#airdropped

Apple isn’t shy about thwarting Facebook’s business plans, but Apple uses that power selectively — they blocked Facebook from spying on Iphone users (yay!) and Apple covertly spied on its customers in exactly the same way as Facebook, for exactly the same purpose, and lied about it:

https://pluralistic.net/2022/11/14/luxury-surveillance/#liar-liar

The ultimately, irresolvable problem of Feudal Security is that the warlord’s mercenaries will protect you against anyone — except the warlord who pays them. When Apple or Google or Facebook decides to attack its users, the company’s security experts will bend their efforts to preventing those users from defending themselves, turning the fortress into a prison:

https://pluralistic.net/2022/10/20/benevolent-dictators/#felony-contempt-of-business-model

Feudal security leaves us at the mercy of giant corporations — fallible and just as vulnerable to temptation as any of us. Both binding arbitration and feudal security assume that the benevolent dictator will always be benevolent, and never make a mistake. Time and again, these assumptions are proven to be nonsense.

Image: Anthony Quintano (modified) https://commons.wikimedia.org/wiki/File:Mark_Zuckerberg_F8_2018_Keynote_%2841118890174%29.jpg

CC BY 2.0: https://creativecommons.org/licenses/by/2.0/deed.en

[Image ID: A painting depicting the Roman sacking of Jerusalem. The Roman leader's head has been replaced with Mark Zuckerberg's head. The wall has Apple's 'Think Different' wordmark and an Ios 'low battery' icon.]

Next week (Feb 8-17), I'll be in Australia, touring my book *Chokepoint Capitalism* with my co-author, Rebecca Giblin. We'll be in Brisbane on Feb 8, and then we're doing a remote event for NZ on Feb 9. Next is Melbourne, Sydney and Canberra. I hope to see you!