#social engineering

Oh, shit. I frantically rewound my conversation, trying to figure out if this could possibly be true. I hadn't given him anything apart from some very anodyne info, like what city I live in (which is in my Wikipedia entry), my date of birth (ditto), and the last four digits of my card.

Wait a sec.

He hadn't asked for the last four digits. He'd asked for the last seven digits. At the time, I'd found that very frustrating, but now – "The first nine digits are the same for every card you issue, right?" I asked the VP.

I'd given him my entire card number.

Goddammit.

The thing is, I know a lot about fraud. I'm writing an entire series of novels about this kind of scam:

https://us.macmillan.com/books/9781250865878/thebezzle

And most summers, I go to Defcon, and I always go to the "social engineering" competitions where an audience listens as a hacker in a soundproof booth cold-calls merchants (with the owner's permission) and tries to con whoever answers the phone into giving up important information.

But I'd been conned.

Now look, I knew I could be conned. I'd been conned before, 13 years ago, by a Twitter worm that successfully phished out of my password via DM:

https://locusmag.com/2010/05/cory-doctorow-persistence-pays-parasites/

That scam had required a miracle of timing. It started the day before, when I'd reset my phone to factory defaults and reinstalled all my apps. That same day, I'd published two big online features that a lot of people were talking about. The next morning, we were late getting out of the house, so by the time my wife and I dropped the kid at daycare and went to the coffee shop, it had a long line. Rather than wait in line with me, my wife sat down to read a newspaper, and so I pulled out my phone and found a Twitter DM from a friend asking "is this you?" with a URL.

Assuming this was something to do with those articles I'd published the day before, I clicked the link and got prompted for my Twitter login again. This had been happening all day because I'd done that mobile reinstall the day before and all my stored passwords had been wiped. I entered it but the page timed out. By that time, the coffees were ready. We sat and chatted for a bit, then went our own ways.

I was on my way to the office when I checked my phone again. I had a whole string of DMs from other friends. Each one read "is this you?" and had a URL.

Oh, shit, I'd been phished.

If I hadn't reinstalled my mobile OS the day before. If I hadn't published a pair of big articles the day before. If we hadn't been late getting out the door. If we had been a little more late getting out the door (so that I'd have seen the multiple DMs, which would have tipped me off).

There's a name for this in security circles: "Swiss-cheese security." Imagine multiple slices of Swiss cheese all stacked up, the holes in one slice blocked by the slice below it. All the slices move around and every now and again, a hole opens up that goes all the way through the stack. Zap!

The fraudster who tricked me out of my credit card number had Swiss cheese security on his side. Yes, he spoofed my bank's caller ID, but that wouldn't have been enough to fool me if I hadn't been on vacation, having just used a pair of dodgy ATMs, in a hurry and distracted. If the 737 Max disaster hadn't happened that day and I'd had more time at the gate, I'd have called my bank back. If my bank didn't use a slightly crappy outsource/out-of-hours fraud center that I'd already had sub-par experiences with. If, if, if.

The next Friday night, at 5:30PM, the fraudster called me back, pretending to be the bank's after-hours center. He told me my card had been compromised again. But: I hadn't removed my card from my wallet since I'd had it replaced. Also, it was half an hour after the bank closed for the long weekend, a very fraud-friendly time. And when I told him I'd call him back and asked for the after-hours fraud number, he got very threatening and warned me that because I'd now been notified about the fraud that any losses the bank suffered after I hung up the phone without completing the fraud protocol would be billed to me. I hung up on him. He called me back immediately. I hung up on him again and put my phone into do-not-disturb.

The following Tuesday, I called my bank and spoke to their head of risk-management. I went through everything I'd figured out about the fraudsters, and she told me that credit unions across America were being hit by this scam, by fraudsters who somehow knew CU customers' phone numbers and names, and which CU they banked at. This was key: my phone number is a reasonably well-kept secret. You can get it by spending money with Equifax or another nonconsensual doxing giant, but you can't just google it or get it at any of the free services. The fact that the fraudsters knew where I banked, knew my name, and had my phone number had really caused me to let down my guard.

The risk management person and I talked about how the credit union could mitigate this attack: for example, by better-training the after-hours card-loss staff to be on the alert for calls from people who had been contacted about supposed card fraud. We also went through the confusing phone-menu that had funneled me to the wrong department when I called in, and worked through alternate wording for the menu system that would be clearer (this is the best part about banking with a small CU – you can talk directly to the responsible person and have a productive discussion!). I even convinced her to buy a ticket to next summer's Defcon to attend the social engineering competitions.

There's a leak somewhere in the CU systems' supply chain. Maybe it's Zelle, or the small number of corresponding banks that CUs rely on for SWIFT transaction forwarding. Maybe it's even those after-hours fraud/card-loss centers. But all across the USA, CU customers are getting calls with spoofed caller IDs from fraudsters who know their registered phone numbers and where they bank.

I've been mulling this over for most of a month now, and one thing has really been eating at me: the way that AI is going to make this kind of problem much worse.

Not because AI is going to commit fraud, though.

One of the truest things I know about AI is: "we're nowhere near a place where bots can steal your job, we're certainly at the point where your boss can be suckered into firing you and replacing you with a bot that fails at doing your job":

https://pluralistic.net/2024/01/15/passive-income-brainworms/#four-hour-work-week

I trusted this fraudster specifically because I knew that the outsource, out-of-hours contractors my bank uses have crummy headsets, don't know how to pronounce my bank's name, and have long-ass, tedious, and pointless standardized questionnaires they run through when taking fraud reports. All of this created cover for the fraudster, whose plausibility was enhanced by the rough edges in his pitch - they didn't raise red flags.

As this kind of fraud reporting and fraud contacting is increasingly outsourced to AI, bank customers will be conditioned to dealing with semi-automated systems that make stupid mistakes, force you to repeat yourself, ask you questions they should already know the answers to, and so on. In other words, AI will groom bank customers to be phishing victims.

This is a mistake the finance sector keeps making. 15 years ago, Ben Laurie excoriated the UK banks for their "Verified By Visa" system, which validated credit card transactions by taking users to a third party site and requiring them to re-enter parts of their password there:

https://web.archive.org/web/20090331094020/http://www.links.org/?p=591

This is exactly how a phishing attack works. As Laurie pointed out, this was the banks training their customers to be phished.

I came close to getting phished again today, as it happens. I got back from Berlin on Friday and my suitcase was damaged in transit. I've been dealing with the airline, which means I've really been dealing with their third-party, outsource luggage-damage service. They have a terrible website, their emails are incoherent, and they officiously demand the same information over and over again.

This morning, I got a scam email asking me for more information to complete my damaged luggage claim. It was a terrible email, from a noreply@ email address, and it was vague, officious, and dishearteningly bureaucratic. For just a moment, my finger hovered over the phishing link, and then I looked a little closer.

On any other day, it wouldn't have had a chance. Today – right after I had my luggage wrecked, while I'm still jetlagged, and after days of dealing with my airline's terrible outsource partner – it almost worked.

So much fraud is a Swiss-cheese attack, and while companies can't close all the holes, they can stop creating new ones.

Meanwhile, I'll continue to post about it whenever I get scammed. I find the inner workings of scams to be fascinating, and it's also important to remind people that everyone is vulnerable sometimes, and scammers are willing to try endless variations until an attack lands at just the right place, at just the right time, in just the right way. If you think you can't get scammed, that makes you especially vulnerable:

https://pluralistic.net/2023/02/24/passive-income/#swiss-cheese-security

Image: Cryteria (modified) https://commons.wikimedia.org/wiki/File:HAL9000.svg

Hustlers are motivated by concrete personal advantage. Most commonly, and most straightforwardly, they want money - as much of it as they can get. They may also be interested in fame, idiosyncratic perks, etc. They do whatever they have to do in order to get what they want.

No surprise: you see huge preponderances of these guys in fields that provide outsize concrete rewards, e.g. finance, the upper echelons of management, etc. But not every natural-born Hustler is in a position to enter a glitzy high-paying field, and in fact you find Hustlers all throughout society and all throughout the economy, finding or making hustles wherever they go.

Having Hustlers working for you is mostly pretty great. They get shit done. They can be induced to work incredibly hard - probably harder than anyone else, under most circumstances - and they'll shank their own mothers if the price is right. If you need anything really important from them, anything at all, it's just a matter of bribing them enough.

...they will also, of course, cheerfully shank you if the price is right. Hustlers aren't the only wellsprings of institutional politics and infighting, but they're the most dangerous ones; they're always potential rivals to everyone around them. Also, you need to keep the tangible rewards flowing in a steady stream in order to get anything out of them, or else they'll put most of their effort into jumping ship (one way or another).

Craftsmen are motivated by the desire to do good work in their chosen fields, for its own sake and for the sake of their treasured self-image as people who do good work.

As you'd expect, for the most part, they're excellent workers and should be prized. But they're not perfect workers. Common weaknesses and downsides include:

They tend to have their own ideas about How Things Should Get Done; they're often resistant to externally-imposed product/service requirements or process changes (and bad at implementing those things) (no matter how important or well-conceived they are), and they're very resistant to "just get it out the door, right now done is better than good."

Being driven chiefly by internal motivation is great, but sometimes it's useful to be able to push things along with external motivators, and Craftsmen are pretty resistant to those. They don't like working more or harder than they're naturally inclined to work, they mostly sneer at carrots, and sticks make them sad and unproductive.

It's important to note that, while noteworthy skill within a field correlates with having a Craftsman temperament and motivation suite - for obvious reasons - those things are not identical at all. Plenty of Craftsmen are bad at their jobs, or just average, and plenty of the best workers are most motivated by things other than the Excellence of the Work Itself.

Fanatics are a relatively rare and specialized group, whom you find mostly within a few specific sorts of culturally-valorized fields. They're motivated by a desire to be part of something Important and Good in a Broader Sense: to Save the World, or some smaller-bore version of that.

They make amazing front-line soldiers, in the sorts of institutions that have "front-line soldiers." They work super hard, and you don't even need to bribe them, you just need to keep them hopped up on inspiration.

The big problem with them is that they're mostly motivated by a feeling - the feeling of Being Righteous - and it's not easy to control where they get that feeling, in any kind of precise way. They're just as resistant to external motivators as Craftsmen are, or even more so, but they're also not being guided by an ideal of effective quality. (No, not even if their chosen cause is theoretically all about an ideal of effective quality, hem hem.) They will happily waste vast amounts of time and money doing useless things, or even counterproductive things, so long as they're engaged in tasks that hit the right psychological buttons for them. There's also a constant risk that a Fanatic will decide that his employer is unrighteous, or that one of his coworkers is unrighteous, and start an internal conflict; the risk scales in a more-than-linear fashion with the number of Fanatics you keep around.

The biggest group, unsurprisingly, is the Normies. In most fields, it is much the biggest group. Normies are motivated by the desire to be members in good standing of their communities, to have positive relationships with the people around them, and to live up to basic norms and expectations.

Managerial skills, in the traditional sense, are incredibly important with Normies. If you want them to do good work for you - and you should want that, as a manager, you've almost certainly got a whole bunch of them - not only do you have to keep them pointed in the right direction, you have to make sure that they're supporting each other. With Hustlers, you just have to throw money at them (and avoid their power plays); with Craftsmen, you just have to let them do their thing, and occasionally badger them into giving you what you need; with Fanatics, you just have to be inspirational; but with Normies, you have to lead, and construct a productive community. You have to set reasonable, achievable norms and expectations that will get you what you need.

This wouldn't be complete if I didn't talk about the Defectors. The Defectors are motivated by not working. They don't want to be there, they resent having to do their jobs, and their primary goal is to shirk as much as possible. They will, by default, put much more effort into shirking than into their assigned tasks.

Obviously, managers don't want to have to deal with them, for good reason. But they're out there, in large numbers - not always in the places and fields where you'd expect to find them - and learning to manage them is sometimes more viable than trying to get rid of them. ("Moving Heaven and Earth to find them jobs that will change their attitude" is often a good plan, although of course it's not always possible and not always worth it.)

Crucially, Defectors are not Normies. If you start with the assumption that the average baseline worker is lazy and sour, you will make some incredibly stupid decisions. There are some fields where, for structural reasons, you can expect that a very large number of your workers will be Defectors; this is a huge and complicated challenge, well beyond the scope of this post, and good luck to you if you have to handle it, but it's not the default.

----------------------

Once you have those categories in your head, and can play with them, a number of obvious-seeming ideas present themselves. Just a couple, for now:

Most high-level executives are Hustlers, or have strong Hustler tendencies, for obvious reasons. Most of the people around them are Hustlers, or have strong Hustler tendencies. This means that they tend to overweight the Hustler outlook, by a lot, when they try to model what their workers are like. More specifically, I'd wager that a lot of them intuitively divide the world into "good workers" ( = Hustlers) and "bad workers" ( = Defectors). This will lead to a heavy overreliance on tangible rewards, a systematic shortchanging of community-building, etc. Which is in fact just what we see.

In particular - crucially - Hustlers and Defectors are the only worker types who ever become more productive under heavy stress. Hustlers actually benefit from it, because it raises the stakes of the game that they're already playing. (If you succeed, you'll be king of the world! If you fail, you'll be shark food! Go go go!) Defectors suffer terribly from stress, of course, but they can sometimes be spooked into doing their jobs as opposed to doing nothing, and sometimes that's the best/easiest way to get something out of them. But stress is terrible for everyone else. Craftsmen lose their focus. Fanatics lose their hope. It's worst of all for Normies, because they take all their cues from the vibes around them; they're productive when they learn to associate work with comfort and happiness, and when you fill their working world with frantic desperation, you just put them in a permanent cringe state.

stop trying to pit your Normies against each other in competitions for status and rewards dear God what are you stupid