RELEASE THE BATS!!!

However, I do think this case is great example of what happens when smart people decide they can use these systems “responsibly.”

Let’s put aside the plagiarism and the energy use for a second (not that these don’t matter, but they’re not relevant to my point at the moment) and pretend these systems were made with entirely properly obtained info and ran on only solar, but otherwise were entirely the same. If you’re a person, you might think “okay, I’m going to only use this to come up with ideas for my next paper: I’m going to do all the research myself and the writing, I’m just gonna brainstorm topics.”

The thing is, even if you’re trying to limit use to just “brainstorming” the chatbot isn’t going to limit itself to just brainstorming. You might ask it “what should I write about involving Greek history” and it may suggest “the plague of Athens” but it’s also going to suggest some “basic info” on the plague of Athens that you might end up picking up and deciding your topic is. It’s going to provide you with “research.” And even if you decide to check that research yourself, that’s still setting you down a specific path you might not have picked organically.

And speaking of double checking its “research”, fact checking is a time heavy process. Not only now do you have to read the original text the bot provided, but you also have to compare it line by line with the summary the bot provided. That’s more time than it would have taken to just read the article by itself. And with any time heavy task, people are going to get lazy. And in original research being lazy might mean skimming a paper or an abstract, chatbots allow you to be lazy without looking at the article at all.

And soon enough, despite telling yourself you were only going to use it for brainstorming, you have offloaded topic generation, research and eventually even drafting because that’s what these bots are intended to do. They are not meant to serve as research tools or help brainstorm or serve as a bouncing off board: they are built to encourage maximum engagement. And that means if you go at them with the best of ethical intentions, they are built to tempt you to use them as much as possible, regardless of whatever line you drew in the sand for yourself.

Before I could even process that, another officer dropped the big surprise: they’d actually been tracking me around town for days via Flock cameras. But they kept losing the trail, so when a camera alerted them that the Range Rover had been spotted turning into Kohl’s that morning, they quickly set up their ambush and waited for me and my wife to walk out of the store and get into the SUV.

[…]

One of the officers pulled out his phone, opened the Flock app, and showed me two photos: a wide shot of the Range Rover going through the intersection and a zoomed-in shot of the New Jersey license plate, which clearly reads 34 10 DTM and says VEHICLE MFR along the bottom. Critically, the number 10 is in a much smaller font than the rest of the tag, which is the non-standard structure New Jersey uses for manufacturer plates.

[…]

The New Jersey plates that were allegedly stolen from the LA dealer were 34 03 DTM, not 34 10 DTM. But when the police report was created and the plate was entered into Flock’s system, it was just recorded as 34 DTM. Just the five large characters, no little number in the middle. And Flock’s AI tech wasn’t registering that non-standard little number when it began picking up the Range Rover around town. It just saw 34 DTM in large type and started alerting the local police.

As we all stood there shaking our heads, including my wife, who was finally allowed to join me, I connected the final dot. A lot of vehicles in JLR’s media fleet have a New Jersey manufacturer plate with the same alphanumeric structure—34 ## DTM—and Officer Ganshyn observed that meant it was now a nationwide issue. Anywhere a police department has a partnership with Flock, any other JLR-owned car with the same plate structure is going to get flagged as stolen. In fact, four other 34 ## DTM cars were being tracked around Minnesota that week, according to Officer Ganshyn. I was just the first one to get nabbed. The only way to stop it would be for the LAPD to correct their initial report and update Flock’s system, which Jaguar Land Rover was now racing to make happen following the phone call.

Still, he warned me to drive straight home, park the Range Rover, and leave it there. If I were to cross into the neighboring town, I’d probably get flagged again and go through this entire ordeal again with a different set of officers. His parting words were ominous: “You’re lucky we’re in Plymouth. If you were in Minneapolis, they definitely would’ve come at you with guns drawn.”