#digital exam

Earlier this week, we covered the news that LSAC is testing out the administration of a digital LSAT. Today, let’s wildly speculate about what this means for the future of the LSAT. Specifically, could the LSAT of the future be a computer-adaptive test like the GMAT and GRE? LSAC has definitely studied the possibility.

Right now, the LSAT is one size fits all. Everyone who takes the test at one particular administration gets the same four scored sections, though the order of those sections varies, as does the content of the experimental section. LSAC could, in theory, offer multiple versions of the LSAT on a given test day, but that wouldn’t change much. As soon as your test booklet gets handed to you, the test you’re going to take that day is fully determined.

This means that the LSAT is longer than it needs to be. From the perspective of the test writers, the point of the LSAT is to figure out how well the test taker can employ the underlying skills. The test needs to have different levels of questions to account for test takers’ different levels of ability. The hardest questions are needed to distinguish the very highest scorers from the merely very good. But those really hard questions don’t tell you very much about lower scorers, who mostly just get them wrong. Conversely, the easy questions are needed to separate average test takers from weaker test takers, but don’t help distinguish between high scorers—the high scorers mostly just get them right.

What if high scorers could skip the easy questions, and low scorers could skip the pointless hard ones they’re likely to get wrong anyway? That’s the promise of a computer-adaptive test, or CAT. Rather than give everyone the same questions, a CAT, uh, adapts. If you get things right, it gives you harder questions to figure out just how good you are. If you have a rough start, the test gives you easier questions to find out what you actually can do. The advantage is that the test can be more accurate, shorter, or both.

This is how the GMAT works. The questions you get within each section aren’t predetermined. The disadvantage to the test taker is that you can’t skip around. Each question demands an answer before you get to see the next one. Once you’ve made your final answer, there’s no going back. But again, the advantage is that the test is shorter than it would otherwise need to be for the same level of precision.

The GRE is adaptive in a kinder, gentler, lesser way. The GRE gives each test taker two sections each of verbal and math. The first section of each is predetermined, and you’re allowed to skip around within a section. But your performance on the first section helps determine what questions you get on the second section. Think of it as a compromise, I guess, between a traditional test like the LSAT and a truer CAT like the GMAT.

If you’re a Blueprint student, your homework is adaptive, too. In this case the point isn’t to give you a score but rather to give you questions that challenge you without going over your head.

The sample digital LSAT that’s been announced doesn’t sound as if it’s adaptive — it has the same five-section format as a standard LSAT, and LSAC hasn’t said anything about an adaptive test. But I suspect that what they’re really testing out is the digital administration method. They’re using tablets connected to a central server, and that could work for a computer-adaptive test or for a non-adaptive computer-based test.

LSAC has definitely considered eventually moving to an adaptive test. How do I know? I took one for the team and dove into the deep, dark section of LSAC’s website where none dare venture: the research reports. Creating a valid and secure CAT presents some challenges, and LSAC has studied how to overcome those challenges.

Since a CAT requires a large pool of questions, it’s unrealistic to write new ones for every test day. That requires re-using questions. How do you detect cheating if people somehow get access to some questions? They’ve studied that. How do you come up with the best, most awesome and comprehensive pool of potential questions? They’ve studied that, too. How do you give test takers their questions in the best possible order? Ditto.

Any change to the LSAT probably won’t happen quickly. Even if the LSAT goes computer-based, it might not be adaptive, at least not at first. But there’s a good chance the future of the LSAT is computer-adaptive. After all, having an adaptive test is one of the best reasons to administer it on a computer. What do you think? Would the prospect of a computer-adaptive LSAT scare you, or would you love a shorter LSAT even if you couldn’t skip around between questions? Comment away!

It’s been rumored for years, the subject of hushed whispers in the shadows. Now, finally, those covert discussions are entering the light of day. That’s right—LSAC is officially making moves toward introducing a digital version of the LSAT.

To be clear, your official LSAT won’t be moving to a digital format any time soon. But LSAC has announced a pilot test of the digital version, which will be taking place at select locations on May 20. (Registration for the digital test is now closed, but it’s worth taking a look at the website just to see the terrible, cheesy stock photo they chose for the top of the page.)

So far, we know that the test will be administered on tablets that will be connected to a central computer, or—as LSAC apparently calls it, with their characteristic lack of irony—a “mother ship.” The questions will be heavily encrypted and will only be stored on the mother ship and the tablets during the test itself, so any would-be LSAT bandits will be out of luck if they attempt to steal the tablets for an early look at the test.

Beyond that pilot program, at this point it’s unclear how soon you might be able to take a digital version of the LSAT for an official test. LSAC is slow in just about everything they do—it takes them a month to score a Scantron, for chrissakes—so don’t hold your breath.

(In fact, in the law.com article, LSAC employee Troy Lowry is quoted as saying that LSAC has been pondering switching to a digital test for more than 20 years. This further cements my perception of LSAC as a group of Ents, ruminating on decisions for decades before arriving at a conclusion.)

Nevertheless, the transition could be big news for test-takers, as it would likely lead to faster score releases and—hopefully, eventually—to the LSAT being offered more than a few times per year, similar to the GRE’s current format. That said, there are benefits to the current analog system, such as easier annotation of questions and Reading Comprehension passages; I wonder if being able to make notes on a tablet will be as easy to refer back to.

We’ll be eagerly awaiting further news, and will report back when we hear more. In the meantime, let us know your thoughts about the prospect of a digital LSAT—are you eagerly anticipating the change, or does the thought of taking the LSAT on a tablet fill you with dread?