#typicalacademic

Here are some of the things that have frustrated me:

Weird assumption that you’re doing everything on the command line instead of in code.  This means that if I want to programmatically control something like training, I either have to circuitously write python that orders commands that start other python sessions, or go into `allennlp.command` and deal with code that seem written “to do the command” and not for my direct use

Relatedly, not set up well for hyperparam tuning, or anything else where you’re trying out different things and doing many training jobs.  Training code is coupled to serialization code in a way unlike anything I’ve seen elsewhere: every single training job must have its own directory, and then it thinks you want a snapshot of the model weights at every epoch and a separate copy of the weights from the best-val epoch and a third copy of those weights inside the final model at the end.

I remember the abstraction with “Fields” for model inputs being constantly frustrating.  I never understood its motivation (I assume there is one) but it puts this weird and complex barrier between the data you have and the actual input your model receives.  Generally I know exactly what it is I want my model to be seeing on an array-by-array level, but if I want my model to see “X” I can’t just say “X,” I have to find the appropriate “Field” and the appropriate Y such that passing Y to the Field returns X.  I remember this causing bugs in things that are trivial anywhere else, like keeping track of your label encoding, adding in “side info” aligned with your tokens, etc.

For me the Field stuff was a obnoxious headache in particular with their BERT implementation: it treats BERT’s tokenizer as a second tokenizer on top of “your model’s” tokenizer, with BERT’s confusingly nested inside a so-called “token indexer,” and at the same pipeline step some arrays will be aligned with what BERT actually sees and others with the “””real””” tokens (see here).

Defines its own abstractions even for things that are quite common and not really NLP-related.  I never really understood what a Predictor and an Archive were for (I think I half understood what a Model was for), and I haven’t really used PyTorch otherwise so maybe it lacks these very basic features? but it limited my range of options when I needed help because I was always dealing with the special AllenNLP trademarked flavor of a thing