Wildcat and Panther Status Meeting
Spent a lot of the day catching up on things after a week away at training. Here's a basic rundown of the things to remember from the day, most of which sprung from a status meeting for the Wildcat and Panther projects.
The work on family history and results is on track per the schedule.
The work on symptoms is behind schedule. We had hoped to have results and error analysis done on that by now, but we don't have a gold standard yet, so nothing there. More on that later.
The documentation of the pipeline is mostly done. It's probably something that will live, though, so we decided I'd review it now and we could make sure it's aimed in the right direction.
Dev has added some (impressive) rules for family history, but we are suffering a data sparsity problem.
We assume that any mention of family history is for colorectal cancer. (Hoping the investigators don't care it doesn't distinguish polyps, though we have some ideas on how we may capture that, but, again, need more data.)
(I sent off some information on the family history that's available from structured data.)
We had a win on case-sensitivity using the Java Patterns library, reducing the size of the jape file nicely.
Some rules from the Pitt pipeline have been combined to make the same output (but still exist as separate rules in case we want to separate later). E.g., A bunch of stuff -> Lower abdominal symptoms.
Some rules from the Pitt pipeline have been separated. E.g., GI Bleeding -> Lower GI Bleeding and Bleeding
Where we don't have an example, the code has been built and is ready for examples to be put into the jape grammar. E.g., Acute bowel obstruction.
The biggest issue is with the lack of a gold standard. I kind of thought we'd have it by sometime last week when I was gone, but there hasn't been good communication/handoff, and so the reports got created on Thursday night and nothing happened with them Friday. Now we need to get them into a user-friendly database and then have abstractors review the the notes to create a gold standard.
Some work has been done on feature selection.
I suggested doing the post-processor before going down into the categories where we only have a few examples.
The post-processor takes the results of each binary classifier and puts it through a filter, where it keeps at most the five "most important" positive classifications, where the "most important" list is compiled by humans.
I pointed out this could be problematic if we let a more important dx with a confidence score of, eg, 51% go through and not a less important dx with a confidence score of, eg, 90%. So that's food for thought.
