Small Learnings

In the last couple of days, we officially admitted the first two students to our new Masters in Machine Learning program!

I'm curious if NiceGUI could be useful for developing easy-to-use apps for our biologist collaborators. I'm not a front-end developer and definitely not a web developer. NiceGUI seems like it be more like writing a JavaFx or Swing GUI than a web app.

Just learned about Optaplanner. This seems like a super cool library for bringing optimization to a larger audience by making it easy to use.

The micro editor seems to be a great replacement for emacs. Sits somewhere between emacs and nano in terms of functionality.

Got Apache Spark reading data from PostgreSQL. Pretty neat!

Officially on break for the next two weeks. I hope everyone has great holidays!

Re-learning everything I forgot about Apache Hadoop and Apache Spark. Spark has definitely improved since I last used it seven years ago.

Recent class prep efforts have been focused on structured logging for services, log collection, and log storage using object stores

I have the D programming language running on my Apple Silicon machine. The LLVM-based D Compiler supports macOS/arm64. Note that I have to build everything in release mode ("-b release") to avoid a linking error. I'm looking forward to playing with D by implementing VLists, hash trees, and tree lists.

VLists are an interesting combination of arrays and linked lists. Each node contains an array of elements. When space is needed, a new array is added to the front of the linked list. To achieve O(log n) random access, each new array is double the size of the previous array. This same strategy is used in scalable bloom filters.

Spent yesterday preparing lectures on statistical regression models and survival analysis for my Winter data science class. Reviewed likelihood functions and ratio tests, Wald tests, and pseudo R squared. Learned about predicting hazard functions with survival trees and SVMs.

Today’s #RecommendationSystems lesson: only return the N items per user with the highest predicted ratings. Otherwise, you’ll generate a query users x all items dense matrix. Oops.

Predicting ratings with truncated SVD gives accurate rankings but not raw scores. Trying to figure out where I should be normalizing / scaling.

“Monoclonal antibody prevents malaria infection in African adults”. Exciting work on an important problem.