I set out to write about the following paper I saw people talk about on twitter and reddit: Hao Li, Zheng Xu, Gavin Taylor, Tom Goldstein Visualizing the Loss Landscape of Neural Nets It's related to this pretty insightful paper: Laurent Dinh, Razvan Pascanu, Samy Bengio, Yoshua Bengio (2017) Sharp
2018/1/24 Today's paper:
Neural Attention Model for Text Summarization: From Facebook AI Research
2018/1/24 Feed Summary
inference
The Generalization Mystery: Sharp vs Flat Minima: Two papers analyze the loss landscape and using flatness as an indicator for generalization are present and also discussed in this post. They are analyzing sharp minima (flatness, in constrast to sharp minima, can be invariant under certain network architecture re-parameterization but still can not be only indicator for generalization) and visualizing loss landscpe (where the flatness can be visualized in 1D or 2D plot through re-parameterization; however, the author contends that this experiment only considers a small portion of reparameterizations and couldn't be conclusive). The author futher developed his own indicator of generalization using ratio of two generalization measurements. In fact, the author derived a "local measure of generalization ability" which considers the loss of two consecutive minibatches and divided by a hyper-paramter $\epsilon$ denoting a restricted region where next minibatch could be moved to (or the region within flatness). The author further proposed using KL divergence as $\epsilon$ measure instead of Euclidean norm to maintain true invariance. All of the assumption based on SGD due to small batch of SGD could yield better generalization. (RW: looks like computing statistics from Adaptive learning rate such as momentum methods)
KDNuggets
ML SaaS comparison: Compare Amazon (ML and SageMaker), Microsoft (Azure) and Google (Preidction API and ML Engine). This comparison is based on what common models can be trained in their cloud and how easy they can be used in terms of automation and parameters tuning (table available). Also the NLP API (speech and text) and Image recognition API provided online. Details included.
GA for hypertuning RNN: tutorial for how to construct GA (DEAP) to optimize RNN (keras)
Excel with Pandas: tutorial article where using pandas with a bundle of excel packages
H2O+R+DeepLearning: Introduce a "Flow" web-based frontend to use DL framework
Cognitive Computing:
evaluate conversational ai: paper
topic-based conversational bots evaluation: paper
About Alexa: podcast and TED talk
Written with StackEdit.
