#topic model

These two are related but not comparable. LDA’s intent isn’t to identify the hidden topics out of corpora (plural of corpus), while word2vec is to represent words in an high dimensional embedding space with reserving of the context. So word2vec is contextual, well at least to some degree.

Some says to compare LDA to doc2vec, those are not comparable either. As doc2vec is just the sum of word2vec. If we really like to do it then word2vec (doc2vec) + k-mean (or other clustering techniques) might be something to compare LDA topic modelling to.

Good blogs about LDA topic model

I have read some great articles about LDA. In particular, I like the posts about LDA gensim example. Gensim is popular library for text mining. It is written in Python and it is easy to use. Here are some good posts that are helpful to learn LDA.  If you are the author, and if you don’t want me to include your post here, please let me know, I will delete it.  Introduction to Latent Dirichlet…

A recently started a project classifying documents with Latent Dirichlet Allocation.  It seems to be a way of generating topic models.  Wikipedia says,

In statistics, latent Dirichlet allocation (LDA) is a generative model that allows sets of observations to be explained by unobserved groups that explain why some parts of the data are similar. 

What is a generative model (incontrast to a discriminative model)?

a generative model is a full probabilistic model of all variables, whereas a discriminative model provides a model only for the target variable(s) conditional on the observed variables. Thus a generative model can be used, for example, to simulate (i.e. generate) values of any variable in the model, whereas a discriminative model allows only sampling of the target variables conditional on the observed quantities

Examples of generative models include:

Gaussian mixture model and other types of mixture model

Hidden Markov model

Probabilistic context-free grammar

Naive Bayes

Averaged one-dependence estimators

Latent Dirichlet allocation

Restricted Boltzmann machine

Examples of discriminative models used in machine learning include:

Logistic regression(aka maximum entropy classifiers)

Linear discriminant analysis

Support vector machines

Boosting (meta-algorithm)

Conditional random fields

Linear regression

Neural networks

Snarxiv uses a context-free grammar built from the original arXiv site.  Which is the real article?

A soluble model for scattering and decay in quaternionic quantum mechanics 1: decay

On Extremal Topological Field Theories on Del-Pezzos

The words or document features are "observed" and the "topics" are unobserved.  I have found a few libraries online which implement these for me:

topicmodels (R)

gensim (Python)

Princeton CS professor David Blei who seems to be the master at topic models. Here's an actual topic analysis of arXiv he gathered, including:

{ group groups ring finite field }

{ surface current transport electronic josephson }

{ energy nuclear model state calculations }

I wonder if I can verify their linguistic relationships with actual ones.  I would like to try this for TechCrunch.  This topic lies in the middle of semantics and statistical algorithms.

To end our overview: Latent Dirichlet Allocation is like this

While probabilistic latent semantic analysis is like this

Gaussian mixture model looks like this: