#backprop

Deep learning vs. machine learning: Understand the differences

Source | www-infoworld-com.cdn.ampproject.org | Martin Heller

Machine learning and deep learning are both forms of artificial intelligence. You can also say, correctly, that deep learning is a specific kind of machine learning. Both machine learning and deep learning start with training and test data and a model and go through an optimization process to find the weights that make the model best…

"The anchovies were nowhere near the sardines and the tuna. That's because they were near the pizza toppings. But it was only a problem because this was a three-dimensional grocery store. If it had been a thirty-dimensional grocery store they could have been near the pizza and the sardines."

Geoff Hinton talks here about a number of different spaces that are not the physical space on which maths intuition develops.

The bag of words model of a document takes each word in a text to be a dimension of the document, with like 80,000 possibilities each or however many words there are in English. (The 80,000 possibilities are the underlying corpus.)

Latent semantic analysis of the bag-of-words type is how Google now does its search rankings. (PageRank only constitutes something like 30% of SERP ranking anymore, because [a] it's too easy to game and [b] it's inspecific to what's being searched on. Domain authority and LSI comprise the rest. <--separate article)

Seeing a pixel-by-pixel representation of a 2-D image as a list vector is problematic because the first pixel in a 200x300 Facebook profile image is next to the second pixel and also next to the 201st pixel. I.e. one needs a 2-array.

Hinton talks about abstract feature space and energies -- equivalently evolutionary fitness or economic utility -- and ravines and mountains upon this manifold.

The number of dimensions here is like the number of parameters (same as free parameters or degrees of freedom or arbitrary parameters in stats class) and in a neural net each "synapse" or graph edge is a lever you can pull.