“The anchovies were nowhere *near* the sardines and the tuna. That’s because they were near the pizza toppings.

But it was only a problem because this was a three-dimensional grocery store. If it had been a **thirty-dimensional grocery store** they could have been near the pizza *and* the sardines.”

I’ve been putting off a post about **phase space** for nearly a year now, and watching this talk made me remember that I’ll soon have to do it.

Geoff Hinton talks here about a number of different spaces that are *not* the physical space that math was initially developed on.

- The
**bag of words**model of a document takes each word in a text to be a dimension of the document, with like 80,000 possibilities each or however many words there are in English. (The 80,000 possibilities are the underlying corpus.) **Latent semantic analysis**of the bag-of-words type is how Google now does its search rankings. (PageRank only constitutes something like 30% of SERP ranking anymore, because [a] it’s too easy to game and [b] it’s inspecific to what’s being searched on. Domain authority and LSI comprise the rest. <—separate article)- Seeing a
**pixel-by-pixel representation**of a 2-D image as a list vector is problematic because the first pixel in a 200×300 Facebook profile image is next to the second pixel and also next to the 201st pixel. I.e. one needs a 2-array. - Hinton talks about abstract
**feature space**and energies — equivalently evolutionary fitness or economic utility — and ravines and mountains upon this manifold. - The number of dimensions here is like the number of parameters (same as free parameters or degrees of freedom or arbitrary parameters in stats class) and in a neural net each “synapse” or graph edge is a lever you can pull.
- The same metaphor — and this is a metaphor in a grand sense which I hope to cover before the year is up — applies to the equalizer on your uncle’s home stereo, i.e. the number of terms in the Fourier decomposition.

(Source: http://www.youtube.com/)

Advertisements