Hinton, G. E. (2007) Learning Multiple Layers of Representation.Trends in Cognitive Sciences, Vol. 11, pp 428-434.
Some considerations on deep learning, i.e., part of the tools that could be used to build vision architectures.
Subscribe to:
Post Comments (Atom)
I had never heard or Restricted Boltzmann Machines (RBM) before, but the proposed deep learning approach looks very promising. I am unsure of how hard the math behind all this machinery will be, as it is conspicuously sidestepped in this paper, but the "unsupervised, layer by layer learning" approach addresses the main problem of multi-layer networks, which is the training stage. The author's last sentence provides a great summary for the reasoning behind this paper: "It is much more sensible first to learn a generative model that infers the hidden variables from the sensory data and then to learn the simpler mapping from the hidden variables to the labels". In the absence of much training data, though, the problem seems to be how to verify if your generative model is outputting something sensible, i.e. what generalization properties this approach has that other approaches did not.
ReplyDeleteThe generated samples from a trained network are really cool to see: http://www.cs.toronto.edu/~hinton/absps/dbm.pdf
ReplyDeleteHowever, somewhat aligning with Alvaro's concerns, the generated samples seem fairly noisy for such a nice dataset (blank background, centered objects) and would like to see what happens with random backgrounds, i.e. can it learn to disregard the background?
Natural next step(?): tying this generative network fed into a discriminative neural net and running backprop over the 2 modules?