Comments on: The $60,000 cat: deep belief networks make less sense for language than vision http://brenocon.com/blog/2012/07/the-60000-cat-deep-belief-networks-make-less-sense-for-language-than-vision/ cognition, language, social systems; statistics, visualization, computation Tue, 25 Nov 2025 13:11:20 +0000 hourly 1 By: Dawen Liang http://brenocon.com/blog/2012/07/the-60000-cat-deep-belief-networks-make-less-sense-for-language-than-vision/#comment-186137 Dawen Liang Sat, 06 Oct 2012 05:11:38 +0000 http://brenocon.com/blog/?p=1336#comment-186137 Just get chance to read your blog. A lot of interesting stuff :) so I subscribed your blog in my Google Reader — under the category of ML, not Stat though (Andrew Gelman is there). If you resist, I could possibly change XD

]]> By: Xinfan Meng http://brenocon.com/blog/2012/07/the-60000-cat-deep-belief-networks-make-less-sense-for-language-than-vision/#comment-164148 Xinfan Meng Fri, 06 Jul 2012 01:17:18 +0000 http://brenocon.com/blog/?p=1336#comment-164148 I can also recognize the frame in {cat, tree, rescue, fireman} immediately; I guess such things are pretty common now. I agree that directly drawing analogy between CV and NLP might be dangerous; but you are drawing your own analogy (character pixel. etc.), right?

]]> By: nope http://brenocon.com/blog/2012/07/the-60000-cat-deep-belief-networks-make-less-sense-for-language-than-vision/#comment-164036 nope Thu, 05 Jul 2012 07:20:56 +0000 http://brenocon.com/blog/?p=1336#comment-164036 The space of words is absurdly high dimensional. This paper deals with a mere 120,000 dimensions (200×200 pixel rgb images). The number of words in English isn’t really well defined but it’s certainly more than 120k. And that’s just to represent one word! The space of sentences is exponentially larger.

Words aren’t dimensionality reduction, they’re just a particularly interpretable “basis” (really a frame but that’s not important).

]]>