One last thing on the Norvig vs. Chomsky thing from a little while ago (http://norvig.com/chomsky.html), which (correctly) casts the issue as Shannon vs. Chomsky.
The relevant seminal publications are:
- Shannon, “Mathematical Theory of Communication,” 1948
- Chomsky, “Syntactic Structures,” 1957
One of those historical figures is still around and representing himself in 2011 — he should get credit just for still showing up to the fight. Are there any historical figures from the Shannon side still around? What I would’ve given to see a Jelinek vs. Chomsky public debate. Though I guess Pereira vs. Chomsky would be pretty great.
I’m pretty sure both Chomsky and Fred (Jelinek) would have viewed it as a waste of their time. It’s a pity that there aren’t any serious intellectuals who are information theorists. Pereira and Norvig are probably the best we’ve got.
Shannon doesn’t have to be around to represent himself because math keeps working even after the discoverer dies.
Norvig’s response was a hoot. It points out how easy it is to turn Chomsky’s rhetoric against itself (watch the CBC doc Manufacturing Consent for a real mind trip if you know anything about the power game in linguistic theory).
Unlike Norvig, I gave up trying to debate these guys years ago and just voted with my feet to get as far away from them as I could. The last talk I gave in linguistics was over 10 years ago and they nearly threw tomatoes at me. I was at NYU and I suggested doing exploratory data analysis (not modeling, mind you) with spoken language corpora. They told me that linguistics wasn’t about modeling what people say, but about “understanding” and whatever I was doing wasn’t linguistics. The only data they considered worthy were the intuitions of “trained linguists”.
The reason the CMU computational linguistics program split from the University of Pittsburgh’s linguistics department is that Pitt (specifically Dan Everett, the one syntactician involved) didn’t consider HPSG linguistics, but instead called it “engineering” and absolutely refused to let one of my and Carl Pollard’s students do a qualifying paper on German word order in HPSG. Polemical words were exchanged and Pitt unilaterally broke all ties. They were a bit surprised all the students wanted to be associated with CMU “engineering” rather than Pitt “science”.
Note to Norvig: Ockham’s razor is not due to Chomsky.
Oops, meant that Ockham’s razor is not due to Einstein.
Interestingly, the right way to think about Ockham’s razor is in terms of a universal computational generalization of Shannon’s theory of entropy, namely Kolmogorov complexity.