Interestingly, the right way to think about Ockham’s razor is in terms of a universal computational generalization of Shannon’s theory of entropy, namely Kolmogorov complexity.

Norvig’s response was a hoot. It points out how easy it is to turn Chomsky’s rhetoric against itself (watch the CBC doc Manufacturing Consent for a real mind trip if you know anything about the power game in linguistic theory).

Unlike Norvig, I gave up trying to debate these guys years ago and just voted with my feet to get as far away from them as I could. The last talk I gave in linguistics was over 10 years ago and they nearly threw tomatoes at me. I was at NYU and I suggested doing exploratory data analysis (not modeling, mind you) with spoken language corpora. They told me that linguistics wasn’t about modeling what people say, but about “understanding” and whatever I was doing wasn’t linguistics. The only data they considered worthy were the intuitions of “trained linguists”.

The reason the CMU computational linguistics program split from the University of Pittsburgh’s linguistics department is that Pitt (specifically Dan Everett, the one syntactician involved) didn’t consider HPSG linguistics, but instead called it “engineering” and absolutely refused to let one of my and Carl Pollard’s students do a qualifying paper on German word order in HPSG. Polemical words were exchanged and Pitt unilaterally broke all ties. They were a bit surprised all the students wanted to be associated with CMU “engineering” rather than Pitt “science”.

Note to Norvig: Ockham’s razor is not due to Chomsky.