Comments on: quick note: cer et al 2010

By: brendano

brendano — Thu, 15 Apr 2010 14:58:26 +0000

But then you need a phrase->dependency extractor for every language. They’re always rule-based. Maybe that’s not too hard? Comparing on CoNLL starts becoming more problematic, but maybe that’s not the point.

By: Daniel Cer

Daniel Cer — Thu, 15 Apr 2010 06:02:52 +0000

I suspect the results may reflect just how good we are at producing phrase structure parses for English. Rather than just focusing on other dependency formalisms, I think the more interesting question might be do the results generalize to other languages.

By: Mihai

Mihai — Thu, 15 Apr 2010 05:41:43 +0000

Yes. Hence the gold POS tags in the experiment.

By: brendano

brendano — Thu, 15 Apr 2010 05:07:55 +0000

Cool.

Your paper only talks about the CoNLL experiment. For the experiment shown on the webpage, where do the Stanford Dependencies data come from? The SD extractor run on gold penn treebank parses?

By: Mihai

Mihai — Thu, 15 Apr 2010 04:41:40 +0000

This is a partial answer to your critique (and also a shameless pitch). Here (http://www.surdeanu.name/mihai/ensemble/) I compared dependency parser performance for Stanford dependencies and CoNLL-2008. The context is slightly different: I used only linear SVMs and I’m interpolating several shift-reduce models. But I think the main observation holds: the performance of dependency parsers is very close on CoNLL-2008 and Stanford dependencies.

I can’t easily evaluate constituent parsers on CoNLL-2008 dependencies because the tokenization is different and the generation of syntactic dependencies for this corpus was not fully automated, but, extrapolating a bit, I don’t think we would see numbers too different than in the Cer et al. paper on CoNLL-2008.