These binomial confidence intervals are defined for i.i.d. samples. This is a particularly lousy assumption for any natural language evaluation involving whole documents, because there’s a huge amount of correlation between words in a document. Given correlation among the items, the confidence intervals need to be fatter because the effective independent sample size is lower. This reduces significance, which may explain why no one ever talks about it.

There’s also sampling variance because the p here are usually only estimated from the corpus, not the entire (super-)population. This also reduces significance.

As we say in CS, “garbage in, garbage out”.