a handy utility. However, have you left out a “last” function i.e.

> binary_eval(two.out.of.3.marker[C!="A"],Test2)
AUC = 0.807
Predictions seem to be real-valued scores, so using naive cutoff 0:
Error in binary_eval(two.out.of.3.marker[C != "A"], Test2) :
could not find function “last”

last<-function(x){ sort(x)[length(x)] }
seems to fix it and gives sensible plots so I am assuming that is all that last does

Bye
Rob

I’d suggest (a) putting recall on the horizontal axis in both of the bottom graphs, (b) making the graphs square rather than rectangular, and (c) fixing axes to the range [0,1].

In a summary plot like this, I’d also like to see number of positive and negative examples in the gold standard, the name of the data set, name of classifier and any params that can be indicated. I always find myself fishing for that in papers. You might also indicate graphs are uninterpolated, though that’s obvious from looking at PR.

There’s no reason you couldn’t overlay multiple evals here in different colors, too.

I’d love to see posterior intervals in addition to single numbers in the plots. Wouldn’t a simple bootstrap work and be simple to implement in R? Of course, that’d get hopelessly messy and prone to misinterpretation with multiple evals on the same plot.

Given our focus on high recall, we’re often zooming in on the 95-100% recall section of the graph, but presumably that’s all settable as in the rest of R’s plotting functions.

Thanks for the LingPipe pointer — there’s also extensive discussion of classifier evaluation in our javadoc for ConfusionMatrix, PrecisionRecallEvaluation, and ScoredPrecisionRecallEvaluation.

ROC Curves and AUC