Wednesday, August 4, 2010

ROC Curve

Receiver operating characteristic (ROC) curve

Exposing the Coadaptive Potential of Protein-protein Interfaces through Computational Sequence Design
Menachem Fromer 1 and Michal Linial 2,*
1School of Computer Science and Engineering, The Hebrew University of Jerusalem, Jerusalem, Israel
2Department of Biological Chemistry, Institute of Life Sciences, Sudarsky Center for Computational Biology, The Hebrew University of Jerusalem, Jerusalem, Israel
*To whom correspondence should be addressed. Michal Linial, E-mail: michall@cc.huji.ac.il
http://bioinformatics.oxfordjournals.org/cgi/content/abstract/btq412v1?etoc

Sample data
http://mark.goadrich.com/programs/AUC/

Example List file
0.9 1
0.8 1
0.7 0
0.6 1
0.55 1
0.54 1
0.53 0
0.52 0
0.51 1
0.505 0
Example ROC file for same dataset as above, using 6 pos and 4 neg examples
0       0
0.0     0.16666666666666666
0.0     0.3333333333333333
0.25    0.3333333333333333
0.25    0.5
0.25    0.6666666666666666
0.25    0.8333333333333334
0.5     0.8333333333333334
0.75    0.8333333333333334
0.75    1.0
1.0     1.0
AUC 
0.75



> AUC
function(ranks, n) {
 #ranks : ranks = c( 1,2,3);
#n = 5 ; total number
Npos =length(ranks)
Nneg = n - Npos
AUC = 1 - ( sum(ranks) - Npos*(Npos+1)/2 ) / (Npos * Nneg)
return(AUC)
}


> ROC
function(ranks , n) {
#doesn't work for ties !
TF = rep(0,n)
TF[ranks] = 1;
fp= cumsum(!TF/sum(!TF))
tp=cumsum(TF/sum(TF))
return( list( fp = fp / max(fp), tp = tp))
}


> ROC(x[x$V2==1,'rank'],nrow(x))
$fp
 [1] 0.25 0.25 0.50 0.75 0.75 0.75 0.75 1.00 1.00 1.00

$tp
 [1] 0.0000000 0.1666667 0.1666667 0.1666667 0.3333333 0.5000000 0.6666667
 [8] 0.6666667 0.8333333 1.0000000

> x$rank <- rank(-1*x$V1)
> ROC(x[x$V2==1,'rank'],nrow(x))
$fp
 [1] 0.00 0.00 0.25 0.25 0.25 0.25 0.50 0.75 0.75 1.00

$tp
 [1] 0.1666667 0.3333333 0.3333333 0.5000000 0.6666667 0.8333333 0.8333333
 [8] 0.8333333 1.0000000 1.0000000

> AUC(x[x$V2==1,'rank'],nrow(x))
[1] 0.75


Comparing experimental and computational alanine scanning techniques for probing a prototypical protein–protein interaction
Richard T. Bradshaw, Bhavesh H. Patel, Edward W. Tate, Robin J. Leatherbarrow and Ian R. Gould1
Department of Chemistry and Chemical Biology Centre, Imperial College London, South Kensington Campus, London SW7 2AZ, UK
1 To whom correspondence should be addressed. E-mail: i.gould@imperial.ac.uk
http://peds.oxfordjournals.org/cgi/content/abstract/gzq047v1?etoc

Comparison study of microarray meta-analysis methods
Anna Campain email and Yee Hwa Yang email
BMC Bioinformatics 2010, 11:408doi:10.1186/1471-2105-11-408
Published: 3 August 2010
http://peds.oxfordjournals.org/cgi/content/abstract/gzq047v1?etoc

Background

Meta-analysis methods exist for combining multiple microarray datasets. However, there are a wide range of issues associated with microarray meta-analysis and a limited ability to compare the performance of different meta-analysis methods.
Results

We compare eight meta-analysis methods, five existing methods, two naive methods and a novel approach (mDEDS). Comparisons are performed using simulated data and two biological case studies with varying degrees of meta-analysis complexity. The performance of meta-analysis methods is assessed via ROC curves and prediction accuracy where applicable.
Conclusions

Existing meta-analysis methods vary in their ability to perform successful meta-analysis. This success is very dependent on the complexity of the data and type of analysis. Our proposed method, mDEDS, performs competitively as a meta-analysis tool even as complexity increases. Because of the varying abilities of compared meta-analysis methods, care should be taken when considering the meta-analysis method used for particular research.


The diagnostic performance of a test, or the accuray of a test to discriminate diseased cases from normal cases is evaluated using Receiver Operating Characteristic (ROC) curve analysis (Metz, 1978; Zweig & Campbell, 1993). ROC curves can also be used to compare the diagnostic performance of two or more laboratory or diagnostic tests (Griner et al., 1981).
http://www.medcalc.be/manual/roc.php

No comments: