|
|
Variable importance-weighted Random Forests |
Yiyi Liu1, Hongyu Zhao1,2( ) |
1. Department of Biostatistics, School of Public Health, Yale University, New Haven, CT 06511, USA 2. Program of Computational Biology and Bioinformatics, Yale University, New Haven, CT 06511, USA |
|
|
Abstract Background: Random Forests is a popular classification and regression method that has proven powerful for various prediction problems in biological studies. However, its performance often deteriorates when the number of features increases. To address this limitation, feature elimination Random Forests was proposed that only uses features with the largest variable importance scores. Yet the performance of this method is not satisfying, possibly due to its rigid feature selection, and increased correlations between trees of forest. Methods: We propose variable importance-weighted Random Forests, which instead of sampling features with equal probability at each node to build up trees, samples features according to their variable importance scores, and then select the best split from the randomly selected features. Results: We evaluate the performance of our method through comprehensive simulation and real data analyses, for both regression and classification. Compared to the standard Random Forests and the feature elimination Random Forests methods, our proposed method has improved performance in most cases. Conclusions: By incorporating the variable importance scores into the random feature selection step, our method can better utilize more informative features without completely ignoring less informative ones, hence has improved prediction accuracy in the presence of weak signals and large noises. We have implemented an R package “viRandomForests” based on the original R package “randomForest” and it can be freely downloaded from http://zhaocenter.org/software.
|
Author Summary Random Forests is a powerful classification and regression method and is commonly used in genomic data analyses. However, when the number of features is very large and the signals are relatively weak, its performance tends to decline. In this article, we propose a modified Random Forests, which instead of considering each feature equally as the original Random Forests does, weights the features according to their informativeness (measured using the variable importance score). This new method, variable importance-weighted Random Forests (viRF), demonstrates improved performance in both simulation and real data analyses studies. |
Keywords
Random Forests
variable importance score
classification
regression
|
Corresponding Author(s):
Hongyu Zhao
|
Online First Date: 06 November 2017
Issue Date: 04 December 2017
|
|
1 |
D. Hanahan, and R. A. Weinberg, (2011) Hallmarks of cancer: the next generation. Cell, 144, 646–674
https://doi.org/10.1016/j.cell.2011.02.013
pmid: 21376230
|
2 |
L. Breiman, (2001) Random forests. Mach. Learn., 45, 5– 32
https://doi.org/10.1023/A:1010933404324
|
3 |
D. S. Palmer, , N. M. O’Boyle, , R. C. Glen, and J. B. Mitchell, (2007) Random forest models to predict aqueous solubility. J. Chem. Inf. Model., 47, 150–158
https://doi.org/10.1021/ci060164k
pmid: 17238260
|
4 |
P. Jiang,, H. Wu,, W. Wang,, W. Ma,, X. Sun,, Z Lu,. (2007) MiPred: classification of real and pseudo microRNA precursors using random forest prediction model with combined features. Nucleic Acids Res. 35, W339–W344
|
5 |
J. W. Lee,, J. B. Lee,, M. Park,, S. H. Song,(2005) An extensive comparison of recent classification tools applied to microarray data. Comput. Stat. Data Anal. 48, 869–885
|
6 |
B. A. Goldstein, , E. C. Polley, and F. B. Briggs, (2011) Random forests for genetic association studies. Stat. Appl. Genet. Mol. Biol., 10, 32
https://doi.org/10.2202/1544-6115.1691
pmid: 22889876
|
7 |
D. Amaratunga, , J. Cabrera, and Y. S. Lee, (2008) Enriched random forests. Bioinformatics, 24, 2010–2014
https://doi.org/10.1093/bioinformatics/btn356
pmid: 18650208
|
8 |
P. M. Granitto, , C. Furlanello, , F. Biasioli, and F. Gasperi, (2006) Recursive feature elimination with random forest for PTR-MS analysis of agroindustrial products. Chemometr. Intell. Lab., 83, 83–90
https://doi.org/10.1016/j.chemolab.2006.01.007
|
9 |
V. Svetnik, , A. Liaw, , C. Tong, and T. Wang, (2004) Application of Breiman’s random forest to modeling structure-activity relationships of pharmaceutical molecules. Lect. Notes Comput. Sci., 3077, 334–343
https://doi.org/10.1007/978-3-540-25966-4_33
|
10 |
R. Díaz-Uriarte, and S.A. de Andrés, (2006) Gene selection and classification of microarray data using random forest. BMC Bioinformatics, 7, 3
https://doi.org/10.1186/1471-2105-7-3
pmid: 16398926
|
11 |
L. Breiman, (2001) Statistical modeling: the two cultures. Stat. Sci., 16, 199–231
https://doi.org/10.1214/ss/1009213726
|
12 |
D. Amaratunga, and J. Cabrera, (2009) A conditional t suite of tests for identifying differentially expressed genes in a DNA microarray experiment with little replication. Stat. Biopharm. Res., 1, 26–38
https://doi.org/10.1198/sbr.2009.0003
|
13 |
G. Biau, (2012) Analysis of a random forests model. J. Mach. Learn. Res., 13, 1063–1095
|
14 |
J. Barretina, , G. Caponigro, , N. Stransky, , K. Venkatesan, , A. A. Margolin, , S. Kim, , C. J. Wilson, , J. Lehár, , G. V. Kryukov, , D. Sonkin, , et al. (2012) The cancer cell line encyclopedia enables predictive modelling of anticancer drug sensitivity. Nature, 483, 603–607
https://doi.org/10.1038/nature11003
pmid: 22460905
|
15 |
I. Guyon, , S. Gunn, , A. Ben-Hur, and G. Dror, (2004) Result Analysis of The Nips 2003 Feature Selection Challenge. In Proceeding NIPS’04 Proceedings of the 17th International Conference on Neural Information Processing Systems. pp. 545–552
|
16 |
S. L. Pomeroy, , P. Tamayo, , M. Gaasenbeek, , L. M. Sturla, , M. Angelo, , M. E. McLaughlin, , J. Y. Kim, , L. C. Goumnerova, , P. M. Black, , C. Lau, , et al. (2002) Prediction of central nervous system embryonal tumour outcome based on gene expression. Nature, 415, 436–442
https://doi.org/10.1038/415436a
pmid: 11807556
|
17 |
D. Singh, , P. G. Febbo, , K. Ross, , D. G. Jackson, , J. Manola, , C. Ladd, , P. Tamayo, A. A. Renshaw, , A. V. D’Amico, , J. P. Richie, , et al. (2002) Gene expression correlates of clinical prostate cancer behavior. Cancer Cell, 1, 203–209
https://doi.org/10.1016/S1535-6108(02)00030-2
pmid: 12086878
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
|
Shared |
|
|
|
|
|
Discussed |
|
|
|
|