Notes on the Random Forests algorithm
The Random Forests algorithm was proposed by Leo Breiman in 1999. The algorithm can be used for both regression and classification, as well as for variable selection, interaction detection, clustering etc.
Here are slides of the guest lecture given on November 26, 2007 for Stat 900 course. The theorem (though stated differently) on the consistency of the bagged 1-nearest neighbour rule can be found in a paper by Biau et al. below.
Below are links to various resources on Random Forest. Some of the links may lead, eventually, to the same resources.
Links
- Leo Breiman's page on Random Forests. An (informal) description of the algorithm as well as links to papers on the algorithm and some of its applications.
- The original paper by Leo Breiman. A preliminary version is available
as Technical
Report 567, Department of Statistics, UC Berkeley, 1999.
Leo Breiman
Random Forests.
Machine Learning, 45(1):5–32, 2001. - One of the first papers that tried to explain the Random Forests
algorithm. A preliminary version is available as
Technical Report 1055,
Department of Statistics, University of Wisconsin, Madison, 2002.
Yi Lin and Yongho Jeon
Random forests and adaptive nearest neighbors.
Journal of the American Statistical Association, 101(474):578–590, 2006. - This paper does not have a word random in its title, but it
proves that a modified version of the Random Forests algorithm is
consistent.
Nicolai Meinshausen
Quantile regression forests.
Journal of Machine Learning Research, 7:983–999, 2006.
abstract | pdf | Journal of Machine Learning Research - A good account on the consistency of various modifications of the Random Forests algorithm.