> For the complete documentation index, see [llms.txt](https://windmising.gitbook.io/liu-yu-bo-play-with-machine-learning/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://windmising.gitbook.io/liu-yu-bo-play-with-machine-learning/src/chapter13-4/13-5.md).

# 13-5 随机森林和extra-trees

Bagging\
Base Estimator: Decision Tree\
只要是以决策树为基础的集成学习算法都叫随机森林。\
scikit-learn提供了随机森林算法，并为算法提供了更多的随机性。\
sickit-learn中，决策树在节点上划分，在随机的特征子集上寻找最优划分的特征。

## 随机森林

使用13-1中的数据，但random\_state=666

```python
import numpy as np
import matplotlib.pyplot as plt
from sklearn import datasets

X, y = datasets.make_moons(n_samples=500, noise=0.3, random_state=666)

plt.scatter(X[y==0,0],X[y==0,1])
plt.scatter(X[y==1,0],X[y==1,1])
plt.show()
```

![](http://windmissing.github.io/images/2019/280.jpg)

```python
from sklearn.ensemble import RandomForestClassifier
rf_clf = RandomForestClassifier(n_estimators=500, random_state=666, oob_score=True)
rf_clf.fit(X, y)
rf_clf.oob_score_
```

输出：0.892

```python
from sklearn.ensemble import RandomForestClassifier
rf_clf2 = RandomForestClassifier(n_estimators=500, max_leaf_nodes=16, random_state=666, oob_score=True)
rf_clf2.fit(X, y)
rf_clf2.oob_score_
```

输出：0.906

## 使用extra-tree

extra-tree = extreme random trees，极其随机的的随机树。 极其随机表现在：决策树在节点上划分，使用随机的特征的随机的阈值。\
提供额外的随机性，抑制过拟合，但增大了bias\
相对于随机森林，速度更快

```python
from sklearn.ensemble import ExtraTreesClassifier
et_clf = ExtraTreesClassifier(n_estimators=500, bootstrap=True, oob_score=True, random_state=666)
et_clf.fit(X, y)
et_clf.oob_score_
```

输出：0.892

## 集成学习解决回归问题

```python
from sklearn.ensemble import BaggingRegressor
from sklearn.ensemble import RandomForestRegressor
from sklearn.ensemble import ExtraTreesRegressor
```