Impurity

Task

Formula

Description

Gini impurity

Classification

${\displaystyle {\sum }_{i=1}^C{f}_i\left(1-f_i\right)}$

f_i is the frequency of label i at node and C is the number of unique labels

Entropy

Classification

${\displaystyle {\sum }_{i=1}^C-f_i\log \left(f_i\right)}$

f_i is the frequency of label i at node and C is the number of unique labels

Variance/Mean Square Error(MSE)

Regression

$\frac{1}{N}{\displaystyle {\sum }_{i=1}^N{\left(y_i-\mu \right)}^2}$

y_i is the label for an instance, N is the number of instance and μ is the mean given by $\frac{1}{N}{\displaystyle {\sum }_{i=1}^N{y}_i}$

Variance/Mean Absoluter Error (MAE) (Scikit-learn only)

Regression

$\frac{1}{N}{\displaystyle {\sum }_{i=1}^N\left|y_i-\mu \right|}$

y_i is the label for an instance, N is the number of instance and μ is the mean given by $\frac{1}{N}{\displaystyle {\sum }_{i=1}^N{y}_i}$