Why and when to use a decision tree - Python Tutorial

- [Man] In order to know when to use a decision tree, we need to understand its strengths and weaknesses. In terms of strengths, decision trees are simple to understand and interpret. The logical structure of a tree is intuitive, and easy to follow. This means we can easily create business rules, based on the structure of a tree. Unlike some other approaches, which work better with either discreet, or continuous features, decision trees are able to handle both types of data, very well. In other words, decision trees are useful, for both classification and regression problems. Decision trees also do very well, in handling missing, noisy and outlier data. This means we don't need to pre-process our data much, before we can use it to build a decision tree model. During each stage of the recursive partitioning process, the feature that reduces impurity the most is chosen. This means that unimportant features, are ignored by the tree. As a result, feature selection is not necessary, when working with decision trees. Decision trees do well on most problems, even if it makes slightly erroneous assumptions, about the nature of the data that is used to build it. Decision trees are useful on both small and large data sets. However, similar to other non-parametric models, the predictive accuracy of a decision tree, tends to improve as it encounters more training examples. There are some weaknesses inherent with decision trees, as well. The recursive partitioning process tends to be biased, toward features with a large number of unique values, when using entropy as a measure of impurity. Decision trees can be rather unstable, because small changes in data can result in large changes, in the structure of the tree. This limitation is mitigated by using decision trees, as part of an ensemble. Decision trees are non-parametric models. This means that they make very few assumptions, about the data. A decision tree attempts to describe the training data, as closely as possible. If not properly remediated, they can easily over fit against the training data. Decision trees can also under fit, if the pruning process is overly aggressive. Decision trees are limited to axis-parallel splits. This means that you can only split data, horizontally or vertically, during the recursive partitioning process. This limits the usefulness of decision trees, in certain problem domains. While decision trees are easy to understand, very large trees can be rather difficult to interpret. Decision trees tend to be biased, if the training data suffers from class imbalance. To mitigate this, we often have to balance the training data, before trying to fit a decision tree to it.