What is Machine Learning Introduction Well-Posed Learning Problems Designing a Learning System Steps for Designing Learning System Are Perspectives and Issues in Machine Learning Concept Learning and the General to Specific Ordering Concept Learning Task Concept Learning as a Search Important Representation Inductive Bias

Decision Tree Learning Types of Decision Tree Strengths and Weakness of Decision Tree Approach Appropriate Problems in Decision Tree Learning Issues in Decision Tree Learning What is Hypothesis Types of Hypothesis ID3 Information Gain for Each Feature

Artificial Neural Networks I Biological Motivation Neural Network Representations Appropriate Problems for Neural Network Learning Perceptron Genetic Algorithms Application Areas What is Reinforcement Learning Terms Used in Reinforcement Learning Approaches to Implement Reinforcement Learning Temporal Difference Learning EM Algorithm GIBS Algorithm

Previous Page

Next Page

Appropriate Problems in Decision Tree Learning

Instances are represented by attribute-value pairs
“Instances are described by a fixed set of attributes (e.g., Temperature) and their values (e.g., Hot). The easiest situation for decision tree learning is when each attribute takes on a small number of disjoint possible values (e.g., Hot, Mild, Cold). However, extensions to the basic algorithm allow handling real-valued attributes as well (e.g., representing Temperature numerically).”
The target function has discrete output values
1. “The decision tree is usually used for Boolean classification (e.g., yes or no) kind of example.
2. Decision tree methods easily extend to learning functions with more than two possible output values. A more substantial extension allows learning target functions with real-valued outputs, though the
3. application of decision trees in this setting is less common.”
Disjunctive descriptions may be required.
Decision trees naturally represent disjunctive expression
The training data may contain errors.
“Decision tree learning methods are robust to errors, both errors in classifications of the training examples and errors in the attribute values that describe these examples.”
The training data may contain missing attribute values.
“Decision tree methods can be used even when some training examples have unknown values (e.g., if the Humidity of the day is known for only some of the training examples).”

Information Gain

Information gain is the measurement of changes in entropy after the segmentation of a dataset based on an attribute.
It calculates how much information a feature provides us about a class.
According to the value of information gain, we split the node and build the decision tree.
A decision tree algorithm always tries to maximize the value of information gain, and a node/attribute having the highest information gain is split first. It can be calculated using the below formula:
Information Gain= Entropy(S)- [(Weighted Avg) *Entropy(each feature)

Entropy:
Entropy is a metric to measure the impurity in a given attribute. It specifies randomness in data. Entropy can be calculated as:

Entropy(s)= -P(yes)log2 P(yes)- P(no) log2 P(no)

Where,

S= Total number of samples

P(yes)= probability of yes

P(no)= probability of no

Previous Page

Next Page