Decision Trees — Classification & Entropy

Decision trees are a simple yet powerful supervised machine learning method. At each node, the tree asks a yes/no question (split) on one feature to reduce "impurity" (entropy or Gini) in the child groups.

Entropy measures uncertainty/messiness: 0 = pure (all one class), 1 = maximum mix.

Information Gain = parent entropy - weighted child entropies. The algorithm greedily picks the split with highest IG.

Overfitting happens when the tree memorizes noise (too many splits); pruning or limiting depth helps generalization.

Interactive goals: Move points to change the data. Drag splits to see entropy drop. Use "Optimal" for best split. Add noise to see accuracy drop, then prune to recover.

Olympiad / Future Skills focus: Compute entropy and IG exactly, understand bias-variance, applications in credit scoring, medical diagnosis, game AI.