Tous les défis
Moyen 0 résolus 0 % de réussite

Information Gain

Compute how much splitting on a feature reduces entropy, the criterion behind ID3 decision trees.

Ce que vous allez apprendre

  • Compute entropy before and after a split
  • Weight subsets by their size
  • Pick the most informative feature

Problème

Afficher la traduction

Entropy measures the uncertainty of a set of labels: H(S) = -Σ pₖ × log₂(pₖ), where pₖ is the fraction of labels of class k. Information gain is how much entropy drops when you split the set by a feature:

IG = H(labels) - Σ (|Sᵥ| / |S|) × H(Sᵥ)

where Sᵥ contains the labels of the samples whose feature equals v. Decision trees pick the feature with the highest gain.

Implement information_gain(feature, labels).

Input

feature: The feature value of each sample.

labels: The class label of each sample (same length).

Output

Return the information gain in bits, as a decimal number.

Contraintes

1 ≤ length ≤ 10,000

Each value and label is 1 to 20 characters

Exemples

Exemple 1
Entrée :
feature = ["sunny", "sunny", "rain", "rain"]
labels = ["no", "no", "yes", "yes"]

Sortie attendue :
expected = 1.0
Entrée :
std::vector<std::string> feature = {"sunny", "sunny", "rain", "rain"};
std::vector<std::string> labels = {"no", "no", "yes", "yes"};

Sortie attendue :
double expected = 1.0;
Entrée :
string[] feature = new string[] {"sunny", "sunny", "rain", "rain"};
string[] labels = new string[] {"no", "no", "yes", "yes"};

Sortie attendue :
double expected = 1.0;

Explication

The labels are half yes and half no, so H = 1 bit. Splitting by weather gives two pure groups (H = 0 each), so the gain is 1 - 0 = 1: the feature tells you everything.

Exemple 2
Entrée :
feature = ["a", "b", "a", "b"]
labels = ["yes", "yes", "no", "no"]

Sortie attendue :
expected = 0.0
Entrée :
std::vector<std::string> feature = {"a", "b", "a", "b"};
std::vector<std::string> labels = {"yes", "yes", "no", "no"};

Sortie attendue :
double expected = 0.0;
Entrée :
string[] feature = new string[] {"a", "b", "a", "b"};
string[] labels = new string[] {"yes", "yes", "no", "no"};

Sortie attendue :
double expected = 0.0;

Explication

Each group a and b still has one yes and one no (H = 1), so the remainder is 0.5 × 1 + 0.5 × 1 = 1 and the gain is 1 - 1 = 0: the feature is useless.

Comment l'aborder

Essayez d'abord seul, puis ouvrez les indices un par un.

Indice 1

First measure the label uncertainty before any split, then compare it to the uncertainty inside each feature group.

Indice 2

Group samples by feature value, count labels within each group, and combine each group's entropy using its share of the data.

Indice 3

Be careful with repeated values and the case where a group has only one label, since its entropy contribution is zero.

Complexité attendue

O(n) time, O(n) extra space

Références

Ce défi a été adapté pour CodeUtility. Suivez l'énoncé et les contraintes ci-dessus.

La meilleure solution validée de chaque codeur. Les plus rapides, les plus légères et les mieux notées passent devant.

Aucune solution validée pour l'instant. Résolvez-le et prenez la première place.

Apprenez des autres

Solutions de la communauté

Aucune solution publiée pour l'instant. Résolvez ce défi pour partager la première.