Alle Herausforderungen
Mittel 0 gelöst 0% Erfolgsquote

Mini-Batch Gradient Descent

Train a linear model by updating the weights after every small batch of samples instead of the whole dataset.

Das lernst du

  • Split data into batches
  • Update weights after each batch
  • Compare batch, mini-batch and stochastic training

Problem

Übersetzung anzeigen

Mini-batch gradient descent is how almost every neural network is trained: split the data into small batches and update the weights after each one.

Train y = w × x + b on the mean squared error, starting from w = 0 and b = 0. In each epoch, split the samples in order into batches of batch_size (the last batch may be smaller). For each batch of size n:

  1. errorᵢ = w × xᵢ + b - yᵢ for each sample in the batch
  2. dw = (2 / n) × Σ errorᵢ × xᵢ and db = (2 / n) × Σ errorᵢ
  3. w ← w - learning_rate × dw and b ← b - learning_rate × db

Implement train_mini_batch(xs, ys, batch_size, learning_rate, epochs).

Input

xs, ys: The data points (same length).

batch_size: Samples per batch.

learning_rate, epochs: The step size and the number of passes over the data.

Output

Return [w, b] after training.

Einschränkungen

1 ≤ number of points ≤ 1,000, 1 ≤ batch_size ≤ number of points, 0 ≤ epochs ≤ 1,000

0 < learning_rate ≤ 1

-100 ≤ each value ≤ 100

Beispiele

Beispiel 1
Eingabe:
xs = [1.0, 2.0, 3.0]
ys = [2.0, 4.0, 6.0]
batch_size = 2
learning_rate = 0.1
epochs = 1

Erwartete Ausgabe:
expected = [2.44, 1.08]
Eingabe:
std::vector<double> xs = {1.0, 2.0, 3.0};
std::vector<double> ys = {2.0, 4.0, 6.0};
long long batch_size = 2;
double learning_rate = 0.1;
long long epochs = 1;

Erwartete Ausgabe:
std::vector<double> expected = {2.44, 1.08};
Eingabe:
double[] xs = new double[] {1.0, 2.0, 3.0};
double[] ys = new double[] {2.0, 4.0, 6.0};
long batch_size = 2;
double learning_rate = 0.1;
long epochs = 1;

Erwartete Ausgabe:
double[] expected = new double[] {2.44, 1.08};

Erläuterung

Batch 1 is x = [1, 2]: errors -2 and -4, dw = -10, db = -6, so w = 1 and b = 0.6. Batch 2 is x = [3] alone: error 1×3 + 0.6 - 6 = -2.4, dw = 2 × (-2.4) × 3 = -14.4 and db = -4.8, so w = 2.44 and b = 1.08. Result: [2.44, 1.08].

Beispiel 2
Eingabe:
xs = [1.0, 2.0, 3.0, 4.0]
ys = [3.0, 5.0, 7.0, 9.0]
batch_size = 4
learning_rate = 0.05
epochs = 1

Erwartete Ausgabe:
expected = [1.75, 0.6]
Eingabe:
std::vector<double> xs = {1.0, 2.0, 3.0, 4.0};
std::vector<double> ys = {3.0, 5.0, 7.0, 9.0};
long long batch_size = 4;
double learning_rate = 0.05;
long long epochs = 1;

Erwartete Ausgabe:
std::vector<double> expected = {1.75, 0.6};
Eingabe:
double[] xs = new double[] {1.0, 2.0, 3.0, 4.0};
double[] ys = new double[] {3.0, 5.0, 7.0, 9.0};
long batch_size = 4;
double learning_rate = 0.05;
long epochs = 1;

Erwartete Ausgabe:
double[] expected = new double[] {1.75, 0.6};

Erläuterung

A batch as big as the dataset is ordinary batch gradient descent: one update per epoch. Result: [1.75, 0.6].

So gehst du vor

Versuch es zuerst allein und öffne dann einen Hinweis nach dem anderen.

Hinweis 1

Process the samples in the given order, updating the parameters after every batch instead of after every single point.

Hinweis 2

For each batch, compute prediction errors first, then use their averages with x values to get the two gradients.

Hinweis 3

The last batch may be smaller than the others, so its gradient scaling must use that batch's size.

Erwartete Komplexität

O(epochs × n) time, O(1) extra space

Referenzen

Diese Aufgabe wurde für CodeUtility angepasst. Beachte die oben genannten Vorgaben und Einschränkungen.

Die beste bestandene Lösung jedes Coders. Schnellere, sparsamere und besser bewertete Lösungen stehen weiter oben.

Noch keine bestandene Lösung. Löse sie und hol dir Platz eins.

Von anderen lernen

Community-Lösungen

Noch keine veröffentlichten Lösungen. Löse die Aufgabe und teile als Erster eine Lösung.