Create SGD neural network optimizer
Instantiates a Stochastic Gradient Descent optimizer, used for training neural networks.
Inputs
-
Learning rate — Float
The learning rate.
-
Momentum — Float
An hyperparameter equal or grater than 0 that accelerates gradient descent in the relevant direction and dampens oscillations.
-
Apply Nesterov momentum — Boolean
Whether to apply Nesterov momentum.
-
Clip norm — Float
If a value is provided, all parameter gradients will be clipped to a maximum norm of Clip norm.
-
Clip value — Float
If a value is provided, all parameter gradients will be clipped to a maximum value of Clip value and a minimum value of minus Clip value.
Outputs
-
Optimizer — NeuralNetworkOptimizer
Resulting optimizer instance.