mxnet.optimizer.DCASGD¶
-
class
mxnet.optimizer.
DCASGD
(momentum=0.0, lamda=0.04, **kwargs)[source]¶ The DCASGD optimizer.
This class implements the optimizer described in Asynchronous Stochastic Gradient Descent with Delay Compensation for Distributed Deep Learning, available at https://arxiv.org/abs/1609.08326.
This optimizer accepts the following parameters in addition to those accepted by
Optimizer
.Parameters: - momentum (float, optional) – The momentum value.
- lamda (float, optional) – Scale DC value.
-
__init__
(momentum=0.0, lamda=0.04, **kwargs)[source]¶ Initialize self. See help(type(self)) for accurate signature.
Methods
__init__
([momentum, lamda])Initialize self. create_optimizer
(name, **kwargs)Instantiates an optimizer with a given name and kwargs. create_state
(index, weight)Creates auxiliary state for a given weight. create_state_multi_precision
(index, weight)Creates auxiliary state for a given weight, including FP32 high precision copy if original weight is FP16. register
(klass)Registers a new optimizer. set_learning_rate
(lr)Sets a new learning rate of the optimizer. set_lr_mult
(args_lr_mult)Sets an individual learning rate multiplier for each parameter. set_lr_scale
(args_lrscale)[DEPRECATED] Sets lr scale. set_wd_mult
(args_wd_mult)Sets an individual weight decay multiplier for each parameter. update
(index, weight, grad, state)Updates the given parameter using the corresponding gradient and state. update_multi_precision
(index, weight, grad, …)Updates the given parameter using the corresponding gradient and state. Attributes
learning_rate
opt_registry