Table Of Contents
Table Of Contents

Xavier

class mxnet.initializer.Xavier(rnd_type='uniform', factor_type='avg', magnitude=3)[source]

Returns an initializer performing “Xavier” initialization for weights.

This initializer is designed to keep the scale of gradients roughly the same in all layers.

By default, rnd_type is 'uniform' and factor_type is 'avg', the initializer fills the weights with random numbers in the range of \([-c, c]\), where \(c = \sqrt{\frac{3.}{0.5 * (n_{in} + n_{out})}}\). \(n_{in}\) is the number of neurons feeding into weights, and \(n_{out}\) is the number of neurons the result is fed to.

If rnd_type is 'uniform' and factor_type is 'in', the \(c = \sqrt{\frac{3.}{n_{in}}}\). Similarly when factor_type is 'out', the \(c = \sqrt{\frac{3.}{n_{out}}}\).

If rnd_type is 'gaussian' and factor_type is 'avg', the initializer fills the weights with numbers from normal distribution with a standard deviation of \(\sqrt{\frac{3.}{0.5 * (n_{in} + n_{out})}}\).

Parameters:
  • rnd_type (str, optional) – Random generator type, can be 'gaussian' or 'uniform'.
  • factor_type (str, optional) – Can be 'avg', 'in', or 'out'.
  • magnitude (float, optional) – Scale of random number.
__init__(rnd_type='uniform', factor_type='avg', magnitude=3)[source]

Initialize self. See help(type(self)) for accurate signature.

Methods

__init__([rnd_type, factor_type, magnitude]) Initialize self.
dumps() Saves the initializer to string
set_verbosity([verbose, print_func]) Switch on/off verbose mode