Table Of Contents
Table Of Contents


class mxnet.metric.Perplexity(ignore_label, axis=-1, name='perplexity', output_names=None, label_names=None)[source]

Computes perplexity.

Perplexity is a measurement of how well a probability distribution or model predicts a sample. A low perplexity indicates the model is good at predicting the sample.

The perplexity of a model q is defined as

\[b^{\big(-\frac{1}{N} \sum_{i=1}^N \log_b q(x_i) \big)} = \exp \big(-\frac{1}{N} \sum_{i=1}^N \log q(x_i)\big)\]

where we let b = e.

\(q(x_i)\) is the predicted value of its ground truth label on sample \(x_i\).

For example, we have three samples \(x_1, x_2, x_3\) and their labels are \([0, 1, 1]\). Suppose our model predicts \(q(x_1) = p(y_1 = 0 | x_1) = 0.3\) and \(q(x_2) = 1.0\), \(q(x_3) = 0.6\). The perplexity of model q is \(exp\big(-(\log 0.3 + \log 1.0 + \log 0.6) / 3\big) = 1.77109762852\).

  • ignore_label (int or None) – Index of invalid label to ignore when counting. By default, sets to -1. If set to None, it will include all entries.

  • axis (int (default -1)) – The axis from prediction that was used to compute softmax. By default use the last axis.

  • name (str) – Name of this metric instance for display.

  • output_names (list of str, or None) – Name of predictions that should be used when updating with update_dict. By default include all predictions.

  • label_names (list of str, or None) – Name of labels that should be used when updating with update_dict. By default include all labels.


>>> predicts = [mx.nd.array([[0.3, 0.7], [0, 1.], [0.4, 0.6]])]
>>> labels   = [mx.nd.array([0, 1, 1])]
>>> perp = mx.metric.Perplexity(ignore_label=None)
>>> perp.update(labels, predicts)
>>> print perp.get()
('Perplexity', 1.7710976285155853)
__init__(ignore_label, axis=-1, name='perplexity', output_names=None, label_names=None)[source]

Initialize self. See help(type(self)) for accurate signature.


__init__(ignore_label[, axis, name, …])

Initialize self.


Returns the current evaluation result.


Save configurations of metric.


Returns the current global evaluation result.


Returns zipped name and value pairs for global results.


Returns zipped name and value pairs.


Resets the internal evaluation result to initial state.


Resets the local portion of the internal evaluation results to initial state.

update(labels, preds)

Updates the internal evaluation result.

update_dict(label, pred)

Update the internal evaluation with named label and pred