# mxnet.metric.Perplexity¶

class mxnet.metric.Perplexity(ignore_label, axis=-1, name='perplexity', output_names=None, label_names=None)[source]

Computes perplexity.

Perplexity is a measurement of how well a probability distribution or model predicts a sample. A low perplexity indicates the model is good at predicting the sample.

The perplexity of a model q is defined as

$b^{\big(-\frac{1}{N} \sum_{i=1}^N \log_b q(x_i) \big)} = \exp \big(-\frac{1}{N} \sum_{i=1}^N \log q(x_i)\big)$

where we let b = e.

$$q(x_i)$$ is the predicted value of its ground truth label on sample $$x_i$$.

For example, we have three samples $$x_1, x_2, x_3$$ and their labels are $$[0, 1, 1]$$. Suppose our model predicts $$q(x_1) = p(y_1 = 0 | x_1) = 0.3$$ and $$q(x_2) = 1.0$$, $$q(x_3) = 0.6$$. The perplexity of model q is $$exp\big(-(\log 0.3 + \log 1.0 + \log 0.6) / 3\big) = 1.77109762852$$.

Parameters: ignore_label (int or None) – Index of invalid label to ignore when counting. By default, sets to -1. If set to None, it will include all entries. axis (int (default -1)) – The axis from prediction that was used to compute softmax. By default use the last axis. name (str) – Name of this metric instance for display. output_names (list of str, or None) – Name of predictions that should be used when updating with update_dict. By default include all predictions. label_names (list of str, or None) – Name of labels that should be used when updating with update_dict. By default include all labels.

Examples

>>> predicts = [mx.nd.array([[0.3, 0.7], [0, 1.], [0.4, 0.6]])]
>>> labels   = [mx.nd.array([0, 1, 1])]
>>> perp = mx.metric.Perplexity(ignore_label=None)
>>> perp.update(labels, predicts)
>>> print perp.get()
('Perplexity', 1.7710976285155853)

__init__(ignore_label, axis=-1, name='perplexity', output_names=None, label_names=None)[source]

Initialize self. See help(type(self)) for accurate signature.

Methods

 __init__(ignore_label[, axis, name, …]) Initialize self. get() Returns the current evaluation result. get_config() Save configurations of metric. get_global() Returns the current global evaluation result. get_global_name_value() Returns zipped name and value pairs for global results. get_name_value() Returns zipped name and value pairs. reset() Resets the internal evaluation result to initial state. reset_local() Resets the local portion of the internal evaluation results to initial state. update(labels, preds) Updates the internal evaluation result. update_dict(label, pred) Update the internal evaluation with named label and pred