Table Of Contents
Table Of Contents

PrefetchingIter

class mxnet.io.PrefetchingIter(iters, rename_data=None, rename_label=None)[source]

Performs pre-fetch for other data iterators.

This iterator will create another thread to perform iter_next and then store the data in memory. It potentially accelerates the data read, at the cost of more memory usage.

Parameters:
  • iters (DataIter or list of DataIter) – The data iterators to be pre-fetched.
  • rename_data (None or list of dict) – The i-th element is a renaming map for the i-th iter, in the form of {‘original_name’ : ‘new_name’}. Should have one entry for each entry in iter[i].provide_data.
  • rename_label (None or list of dict) – Similar to rename_data.

Examples

>>> iter1 = mx.io.NDArrayIter({'data':mx.nd.ones((100,10))}, batch_size=25)
>>> iter2 = mx.io.NDArrayIter({'data':mx.nd.ones((100,10))}, batch_size=25)
>>> piter = mx.io.PrefetchingIter([iter1, iter2],
...                               rename_data=[{'data': 'data_1'}, {'data': 'data_2'}])
>>> print(piter.provide_data)
[DataDesc[data_1,(25, 10L),<type 'numpy.float32'>,NCHW],
 DataDesc[data_2,(25, 10L),<type 'numpy.float32'>,NCHW]]
__init__(iters, rename_data=None, rename_label=None)[source]

Initialize self. See help(type(self)) for accurate signature.

Methods

__init__(iters[, rename_data, rename_label]) Initialize self.
getdata() Get data of current batch.
getindex() Get index of the current batch.
getlabel() Get label of the current batch.
getpad() Get the number of padding examples in the current batch.
iter_next() Move to the next batch.
next() Get next data batch from iterator.
reset() Reset the iterator to the begin of the data.

Attributes

provide_data
provide_label