mirror of
https://github.com/facebookresearch/fairseq.git
synced 2024-09-11 17:25:31 +03:00
803c0a6d11
Summary: A few changes here: - update GroupedIterator and ShardedIterator to support counting. This will be useful on TPUs, since the TPU dataloading threads may advance faster than we can process them. - add tests for the above - in CountingIterator, rename `count` -> `n`. This is needed because `count` is overloaded for iterables (e.g., `list` defines a different `count` method, which is actually a search function). - in CountingIterator, rename `override_len` -> `total` to be more consistent with other iterators (e.g., tqdm). This functionality was unused previously (it's only needed for TPUs), so the rename is easy. Pull Request resolved: https://github.com/fairinternal/fairseq-py/pull/1166 Reviewed By: ngoyal2707 Differential Revision: D21373525 Pulled By: myleott fbshipit-source-id: 102f3d50ed1a5163a7d1216ca5a179564a05dfe4 |
||
|---|---|---|
| .. | ||
| speech_recognition | ||
| __init__.py | ||
| test_average_checkpoints.py | ||
| test_backtranslation_dataset.py | ||
| test_binaries.py | ||
| test_bmuf.py | ||
| test_character_token_embedder.py | ||
| test_concat_dataset.py | ||
| test_convtbc.py | ||
| test_dictionary.py | ||
| test_export.py | ||
| test_file_io.py | ||
| test_iterators.py | ||
| test_label_smoothing.py | ||
| test_lstm_jitable.py | ||
| test_memory_efficient_fp16.py | ||
| test_metrics.py | ||
| test_multi_corpus_sampled_dataset.py | ||
| test_multihead_attention.py | ||
| test_noising.py | ||
| test_reproducibility.py | ||
| test_resampling_dataset.py | ||
| test_sequence_generator.py | ||
| test_sequence_scorer.py | ||
| test_sparse_multihead_attention.py | ||
| test_token_block_dataset.py | ||
| test_train.py | ||
| test_utils.py | ||
| transformer_quantization_config.yaml | ||
| utils.py | ||