mirror of
https://github.com/facebookresearch/fairseq.git
synced 2024-09-11 17:25:31 +03:00
Summary: # Before submitting - [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements) - [x] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/master/CONTRIBUTING.md)? - [x] Did you make sure to update the docs? - [x] Did you write any new necessary tests? ## What does this PR do? Implements the multiply_factor optimization used in memory efficient fp16 training to mixed precision training. The methods multiply_grads and clip_grad_norm do not touch each gradient, but rather a "multiply factor" that is then factored in when unscaling gradients. ## PR review Anyone in the community is free to review the PR once the tests have passed. If we didn't discuss your PR in Github issues there's a high chance it will not be merged. ## Did you have fun? Make sure you had fun coding � Pull Request resolved: https://github.com/fairinternal/fairseq-py/pull/1248 Reviewed By: myleott Differential Revision: D23201396 Pulled By: andersonic fbshipit-source-id: 6c6f64542893e0ecac72e132464bb334dcb9874d |
||
---|---|---|
.. | ||
gpu | ||
speech_recognition | ||
__init__.py | ||
test_average_checkpoints.py | ||
test_backtranslation_dataset.py | ||
test_binaries.py | ||
test_bmuf.py | ||
test_character_token_embedder.py | ||
test_concat_dataset.py | ||
test_convtbc.py | ||
test_dictionary.py | ||
test_export.py | ||
test_file_io.py | ||
test_fp16_optimizer.py | ||
test_inference_dropout.py | ||
test_iterators.py | ||
test_label_smoothing.py | ||
test_lstm_jitable.py | ||
test_memory_efficient_fp16.py | ||
test_metrics.py | ||
test_multi_corpus_sampled_dataset.py | ||
test_multihead_attention.py | ||
test_noising.py | ||
test_reproducibility.py | ||
test_resampling_dataset.py | ||
test_sequence_generator.py | ||
test_sequence_scorer.py | ||
test_sparse_multihead_attention.py | ||
test_token_block_dataset.py | ||
test_train.py | ||
test_utils.py | ||
utils.py |