Summary:
Pull Request resolved: https://github.com/facebookresearch/fairseq/pull/4513
With some fixes to torchscript using dual copies.
Reland this diff.
Reviewed By: erichan1
Differential Revision: D37371293
fbshipit-source-id: 4fcfc4083955b6f5fc4ef8600f1b517b6ba69aae
Summary:
Context: https://fburl.com/7vdj7vhl
Backing out due to breaking our TorchScript test:
```
RuntimeError:
method cannot be used as a value:
File "/dev/shm/uid-30041/54641b26-seed-nspid4026533396_cgpid7154327-ns-4026533393/fairseq/modules/transformer_layer.py", line 307
self.in_proj_weight,
self.in_proj_bias,
self.self_attn.out_proj.weight,
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
self.self_attn.out_proj.bias,
self.activation_relu_or_gelu == 2,
Stack trace:
Exception type: torch::jit::ErrorReport
```
https://fburl.com/sandcastle/4pzqemf5
Original commit changeset: 984266f850fc
Original Phabricator Diff: D37082681 (3a757d7ab2)
Differential Revision: D37303846
fbshipit-source-id: 1757ea5dae98be5beb4d08f70b0c3001d6ea336f
Summary:
Pull Request resolved: https://github.com/facebookresearch/fairseq/pull/4480
as titled and depends on D36057338
Fork the inference path inside the forward function. If loaded the checkpoint file and perform the inference, we will deploy BT. Otherwise, fairseq take the position.
In summary:
Accuracy: accuracy loss due to the fp16, the maximum diff is around 0.009. If we set it to fp32, there is no accuracy loss
Perf: the current fairseq has similar speed as vanilla version. After the enablement, the speedup is similar to standalone BT test.
With batch size=64
For V100, the speedup reaches to 1.23x
For A100, the speedup reaches to 1.38x
After enable nested tensor,
For V100, the speedup reaches to 2.46x
Reviewed By: mikekgfb
Differential Revision: D37082681
fbshipit-source-id: 984266f850fc30603e48be56e41ac2c67da080f5
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [ ] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
## What does this PR do?
fix `black` failures
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
Pull Request resolved: https://github.com/fairinternal/fairseq-py/pull/2816
Reviewed By: alexeib
Differential Revision: D33172615
Pulled By: dianaml0
fbshipit-source-id: 36b141f42941670f1bfa981041d878042feb0428
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [ ] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
## What does this PR do?
reverting to fix issue mentioned [here](https://github.com/pytorch/fairseq/issues/3913). Having another PR for fixing the original issue later.
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
Pull Request resolved: https://github.com/fairinternal/fairseq-py/pull/2763
Reviewed By: myleott
Differential Revision: D33000411
Pulled By: jingfeidu
fbshipit-source-id: 95a54cbdc612129a0eab4b5e6aa576a5bcf00588
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [ ] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
## What does this PR do?
- [x] applies flake8 fixes to main branch (https://github.com/fairinternal/fairseq-py/issues/2546) - still more to be fixed
Fix GPU tests:
- [x] when torch.ao.quantization import doesn't work use torch.quantization
- [x] build apex from earlier commit in circleci so that its compatible with pytorch 1.8 and 1.9
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
Pull Request resolved: https://github.com/fairinternal/fairseq-py/pull/2570
Reviewed By: Mortimerp9
Differential Revision: D32955312
Pulled By: dianaml0
fbshipit-source-id: e163cbd4998f171f819e31b0682c1c0f1986f9e1
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [ ] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
## What does this PR do?
Fixes # (issue).
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
Pull Request resolved: https://github.com/fairinternal/fairseq-py/pull/2678
Reviewed By: Mortimerp9
Differential Revision: D32653381
Pulled By: dianaml0
fbshipit-source-id: 2810d14867cd7d64f4d340740e2b590b82de47fe
Summary:
1. added test for genereting pad tokens during beam search with prefix
tokens
2. modified lprobs for pad token and prefix tokens to avoid generating
pad
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [ ] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/master/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
## What does this PR do?
Fixes # (issue).
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
Pull Request resolved: https://github.com/fairinternal/fairseq-py/pull/2227
Reviewed By: xianxl
Differential Revision: D30649356
Pulled By: jingfeidu
fbshipit-source-id: d94903a912e767391c8fca61f98f65b5cea3b56e
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [x] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/master/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [x] Did you write any new necessary tests?
## What does this PR do?
Fixes https://github.com/pytorch/fairseq/issues/2022.
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
Pull Request resolved: https://github.com/pytorch/fairseq/pull/2090
Reviewed By: cndn
Differential Revision: D21385984
Pulled By: myleott
fbshipit-source-id: 1428e02e625b8625df71a83c05dcf933c3f899df
Summary:
Pull Request resolved: https://github.com/pytorch/fairseq/pull/2059
test_ensemble_sequence_generator and test_export_ensemble_model are green on fbcode master but Pytorch 1.5 release cut happened before the TorchScript fix, so updating the gate to 1.6
Remove quantization test from fairseq as FBGEMMS is binded at OSS side. Will add the test back in fbtranslate but land this first to fix OSS side failures.
Reviewed By: myleott
Differential Revision: D21231873
fbshipit-source-id: 8a2ad7dbed118ca8e3f4c351c399a82fd9740445
Summary: The fix in MHA is suggested by driazati, to avoid JIT compilation for if branch in MHA forward when in scripting. Without this quantization wouldn't work. Details in https://fb.workplace.com/groups/2240361332735959/permalink/626166461295703/
Reviewed By: jhcross
Differential Revision: D20881076
fbshipit-source-id: b50347b45cd7dbdef02ac7b71316ba734019f57e
Summary:
Pull Request resolved: https://github.com/fairinternal/fairseq-py/pull/1127
Pull Request resolved: https://github.com/pytorch/fairseq/pull/1953
Script the `reorder_incremental_states` in the base FairseqModel
Remove the overwrite scriptable `reorder_incremental_states` in the TransformerModel
Change the decoder_len, since len(Tuple) is supported in Script
Relanded reverted diff D20797390
Reviewed By: myleott
Differential Revision: D20896200
fbshipit-source-id: cc4ae34f89f16007656cce6ec6f7e01b13899278
Summary:
Pull Request resolved: https://github.com/fairinternal/fairseq-py/pull/1120
Pull Request resolved: https://github.com/pytorch/fairseq/pull/1940
Deprecate the SequenceGenerator in Fairseq with the Scripted vision.
Pass all integration unit tests
- Copy ScriptSequenceGenerator to SequenceGenerator:
- Modified the forward_decoder to fix bug when using adaptive_softmax in `get_prob_normalize` (marked with the inline comment)
- Add support for other EnsembleModels as input arg (marked with the inline comment)
- Add `FBEnsembleModelWithFork` to support folk/join in ensemblemodel
- Add `test_fb_ensemble_model` to test folk/join feature
- Still have bugs in folk/join feature when running in the Fairseq interface (like generation and interactive). Need further investigation P128130029. cc cndn, jhcross
- Modified SequenceGenerator initialization the interface
- Clear up the codes: delete unused functions `get_normalized_probs` and `_decode`
Reland reverted diff D20685075
Reviewed By: cndn
Differential Revision: D20895977
fbshipit-source-id: 424ee318e67d5d6ffed3edb92c7fa78485ba34af
Summary:
Pull Request resolved: https://github.com/fairinternal/fairseq-py/pull/1127
Pull Request resolved: https://github.com/pytorch/fairseq/pull/1953
Script the `reorder_incremental_states` in the base FairseqModel
Remove the overwrite scriptable `reorder_incremental_states` in the TransformerModel
Change the decoder_len, since len(Tuple) is supported in Script
Reviewed By: myleott
Differential Revision: D20797390
fbshipit-source-id: ab29874973adc5dbd556c591942a0e071c81fc52
Summary:
Pull Request resolved: https://github.com/fairinternal/fairseq-py/pull/1120
Pull Request resolved: https://github.com/pytorch/fairseq/pull/1940
Deprecate the SequenceGenerator in Fairseq with the Scripted vision.
Pass all integration unit tests
- Copy ScriptSequenceGenerator to SequenceGenerator:
- Modified the forward_decoder to fix bug when using adaptive_softmax in `get_prob_normalize` (marked with the inline comment)
- Add support for other EnsembleModels as input arg (marked with the inline comment)
- Add `FBEnsembleModelWithFork` to support folk/join in ensemblemodel
- Add `test_fb_ensemble_model` to test folk/join feature
- Still have bugs in folk/join feature when running in the Fairseq interface (like generation and interactive). Need further investigation P128130029. cc cndn, jhcross
- Modified SequenceGenerator initialization the interface
- Clear up the codes: delete unused functions `get_normalized_probs` and `_decode`
Reviewed By: myleott
Differential Revision: D20685075
fbshipit-source-id: 046b76874465a70d8118a97ad670311c6ce1d1c8
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [x] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/master/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [x] Did you write any new necessary tests?
## What does this PR do?
Fixes https://github.com/pytorch/fairseq/issues/1791.
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
Pull Request resolved: https://github.com/pytorch/fairseq/pull/1792
Reviewed By: jmp84
Differential Revision: D20322704
Pulled By: myleott
fbshipit-source-id: 3cfa1bddda06b966e9dc9bc8ff183009d844b23c
Summary:
This PR implements a new generation strategy that we experimented with in project Pinocchio (https://github.com/fairinternal/Pinocchio), see the paper submission in: https://fburl.com/hduj2me7.
Specifically in this PR:
- added a Diverse Beam Search variant as described in https://arxiv.org/abs/1611.08562
- moved the Search object generation out of `sequence_generation.py`, which allows for limiting the number of kwargs passes around
- made sure the above changes are backward compatible based on grep - P124083926
- added test cases covering these scenarios
Pull Request resolved: https://github.com/fairinternal/fairseq-py/pull/953
Test Plan:
- `python -m unittest tests.test_binaries -v`- including added test cases, see issues below for some details
- `python -m unittest tests.test_sequence_generator -v` - including added test cases
- tested locally in conjunction with the Pinocchio repo
- grepped for all instantiations of `SequenceGeneration`, made sure they're backward compatible
# Issues
- when I try to run all tests with `python -m unittest tests.test_binaries -v` command, the execution gets stuck on `test_binaries.TestTranslation.test_generation` - the test otherwise passes without problems when ran individually. Is this a known problem?
- discovered T59235948 - assigned to fairseq oncall
Reviewed By: myleott, fabiopetroni
Differential Revision: D19142394
Pulled By: ola13
fbshipit-source-id: d24543424c14a9537e7b6485951d9f841da62b07
Summary:
The previous BSD+PATENTS license was controversial. We have been
approved to relicense fairseq under the MIT license.
Pull Request resolved: https://github.com/fairinternal/fairseq-py/pull/786
Differential Revision: D16560654
Pulled By: myleott
fbshipit-source-id: f78b1beb4f2895dd7b9bfc79f5f952a2bfb94034
A Task defines the data format, stores shared state (e.g., dictionaries) and provides helpers for building the model/criterion and calculating the loss.
Changes:
- Add TranslationTask and LanguageModelingTask. New tasks can be registered with @register_task decorator.
- Add EpochBatchIterator to encapsulate batching and saving/restoring dataloader position
- Remove LEFT_PAD_* constants and make them configurable per task
Changes:
- 7d19e36: Add `--sampling` flag to generate.py to sample instead of doing beam search
- c777340: Add `scripts/average_checkpoints.py` to average multiple checkpoints into a combined model
- 3ea882c: Add `--max-update` option to train.py to stop training after a given number of updates
- small bugfixes for distributed training, LSTM, inverse square root LR scheduler
This PR includes breaking API changes to modularize fairseq-py and adds support for distributed training across multiple nodes.
Changes:
- c7033ef: add support for distributed training! See updated README for usage.
- e016299: modularize fairseq-py, adding support for register_model, register_criterion, register_optimizer, etc.
- 154e440: update LSTM implementation to use PackedSequence objects in the encoder, better following best practices and improving perf
- 90c2973 and 1da6265: improve unit test coverage
