Summary:
https://github.com/facebookresearch/fairseq/issues/4501
# Before submitting
- [x] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [x] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [x] Did you make sure to update the docs?
- [x] Did you write any new necessary tests?
## What does this PR do?
Fixes https://github.com/facebookresearch/fairseq/issues/4501 .
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
Pull Request resolved: https://github.com/facebookresearch/fairseq/pull/4511
Reviewed By: cbalioglu
Differential Revision: D37415937
Pulled By: dianaml0
fbshipit-source-id: bed8cd8e6db23f161fc96e32a0b14535b2373a43
Summary:
Pull Request resolved: https://github.com/facebookresearch/fairseq/pull/4513
With some fixes to torchscript using dual copies.
Reland this diff.
Reviewed By: erichan1
Differential Revision: D37371293
fbshipit-source-id: 4fcfc4083955b6f5fc4ef8600f1b517b6ba69aae
Summary:
Alternatively, we could pin a version of omegaconf
# Before submitting
- [X ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements) No
- [X ] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)? Yes
- [X ] Did you make sure to update the docs? N/A
- [ X] Did you write any new necessary tests? N/A
## What does this PR do?
No issue opened, but noticed when running torchbenchmark
## PR review
## Did you have fun?
The most fun you can have with your clothes on
Pull Request resolved: https://github.com/facebookresearch/fairseq/pull/4440
Reviewed By: dianaml0
Differential Revision: D36694085
Pulled By: alexeib
fbshipit-source-id: 1175a757868aa0495644e3c782a4321e4ac85169
Summary:
Explicitly convert sequence length to int. Sometimes the sequence length is float tensor.
Here is the exception that I was getting:
Traceback (most recent call last):
File "/data/sandcastle/boxes/fbsource/fbcode/buck-out/dev/gen/aab7ed39/deeplearning/projects/fairseq-py/hydra_train#link-tree/torch/multiprocessing/spawn.py", line 69, in _wrap
fn(i, *args)
File "/data/sandcastle/boxes/fbsource/fbcode/buck-out/dev/gen/aab7ed39/deeplearning/projects/fairseq-py/hydra_train#link-tree/fairseq/distributed/utils.py", line 328, in distributed_main
main(cfg, **kwargs)
File "/data/sandcastle/boxes/fbsource/fbcode/buck-out/dev/gen/aab7ed39/deeplearning/projects/fairseq-py/hydra_train#link-tree/fairseq_cli/train.py", line 190, in main
valid_losses, should_stop = train(cfg, trainer, task, epoch_itr)
File "/usr/local/fbcode/platform010/lib/python3.8/contextlib.py", line 75, in inner
return func(*args, **kwds)
File "/data/sandcastle/boxes/fbsource/fbcode/buck-out/dev/gen/aab7ed39/deeplearning/projects/fairseq-py/hydra_train#link-tree/fairseq_cli/train.py", line 330, in train
valid_losses, should_stop = validate_and_save(
File "/data/sandcastle/boxes/fbsource/fbcode/buck-out/dev/gen/aab7ed39/deeplearning/projects/fairseq-py/hydra_train#link-tree/fairseq_cli/train.py", line 421, in validate_and_save
valid_losses = validate(cfg, trainer, task, epoch_itr, valid_subsets)
File "/data/sandcastle/boxes/fbsource/fbcode/buck-out/dev/gen/aab7ed39/deeplearning/projects/fairseq-py/hydra_train#link-tree/fairseq_cli/train.py", line 505, in validate
trainer.valid_step(sample)
File "/usr/local/fbcode/platform010/lib/python3.8/contextlib.py", line 75, in inner
return func(*args, **kwds)
File "/data/sandcastle/boxes/fbsource/fbcode/buck-out/dev/gen/aab7ed39/deeplearning/projects/fairseq-py/hydra_train#link-tree/fairseq/trainer.py", line 1157, in valid_step
logging_output = self._reduce_and_log_stats(logging_outputs, sample_size)
File "/data/sandcastle/boxes/fbsource/fbcode/buck-out/dev/gen/aab7ed39/deeplearning/projects/fairseq-py/hydra_train#link-tree/fairseq/trainer.py", line 1527, in _reduce_and_log_stats
logging_output = agg.get_smoothed_values()
File "/data/sandcastle/boxes/fbsource/fbcode/buck-out/dev/gen/aab7ed39/deeplearning/projects/fairseq-py/hydra_train#link-tree/fairseq/logging/meters.py", line 300, in get_smoothed_values
[
File "/data/sandcastle/boxes/fbsource/fbcode/buck-out/dev/gen/aab7ed39/deeplearning/projects/fairseq-py/hydra_train#link-tree/fairseq/logging/meters.py", line 301, in <listcomp>
(key, self.get_smoothed_value(key))
File "/data/sandcastle/boxes/fbsource/fbcode/buck-out/dev/gen/aab7ed39/deeplearning/projects/fairseq-py/hydra_train#link-tree/fairseq/logging/meters.py", line 293, in get_smoothed_value
return meter.fn(self)
File "/data/sandcastle/boxes/fbsource/fbcode/buck-out/dev/gen/aab7ed39/deeplearning/projects/fairseq-py/hydra_train#link-tree/fairseq/tasks/translation.py", line 438, in compute_bleu
bleu = comp_bleu(
File "/data/sandcastle/boxes/fbsource/fbcode/buck-out/dev/gen/aab7ed39/deeplearning/projects/fairseq-py/hydra_train#link-tree/sacrebleu/metrics/bleu.py", line 282, in compute_bleu
return BLEUScore(score, correct, total, precisions, bp, sys_len, ref_len)
File "/data/sandcastle/boxes/fbsource/fbcode/buck-out/dev/gen/aab7ed39/deeplearning/projects/fairseq-py/hydra_train#link-tree/sacrebleu/metrics/bleu.py", line 103, in _init_
self._verbose += f"ratio = {self.ratio:.3f} hyp_len = {self.sys_len:d} "
File "/data/sandcastle/boxes/fbsource/fbcode/buck-out/dev/gen/aab7ed39/deeplearning/projects/fairseq-py/hydra_train#link-tree/torch/_tensor.py", line 664, in _format_
return self.item().__format__(format_spec)
ValueError: Unknown format code 'd' for object of type 'float'
Differential Revision: D37377076
fbshipit-source-id: 2e7bb22018a446fdc8d223d8c81bea742a249ec7
Summary:
One line change at transformer decoder to enable jit script of 8 bit dynamic quantization of embedding for translation model.
f351446639. Error replicated at Bento: N2127304
{F745783045}
Add option for translation model to export model with/without embedding quantization.
Reviewed By: theweiho
Differential Revision: D37334811
fbshipit-source-id: a95bbe70f5eba6445c2e40751bdfac24396b710c
Summary: fairseq patches a omegaconf internal util function that no longer exists in OmegaConf 2.2. This is a fix to make it compatible with both versions.
Reviewed By: dianaml0
Differential Revision: D37323720
fbshipit-source-id: 1b15b86decc70776303afe4a9a4c63acfef27ffc
Summary:
Context: https://fburl.com/7vdj7vhl
Backing out due to breaking our TorchScript test:
```
RuntimeError:
method cannot be used as a value:
File "/dev/shm/uid-30041/54641b26-seed-nspid4026533396_cgpid7154327-ns-4026533393/fairseq/modules/transformer_layer.py", line 307
self.in_proj_weight,
self.in_proj_bias,
self.self_attn.out_proj.weight,
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
self.self_attn.out_proj.bias,
self.activation_relu_or_gelu == 2,
Stack trace:
Exception type: torch::jit::ErrorReport
```
https://fburl.com/sandcastle/4pzqemf5
Original commit changeset: 984266f850fc
Original Phabricator Diff: D37082681 (3a757d7ab2)
Differential Revision: D37303846
fbshipit-source-id: 1757ea5dae98be5beb4d08f70b0c3001d6ea336f
Summary:
this PR reuses a dataloader instead of recreating it for every epoch. Creating a dataloader is expensive, and also leads to crashes with latest pytorch when spawning 1 task per gpu (e.g. using slurm). Otherwise, the behavior should remain the same.
X-link: https://github.com/fairinternal/fairseq-py/pull/3442
Reviewed By: arbabu123
Differential Revision: D37162558
Pulled By: alexeib
fbshipit-source-id: ea4dd0955dbf01d74aebb067116d7e38b60a031b
Summary:
Pull Request resolved: https://github.com/facebookresearch/fairseq/pull/4480
as titled and depends on D36057338
Fork the inference path inside the forward function. If loaded the checkpoint file and perform the inference, we will deploy BT. Otherwise, fairseq take the position.
In summary:
Accuracy: accuracy loss due to the fp16, the maximum diff is around 0.009. If we set it to fp32, there is no accuracy loss
Perf: the current fairseq has similar speed as vanilla version. After the enablement, the speedup is similar to standalone BT test.
With batch size=64
For V100, the speedup reaches to 1.23x
For A100, the speedup reaches to 1.38x
After enable nested tensor,
For V100, the speedup reaches to 2.46x
Reviewed By: mikekgfb
Differential Revision: D37082681
fbshipit-source-id: 984266f850fc30603e48be56e41ac2c67da080f5
Summary:
Because of the way language ID tokens were introduced for training 1:N MT models, we sometimes see the artifact of the model producing language tokens in the output (see T119348697 for details). This change prevents that by simply zeroing out their probabilities during beam search.
Edit: it turns out that the majority of cases of target language token appearing in the output are due to UNK replacement, where the language ID in source prefix was the "most-attended-to" source token at the step where the UNK was produced. Thus, this change zeros out the attention weights for prefix tokens in the source sequence.
Reviewed By: theweiho
Differential Revision: D36282175
fbshipit-source-id: fb7b2bfd3a8c1c66563ea509e68ab742a831ba4a
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [ x ] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ x ] Did you make sure to update the docs?
- [ x ] Did you write any new necessary tests?
## What does this PR do?
include wav2vec-u 2.0
!!! TODO !!! update title/link of paper in readme
X-link: https://github.com/fairinternal/fairseq-py/pull/2826
Reviewed By: michaelauli
Differential Revision: D37162174
Pulled By: alexeib
fbshipit-source-id: b985ebb9bb94c25d30b6fc53d8c79088cb9798f9
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [ ] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
## What does this PR do?
Had to rerun release, created new PR.
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
Pull Request resolved: https://github.com/facebookresearch/fairseq/pull/4486
Reviewed By: cbalioglu
Differential Revision: D37110232
Pulled By: dianaml0
fbshipit-source-id: e5ce9d8c4d37a27e256a17458808312ab3b02b53
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [ ] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
## What does this PR do?
Fixes # (issue).
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
Pull Request resolved: https://github.com/facebookresearch/fairseq/pull/4483
Reviewed By: cbalioglu
Differential Revision: D37098170
Pulled By: dianaml0
fbshipit-source-id: 6413fe0d03586e8b8b81701284cc010eafce7e4a
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [ ] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
## What does this PR do?
Fixes # (issue).
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
Pull Request resolved: https://github.com/facebookresearch/fairseq/pull/4475
Reviewed By: anupambhatnagar
Differential Revision: D37081823
Pulled By: dianaml0
fbshipit-source-id: b5167aa66e0b873c5f466e1b16a27ca779802780
Summary:
One command line was missing and it broke the test.
It will address issue reported in https://github.com/facebookresearch/fairseq/issues/4462
Reviewed By: kahne
Differential Revision: D37043373
fbshipit-source-id: e3f14b6aa1cbb603e4b23dfbaf8f57564b73d470
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [ ] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
## What does this PR do?
Fixes # (issue).
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
Pull Request resolved: https://github.com/facebookresearch/fairseq/pull/4476
Reviewed By: shruti-bh
Differential Revision: D37072342
Pulled By: dianaml0
fbshipit-source-id: c5343a766641d3939e3af1b61b62d8a03f4dae28
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [ ] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
## What does this PR do?
Fixes # (issue).
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
Pull Request resolved: https://github.com/facebookresearch/fairseq/pull/4473
Reviewed By: cbalioglu
Differential Revision: D37052250
Pulled By: dianaml0
fbshipit-source-id: e5e4c38a9108c769953ef2202c7adb8aa335771a
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [ ] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
## What does this PR do?
Fixes # (issue).
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
Pull Request resolved: https://github.com/facebookresearch/fairseq/pull/4472
Reviewed By: cbalioglu
Differential Revision: D37047097
Pulled By: dianaml0
fbshipit-source-id: 7cc67bb6661a393d7e5d15f1be7d35646cef5181
Summary: New regression loss for end point detection study
Differential Revision: D36887730
fbshipit-source-id: d76eec81f686ecb2630f3a8be3aafdc7d222af61
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [ ] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
## What does this PR do?
Fixes # (issue).
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
Pull Request resolved: https://github.com/facebookresearch/fairseq/pull/4466
Reviewed By: cbalioglu
Differential Revision: D37040595
Pulled By: dianaml0
fbshipit-source-id: b834594c047fa786111a0ade81a7c7e4e2909625
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [ ] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
## What does this PR do?
Fixes # (issue).
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
Pull Request resolved: https://github.com/facebookresearch/fairseq/pull/4471
Reviewed By: cbalioglu
Differential Revision: D37040680
Pulled By: dianaml0
fbshipit-source-id: f2ab6ab4dd410ea7784fe94d1386bccfd1956909
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [ ] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
## What does this PR do?
Automates release process and allows it to be triggered directly. Heavily inspired by fairscale's release setup. Few improvements in followup PR.
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
Pull Request resolved: https://github.com/facebookresearch/fairseq/pull/4455
Reviewed By: cbalioglu
Differential Revision: D36993777
Pulled By: dianaml0
fbshipit-source-id: bfa9663c3a7d20dd7ebf690e182d7f8102328c08
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [ ] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
## What does this PR do?
Build wheels was broken.
- [x] get build wheels to work successfully
- [x] update the fairseq version
- [x] Bump up the python version
- [x] skip musllinux builds
followup improvements incoming in another PR
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
Pull Request resolved: https://github.com/facebookresearch/fairseq/pull/4206
Reviewed By: cbalioglu
Differential Revision: D36880560
Pulled By: dianaml0
fbshipit-source-id: 91c538e785c5ed107b17b8093afa1d3ce387f56a
Summary:
First part of the diff is the copy of https://github.com/fairinternal/fairseq-py/pull/3308 with slight change: now check arg where 'decoder' appears anywhere in the arg in order to catch args like '--share-decoder-input-output-embed`.
This is how it looks like in the log:
```
2022-05-20 16:19:52 | WARNING | fairseq.models.speech_to_text.xm_transformer | Overriding decoder arg decoder_normalize_before: from False to True
2022-05-20 16:19:52 | WARNING | fairseq.models.speech_to_text.xm_transformer | Overriding decoder arg decoder_layers: from 12 to 6
2022-05-20 16:19:52 | WARNING | fairseq.models.speech_to_text.xm_transformer | Overriding decoder arg decoder_attention_heads: from 16 to 12
2022-05-20 16:19:52 | WARNING | fairseq.models.speech_to_text.xm_transformer | Overriding decoder arg decoder_embed_dim: from 1024 to 768
2022-05-20 16:19:52 | WARNING | fairseq.models.speech_to_text.xm_transformer | Overriding decoder arg decoder_ffn_embed_dim: from 4096 to 3072
2022-05-20 16:19:52 | WARNING | fairseq.models.speech_to_text.xm_transformer | Overriding decoder arg share_decoder_input_output_embed: from False to True
2022-05-20 16:19:52 | WARNING | fairseq.models.speech_to_text.xm_transformer | Overriding decoder arg decoder_output_dim: from 1024 to 768
2022-05-20 16:19:52 | WARNING | fairseq.models.speech_to_text.xm_transformer | Overriding decoder arg decoder_input_dim: from 1024 to 768
```
Second part of the diff is about `--remove-weight-norm` encoder argument. Weight norm happens only *once* in w2v. It breaks any code which tries to use deepcopy of the model such as stochastic weight averaging or exponential moving average (EMA) of the model. Removing weightnorm has no effect on the weights, so it does not corrupt model weights.
Example log:
```
2022-05-20 16:06:09 | WARNING | fairseq.models.speech_to_text.xm_transformer | Removing weight norm from wav2vec encoder
2022-05-20 16:06:09 | WARNING | fairseq.models.speech_to_text.xm_transformer | Weight norm removed from module with Conv1d(1024, 1024, kernel_size=(128,), stride=(1,), padding=(64,), groups=16)
```
Gpu tests are passed for this diff.
X-link: https://github.com/fairinternal/fairseq-py/pull/3400
Reviewed By: kahne
Differential Revision: D36785559
Pulled By: uralik
fbshipit-source-id: 053a2f79549ceeb1357f09d1aefab77ac5581c20
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [ ] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
## What does this PR do?
Fixes # (issue).
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
Pull Request resolved: https://github.com/facebookresearch/fairseq/pull/4218
Reviewed By: cbalioglu
Differential Revision: D36681610
Pulled By: dianaml0
fbshipit-source-id: fb47c1df91cf73eabe082783c852a6a66b8831e3
Summary:
Add a link and bibtex citation to the X-MOD paper
X-link: https://github.com/fairinternal/fairseq-py/pull/3386
Reviewed By: ngoyal2707
Differential Revision: D36377376
Pulled By: artetxem
fbshipit-source-id: 84540b90dbea8502ba5fecfcbdb60164e4fb801e
Summary:
fixes memory leak in ema module by making sure the update happens in no_grad regime
X-link: https://github.com/fairinternal/fairseq-py/pull/3384
Reviewed By: arbabu123
Differential Revision: D36352890
Pulled By: alexeib
fbshipit-source-id: 0f3575ac356a13483e00ed431375b2c798621a3a
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [ ] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
## What does this PR do?
Fixes failing build error.
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
Pull Request resolved: https://github.com/pytorch/fairseq/pull/4418
Reviewed By: yuntang
Differential Revision: D36325742
Pulled By: dianaml0
fbshipit-source-id: d68d04dc01eac0f776ba06534cdf846d9d45fe0b
Summary:
Release code and models from the paper "Lifting the Curse of Multilinguality by Pre-training Modular Transformers".
X-link: https://github.com/fairinternal/fairseq-py/pull/3333
Reviewed By: ngoyal2707
Differential Revision: D36319556
Pulled By: artetxem
fbshipit-source-id: 9e373fa8811da28e78328102b8224f9e0d05ab9e
Summary:
Follow up to diff D35864490
Currently speech_to_text task only supports prepending language tag but we append the language tag in unit mBART. To support that use case here,
I add an option called 'prepend_bos_and_append_tgt_lang_tag' to S2TDataConfig to support the case where to want to prepend the BOS token and append the language tag.
Update the sequence generator to read the tgt_lang tag from the data config and replace EOS token with it during inference.
Example config file for S2UT pretraining with wav2vec 2.0 encoder and unit mBART looks like
```
audio_root: $AUDIO_ROOT
standardize_audio: true
use_audio_input: true
vocab_filename: dict_1003_unitmbart.txt
prepend_bos_and_append_tgt_lang_tag: true
eos_token: <lang:en>
```
X-link: https://github.com/fairinternal/fairseq-py/pull/3353
Reviewed By: kahne
Differential Revision: D36112014
Pulled By: sravyapopuri388
fbshipit-source-id: 56294d8fd6071ad6e95c797bd221467e4cf3d035
Summary:
1. Add joint pre-training scripts
2. Replace prepend_tgt_lang_tag_no_change with prepend_tgt_lang_tag_as_bos
3. Add readme for the joint pre-training
4. Add test case for the Librispeech model
Reviewed By: hygong-fb
Differential Revision: D36300953
fbshipit-source-id: cb749689787ed97c1250d122bdefb7f7a2252292
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [ ] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
## What does this PR do?
Blocksparse attention no longer accepts masks.
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
Pull Request resolved: https://github.com/pytorch/fairseq/pull/4401
Reviewed By: blefaudeux
Differential Revision: D36208195
Pulled By: dianaml0
fbshipit-source-id: 0d0c57533cb9346724e8e8b0b9c28a2e57759135
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [ ] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
## What does this PR do?
Fixes # (issue).
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
Pull Request resolved: https://github.com/pytorch/fairseq/pull/4402
Reviewed By: xwhan
Differential Revision: D36208103
Pulled By: dianaml0
fbshipit-source-id: 1600356d20dc32340935c0c88c1f700a1cdefa14
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [ ] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
## What does this PR do?
- [x] formatting fix
- [x] optional import of xFormers
- [x] enabled doc building as part of CI
- [x] remove mask arguments for attentions that do not support them
- [x] remove masks for blocksparse tests, no longer supported
- [ ] use pytest instead of deprecated `setup.py test`
- [ ] CircleCI xFormers tests
Will submit without the last two done to unblock people using the repo
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
X-link: https://github.com/fairinternal/fairseq-py/pull/3362
Reviewed By: blefaudeux
Differential Revision: D36169572
Pulled By: dianaml0
fbshipit-source-id: 3b20ae5f377144a0854e016771af703f0d0d694b
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [x] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/master/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [x] Did you write any new necessary tests?
## What does this PR do?
This PR is a cleaned up version of https://github.com/fairinternal/fairseq-py/issues/2138. It is based on the `main` branch instead of the `gshard` branch. Removed call to xFormers MultiHeadDispatch, only using xFormers Attention.
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
X-link: https://github.com/fairinternal/fairseq-py/pull/2263
Reviewed By: blefaudeux
Differential Revision: D33800377
Pulled By: dianaml0
fbshipit-source-id: 658d52214c782212b12881b30c4d908a763b4cf2
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [ ] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
## What does this PR do?
Fixes # (issue).
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
X-link: https://github.com/fairinternal/fairseq-py/pull/3350
Reviewed By: shruti-bh
Differential Revision: D36009526
Pulled By: dianaml0
fbshipit-source-id: 9cdc3d53086b8d40a780bcb64cfe28108091ab98
Summary:
# Before submitting
- [x] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [x] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [x] Did you make sure to update the docs?
- [x] Did you write any new necessary tests?
## What does this PR do?
Fixes https://github.com/pytorch/fairseq/issues/4302 (issue).
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
I had fun when I figured out why torchrun was failing :)
Pull Request resolved: https://github.com/pytorch/fairseq/pull/4351
Reviewed By: shruti-bh
Differential Revision: D35784181
Pulled By: dianaml0
fbshipit-source-id: 560c7af12b2f9278cba6c85711b98b9e043d0ec9
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [ ] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
## What does this PR do?
Pulling out some changes from https://github.com/fairinternal/fairseq-py/pull/2263 unrelated to xformers to make the PR cleaner
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
X-link: https://github.com/fairinternal/fairseq-py/pull/3068
Reviewed By: blefaudeux
Differential Revision: D34149016
Pulled By: dianaml0
fbshipit-source-id: 6442a5f451d56cc47106227298a624516b19a9ad
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [ ] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
## What does this PR do?
Breaking build.
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
X-link: https://github.com/fairinternal/fairseq-py/pull/3346
Reviewed By: anchit
Differential Revision: D35979333
Pulled By: dianaml0
fbshipit-source-id: 929e1d4a0b94d7f214646a5d5c226a790c563573
