Summary:
1. Add joint pre-training scripts
2. Replace prepend_tgt_lang_tag_no_change with prepend_tgt_lang_tag_as_bos
3. Add readme for the joint pre-training
4. Add test case for the Librispeech model
Reviewed By: hygong-fb
Differential Revision: D36300953
fbshipit-source-id: cb749689787ed97c1250d122bdefb7f7a2252292
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [ ] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
## What does this PR do?
Blocksparse attention no longer accepts masks.
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
Pull Request resolved: https://github.com/pytorch/fairseq/pull/4401
Reviewed By: blefaudeux
Differential Revision: D36208195
Pulled By: dianaml0
fbshipit-source-id: 0d0c57533cb9346724e8e8b0b9c28a2e57759135
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [ ] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
## What does this PR do?
Fixes # (issue).
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
Pull Request resolved: https://github.com/pytorch/fairseq/pull/4402
Reviewed By: xwhan
Differential Revision: D36208103
Pulled By: dianaml0
fbshipit-source-id: 1600356d20dc32340935c0c88c1f700a1cdefa14
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [ ] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
## What does this PR do?
- [x] formatting fix
- [x] optional import of xFormers
- [x] enabled doc building as part of CI
- [x] remove mask arguments for attentions that do not support them
- [x] remove masks for blocksparse tests, no longer supported
- [ ] use pytest instead of deprecated `setup.py test`
- [ ] CircleCI xFormers tests
Will submit without the last two done to unblock people using the repo
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
X-link: https://github.com/fairinternal/fairseq-py/pull/3362
Reviewed By: blefaudeux
Differential Revision: D36169572
Pulled By: dianaml0
fbshipit-source-id: 3b20ae5f377144a0854e016771af703f0d0d694b
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [x] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/master/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [x] Did you write any new necessary tests?
## What does this PR do?
This PR is a cleaned up version of https://github.com/fairinternal/fairseq-py/issues/2138. It is based on the `main` branch instead of the `gshard` branch. Removed call to xFormers MultiHeadDispatch, only using xFormers Attention.
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
X-link: https://github.com/fairinternal/fairseq-py/pull/2263
Reviewed By: blefaudeux
Differential Revision: D33800377
Pulled By: dianaml0
fbshipit-source-id: 658d52214c782212b12881b30c4d908a763b4cf2
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [ ] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
## What does this PR do?
Fixes # (issue).
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
X-link: https://github.com/fairinternal/fairseq-py/pull/3350
Reviewed By: shruti-bh
Differential Revision: D36009526
Pulled By: dianaml0
fbshipit-source-id: 9cdc3d53086b8d40a780bcb64cfe28108091ab98
Summary:
# Before submitting
- [x] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [x] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [x] Did you make sure to update the docs?
- [x] Did you write any new necessary tests?
## What does this PR do?
Fixes https://github.com/pytorch/fairseq/issues/4302 (issue).
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
I had fun when I figured out why torchrun was failing :)
Pull Request resolved: https://github.com/pytorch/fairseq/pull/4351
Reviewed By: shruti-bh
Differential Revision: D35784181
Pulled By: dianaml0
fbshipit-source-id: 560c7af12b2f9278cba6c85711b98b9e043d0ec9
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [ ] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
## What does this PR do?
Pulling out some changes from https://github.com/fairinternal/fairseq-py/pull/2263 unrelated to xformers to make the PR cleaner
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
X-link: https://github.com/fairinternal/fairseq-py/pull/3068
Reviewed By: blefaudeux
Differential Revision: D34149016
Pulled By: dianaml0
fbshipit-source-id: 6442a5f451d56cc47106227298a624516b19a9ad
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [ ] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
## What does this PR do?
Breaking build.
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
X-link: https://github.com/fairinternal/fairseq-py/pull/3346
Reviewed By: anchit
Differential Revision: D35979333
Pulled By: dianaml0
fbshipit-source-id: 929e1d4a0b94d7f214646a5d5c226a790c563573
Summary:
Implements beamable encoder-decoder cross attention. This removes the need to duplicate the encoder states beam_size # of times during inference. Which gives both a big memory improvement enabling larger batch sizes while on GPU and also compute efficiency by greatly reducing time spent in reorder_encoder_out.
This is inspired from work in [fastseq](https://arxiv.org/abs/2106.04718) which has more in-depth analysis.
There was an old [PR](https://github.com/pytorch/fairseq/pull/1958) for fairseq as well to implement this feature but was not merged and eventually closed. I revive+refactor that PR and also add support for dynamically changing the beam_size while calling `hub_interface.generate()`
## Benchmarking
**CPU Performance** (On-demand devserver)
batch size: 1 | beam size: 4
50.4s/it -> 22.3s/it | **2.25X Speedup**
batch size: 2 | beam size: 4
53.1s/it -> 25.8s/it | **2.06X Speedup**
batch size: 1 | beam size: 8
65.8s/it -> 23.8s/it | **2.76X Speedup**
**GPU Performance**
Reported in detail [here](https://github.com/pytorch/fairseq/issues/1957)
Currently this optimization is only enabled for our custom BART model used in the workplace summarization demo to unblock landing this fast.
This should be up-streamed to TransformerModel after syncing with fairseq folk.
Reviewed By: xwhan
Differential Revision: D35722467
fbshipit-source-id: a420f73ff5b9ec0cdf40c59464b6ed1794114906
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [ ] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
## What does this PR do?
Fixes # (issue).
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
X-link: https://github.com/fairinternal/fairseq-py/pull/3320
Reviewed By: an918tw
Differential Revision: D35752980
Pulled By: sravyapopuri388
fbshipit-source-id: da59d0621f6fa5d981701802f69a89495bcb9599
Summary:
# Before submitting
- [X] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [X] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [X] Did you make sure to update the docs?
- [X] Did you write any new necessary tests?
## What does this PR do?
Fixes https://github.com/pytorch/fairseq/issues/4300
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Big time!
Note:
I had to update `black` because of [this known issue](https://github.com/psf/black/issues/2964):
```
black....................................................................Failed
- hook id: black
- exit code: 1
Traceback (most recent call last):
File "/Users/azzhipa/.cache/pre-commit/repoxt83whf2/py_env-python3.8/bin/black", line 8, in <module>
sys.exit(patched_main())
File "/Users/azzhipa/.cache/pre-commit/repoxt83whf2/py_env-python3.8/lib/python3.8/site-packages/black/__init__.py", line 1423, in patched_main
patch_click()
File "/Users/azzhipa/.cache/pre-commit/repoxt83whf2/py_env-python3.8/lib/python3.8/site-packages/black/__init__.py", line 1409, in patch_click
from click import _unicodefun
ImportError: cannot import name '_unicodefun' from 'click' (/Users/azzhipa/.cache/pre-commit/repoxt83whf2/py_env-python3.8/lib/python3.8/site-packages/click/__init__.py)
```
Pull Request resolved: https://github.com/pytorch/fairseq/pull/4344
Reviewed By: zhengwy888
Differential Revision: D35691648
Pulled By: dianaml0
fbshipit-source-id: 4bdf408bc9d9cca76c9c08e138cf85b1d00d14d4
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [x] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/master/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [x] Did you write any new necessary tests?
## What does this PR do?
Fixes a bug in the no_overlap case when computing mask indices for wav2vec
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
Pull Request resolved: https://github.com/pytorch/fairseq/pull/3266
Reviewed By: arbabu123
Differential Revision: D35704063
Pulled By: alexeib
fbshipit-source-id: 3b77dc4cc50e539b57e6ad5f38f59eb975356adb
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [x] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
## What does this PR do?
Fixes typo
## PR review
## Did you have fun?
Pull Request resolved: https://github.com/pytorch/fairseq/pull/4334
Reviewed By: Mortimerp9
Differential Revision: D35503972
Pulled By: dianaml0
fbshipit-source-id: 09893de009d398e7a048ec89f757634ddc10139d
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [ ] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
## What does this PR do?
Fixes # (issue).
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
X-link: https://github.com/fairinternal/fairseq-py/pull/3280
Reviewed By: jmp84
Differential Revision: D35451365
Pulled By: sravyapopuri388
fbshipit-source-id: 691480e2f568922c1bf29f5d109dc042c0588a67
Summary:
Detecting repeated ngram is currently super slow in fairseq. I discovered this while reading the [fastseq paper](https://arxiv.org/abs/2106.04718)
While this was partially solved in fairseq by borrowing their optimized cuda kernel in [PR](https://github.com/fairinternal/fairseq-py/pull/1509) there was no optimization made for the CPU case. Moreover most users (including me) don't know about this obscure ngram kernel and how to compile it. Also the kernel isn't torchscriptable
Diving through the fastseq code i discovered this [PR](https://github.com/microsoft/fastseq/pull/18) and re-implemented the same optimization. This does away with slow dictionaries and relies on much faster lists as well as simplifies the code.
# Performance Benchmarking
We get **1.7X** improvement in E2E inference throughput without scripting and **3.2X** with scripting
Data/Task: summarization task (BookSum)
Hardware: A100
(I used batch size 5, expect much larger gains with larger batch sizes)
Without torchscripting
**Before: 15.7s/it**
**After: 9.5s/it**
**With kernel: 9.3s/it**
With torchscripted NGramRepeatBlock
**Before: 32.99s/it
After: 10.1s/it**
The dictionary handling in torchscript is especially slow hurting the existing implementation by 2 times! The new one doesn't suffer as much of a slowdown
This new implementation comes very close to the optimized CUDA kernel but works on CPU and supports torchscripting.
Reviewed By: xwhan
Differential Revision: D35517508
fbshipit-source-id: 4fd9dcbc0076064601af0621b76113b70835fb02
Summary:
Make label_rate be of type float in Hubert pretraining to support decimal label rate (e.g. 33.3Hz, otherwise verify_label_lengths() will give warnings if the undelying label rate is 33.3Hz)
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [x] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [x] Did you write any new necessary tests?
## What does this PR do?
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
Pull Request resolved: https://github.com/pytorch/fairseq/pull/3937
Reviewed By: zhengwy888
Differential Revision: D31489119
Pulled By: dianaml0
fbshipit-source-id: 3f9fa76b0fb07affbb947d5c7c09b6e48fbba231
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [x] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
## What does this PR do?
Enables logging of params and metrics with Aim. Aim is an open-source experiment tracker - https://github.com/aimhubio/aim
1. Added two arguments to CommonConfig:
- aim_repo: defines Aim repository location, can be set to remote URL as well(i.e. `aim://<ip>:<port>`)
- aim_run_hash: defines run hash. If skipped, run will be created or continued based on `save_dir` argument. If there is an existing run which has the same `save_dir`, it will be reopened/continued, otherwise a new run will be created.
2. Implemented AimProgressBarWrapper class to handle logging
Pull Request resolved: https://github.com/pytorch/fairseq/pull/4311
Reviewed By: ArmenAg
Differential Revision: D35177412
Pulled By: dianaml0
fbshipit-source-id: 287afe3a77e1048e497a4e1bdc42efd46ec9c2fe
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [ ] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
## What does this PR do?
Fixes # (issue).
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
Pull Request resolved: https://github.com/pytorch/fairseq/pull/4313
Reviewed By: shruti-bh
Differential Revision: D35200613
Pulled By: dianaml0
fbshipit-source-id: c011f89f4a7ee9404bec61728b52fcea8640d292
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [ ] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
## What does this PR do?
Fix issue with `black` causing build error.
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
Pull Request resolved: https://github.com/pytorch/fairseq/pull/4310
Reviewed By: shruti-bh
Differential Revision: D35151101
Pulled By: dianaml0
fbshipit-source-id: 63d80b848fdd3c004d784add3bf74e4c5281e952
Summary:
Releasing pre-trained mHuBERT, vocoder, speech normalizer for the paper "Textless Speech-to-Speech Translation on Real Data"
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [ ] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
## What does this PR do?
Fixes # (issue).
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
X-link: https://github.com/fairinternal/fairseq-py/pull/3245
Reviewed By: sravyapopuri388
Differential Revision: D35135891
Pulled By: an918tw
fbshipit-source-id: 96e0a6354dc61d5cbfce9943893bebadfb21b642
Summary:
As per title
Created from CodeHub with https://fburl.com/edit-in-codehub
Reviewed By: arbabu123
Differential Revision: D35151134
fbshipit-source-id: bb97ae583542c8e7983b9d9042d8a3084b8fbef5
Summary:
OSS "Enhanced Direct Speech-to-Speech Translation Using Self-supervised Pre-training and Data Augmentation" paper code
- Update xm_transformer to add a new arguments called encoder_proj (which ensures the encoder embedding dim and decoder embedding dim are matched) and max_positions (related to embedding size of conformer).
- Add documentation and pretrained models related to the paper
X-link: https://github.com/fairinternal/fairseq-py/pull/3233
Reviewed By: pipibjc
Differential Revision: D35119604
Pulled By: sravyapopuri388
fbshipit-source-id: bbe517c4803c5808f8cce0e5d16cf5ffa96f425c
Summary:
Per anj-s 's suggestion - this seems to fix the
```
assert len(self.flat_params) == 1, "Incorrect access to flat_param"
AssertionError: Incorrect access to flat_param
```
error when training transformer models w/ large number of params
~~(not sure why the number of params affect fairscale FSDP wrapping???)~~ Did this maybe only manifest when the encoder/decoder individually had > 1e8 params due to the default of `min_params_to_wrap`?
Looking at D26771144 (656d7e5779) & https://github.com/fairinternal/fairseq-py/pull/1667 where this code was added - it's unclear why wrapping was specifically necessary when share_all_embeddings=False? Is it OK to just delete this code?
(And did the gshard model avoid this issue b/c it used share_all_embeddings=True?)
Reviewed By: huihuifan
Differential Revision: D35084649
fbshipit-source-id: ad5b394c9920e3bea2767a0771f6de36aecb3687
Summary: Replace "prepend-tgt-lang-tag" with "prepend-tgt-lang-tag-as-bos" in s2s data loading and s2s task.
Reviewed By: yuntang
Differential Revision: D34912239
fbshipit-source-id: 654d0eafafc275be6c2470b08a323f57a4f9b9cb
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [ ] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
## What does this PR do?
Support tgt-lang-tag in speech-to-speech task.
1. If we set prepend_tgt_lang_tag: true, a dictionary with units and lang tags would be loaded from vocab_filename; otherwise, a dictionary is created with units only in setup_task.
2. prepend_tgt_lang_tag would add the target language token to the beginning of prev_output_tokens during data loading.
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
X-link: https://github.com/fairinternal/fairseq-py/pull/3187
Reviewed By: yuntang
Differential Revision: D34768755
Pulled By: hygong-fb
fbshipit-source-id: fa395c3319907221f95333283689671b194f3ccc
Summary:
Our mission at Meta Open Source is to empower communities through open source, and we believe that it means building a welcoming and safe environment for all. As a part of this work, we are adding this banner in support for Ukraine during this crisis.
Pull Request resolved: https://github.com/pytorch/fairseq/pull/4249
Reviewed By: arbabu123
Differential Revision: D34635479
Pulled By: dmitryvinn-fb
fbshipit-source-id: 488d30f0967ae9542ead968c5cb951ecf0e02a64
Summary:
## What does this PR do?
Avoid throwing ValueError when attempting to load a user defined module from common.user_dir that has the same module name and same module path as some loaded module. This occurs when a job is preempted and restarts using submitit_slurm
X-link: https://github.com/fairinternal/fairseq-py/pull/3144
Reviewed By: Abdel-rahmanMohamed
Differential Revision: D34521450
Pulled By: wnhsu
fbshipit-source-id: eed00d4238a66dc524eee400a55ad2c011e1543c
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [ ] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
## What does this PR do?
release training instructions for unit-based HiFi-GAN vocoder with duration prediction
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
X-link: https://github.com/fairinternal/fairseq-py/pull/3156
Reviewed By: sravyapopuri388
Differential Revision: D34582951
Pulled By: an918tw
fbshipit-source-id: 2e575fb15aa8cd5444272c3c31426ac64da84e97
Summary:
# Before submitting
- [x] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
https://groups.google.com/g/fairseq-users/c/YoSm5J2To1A
- [ ] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
## What does this PR do?
Fixes https://github.com/pytorch/fairseq/issues/4242
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
Pull Request resolved: https://github.com/pytorch/fairseq/pull/4243
Reviewed By: arbabu123
Differential Revision: D34538164
Pulled By: alexeib
fbshipit-source-id: cf2fdaa7663bee34571fb3d3bd9bdaf79d756206
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [x] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
## What does this PR do?
Fixes https://github.com/pytorch/fairseq/issues/4058
While using the library the following warnings are shown which sometimes hinder the workflow. The warnings are
`<USER_PATH>/fairseq/search.py:140: UserWarning: __floordiv__ is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').
beams_buf = indices_buf // vocab_size`
`<USER_PATH>/fairseq/sequence_generator.py:666: UserWarning: __floordiv__ is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').
unfin_idx = bbsz_idx // beam_size`
The methodology was simple, instead of using the `//`, it was replaced by `torch.div(arg1, arg2, rounding_mode='trunc')` and the variable alues do not change for both before and after, just the warning is resolved.
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
Yes, I did! Thanks!
Pull Request resolved: https://github.com/pytorch/fairseq/pull/4221
Reviewed By: arbabu123
Differential Revision: D34538147
Pulled By: alexeib
fbshipit-source-id: 143897a249129a163b6a30ba9b5cf5595ef42330
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [ ] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
## What does this PR do?
Fixes # (issue).
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
X-link: https://github.com/fairinternal/fairseq-py/pull/3113
Reviewed By: an918tw, kahne
Differential Revision: D34365606
Pulled By: sravyapopuri388
fbshipit-source-id: aa4f0ab24ca191101b9eca0f5e08dcbedf9fadbb
Summary:
Best metric is now only logged for the first of all the validation subsets
# Before submitting
- [x ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
https://groups.google.com/g/fairseq-users/c/7nk3rJmvlg8
- [ ] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
## What does this PR do?
Fixes https://github.com/pytorch/fairseq/issues/4162
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
Pull Request resolved: https://github.com/pytorch/fairseq/pull/4180
Reviewed By: michaelauli
Differential Revision: D34365416
Pulled By: alexeib
fbshipit-source-id: 872f77da2cbf064ed838ebc7959365b0b33fe723
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [ ] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
## What does this PR do?
Fixes # (issue).
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
X-link: https://github.com/fairinternal/fairseq-py/pull/3104
Reviewed By: kahne
Differential Revision: D34323889
Pulled By: sravyapopuri388
fbshipit-source-id: da7216bc5918fd0e57e10395044088a555af2e07
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [ ] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
## What does this PR do?
Fixes # (issue).
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
Pull Request resolved: https://github.com/fairinternal/fairseq-py/pull/3063
Reviewed By: eugene-kharitonov
Differential Revision: D34323605
Pulled By: wnhsu
fbshipit-source-id: 9dc779a6c399cda710863596e0880b9277ff2919
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [ ] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
## What does this PR do?
Fixes # (issue).
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
Pull Request resolved: https://github.com/fairinternal/fairseq-py/pull/3107
Reviewed By: cndn
Differential Revision: D34354339
Pulled By: sravyapopuri388
fbshipit-source-id: 50888706123d246c13d2cbb22d0e043740ff6bf5
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [ ] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
## What does this PR do?
Fixes # (issue).
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
Pull Request resolved: https://github.com/fairinternal/fairseq-py/pull/3065
Reviewed By: Mortimerp9
Differential Revision: D34144674
Pulled By: dianaml0
fbshipit-source-id: 842b0d29c9c85d4b56b640f2823fcb4e3f912f98
Summary:
The only difference with plain list/dict now is that nn.Parameters are
handled specially and registered as parameters properly.
test_nn and parametrization works locally.
Will see in CI if DP is fixed as well.
Tentative fix for https://github.com/pytorch/pytorch/issues/36035
Pull Request resolved: https://github.com/pytorch/pytorch/pull/70499
Reviewed By: jbschlosser, alexeib
Differential Revision: D34005332
Pulled By: albanD
fbshipit-source-id: 7e76b0873d0fec345cb537e2a6ecba0258e662b9
Summary:
# Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [ ] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/main/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
## What does this PR do?
Fixes # (issue).
## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
## Did you have fun?
Make sure you had fun coding �
Pull Request resolved: https://github.com/fairinternal/fairseq-py/pull/3059
Reviewed By: kahne
Differential Revision: D34083178
Pulled By: sravyapopuri388
fbshipit-source-id: a33af1696570be4826973b19fe34177bcf851e06
