fairseq/tests
Jingfei Du 932a3d4aad fix beam search with prefix tokens (#2227)
Summary:
1. added test for genereting pad tokens during beam search with prefix
tokens
2. modified lprobs for pad token and prefix tokens to avoid generating
pad

# Before submitting

- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [ ] Did you read the [contributor guideline](https://github.com/pytorch/fairseq/blob/master/CONTRIBUTING.md)?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?

## What does this PR do?
Fixes # (issue).

## PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

## Did you have fun?
Make sure you had fun coding �

Pull Request resolved: https://github.com/fairinternal/fairseq-py/pull/2227

Reviewed By: xianxl

Differential Revision: D30649356

Pulled By: jingfeidu

fbshipit-source-id: d94903a912e767391c8fca61f98f65b5cea3b56e
2021-08-30 18:07:13 -07:00
..
distributed Add tests for fairseq.distributed.utils.all_gather_list (#1548) 2021-01-28 14:21:10 -08:00
gpu Add torch.cuda.amp support (#3460) 2021-05-26 14:39:10 -07:00
speech_recognition Enable Hydra configs in fairseq (#1343) (#1510) 2020-10-20 00:32:26 -07:00
__init__.py remediation of S205607 2020-07-17 17:21:51 -07:00
test_activation_checkpointing.py Make checkpoint wrapper pickleable (#1603) 2021-02-06 08:07:32 -08:00
test_amp_optimizer.py Add torch.cuda.amp support (#3460) 2021-05-26 14:39:10 -07:00
test_average_checkpoints.py Apply black+isort (#1357) 2020-10-18 18:14:51 -07:00
test_backtranslation_dataset.py Apply black+isort (#1357) 2020-10-18 18:14:51 -07:00
test_binaries.py MultiGPU test and --log-file workaround (#1793) 2021-04-21 06:39:00 -07:00
test_character_token_embedder.py Apply black+isort (#1357) 2020-10-18 18:14:51 -07:00
test_checkpoint_utils.py Move checkpoint state_dict creation into Trainer (#1666) 2021-03-04 13:32:44 -08:00
test_concat_dataset.py Apply black+isort (#1357) 2020-10-18 18:14:51 -07:00
test_constraints.py Apply black+isort (#1357) 2020-10-18 18:14:51 -07:00
test_convtbc.py Apply black+isort (#1357) 2020-10-18 18:14:51 -07:00
test_data_utils.py batch_by_size refactoring: 100x speedup and optimization of memory footprint 2020-12-28 21:05:51 -08:00
test_dataclass_utils.py Hierarchical Configs 2021-07-16 04:56:12 -07:00
test_dataset.py Add support for FullyShardedDataParallel (--ddp-backend=fully_sharded) (#1667) 2021-03-04 13:32:46 -08:00
test_dictionary.py Extract File Chunking to its own utils (#1955) 2021-06-28 01:46:32 -07:00
test_export.py Improve torchscript compatibility of transfomer and transformer pg (#3247) 2021-02-22 14:22:54 -08:00
test_file_chunker_utils.py Extract File Chunking to its own utils (#1955) 2021-06-28 01:46:32 -07:00
test_file_io.py Delete line that breaks gh ci (#1814) 2021-04-19 16:31:11 -07:00
test_fp16_optimizer.py end to end hydra configs (#1393) 2020-11-04 18:20:12 -08:00
test_inference_dropout.py Enable Hydra configs in fairseq (#1343) (#1510) 2020-10-20 00:32:26 -07:00
test_iopath.py Support atomic saves for checkpoints (#1520) 2020-12-18 07:40:49 -08:00
test_iterators.py Simplify CountingIterator 2021-04-29 16:17:00 -07:00
test_label_smoothing.py Apply black+isort (#1357) 2020-10-18 18:14:51 -07:00
test_lm_context_window.py Fix --context-window and add test (#1526) 2020-12-23 18:35:54 -08:00
test_lstm_jitable.py Apply black+isort (#1357) 2020-10-18 18:14:51 -07:00
test_memory_efficient_fp16.py Enable Hydra configs in fairseq (#1343) (#1510) 2020-10-20 00:32:26 -07:00
test_metrics.py Apply black+isort (#1357) 2020-10-18 18:14:51 -07:00
test_multi_corpus_dataset.py optimize sampling process of multi_corpus_dataset 2021-03-03 19:31:40 -08:00
test_multi_corpus_sampled_dataset.py Relicense fairseq under MIT license (#786) 2019-07-30 07:48:23 -07:00
test_multihead_attention.py Adding check for filler size (#3495) 2021-04-21 09:09:19 -07:00
test_noising.py Apply black+isort (#1357) 2020-10-18 18:14:51 -07:00
test_online_backtranslation.py Obt 2 (#1614) 2021-03-30 09:56:03 -07:00
test_plasma_utils.py Plasma tests: ask for less disk (#1893) 2021-05-24 09:00:18 -07:00
test_reproducibility.py Add torch.cuda.amp support (#3460) 2021-05-26 14:39:10 -07:00
test_resampling_dataset.py Apply black+isort (#1357) 2020-10-18 18:14:51 -07:00
test_roberta.py Obt 2 (#1614) 2021-03-30 09:56:03 -07:00
test_sequence_generator.py fix beam search with prefix tokens (#2227) 2021-08-30 18:07:13 -07:00
test_sequence_scorer.py Apply black+isort (#1357) 2020-10-18 18:14:51 -07:00
test_sparse_multihead_attention.py Apply black+isort (#1357) 2020-10-18 18:14:51 -07:00
test_token_block_dataset.py TokenBlockDataset np type promotion issue (#1658) 2021-02-26 21:00:38 -08:00
test_train.py fixes tests/test_train.py to mock checkpoint.save_dir config node (#3675) 2021-07-06 15:07:31 -07:00
test_transformer.py fix MultiHeadAttention assert (#1798) 2021-04-14 04:59:59 -07:00
test_utils.py Apply black+isort (#1357) 2020-10-18 18:14:51 -07:00
test_valid_subset_checks.py Migrate DummyMaskedLMTask to FairseqTask (#3593) 2021-06-10 09:43:08 -07:00
utils.py MultiGPU test and --log-file workaround (#1793) 2021-04-21 06:39:00 -07:00