5 Commits

Author	SHA1	Message	Date
Sam Shleifer	4f881a760e	TokenBlockDataset np type promotion issue (#1658) ... Summary: Pull Request resolved: https://github.com/fairinternal/fairseq-py/pull/1658 Reviewed By: jxmsML Differential Revision: D26701840 Pulled By: sshleifer fbshipit-source-id: 90d631c3cd775ab847366fe7a05136c29d90cd63	2021-02-26 21:00:38 -08:00
Myle Ott	a48f235636	Apply black+isort (#1357) ... Summary: Pull Request resolved: https://github.com/fairinternal/fairseq-py/pull/1357 Reviewed By: alexeib Differential Revision: D24377772 fbshipit-source-id: 51581af041d42d62166b33a35a1a4228b1a76f0c	2020-10-18 18:14:51 -07:00
Myle Ott	e75cff5f2c	Relicense fairseq under MIT license (#786) ... Summary: The previous BSD+PATENTS license was controversial. We have been approved to relicense fairseq under the MIT license. Pull Request resolved: https://github.com/fairinternal/fairseq-py/pull/786 Differential Revision: D16560654 Pulled By: myleott fbshipit-source-id: f78b1beb4f2895dd7b9bfc79f5f952a2bfb94034	2019-07-30 07:48:23 -07:00
Myle Ott	e4edf27a97	Improve init speed of TokenBlockDataset and EpochBatchIterator ... Summary: Pull Request resolved: https://github.com/pytorch/fairseq/pull/704 Differential Revision: D15221549 Pulled By: myleott fbshipit-source-id: b0021acdc2d7792ce51421f1432e1f2bd8218f7b	2019-05-07 07:08:53 -07:00
Myle Ott	42be3ebd41	Merge internal changes (#483) ... Summary: Changelog: - `4889802`: can now remove detokenize sentencepiece output with `--remove-bpe=sentencepiece` (fixes #331). Also added `--sacrebleu` for computing detokenized BLEU. - `0d76427`: fix assertion error when training language model with dataset containing empty sentences - minor bug and style fixes Pull Request resolved: https://github.com/pytorch/fairseq/pull/483 Differential Revision: D13867899 Pulled By: myleott fbshipit-source-id: 25c940b847fe270262ac8f5ac838407b3977fdda	2019-01-30 09:01:10 -08:00