mirror of
https://github.com/facebookresearch/fairseq.git
synced 2024-09-11 17:25:31 +03:00
Summary: Imported from https://github.com/fairinternal/fairseq-py/pull/1284. Updated according to PR comments. Main changes: * New task: `fairseq.tasks.speech_to_text` * Multilingual support: multiple train sub-splits, temperature-based sampling, language ID tokens * New dataset: `fairseq.data.audio.speech_to_text_dataset` * Added accuracy metrics and BOS prefix removal to label smoothed cross entropy * New models: Transformer (`fairseq.models.speech_to_text.s2t_transformer`) and BLSTM (`fairseq.models.speech_to_text.berard`) * Extended scorers: * Added a base scorer class: `fairseq.scorers.BaseScorer` (the parent class for all scorers except the BLEU scorer in CPP) * Added an evaluation tokenizer: `fairseq.scorers.eval_tokenizer` which leverages sacreBLEU's built-in tokenizers and allows character-level tokenization as well as punctuation removal (for WER scoring). * Added chrF scorer: `fairseq.scorers.chrf` * Online Mel-filter bank speech feature extraction (via CPP-based pyKaldi or Python-based TorchAudio): `fairseq.data.audio.audio_utils` * Online speech feature transforms: `fairseq.data.audio.feature_transforms.*` * Fixed the subsampled sequence lengths in VGGTransformer (`examples.speech_recognition.models.vggtransformer`) * Examples under `examples/speech_to_text`: * LibriSpeech (ASR): better results than VGGTransformer with smaller Transformer-based models * MuST-C (ST): comparable to [SOTA results](https://arxiv.org/pdf/2004.10234.pdf) but with less tricks Reviewed By: jmp84 Differential Revision: D24065273 fbshipit-source-id: 5f842ca9c826f92d4af660705611885fe440a9ab |
||
---|---|---|
.. | ||
gpu | ||
speech_recognition | ||
__init__.py | ||
test_average_checkpoints.py | ||
test_backtranslation_dataset.py | ||
test_binaries.py | ||
test_bmuf.py | ||
test_character_token_embedder.py | ||
test_concat_dataset.py | ||
test_constraints.py | ||
test_convtbc.py | ||
test_dictionary.py | ||
test_export.py | ||
test_file_io.py | ||
test_fp16_optimizer.py | ||
test_inference_dropout.py | ||
test_iterators.py | ||
test_label_smoothing.py | ||
test_lstm_jitable.py | ||
test_memory_efficient_fp16.py | ||
test_metrics.py | ||
test_multi_corpus_sampled_dataset.py | ||
test_multihead_attention.py | ||
test_noising.py | ||
test_reproducibility.py | ||
test_resampling_dataset.py | ||
test_sequence_generator.py | ||
test_sequence_scorer.py | ||
test_sparse_multihead_attention.py | ||
test_token_block_dataset.py | ||
test_train.py | ||
test_utils.py | ||
utils.py |