mirror of
https://github.com/facebookresearch/fairseq.git
synced 2024-09-11 17:25:31 +03:00
1d1c145387
Summary: Imported from https://github.com/fairinternal/fairseq-py/pull/1284. Updated according to PR comments. Main changes: * New task: `fairseq.tasks.speech_to_text` * Multilingual support: multiple train sub-splits, temperature-based sampling, language ID tokens * New dataset: `fairseq.data.audio.speech_to_text_dataset` * Added accuracy metrics and BOS prefix removal to label smoothed cross entropy * New models: Transformer (`fairseq.models.speech_to_text.s2t_transformer`) and BLSTM (`fairseq.models.speech_to_text.berard`) * Extended scorers: * Added a base scorer class: `fairseq.scorers.BaseScorer` (the parent class for all scorers except the BLEU scorer in CPP) * Added an evaluation tokenizer: `fairseq.scorers.eval_tokenizer` which leverages sacreBLEU's built-in tokenizers and allows character-level tokenization as well as punctuation removal (for WER scoring). * Added chrF scorer: `fairseq.scorers.chrf` * Online Mel-filter bank speech feature extraction (via CPP-based pyKaldi or Python-based TorchAudio): `fairseq.data.audio.audio_utils` * Online speech feature transforms: `fairseq.data.audio.feature_transforms.*` * Fixed the subsampled sequence lengths in VGGTransformer (`examples.speech_recognition.models.vggtransformer`) * Examples under `examples/speech_to_text`: * LibriSpeech (ASR): better results than VGGTransformer with smaller Transformer-based models * MuST-C (ST): comparable to [SOTA results](https://arxiv.org/pdf/2004.10234.pdf) but with less tricks Reviewed By: jmp84 Differential Revision: D24065273 fbshipit-source-id: 5f842ca9c826f92d4af660705611885fe440a9ab |
||
|---|---|---|
| .. | ||
| gpu | ||
| speech_recognition | ||
| __init__.py | ||
| test_average_checkpoints.py | ||
| test_backtranslation_dataset.py | ||
| test_binaries.py | ||
| test_bmuf.py | ||
| test_character_token_embedder.py | ||
| test_concat_dataset.py | ||
| test_constraints.py | ||
| test_convtbc.py | ||
| test_dictionary.py | ||
| test_export.py | ||
| test_file_io.py | ||
| test_fp16_optimizer.py | ||
| test_inference_dropout.py | ||
| test_iterators.py | ||
| test_label_smoothing.py | ||
| test_lstm_jitable.py | ||
| test_memory_efficient_fp16.py | ||
| test_metrics.py | ||
| test_multi_corpus_sampled_dataset.py | ||
| test_multihead_attention.py | ||
| test_noising.py | ||
| test_reproducibility.py | ||
| test_resampling_dataset.py | ||
| test_sequence_generator.py | ||
| test_sequence_scorer.py | ||
| test_sparse_multihead_attention.py | ||
| test_token_block_dataset.py | ||
| test_train.py | ||
| test_utils.py | ||
| utils.py | ||