mirror of
https://github.com/facebookresearch/fairseq.git
synced 2024-09-11 09:15:43 +03:00
Merge branch 'main' into fix-pdb-os-error
This commit is contained in:
commit
c87cc37aac
@ -2,7 +2,7 @@
|
||||
|
||||
## Model details
|
||||
|
||||
**Organization developing the model** The FAIR team of Meta AI.
|
||||
**Organization developing the model** The FAIR team
|
||||
|
||||
**Model version** This is version 1 of the model.
|
||||
|
||||
|
@ -6,12 +6,12 @@ We follow the recommendations of Gebru et al. (2018) and provide a datacard for
|
||||
## Motivation
|
||||
* **For what purpose was the dataset created? Was there a specific task in mind? Was there a specific gap that needed to be filled? Please provide a description.**
|
||||
The pre-training data for training the 1.1 T model was created by a union of six English language datasets, including five datasets used by RoBERTa (Liu et al 2019) and the English subset of CC 100. These purpose of creating this dataset was to pre-train the language model.
|
||||
|
||||
|
||||
* **Who created the dataset (e.g., which team, research group) and on behalf of which entity (e.g., company, institution, organization)?**
|
||||
Meta AI.
|
||||
|
||||
FAIR (Fundamental Artificial Intelligence Research)
|
||||
|
||||
* **Who funded the creation of the dataset? If there is an associated grant, please provide the name of the grantor and the grant name and number.**
|
||||
Meta AI.
|
||||
FAIR (Fundamental Artificial Intelligence Research)
|
||||
|
||||
* **Any other comments?**
|
||||
No.
|
||||
@ -183,7 +183,7 @@ No.
|
||||
## Maintenance
|
||||
|
||||
* **Who is supporting/hosting/maintaining the dataset?**
|
||||
Meta AI.
|
||||
FAIR (Fundamental Artificial Intelligence Research)
|
||||
|
||||
* **How can the owner/curator/manager of the dataset be contacted (e.g., email address)?**
|
||||
Refer to the main document.
|
||||
|
@ -2,7 +2,7 @@
|
||||
## Version 1.0.0
|
||||
|
||||
### Model developer
|
||||
Meta AI
|
||||
FAIR (Fundamental Artificial Intelligence Research)
|
||||
|
||||
### Model type
|
||||
An autoregressive English language model trained on a union of six English language models. We explore dense and sparse (MoE based) architectures in the paper.
|
||||
@ -132,7 +132,7 @@ A dataset extracted from CommonCrawl snapshots between January 2018 and December
|
||||
The 1.1T parameter model was evaluated on the StereoSet and CrowS pairs dataset for inherent bias in the model, and bias as a result of the data. Similar to StereoSet, we observe that both the dense and MoE models get worse in terms of the Stereotype Score (SS) with scale.
|
||||
|
||||
### Privacy and security
|
||||
The 1.1T model did not have any special Privacy and Security considerations. The training data and evaluation data were both public and went through standard Meta AI Privacy and licensing procedures.
|
||||
The 1.1T model did not have any special Privacy and Security considerations. The training data and evaluation data were both public and went through standard Meta privacy and licensing procedures.
|
||||
|
||||
### Transparency and control
|
||||
In the spirit of transparency and accountability we have created this model card for the 1.1T parameter model and a data card for the training data (referenced in Artetxe et al. (2021)).
|
||||
|
@ -1,6 +1,6 @@
|
||||
# ASR-BLEU evaluation toolkit
|
||||
|
||||
This toolkit provides a set of public ASR models used for evaluation of different speech-to-speech translation systems at Meta AI. It enables easier score comparisons between different system's outputs.
|
||||
This toolkit provides a set of public ASR models used for evaluation of different speech-to-speech translation systems at FAIR. It enables easier score comparisons between different system's outputs.
|
||||
|
||||
The ASRGenerator wraps different CTC-based ASR models from HuggingFace and fairseq code bases. Torchaudio CTC decoder is built on top of it to decode given audio files.
|
||||
|
||||
@ -31,4 +31,4 @@ python compute_asr_bleu.py --lang <LANG> \
|
||||
--reference_format txt
|
||||
```
|
||||
|
||||
For more details about arguments please see the script argparser help.
|
||||
For more details about arguments please see the script argparser help.
|
||||
|
@ -1,4 +1,4 @@
|
||||
XStoryCloze consists of professional translation of the validation split of the [English StoryCloze dataset](https://cs.rochester.edu/nlp/rocstories/) (Spring 2016 version) to 10 other languages. This dataset is released by Meta AI alongside the paper [Few-shot Learning with Multilingual Generative Language Models. EMNLP 2022](https://arxiv.org/abs/2112.10668).
|
||||
XStoryCloze consists of professional translation of the validation split of the [English StoryCloze dataset](https://cs.rochester.edu/nlp/rocstories/) (Spring 2016 version) to 10 other languages. This dataset is released by FAIR (Fundamental Artificial Intelligence Research) alongside the paper [Few-shot Learning with Multilingual Generative Language Models. EMNLP 2022](https://arxiv.org/abs/2112.10668).
|
||||
|
||||
# Languages
|
||||
ru, zh (Simplified), es (Latin America), ar, hi, id, te, sw, eu, my.
|
||||
|
@ -2,7 +2,7 @@
|
||||
## Version 1.0.0
|
||||
|
||||
### Model developer
|
||||
Meta AI
|
||||
FAIR (Fundamental Artificial Intelligence Research)
|
||||
|
||||
### Model type
|
||||
A family of multilingual autoregressive language models (ranging from 564 million to 7.5 billion parameters) trained on a balanced corpus of a diverse set of languages. The language model can learn tasks from natural language descriptions and a few examples.
|
||||
@ -31,7 +31,7 @@ The model was evaluated on hate speech detection and occupation identification.
|
||||
## Metrics
|
||||
### Model performance measures
|
||||
The XGLM model was primarily evaluated on
|
||||
1. Zero shot and few shot learning by looking at per-language performance on tasks spanning commonsense reasoning (XCOPA, XWinograd), natural language inference (XNLI) and paraphrasing (PAWS-X). The model is also evaluated on XStoryCloze, a new dataset created by Meta AI.
|
||||
1. Zero shot and few shot learning by looking at per-language performance on tasks spanning commonsense reasoning (XCOPA, XWinograd), natural language inference (XNLI) and paraphrasing (PAWS-X). The model is also evaluated on XStoryCloze, a new dataset created by FAIR (Fundamental Artificial Intelligence Research).
|
||||
2. Cross lingual transfer through templates and few-shot examples.
|
||||
3. Knowledge probing - Evaluate to what extent the XGLM model can effectively store factual knowledge in different languages using the mLAMA benchmark.
|
||||
4. Translation - We report machine translation results on WMT benchmarks and a subset of FLORES-101 in the main paper.
|
||||
@ -50,7 +50,7 @@ The Cross-lingual Natural Language Inference (XNLI) corpus is the extension of t
|
||||
|
||||
### XStoryCloze
|
||||
#### Description
|
||||
A new dataset created by Meta AI along side this work by translating the validation split of the English StoryCloze dataset (Mostafazadeh et al., 2016) (Spring 2016 version) to 10 other typologically diverse languages (ru, zh Simplified, es Latin America, ar, hi, id, te, sw, eu, my).
|
||||
A new dataset created by FAIR along side this work by translating the validation split of the English StoryCloze dataset (Mostafazadeh et al., 2016) (Spring 2016 version) to 10 other typologically diverse languages (ru, zh Simplified, es Latin America, ar, hi, id, te, sw, eu, my).
|
||||
|
||||
### XCOPA (Ponti et al., 2020)
|
||||
#### Description
|
||||
@ -85,7 +85,7 @@ More details on the CC100-XL dataset can be found in the Appendix section of the
|
||||
The XGLM model was evaluated on Hate speech and bias identification datasets. For hate speech, we observe that across the 5 languages in the dataset, in context learning results are only slightly better than random (50%). Another interesting observation is that most few shot results are worse than zero-shot, which indicates that the model is not able to utilize examples using the templates described in the paper. For bias identification, the XGLM (6.7B) English only model achieves the best performance on English and Spanish, while the GPT-3 model of comparable size (6.7B) model achieves the best in French. On certain occupations (e.g. model and teacher), XGLM 6.7B En only model and GPT-3 (6.7B) have very significant bias while XGLM 7.5B is much less biased.
|
||||
|
||||
### Privacy and security
|
||||
The XGLM model did not have any special Privacy and Security considerations. The training data and evaluation data were both public and went through standard Meta AI Privacy and licensing procedures.
|
||||
The XGLM model did not have any special Privacy and Security considerations. The training data and evaluation data were both public and went through standard Meta privacy and licensing procedures.
|
||||
|
||||
### Transparency and control
|
||||
In the spirit of transparency and accountability we have created this model card and a data card for the CC100-XL which can be found in the Appendix section of the paper.
|
||||
|
@ -524,7 +524,7 @@ class EpochBatchIterator(EpochBatchIterating):
|
||||
# TODO: Below is a lazy implementation which discard the final batch regardless
|
||||
# of whether it is a full batch or not.
|
||||
|
||||
total_num_itrs = len(self.epoch_batch_sampler) - 1
|
||||
total_num_itrs = len(itr) - 1
|
||||
itr.take(total_num_itrs)
|
||||
logger.info(f"skip final residual batch, total_num_itrs = {total_num_itrs}")
|
||||
|
||||
|
Loading…
Reference in New Issue
Block a user