OPUS-MT-train/models/sg-en/README.md

opus-2020-01-16.zip

• dataset: opus
• model: transformer-align
• pre-processing: normalization + SentencePiece
• download: opus-2020-01-16.zip
• test set translations: opus-2020-01-16.test.txt
• test set scores: opus-2020-01-16.eval.txt

Benchmarks

testset	BLEU	chr-F
JW300.sg.en	32.0	0.477

opus+bt-2020-05-23.zip

• dataset: opus+bt
• model: transformer-align
• source language(s): sg
• target language(s): en
• model: transformer-align
• pre-processing: normalization + SentencePiece (spm4k,spm4k)
• download: opus+bt-2020-05-23.zip
• test set translations: opus+bt-2020-05-23.test.txt
• test set scores: opus+bt-2020-05-23.eval.txt

Training data: opus+bt

• sg-en: Tatoeba (10)
• sg-en: total size = 10
• unused dev/test data is added to training data
• total size (opus+bt): 176275

Validation data

• en-sg: JW300

• devset = top 2500 lines of JW300.src.shuffled!

• testset = next 2500 lines of JW300.src.shuffled!

• remaining lines are added to traindata

Benchmarks

testset	BLEU	chr-F
JW300.sg.en	31.5	0.478