OPUS-MT-train/models/en-to/README.md

opus-2020-01-08.zip

• dataset: opus
• model: transformer-align
• pre-processing: normalization + SentencePiece
• download: opus-2020-01-08.zip
• test set translations: opus-2020-01-08.test.txt
• test set scores: opus-2020-01-08.eval.txt

Benchmarks

testset	BLEU	chr-F
JW300.en.to	56.3	0.689

opus+bt-2020-05-23.zip

• dataset: opus+bt
• model: transformer-align
• source language(s): en
• target language(s): to
• model: transformer-align
• pre-processing: normalization + SentencePiece (spm4k,spm4k)
• download: opus+bt-2020-05-23.zip
• test set translations: opus+bt-2020-05-23.test.txt
• test set scores: opus+bt-2020-05-23.eval.txt

Training data: opus+bt

• en-to: QED (15) Tatoeba (15) wikimedia (1)
• en-to: total size = 31
• unused dev/test data is added to training data
• total size (opus+bt): 333911

Validation data

• en-to: JW300

• devset = top 2500 lines of JW300.src.shuffled!

• testset = next 2500 lines of JW300.src.shuffled!

• remaining lines are added to traindata

Benchmarks

testset	BLEU	chr-F
JW300.en.to	59.1	0.716