Helsinki-NLP/OPUS-MT-train
1
1
Fork 0
mirror of https://github.com/Helsinki-NLP/OPUS-MT-train.git synced 2024-09-11 20:27:19 +03:00
Code Issues Packages Projects Releases Wiki Activity

removed old work-spm files

Browse Source
This commit is contained in:
Joerg Tiedemann 2022-02-07 00:10:10 +02:00
parent 2d2663a332
commit 8b05bb352a
12072 changed files with 0 additions and 2610581 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
work-spm/aa-en/test/README
View File

@ -1,2 +0,0 @@
testset = top 250 lines of ../val/GNOME.src.shuffled!
testset = top 250 lines of ../val/bible-uedin.src.shuffled!

2
work-spm/aa-en/val/README
View File

@ -1,2 +0,0 @@
devset = top 250 lines of GNOME.src.shuffled!
devset = top 250 lines of bible-uedin.src.shuffled!

1
work-spm/aa-fi/test/README
View File

@ -1 +0,0 @@
testset = top 250 lines of ../val/bible-uedin.src.shuffled!

1
work-spm/aa-fi/val/README
View File

@ -1 +0,0 @@
devset = top 250 lines of bible-uedin.src.shuffled!

15
work-spm/ab-en/README.md
View File

@ -1,15 +0,0 @@
# opus-2019-12-18.zip
* dataset: opus
* model: transformer-align
* pre-processing: normalization + SentencePiece
* download: [opus-2019-12-18.zip](https://object.pouta.csc.fi/OPUS-MT-models/ab-en/opus-2019-12-18.zip)
* test set translations: [opus-2019-12-18.test.txt](https://object.pouta.csc.fi/OPUS-MT-models/ab-en/opus-2019-12-18.test.txt)
* test set scores: [opus-2019-12-18.eval.txt](https://object.pouta.csc.fi/OPUS-MT-models/ab-en/opus-2019-12-18.eval.txt)
## Benchmarks
| testset | BLEU | chr-F |
|-----------------------|-------|-------|
| JW300.ab.en | 2.9 | 0.144 |
| Tatoeba.ab.en | 2.3 | 0.097 |

1
work-spm/ab-en/test/README
View File

@ -1 +0,0 @@
testset = second top 5000 lines of ../val/JW300.src.shuffled!

27
work-spm/ab-en/test/Tatoeba.src
View File

@ -1,27 +0,0 @@
Сара азба сакуеит.
Акәашара бзиa бoба?
Cаҟара cыцлеит.
Маҷк cааҧсеит.
Абиблиотека абаҟоу?
Сыцәар сҭахыуп.
Иҭабуп!
Англыз бызшәа жәдыруама?
Асааҭ шаҟоузеи?
Сара сцоит.
Иахьа шәахьоуп.
Бурӡ Халифа иахьатәила зегь реицкис иреиҳау хыб ҳәа иашьҭоуп.
Маҷк cааҧсеит.
Сара азба сакуеит.
Иуфазеи?
Шьыжьбзиа!
Сара ашә сфоит.
Сара акәац сфоит.
Сыӡбалқәас рхы надырхәозеи аҧсуаа?
Сара Лори сыхьӡуп.
Сара цәыкьа сышьҭалоит.
Aиашьа дyмоумаҵ?
Cара yаҵәы аpахь caaуеит.
Уи cыҧcықәаpа ауп.
Сара исҳәоит.
Дaба сыҧшаауeиҵ?
Бзиала шәаабеит Авикипедиахь.

27
work-spm/ab-en/test/Tatoeba.trg
View File

@ -1,27 +0,0 @@
I'm thirsty.
Do you like dancing?
I'm very happy.
I'm a little tired.
Where is the library?
I want to sleep.
Thank you!
Do you speak English?
What time is it?
I'm going.
Today is Monday.
Burj Khalifa is currently the tallest skyscraper in the world.
I'm a little bit tired.
I am thirsty.
What did you eat?
Good morning.
I eat cheese.
I eat meat.
Which sauces do the Abkhazians use?
My name is Laurie.
I'll go to bed early.
Do you have a brother?
I'll be here tomorrow.
I really like it.
I will say.
Where can I find it?
Welcome to Wikipedia.

1
work-spm/ab-en/val/README
View File

@ -1 +0,0 @@
devset = top 5000 lines of JW300.src.shuffled!

1
work-spm/ab-fi/test/README
View File

@ -1 +0,0 @@
testset = second top 5000 lines of ../val/JW300.src.shuffled!

1
work-spm/ab-fi/val/README
View File

@ -1 +0,0 @@
devset = top 5000 lines of JW300.src.shuffled!

2
work-spm/ace-en/test/README
View File

@ -1,2 +0,0 @@
testset = top 250 lines of ../val/GNOME.src.shuffled!
testset = top 250 lines of ../val/bible-uedin.src.shuffled!

2
work-spm/ace-en/val/README
View File

@ -1,2 +0,0 @@
devset = top 250 lines of GNOME.src.shuffled!
devset = top 250 lines of bible-uedin.src.shuffled!

1
work-spm/ace-fi/test/README
View File

@ -1 +0,0 @@
testset = top 250 lines of ../val/bible-uedin.src.shuffled!

1
work-spm/ace-fi/val/README
View File

@ -1 +0,0 @@
devset = top 250 lines of bible-uedin.src.shuffled!

14
work-spm/ach-en/README.md
View File

@ -1,14 +0,0 @@
# opus-2019-12-18.zip
* dataset: opus
* model: transformer-align
* pre-processing: normalization + SentencePiece
* download: [opus-2019-12-18.zip](https://object.pouta.csc.fi/OPUS-MT-models/ach-en/opus-2019-12-18.zip)
* test set translations: [opus-2019-12-18.test.txt](https://object.pouta.csc.fi/OPUS-MT-models/ach-en/opus-2019-12-18.test.txt)
* test set scores: [opus-2019-12-18.eval.txt](https://object.pouta.csc.fi/OPUS-MT-models/ach-en/opus-2019-12-18.eval.txt)
## Benchmarks
| testset | BLEU | chr-F |
|-----------------------|-------|-------|
| JW300.ach.en | 5.4 | 0.207 |

1
work-spm/ach-en/test/README
View File

@ -1 +0,0 @@
testset = second top 5000 lines of ../val/JW300.src.shuffled!

0
work-spm/ach-en/test/Tatoeba.src
View File

0
work-spm/ach-en/test/Tatoeba.trg
View File

1
work-spm/ach-en/val/README
View File

@ -1 +0,0 @@
devset = top 5000 lines of JW300.src.shuffled!

1
work-spm/ach-fi/test/README
View File

@ -1 +0,0 @@
testset = second top 5000 lines of ../val/JW300.src.shuffled!

1
work-spm/ach-fi/val/README
View File

@ -1 +0,0 @@
devset = top 5000 lines of JW300.src.shuffled!

2
work-spm/acm-en/test/README
View File

@ -1,2 +0,0 @@
testset = top 250 lines of ../val/GNOME.src.shuffled!
testset = top 250 lines of ../val/bible-uedin.src.shuffled!

2
work-spm/acm-en/val/README
View File

@ -1,2 +0,0 @@
devset = top 250 lines of GNOME.src.shuffled!
devset = top 250 lines of bible-uedin.src.shuffled!

1
work-spm/acm-fi/test/README
View File

@ -1 +0,0 @@
testset = top 250 lines of ../val/bible-uedin.src.shuffled!

1
work-spm/acm-fi/val/README
View File

@ -1 +0,0 @@
devset = top 250 lines of bible-uedin.src.shuffled!

14
work-spm/acu-en/README.md
View File

@ -1,14 +0,0 @@
# opus-2019-12-18.zip
* dataset: opus
* model: transformer-align
* pre-processing: normalization + SentencePiece
* download: [opus-2019-12-18.zip](https://object.pouta.csc.fi/OPUS-MT-models/acu-en/opus-2019-12-18.zip)
* test set translations: [opus-2019-12-18.test.txt](https://object.pouta.csc.fi/OPUS-MT-models/acu-en/opus-2019-12-18.test.txt)
* test set scores: [opus-2019-12-18.eval.txt](https://object.pouta.csc.fi/OPUS-MT-models/acu-en/opus-2019-12-18.eval.txt)
## Benchmarks
| testset | BLEU | chr-F |
|-----------------------|-------|-------|
| bible-uedin.acu.en | 3.8 | 0.202 |

2
work-spm/acu-en/test/README
View File

@ -1,2 +0,0 @@
testset = top 250 lines of ../val/GNOME.src.shuffled!
testset = second top 5000 lines of ../val/bible-uedin.src.shuffled!

0
work-spm/acu-en/test/Tatoeba.src
View File

0
work-spm/acu-en/test/Tatoeba.trg
View File

2
work-spm/acu-en/val/README
View File

@ -1,2 +0,0 @@
devset = top 250 lines of GNOME.src.shuffled!
devset = top 5000 lines of bible-uedin.src.shuffled!

1
work-spm/acu-fi/test/README
View File

@ -1 +0,0 @@
testset = top 1000 lines of ../val/bible-uedin.src.shuffled!

1
work-spm/acu-fi/val/README
View File

@ -1 +0,0 @@
devset = top 1000 lines of bible-uedin.src.shuffled!

14
work-spm/ada-en/README.md
View File

@ -1,14 +0,0 @@
# opus-2019-12-18.zip
* dataset: opus
* model: transformer-align
* pre-processing: normalization + SentencePiece
* download: [opus-2019-12-18.zip](https://object.pouta.csc.fi/OPUS-MT-models/ada-en/opus-2019-12-18.zip)
* test set translations: [opus-2019-12-18.test.txt](https://object.pouta.csc.fi/OPUS-MT-models/ada-en/opus-2019-12-18.test.txt)
* test set scores: [opus-2019-12-18.eval.txt](https://object.pouta.csc.fi/OPUS-MT-models/ada-en/opus-2019-12-18.eval.txt)
## Benchmarks
| testset | BLEU | chr-F |
|-----------------------|-------|-------|
| JW300.ada.en | 4.3 | 0.182 |

1
work-spm/ada-en/test/README
View File

@ -1 +0,0 @@
testset = second top 5000 lines of ../val/JW300.src.shuffled!

0
work-spm/ada-en/test/Tatoeba.src
View File

0
work-spm/ada-en/test/Tatoeba.trg
View File

1
work-spm/ada-en/val/README
View File

@ -1 +0,0 @@
devset = top 5000 lines of JW300.src.shuffled!

1
work-spm/ada-fi/test/README
View File

@ -1 +0,0 @@
testset = second top 5000 lines of ../val/JW300.src.shuffled!

1
work-spm/ada-fi/val/README
View File

@ -1 +0,0 @@
devset = top 5000 lines of JW300.src.shuffled!

2
work-spm/ady-en/test/README
View File

@ -1,2 +0,0 @@
testset = top 250 lines of ../val/GNOME.src.shuffled!
testset = top 250 lines of ../val/bible-uedin.src.shuffled!

2
work-spm/ady-en/val/README
View File

@ -1,2 +0,0 @@
devset = top 250 lines of GNOME.src.shuffled!
devset = top 250 lines of bible-uedin.src.shuffled!

1
work-spm/ady-fi/test/README
View File

@ -1 +0,0 @@
testset = top 250 lines of ../val/bible-uedin.src.shuffled!

1
work-spm/ady-fi/val/README
View File

@ -1 +0,0 @@
devset = top 250 lines of bible-uedin.src.shuffled!

2
work-spm/ae-en/test/README
View File

@ -1,2 +0,0 @@
testset = top 250 lines of ../val/GNOME.src.shuffled!
testset = top 250 lines of ../val/bible-uedin.src.shuffled!

2
work-spm/ae-en/val/README
View File

@ -1,2 +0,0 @@
devset = top 250 lines of GNOME.src.shuffled!
devset = top 250 lines of bible-uedin.src.shuffled!

1
work-spm/ae-fi/test/README
View File

@ -1 +0,0 @@
testset = top 250 lines of ../val/bible-uedin.src.shuffled!

1
work-spm/ae-fi/val/README
View File

@ -1 +0,0 @@
devset = top 250 lines of bible-uedin.src.shuffled!

2
work-spm/aeb-en/test/README
View File

@ -1,2 +0,0 @@
testset = top 250 lines of ../val/GNOME.src.shuffled!
testset = top 250 lines of ../val/bible-uedin.src.shuffled!

2
work-spm/aeb-en/val/README
View File

@ -1,2 +0,0 @@
devset = top 250 lines of GNOME.src.shuffled!
devset = top 250 lines of bible-uedin.src.shuffled!

1
work-spm/aeb-fi/test/README
View File

@ -1 +0,0 @@
testset = top 250 lines of ../val/bible-uedin.src.shuffled!

1
work-spm/aeb-fi/val/README
View File

@ -1 +0,0 @@
devset = top 250 lines of bible-uedin.src.shuffled!

14
work-spm/aed-en/README.md
View File

@ -1,14 +0,0 @@
# opus-2019-12-18.zip
* dataset: opus
* model: transformer-align
* pre-processing: normalization + SentencePiece
* download: [opus-2019-12-18.zip](https://object.pouta.csc.fi/OPUS-MT-models/aed-en/opus-2019-12-18.zip)
* test set translations: [opus-2019-12-18.test.txt](https://object.pouta.csc.fi/OPUS-MT-models/aed-en/opus-2019-12-18.test.txt)
* test set scores: [opus-2019-12-18.eval.txt](https://object.pouta.csc.fi/OPUS-MT-models/aed-en/opus-2019-12-18.eval.txt)
## Benchmarks
| testset | BLEU | chr-F |
|-----------------------|-------|-------|
| JW300.aed.en | 4.0 | 0.177 |

1
work-spm/aed-en/test/README
View File

@ -1 +0,0 @@
testset = second top 5000 lines of ../val/JW300.src.shuffled!

0
work-spm/aed-en/test/Tatoeba.src
View File

0
work-spm/aed-en/test/Tatoeba.trg
View File

1
work-spm/aed-en/val/README
View File

@ -1 +0,0 @@
devset = top 5000 lines of JW300.src.shuffled!

1
work-spm/aed-fi/test/README
View File

@ -1 +0,0 @@
testset = second top 5000 lines of ../val/JW300.src.shuffled!

1
work-spm/aed-fi/val/README
View File

@ -1 +0,0 @@
devset = top 5000 lines of JW300.src.shuffled!

1
work-spm/af-de/test/Tatoeba.src
View File

@ -1 +0,0 @@
/scratch/project_2001194/Opus-MT-train/work-spm/de-af/test/Tatoeba.trg

1
work-spm/af-de/test/Tatoeba.trg
View File

@ -1 +0,0 @@
/scratch/project_2001194/Opus-MT-train/work-spm/de-af/test/Tatoeba.src

1
work-spm/af-de/val/Tatoeba.src
View File

@ -1 +0,0 @@
/scratch/project_2001194/Opus-MT-train/work-spm/de-af/val/Tatoeba.trg

1
work-spm/af-de/val/Tatoeba.src.shuffled.gz
View File

@ -1 +0,0 @@
/scratch/project_2001194/Opus-MT-train/work-spm/de-af/val/Tatoeba.src.shuffled.gz

1
work-spm/af-de/val/Tatoeba.trg
View File

@ -1 +0,0 @@
/scratch/project_2001194/Opus-MT-train/work-spm/de-af/val/Tatoeba.src

14
work-spm/af-en/README.md
View File

@ -1,14 +0,0 @@
# opus-2019-12-18.zip
* dataset: opus
* model: transformer-align
* pre-processing: normalization + SentencePiece
* download: [opus-2019-12-18.zip](https://object.pouta.csc.fi/OPUS-MT-models/af-en/opus-2019-12-18.zip)
* test set translations: [opus-2019-12-18.test.txt](https://object.pouta.csc.fi/OPUS-MT-models/af-en/opus-2019-12-18.test.txt)
* test set scores: [opus-2019-12-18.eval.txt](https://object.pouta.csc.fi/OPUS-MT-models/af-en/opus-2019-12-18.eval.txt)
## Benchmarks
| testset | BLEU | chr-F |
|-----------------------|-------|-------|
| Tatoeba.af.en | 60.8 | 0.736 |

1
work-spm/af-en/test/README
View File

@ -1 +0,0 @@
testset = top 1000 lines of ../val/Tatoeba.src.shuffled!

1345
work-spm/af-en/test/Tatoeba.src
View File

File diff suppressed because it is too large Load Diff

1345
work-spm/af-en/test/Tatoeba.trg
View File

File diff suppressed because it is too large Load Diff

1
work-spm/af-en/val/README
View File

@ -1 +0,0 @@
devset = top 1000 lines of Tatoeba.src.shuffled!

1000
work-spm/af-en/val/Tatoeba.src
View File

File diff suppressed because it is too large Load Diff

BIN
work-spm/af-en/val/Tatoeba.src.shuffled.gz
View File

Binary file not shown.

1000
work-spm/af-en/val/Tatoeba.trg
View File

File diff suppressed because it is too large Load Diff

14
work-spm/af-fi/README.md
View File

@ -1,14 +0,0 @@
# opus-2020-01-08.zip
* dataset: opus
* model: transformer-align
* pre-processing: normalization + SentencePiece
* download: [opus-2020-01-08.zip](https://object.pouta.csc.fi/OPUS-MT-models/af-fi/opus-2020-01-08.zip)
* test set translations: [opus-2020-01-08.test.txt](https://object.pouta.csc.fi/OPUS-MT-models/af-fi/opus-2020-01-08.test.txt)
* test set scores: [opus-2020-01-08.eval.txt](https://object.pouta.csc.fi/OPUS-MT-models/af-fi/opus-2020-01-08.eval.txt)
## Benchmarks
| testset | BLEU | chr-F |
|-----------------------|-------|-------|
| JW300.af.fi | 32.3 | 0.576 |

1
work-spm/af-fi/test/README
View File

@ -1 +0,0 @@
testset = second top 5000 lines of ../val/JW300.src.shuffled!

1
work-spm/af-fi/val/README
View File

@ -1 +0,0 @@
devset = top 5000 lines of JW300.src.shuffled!

14
work-spm/af-fr/README.md
View File

@ -1,14 +0,0 @@
# opus-2020-01-08.zip
* dataset: opus
* model: transformer-align
* pre-processing: normalization + SentencePiece
* download: [opus-2020-01-08.zip](https://object.pouta.csc.fi/OPUS-MT-models/af-fr/opus-2020-01-08.zip)
* test set translations: [opus-2020-01-08.test.txt](https://object.pouta.csc.fi/OPUS-MT-models/af-fr/opus-2020-01-08.test.txt)
* test set scores: [opus-2020-01-08.eval.txt](https://object.pouta.csc.fi/OPUS-MT-models/af-fr/opus-2020-01-08.eval.txt)
## Benchmarks
| testset | BLEU | chr-F |
|-----------------------|-------|-------|
| JW300.af.fr | 35.3 | 0.543 |

14
work-spm/af-sv/README.md
View File

@ -1,14 +0,0 @@
# opus-2020-01-08.zip
* dataset: opus
* model: transformer-align
* pre-processing: normalization + SentencePiece
* download: [opus-2020-01-08.zip](https://object.pouta.csc.fi/OPUS-MT-models/af-sv/opus-2020-01-08.zip)
* test set translations: [opus-2020-01-08.test.txt](https://object.pouta.csc.fi/OPUS-MT-models/af-sv/opus-2020-01-08.test.txt)
* test set scores: [opus-2020-01-08.eval.txt](https://object.pouta.csc.fi/OPUS-MT-models/af-sv/opus-2020-01-08.eval.txt)
## Benchmarks
| testset | BLEU | chr-F |
|-----------------------|-------|-------|
| JW300.af.sv | 40.4 | 0.599 |

2
work-spm/afb-en/test/README
View File

@ -1,2 +0,0 @@
testset = top 250 lines of ../val/GNOME.src.shuffled!
testset = top 250 lines of ../val/bible-uedin.src.shuffled!

2
work-spm/afb-en/val/README
View File

@ -1,2 +0,0 @@
devset = top 250 lines of GNOME.src.shuffled!
devset = top 250 lines of bible-uedin.src.shuffled!

1
work-spm/afb-fi/test/README
View File

@ -1 +0,0 @@
testset = top 250 lines of ../val/bible-uedin.src.shuffled!

1
work-spm/afb-fi/val/README
View File

@ -1 +0,0 @@
devset = top 250 lines of bible-uedin.src.shuffled!

2
work-spm/afh-en/test/README
View File

@ -1,2 +0,0 @@
testset = top 250 lines of ../val/GNOME.src.shuffled!
testset = top 250 lines of ../val/bible-uedin.src.shuffled!

2
work-spm/afh-en/val/README
View File

@ -1,2 +0,0 @@
devset = top 250 lines of GNOME.src.shuffled!
devset = top 250 lines of bible-uedin.src.shuffled!

1
work-spm/afh-fi/test/README
View File

@ -1 +0,0 @@
testset = top 250 lines of ../val/bible-uedin.src.shuffled!

1
work-spm/afh-fi/val/README
View File

@ -1 +0,0 @@
devset = top 250 lines of bible-uedin.src.shuffled!

14
work-spm/agr-en/README.md
View File

@ -1,14 +0,0 @@
# opus-2019-12-18.zip
* dataset: opus
* model: transformer-align
* pre-processing: normalization + SentencePiece
* download: [opus-2019-12-18.zip](https://object.pouta.csc.fi/OPUS-MT-models/agr-en/opus-2019-12-18.zip)
* test set translations: [opus-2019-12-18.test.txt](https://object.pouta.csc.fi/OPUS-MT-models/agr-en/opus-2019-12-18.test.txt)
* test set scores: [opus-2019-12-18.eval.txt](https://object.pouta.csc.fi/OPUS-MT-models/agr-en/opus-2019-12-18.eval.txt)
## Benchmarks
| testset | BLEU | chr-F |
|-----------------------|-------|-------|
| bible-uedin.agr.en | 4.5 | 0.222 |

2
work-spm/agr-en/test/README
View File

@ -1,2 +0,0 @@
testset = top 250 lines of ../val/GNOME.src.shuffled!
testset = second top 5000 lines of ../val/bible-uedin.src.shuffled!

0
work-spm/agr-en/test/Tatoeba.src
View File

0
work-spm/agr-en/test/Tatoeba.trg
View File

2
work-spm/agr-en/val/README
View File

@ -1,2 +0,0 @@
devset = top 250 lines of GNOME.src.shuffled!
devset = top 5000 lines of bible-uedin.src.shuffled!

1
work-spm/agr-fi/test/README
View File

@ -1 +0,0 @@
testset = top 1000 lines of ../val/bible-uedin.src.shuffled!

1
work-spm/agr-fi/val/README
View File

@ -1 +0,0 @@
devset = top 1000 lines of bible-uedin.src.shuffled!

2
work-spm/aha-en/test/README
View File

@ -1,2 +0,0 @@
testset = top 250 lines of ../val/GNOME.src.shuffled!
testset = top 250 lines of ../val/bible-uedin.src.shuffled!

2
work-spm/aha-en/val/README
View File

@ -1,2 +0,0 @@
devset = top 250 lines of GNOME.src.shuffled!
devset = top 250 lines of bible-uedin.src.shuffled!

1
work-spm/aha-fi/test/README
View File

@ -1 +0,0 @@
testset = top 250 lines of ../val/bible-uedin.src.shuffled!

1
work-spm/aha-fi/val/README
View File

@ -1 +0,0 @@
devset = top 250 lines of bible-uedin.src.shuffled!

2
work-spm/aii-en/test/README
View File

@ -1,2 +0,0 @@
testset = top 250 lines of ../val/GNOME.src.shuffled!
testset = top 250 lines of ../val/bible-uedin.src.shuffled!

2
work-spm/aii-en/val/README
View File

@ -1,2 +0,0 @@
devset = top 250 lines of GNOME.src.shuffled!
devset = top 250 lines of bible-uedin.src.shuffled!

1
work-spm/aii-fi/test/README
View File

@ -1 +0,0 @@
testset = top 250 lines of ../val/bible-uedin.src.shuffled!

1
work-spm/aii-fi/val/README
View File

@ -1 +0,0 @@
devset = top 250 lines of bible-uedin.src.shuffled!

Some files were not shown because too many files have changed in this diff Show More