153 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Traubert	ec69daa989	Some puhti modules needed for installing	2020-09-24 13:18:18 +03:00
Traubert	21433867ab	If we include lib/config.mk before prerequisites are made, make fails ... Therefore, the install target needs to come first (though this is not necessarily sufficient).	2020-09-24 12:25:24 +03:00
Traubert	1e349f269c	Fix type	2020-09-24 12:15:08 +03:00
Joerg Tiedemann	f9a44bdb99	merged	2020-09-18 23:08:56 +03:00
Joerg Tiedemann	87a5354de5	changes to tatoeba recipes	2020-09-18 23:05:46 +03:00
Jörg Tiedemann	a61cf48443	add option to skip sentence piecce vocabs but use marian_vocab instead	2020-09-16 19:33:19 +03:00
tiedemann	913d31472e	Merge pull request #25 from Helsinki-NLP/sam-fixes ... Fix typo	2020-09-16 09:28:46 +03:00
Joerg Tiedemann	c564bd1f56	fix in fetching data for Sami languages	2020-09-16 09:25:36 +03:00
Traubert	1da54c4155	Fix typo	2020-09-15 15:02:20 +03:00
Jörg Tiedemann	58fbf0bdd8	back to old subword model names	2020-09-14 08:53:57 +03:00
Jörg Tiedemann	c2798e9758	plain text vocab files from spm models	2020-09-13 22:17:21 +03:00
Jörg Tiedemann	24e92de56a	proper release packages for models with internal sentence piece vocabs	2020-09-13 00:00:15 +03:00
Jörg Tiedemann	666b2b8462	internal sentence piece models in transformers	2020-09-12 16:16:01 +03:00
Jörg Tiedemann	ddafb43d66	removed dependence on moses tools in preprocessing script for released spm packages	2020-09-12 14:42:10 +03:00
Jörg Tiedemann	c0cb356417	added acknowledgements	2020-09-12 12:01:02 +03:00
Jörg Tiedemann	16eef8e45d	moved project makefiles to lib/projects	2020-09-10 12:12:44 +03:00
Jörg Tiedemann	1a6e29275d	dev data is now uniq to avoid overlaps with test data	2020-09-09 23:21:07 +03:00
Jörg Tiedemann	3735af4ec1	more documentation	2020-09-07 23:00:01 +03:00
Jörg Tiedemann	3367ad2e34	documentation of low-resource languages	2020-09-06 23:56:16 +03:00
Jörg Tiedemann	909e525a2d	keep translations even if uncomplete in pivoting	2020-09-06 00:22:48 +03:00
Jörg Tiedemann	a47c292152	pivoting and documentation	2020-09-05 22:19:00 +03:00
Jörg Tiedemann	ad828c3124	started tutorial and fixes to backtranslate makefile	2020-09-05 00:16:22 +03:00
Tiedemann Jörg	d11f74ce41	added bpe submodule	2020-09-04 15:34:20 +03:00
Tiedemann	96eaad2d05	added possibility to fetch moses file from ObjectStore (instead of reading with opus_read)	2020-09-03 22:04:44 +03:00
Joerg Tiedemann	971ece9606	fix tatoeba data labels	2020-09-03 07:55:44 +03:00
Tiedemann	7e97f4bc19	setup and installation information added	2020-09-02 16:49:22 +03:00
Tiedemann	1435b7849a	moved allas recipes to a different makefile	2020-09-02 16:35:35 +03:00
Tiedemann	2332732577	make compatible with mac osx and include submodules for required tools	2020-09-02 15:52:34 +03:00
Joerg Tiedemann	cde8e65a5b	new buckets for fetch and store, uncompressed now and follow-links	2020-09-01 16:15:01 +03:00
Joerg Tiedemann	1a279ce6f1	started documentation of project specific models	2020-08-28 15:53:23 +03:00
Joerg Tiedemann	639bd2adda	started documentation of project specific models	2020-08-28 15:51:37 +03:00
Joerg Tiedemann	2c04e48dbe	fixed an important bug in data merging	2020-08-28 11:52:46 +03:00
Joerg Tiedemann	e31550a3ad	enabled fetching OPUS data instead of reading local files if necessary	2020-08-28 10:53:11 +03:00
Joerg Tiedemann	94eeec13eb	take away dependence on local OPUS files for finding data	2020-08-27 22:36:50 +03:00
Joerg Tiedemann	831ee89f76	fixed bug in env.mk	2020-08-26 22:18:12 +03:00
Joerg Tiedemann	596dd993a5	more documentation	2020-08-26 21:45:03 +03:00
Joerg Tiedemann	a8b54f5311	some info about training added	2020-08-26 15:12:38 +03:00
Joerg Tiedemann	2f8a37cc92	more details about data compilation added	2020-08-26 14:31:50 +03:00
Joerg Tiedemann	dac6070069	started some more documentation	2020-08-26 09:59:24 +03:00
Joerg Tiedemann	f2a413b740	minor cleanup in env	2020-08-26 01:01:44 +03:00
Joerg Tiedemann	4c35456038	cleanup in data makefile	2020-08-26 00:44:02 +03:00
Joerg Tiedemann	9375f37886	missing makefile added	2020-08-25 22:42:33 +03:00
Joerg Tiedemann	308bf647f0	fetchdata src and dest dir	2020-08-25 22:07:32 +03:00
Joerg Tiedemann	1e23566f30	fixed fetch and store from and to allas	2020-08-23 10:08:06 +03:00
Joerg Tiedemann	0e27198048	store and fetch work data	2020-08-22 23:51:37 +03:00
Joerg Tiedemann	d7252e32b7	tatoeba monolingual data	2020-08-05 00:00:24 +03:00
Joerg Tiedemann	6bf0207cc6	list of models added	2020-08-03 11:58:51 +03:00
Joerg Tiedemann	c9fcb7f35d	tatoeba langgroup models	2020-08-02 11:38:42 +03:00
Joerg Tiedemann	1b913277b3	tatoeba language group models with various sample sizews	2020-07-25 22:52:33 +03:00
Joerg Tiedemann	5493aeddb4	fixed a problem with lang group targets	2020-07-14 21:40:49 +03:00