8 Commits

Author	SHA1	Message	Date
Rico Sennrich	0607b5443f	byte-level BPE. Still not fully tested, fails some unit tests with glossaries.	2024-07-24 14:12:50 +02:00
Rico Sennrich	75a69fc153	add some more umlauts to tests to check behavior in different locales	2020-02-21 17:39:42 +01:00
Rico Sennrich	5c7b56ea97	apply BPE dropout on list, not set of symbol pairs (in line with what Provilkov et al. did) ... simplify and optimize apply_bpe code	2019-11-14 15:14:39 +01:00
Rico Sennrich	a40db4510c	documentation	2019-10-30 09:07:54 +01:00
Proyag	ba1db43457	add unittest (and fix python3 integer division in unittest)	2018-07-09 11:12:25 +02:00
Rico Sennrich	65db9c5407	create symlink in old script location (with deprecation warning)	2018-05-16 14:47:23 +01:00
Rico Sennrich	4a1d3a777b	modify files for packaging; thanks to universome	2018-05-16 14:35:23 +01:00
Rico Sennrich	2a4a44b5c0	move files to package structure; add setup.py	2018-05-16 11:44:24 +01:00