Commit Graph

10 Commits

Author SHA1 Message Date
Rico Sennrich
0607b5443f byte-level BPE. Still not fully tested, fails some unit tests with glossaries. 2024-07-24 14:12:50 +02:00
Rico Sennrich
810ee1487a suppress extra warnings 2022-09-05 14:31:23 +02:00
yimmon
bce5f8abcf Add parallel support (--num-workers) 2020-06-18 01:04:38 +08:00
Rico Sennrich
73a6e55d5b suppert argument --total-symbols in learn_joint_bpe_and_vocab 2018-08-20 12:07:45 +01:00
Jean A. Senellart
8450bd3231 condition parameter conversion to python 2 2018-07-18 07:36:11 +10:00
Jean A. Senellart
9df8997c78 enable unicode separator 2018-07-12 11:52:30 +09:00
Rico Sennrich
8012fd6607 fix pip package with Python3 2018-05-21 10:53:59 +01:00
Rico Sennrich
65db9c5407 create symlink in old script location (with deprecation warning) 2018-05-16 14:47:23 +01:00
Rico Sennrich
4a1d3a777b modify files for packaging; thanks to universome 2018-05-16 14:35:23 +01:00
Rico Sennrich
2a4a44b5c0 move files to package structure; add setup.py 2018-05-16 11:44:24 +01:00