moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2024-09-20 07:42:21 +03:00
Code Issues Packages Projects Releases Wiki Activity
57688ae5b2
mosesdecoder/scripts/tokenizer
History
Hieu Hoang 038892601e reminder of language codes
2011-11-30 14:29:34 +07:00
..
nonbreaking_prefixes reminder of language codes 2011-11-30 14:29:34 +07:00
detokenizer.perl when detokenizing, remove whitespace between a pair of CJK (Chinese/Japanese/Korean) words 2011-08-08 15:30:54 +00:00
lowercase.perl add lowercaser 2010-08-02 14:05:23 +00:00
tokenizer.perl official release of experiment.perl 2010-05-04 23:04:10 +00:00