mirror of
https://github.com/moses-smt/mosesdecoder.git
synced 2025-01-08 20:46:59 +03:00
d04bdc7440
Separate "," after a number if it's the end of a sentence. Example: He is tall, He was born in 1800, He wants to go there in 2000. He is tall , He was born in 1800 , He wants to go there in 2000 . |
||
|---|---|---|
| .. | ||
| basic-protected-patterns | ||
| deescape-special-chars-PTB.perl | ||
| deescape-special-chars.perl | ||
| delete-long-words.perl | ||
| detokenizer.perl | ||
| escape-special-chars.perl | ||
| lowercase.perl | ||
| normalize-punctuation.perl | ||
| pre_tokenize_cleaning.py | ||
| pre-tok-clean.perl | ||
| pre-tokenizer.perl | ||
| remove-non-printing-char.perl | ||
| replace-unicode-punctuation.perl | ||
| tokenizer_PTB.perl | ||
| tokenizer.perl | ||