moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2024-10-05 15:58:03 +03:00
Code Issues Packages Projects Releases Wiki Activity

Single quotes should be escaped as single quotes.

Browse Source
This commit is contained in:
alvations 2019-11-25 10:10:40 +08:00 committed by GitHub
parent 74d54b54c3
commit f6d7adde15
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 2 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
scripts/tokenizer/normalize-punctuation.perl
View File

@ -42,8 +42,8 @@ while(<STDIN>) {
s/´/\'/g;
s/([a-z])‘([a-z])/$1\'$2/gi;
s/([a-z])’([a-z])/$1\'$2/gi;
s/‘/\"/g;
s/‚/\"/g;
s/‘/\'/g;
s/‚/\'/g;
s/’/\"/g;
s/''/\"/g;
s/´´/\"/g;