moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2024-08-17 07:20:48 +03:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #209 from joelb-git/multi-bleu-detok-non-ascii-fix

Browse Source 
Fix non-ASCII lowercasing
This commit is contained in:
Hieu Hoang 2019-03-01 23:26:12 +00:00 committed by GitHub
commit 187a75cb55
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 3 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
scripts/generic/multi-bleu-detok.perl
View File

@ -14,6 +14,9 @@
use warnings;
use strict;
binmode(STDIN, ":utf8");
use open ':encoding(UTF-8)';
my $lowercase = 0;
if ($ARGV[0] eq "-lc") {
$lowercase = 1;