Kenneth Heafield
7c507bfa74
May is not an abbreviation
2015-01-19 16:37:57 -05:00
Hieu Hoang
c48a3aadc1
chmod
2014-12-11 16:54:19 +00:00
Kenneth Heafield
f97ed79a70
Month abbreviations shouldn't be causing a sentence split.
...
Yes this will break existing tokenized data :-(.
2014-12-05 03:41:01 -05:00
Hieu Hoang
eb75e58820
Merge pull request #72 from flammie/master
...
Add Finnish non-breaking prefixes
2014-09-04 16:31:14 +01:00
Flammie Pirinen
1da3df93bc
fix location and remove english notes
2014-09-04 16:01:10 +01:00
Hieu Hoang
23f10cc73f
move notice about czech prefixes to share/README
2014-08-06 15:03:37 +01:00
Hieu Hoang
a8cb0f7f13
Tamil tokenization /P.Arththika
2014-01-05 14:07:28 +00:00
Dimitris Mavroeidis
f9abcb55d4
Update nonbreaking_prefix.el
...
Added non-breaking prefixes for Greek.
2013-10-07 20:51:02 +03:00
Jeremy Gwinnup
a5fb4d1550
Fixed bug in tokenizer.perl where comma separated lists of single
...
characters aren't handled correctly
input> A,B,C,D,E,F
yielded> A, B,C , D,E , F
now yields> A, B, C, D, E, F
Updated Russian nonbreaking prefixes list with capital letters
2013-08-16 14:39:50 -04:00
Achim
038871fdb3
Hungarian and Latvian non-breaking prefix files
2013-03-18 17:17:35 -04:00
Barry Haddow
2b4e61d826
Merge branch 'trunk' into miramerge
...
Compiles, not tested.
Conflicts:
Jamroot
OnDiskPt/PhraseNode.h
OnDiskPt/TargetPhrase.cpp
OnDiskPt/TargetPhrase.h
OnDiskPt/TargetPhraseCollection.cpp
mert/BleuScorer.cpp
mert/Data.cpp
mert/FeatureData.cpp
moses-chart-cmd/src/Main.cpp
moses/src/AlignmentInfo.h
moses/src/ChartManager.cpp
moses/src/LM/Ken.cpp
moses/src/LM/Ken.h
moses/src/LMList.h
moses/src/LexicalReordering.h
moses/src/PhraseDictionaryTree.h
moses/src/ScoreIndexManager.h
moses/src/StaticData.h
moses/src/TargetPhrase.h
moses/src/Word.cpp
scripts/ems/experiment.meta
scripts/ems/experiment.perl
scripts/training/train-model.perl
2012-07-17 13:36:50 +01:00
Karel Bílek
f461dc3e96
czech prefixes
2012-07-14 03:32:10 +02:00
Hieu Hoang
93bff3f201
lock m_vocab variable access in Encode() and Lookup(). Other functions are still not threadsafe
2012-06-26 13:33:34 -04:00
