moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2024-09-20 23:58:15 +03:00
Code Issues Packages Projects Releases Wiki Activity
1,925 Commits 136 Branches 26 Tags 147 MiB
ad62e27c90
Commit Graph

464 Commits

Author SHA1 Message Date
rosasjolu
ad62e27c90 git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3546 1f5c12ca-751b-0410-a591-d2e778427230 2010-09-22 10:24:52 +00:00
hieuhoang1972
ee842f578c delete data files 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3545 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-22 10:05:11 +00:00
phkoehn
f34b37bad3 added hierarchical alignment view to web analysis tool 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3514 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-17 13:28:04 +00:00
hieuhoang1972
a582d483cc add lowercase script 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3475 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-16 07:32:32 +00:00
phkoehn
fb8b0eb180 new prefix files for tokenizer 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3467 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-15 16:06:04 +00:00
rosasjolu
128a885406 git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3446 1f5c12ca-751b-0410-a591-d2e778427230 2010-09-14 19:27:15 +00:00
suzyh
fa4eca6ccb Added loop to check_if_crashed in EMS experiment.perl to wait in case the .STDERR file is slow in appearing after the step has completed. Reinstated --old-sge and --filterfile command-line arguments to mert-moses.pl. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3420 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-09 11:40:40 +00:00
bhaddow
fd7997dbf5 Fix for mert script from Yu Chen, to make sure it reads the correct 
feature order from the nbest list.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3419 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-08 08:17:05 +00:00
hieuhoang1972
e53aeb903c need non-empty arg in mert-moses.pl otherwise it crashes. The wonders of a non-typechecked, arg checking scripting language 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3417 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-04 00:16:26 +00:00
phkoehn
4a85fd95ce srilm as setting in interpolation 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3416 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-03 12:57:16 +00:00
hieuhoang1972
51b99ede7a delete old qsub args 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3415 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-02 20:42:55 +00:00
bhaddow
12269b062c Only add queue-flags if non-empty. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3414 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-02 09:01:39 +00:00
bhaddow
7efec1a087 Set default max mert iterations to 25. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3413 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-01 19:57:55 +00:00
bhaddow
5f2f345165 max iterations option 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3412 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-01 17:02:12 +00:00
bgottesman
e409b6827c add --max-word-length option to cleaning script, with default value 1000; any segment with a word (or factor) exceeding this length in chars is discarded; motivated by symal.cpp, which has its own such parameter (hardcoded to 1000) and crashes if it encounters a word that exceeds it 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3410 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-23 16:35:14 +00:00
hieuhoang1972
083a9af215 delete alignment info for terminals 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3405 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-13 10:03:13 +00:00
rafpayen
b431f951c5 git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3404 1f5c12ca-751b-0410-a591-d2e778427230 2010-08-13 09:58:17 +00:00
hieuhoang1972
382799dd38 delete win32 make. out-of-date. find a better way of doing this 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3400 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-10 14:41:49 +00:00
hieuhoang1972
9600b6473c delete win32 make. out-of-date. find a better way of doing this 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3399 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-10 14:41:28 +00:00
bhaddow
08a8480136 rename mert-moses-new.pl to mert-moses.pl 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3398 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-10 14:29:07 +00:00
bhaddow
321f528ff5 remove zmert and cmert 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3397 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-10 14:28:03 +00:00
bhaddow
904133fcb7 Merge in the multiple models branch. These changes allow the moses server 
to support multiple translation, language and generation models within the
same process. The main design change is the introduction of a TranslationSystem
object to manage the models, which have been moved out of StaticData.
The changes should have no effect on existing systems.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3394 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-10 13:12:00 +00:00
bhaddow
d31b030bc5 Write correct ttable type when binarising a phrase table 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3392 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-09 12:28:33 +00:00
bhaddow
f2660e8d41 Fix glue grammar generation for new ttable format 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3386 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-06 14:45:37 +00:00
hieuhoang1972
7e6b3766dd visual studio 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3385 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-03 15:56:41 +00:00
bhaddow
faf65dfcd2 Remove unused options. 
Merge in some changes from mert-moses.perl


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3384 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-03 15:54:31 +00:00
hieuhoang1972
579253d3cd add lowercaser 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3380 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-02 14:05:23 +00:00
rafpayen
2ef133e02b add empty fields in glue grammar to accomodate the new phrase table format 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3378 1f5c12ca-751b-0410-a591-d2e778427230
 2010-07-30 15:55:14 +00:00
nicolabertoldi
621428de44 improved description of configuration file for [ttable-file] parameter 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3376 1f5c12ca-751b-0410-a591-d2e778427230
 2010-07-30 09:10:17 +00:00
hieuhoang1972
8adef921ed new format for consolidate-direct 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3374 1f5c12ca-751b-0410-a591-d2e778427230
 2010-07-29 23:20:37 +00:00
hieuhoang1972
0ee6d75566 bug in Good turing 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3372 1f5c12ca-751b-0410-a591-d2e778427230
 2010-07-28 22:49:37 +00:00
hieuhoang1972
340ebbd333 bug in Good turing 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3370 1f5c12ca-751b-0410-a591-d2e778427230
 2010-07-28 21:52:32 +00:00
hieuhoang1972
ae9779dd7f separate PhraseAlignment class into separate file 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3369 1f5c12ca-751b-0410-a591-d2e778427230
 2010-07-28 21:28:14 +00:00
hieuhoang1972
3d9d756055 alignment info, new format 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3363 1f5c12ca-751b-0410-a591-d2e778427230
 2010-07-27 11:04:03 +00:00
rafpayen
b9e74aab90 change phrase-word-alignment to boolean flag instead of string 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3362 1f5c12ca-751b-0410-a591-d2e778427230
 2010-07-22 13:08:46 +00:00
hieuhoang1972
881117d9f5 alignment info in pt 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3361 1f5c12ca-751b-0410-a591-d2e778427230
 2010-07-18 19:49:08 +00:00
hieuhoang1972
31930eb6fc alignment info in pt 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3358 1f5c12ca-751b-0410-a591-d2e778427230
 2010-07-17 22:29:06 +00:00
pjwilliams
fab2e96d2f In extract-rules, if the source or target syntax contains an unsupported 
escape sequence (anything other than "<", ">", "&", "&apos",
and "&quot") then write a warning message and skip the sentence pair
(instead of asserting).


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3350 1f5c12ca-751b-0410-a591-d2e778427230
 2010-06-29 10:41:42 +00:00
mphi
1f6e9b488b the script now calculates the p-value and confidence intervals not only using BLEU, but also the NIST score; 
improved confidence interval representation (avg+-stddev);

fixed bugs



git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3345 1f5c12ca-751b-0410-a591-d2e778427230
 2010-06-22 20:17:42 +00:00
phkoehn
4e0bc582f6 minor improvements: binarizing rule tables in filter script, multiple reference translation in ems analysis 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3284 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-28 22:19:58 +00:00
pjwilliams
05eb33d5ac In train-model.perl, write an 'unknown-lhs' line out to the config file if 
an unknown word label file has been generated.  Also, disable this option
by default since it can greatly increase number of hypotheses generated
and hasn't been shown to help translation yet.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3277 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-27 15:11:16 +00:00
pjwilliams
7d2d79022a Remove temporary file from scripts/released-files. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3267 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-24 15:25:32 +00:00
phkoehn
b271862d7c various updates, mostly related to experiment.perl 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3262 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-18 17:39:16 +00:00
phkoehn
c15fc6f104 minor bug fixes 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3247 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-10 22:41:54 +00:00
phkoehn
524b1b12d2 added info for input phrase coverage 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3245 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-10 03:05:35 +00:00
phkoehn
883d12d482 added info for input word coverage to analysis + fixes 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3244 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-10 00:19:40 +00:00
bojar
dcb8aafca7 rename train-factored-....perl to train-model.perl also in the Makefile and releasing 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3242 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-07 23:17:55 +00:00
phkoehn
447dccfc59 more analysis in experiment.perl 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3234 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-07 11:28:55 +00:00
phkoehn
45ecfa72d2 minor changes to experiment.perl 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3225 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-05 16:06:34 +00:00
phkoehn
2ed6804f12 official release of experiment.perl 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3224 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-04 23:04:10 +00:00