moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2025-01-08 12:36:35 +03:00
Code Issues Packages Projects Releases Wiki Activity
6,134 Commits 136 Branches 26 Tags 147 MiB
ecfb8465d4
Commit Graph

514 Commits

Author SHA1 Message Date
rosasjolu
ad62e27c90 git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3546 1f5c12ca-751b-0410-a591-d2e778427230 2010-09-22 10:24:52 +00:00
hieuhoang1972
ee842f578c delete data files 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3545 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-22 10:05:11 +00:00
phkoehn
f34b37bad3 added hierarchical alignment view to web analysis tool 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3514 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-17 13:28:04 +00:00
hieuhoang1972
a582d483cc add lowercase script 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3475 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-16 07:32:32 +00:00
phkoehn
fb8b0eb180 new prefix files for tokenizer 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3467 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-15 16:06:04 +00:00
rosasjolu
128a885406 git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3446 1f5c12ca-751b-0410-a591-d2e778427230 2010-09-14 19:27:15 +00:00
suzyh
fa4eca6ccb Added loop to check_if_crashed in EMS experiment.perl to wait in case the .STDERR file is slow in appearing after the step has completed. Reinstated --old-sge and --filterfile command-line arguments to mert-moses.pl. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3420 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-09 11:40:40 +00:00
bhaddow
fd7997dbf5 Fix for mert script from Yu Chen, to make sure it reads the correct 
feature order from the nbest list.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3419 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-08 08:17:05 +00:00
hieuhoang1972
e53aeb903c need non-empty arg in mert-moses.pl otherwise it crashes. The wonders of a non-typechecked, arg checking scripting language 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3417 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-04 00:16:26 +00:00
phkoehn
4a85fd95ce srilm as setting in interpolation 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3416 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-03 12:57:16 +00:00
hieuhoang1972
51b99ede7a delete old qsub args 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3415 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-02 20:42:55 +00:00
bhaddow
12269b062c Only add queue-flags if non-empty. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3414 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-02 09:01:39 +00:00
bhaddow
7efec1a087 Set default max mert iterations to 25. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3413 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-01 19:57:55 +00:00
bhaddow
5f2f345165 max iterations option 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3412 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-01 17:02:12 +00:00
bgottesman
e409b6827c add --max-word-length option to cleaning script, with default value 1000; any segment with a word (or factor) exceeding this length in chars is discarded; motivated by symal.cpp, which has its own such parameter (hardcoded to 1000) and crashes if it encounters a word that exceeds it 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3410 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-23 16:35:14 +00:00
hieuhoang1972
083a9af215 delete alignment info for terminals 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3405 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-13 10:03:13 +00:00
rafpayen
b431f951c5 git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3404 1f5c12ca-751b-0410-a591-d2e778427230 2010-08-13 09:58:17 +00:00
hieuhoang1972
382799dd38 delete win32 make. out-of-date. find a better way of doing this 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3400 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-10 14:41:49 +00:00
hieuhoang1972
9600b6473c delete win32 make. out-of-date. find a better way of doing this 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3399 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-10 14:41:28 +00:00
bhaddow
08a8480136 rename mert-moses-new.pl to mert-moses.pl 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3398 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-10 14:29:07 +00:00
bhaddow
321f528ff5 remove zmert and cmert 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3397 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-10 14:28:03 +00:00
bhaddow
904133fcb7 Merge in the multiple models branch. These changes allow the moses server 
to support multiple translation, language and generation models within the
same process. The main design change is the introduction of a TranslationSystem
object to manage the models, which have been moved out of StaticData.
The changes should have no effect on existing systems.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3394 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-10 13:12:00 +00:00
bhaddow
d31b030bc5 Write correct ttable type when binarising a phrase table 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3392 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-09 12:28:33 +00:00
bhaddow
f2660e8d41 Fix glue grammar generation for new ttable format 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3386 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-06 14:45:37 +00:00
hieuhoang1972
7e6b3766dd visual studio 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3385 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-03 15:56:41 +00:00
bhaddow
faf65dfcd2 Remove unused options. 
Merge in some changes from mert-moses.perl


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3384 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-03 15:54:31 +00:00
hieuhoang1972
579253d3cd add lowercaser 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3380 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-02 14:05:23 +00:00
rafpayen
2ef133e02b add empty fields in glue grammar to accomodate the new phrase table format 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3378 1f5c12ca-751b-0410-a591-d2e778427230
 2010-07-30 15:55:14 +00:00
nicolabertoldi
621428de44 improved description of configuration file for [ttable-file] parameter 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3376 1f5c12ca-751b-0410-a591-d2e778427230
 2010-07-30 09:10:17 +00:00
hieuhoang1972
8adef921ed new format for consolidate-direct 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3374 1f5c12ca-751b-0410-a591-d2e778427230
 2010-07-29 23:20:37 +00:00
hieuhoang1972
0ee6d75566 bug in Good turing 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3372 1f5c12ca-751b-0410-a591-d2e778427230
 2010-07-28 22:49:37 +00:00
hieuhoang1972
340ebbd333 bug in Good turing 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3370 1f5c12ca-751b-0410-a591-d2e778427230
 2010-07-28 21:52:32 +00:00
hieuhoang1972
ae9779dd7f separate PhraseAlignment class into separate file 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3369 1f5c12ca-751b-0410-a591-d2e778427230
 2010-07-28 21:28:14 +00:00
hieuhoang1972
3d9d756055 alignment info, new format 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3363 1f5c12ca-751b-0410-a591-d2e778427230
 2010-07-27 11:04:03 +00:00
rafpayen
b9e74aab90 change phrase-word-alignment to boolean flag instead of string 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3362 1f5c12ca-751b-0410-a591-d2e778427230
 2010-07-22 13:08:46 +00:00
hieuhoang1972
881117d9f5 alignment info in pt 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3361 1f5c12ca-751b-0410-a591-d2e778427230
 2010-07-18 19:49:08 +00:00
hieuhoang1972
31930eb6fc alignment info in pt 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3358 1f5c12ca-751b-0410-a591-d2e778427230
 2010-07-17 22:29:06 +00:00
pjwilliams
fab2e96d2f In extract-rules, if the source or target syntax contains an unsupported 
escape sequence (anything other than "<", ">", "&", "&apos",
and "&quot") then write a warning message and skip the sentence pair
(instead of asserting).


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3350 1f5c12ca-751b-0410-a591-d2e778427230
 2010-06-29 10:41:42 +00:00
mphi
1f6e9b488b the script now calculates the p-value and confidence intervals not only using BLEU, but also the NIST score; 
improved confidence interval representation (avg+-stddev);

fixed bugs



git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3345 1f5c12ca-751b-0410-a591-d2e778427230
 2010-06-22 20:17:42 +00:00
phkoehn
4e0bc582f6 minor improvements: binarizing rule tables in filter script, multiple reference translation in ems analysis 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3284 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-28 22:19:58 +00:00
pjwilliams
05eb33d5ac In train-model.perl, write an 'unknown-lhs' line out to the config file if 
an unknown word label file has been generated.  Also, disable this option
by default since it can greatly increase number of hypotheses generated
and hasn't been shown to help translation yet.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3277 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-27 15:11:16 +00:00
pjwilliams
7d2d79022a Remove temporary file from scripts/released-files. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3267 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-24 15:25:32 +00:00
phkoehn
b271862d7c various updates, mostly related to experiment.perl 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3262 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-18 17:39:16 +00:00
phkoehn
c15fc6f104 minor bug fixes 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3247 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-10 22:41:54 +00:00
phkoehn
524b1b12d2 added info for input phrase coverage 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3245 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-10 03:05:35 +00:00
phkoehn
883d12d482 added info for input word coverage to analysis + fixes 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3244 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-10 00:19:40 +00:00
bojar
dcb8aafca7 rename train-factored-....perl to train-model.perl also in the Makefile and releasing 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3242 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-07 23:17:55 +00:00
phkoehn
447dccfc59 more analysis in experiment.perl 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3234 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-07 11:28:55 +00:00
phkoehn
45ecfa72d2 minor changes to experiment.perl 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3225 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-05 16:06:34 +00:00
phkoehn
2ed6804f12 official release of experiment.perl 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3224 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-04 23:04:10 +00:00
bojar
4cc43c61f2 without parallelizer, -nbest-list must not be quoted 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3214 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-29 20:25:36 +00:00
bojar
21de1e121f Merge branch 'master' into moses-svn 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3192 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-21 14:48:32 +00:00
bojar
db8e3357f5 Merge branch 'zmert' into moses-svn 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3152 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-20 22:03:04 +00:00
bojar
aa6043a556 mert-moses must quote -n-best-list *if* passed through parallelizer (and must not otherwise) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3151 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-20 17:02:58 +00:00
bojar
8ea058bfc7 fixed handling of the lmodel section; has only 3 ints 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3150 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-20 17:02:41 +00:00
bojar
a7677e7fa2 releasing zmert-moses as well 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3145 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-20 11:11:49 +00:00
bojar
f36f347014 safer everything: tempdir, open, execution 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3144 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-20 11:10:31 +00:00
pjwilliams
2edfc16912 Merge remaining script support for tree-based models from mt3_chart. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3137 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-16 09:45:51 +00:00
hieuhoang1972
a2233d0f8d xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3136 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-14 16:53:39 +00:00
pjwilliams
264c9150d1 Use external consolidate' program in train-factored-phrase-model.perl.' 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3135 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-14 15:51:20 +00:00
pjwilliams
5faaedc0df Copy in consolidate,' consolidate-direct,' and the new version of 
`score' from mt3_chart.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3134 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-14 15:50:17 +00:00
leven101
929bcf25fa added traing/lexical-reordering subdir to makefile 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3133 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-13 17:18:46 +00:00
hieuhoang1972
06ee9a3be3 vs 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3132 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-13 16:50:44 +00:00
pjwilliams
53cb08efca Use a generic version of the SAFE_GETLINE macro in scripts/phrase-extract 
instead of defining one per source file.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3131 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-13 16:29:55 +00:00
hieuhoang1972
0440dfe079 xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3130 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-13 16:13:56 +00:00
pjwilliams
580acce9e2 Integrate rule extraction code from mt3_chart. There are now two extract 
programs: `extract' for the phrase-based model and `extract-rules' for
tree-based models.  They could be combined into a single program, but
they're probably sufficiently different that it isn't worthwhile.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3129 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-13 15:34:39 +00:00
pjwilliams
51ae927ede Start merging in rule extraction code from mt3_chart branch. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3126 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-12 15:22:50 +00:00
pjwilliams
9c2536417f Remove file limit option for phrase extraction. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3122 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-12 11:56:54 +00:00
pjwilliams
99f1c92edb Remove redundant --ZipFiles option from extract. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3120 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-12 10:53:08 +00:00
pjwilliams
4c6c4b71cf Remove redundant --ProperConditioning option from extract. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3118 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-12 10:41:32 +00:00
bojar
7172d05a43 fixed a bug, too eager check for preprocessing type; not needed in some cases 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3111 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 23:21:13 +00:00
bojar
82d6cc714e use qruncmd to parallelize srunblocks 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3110 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 23:20:59 +00:00
bojar
c5f44a2abf better verbosity level for srunblocks: emit some (most importantly fatal) msgs 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3109 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 23:20:45 +00:00
bojar
390ee866d8 require TMT_ROOT only if TectoMT will be actually needed 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3108 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 23:20:30 +00:00
bojar
9fe574f6ed fixed weights for SemPOS_BLEU metric 
usage: MERTFLAGS="--semposbleu-weights <sempos_weight>:<bleu_weight>"
e.g. --semposbleu-weights 2:1 to increase the weight of SemPOS


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3107 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:47:22 +00:00
bojar
26d77d15b2 added option --semposbleu-weights to specify weight of SemPOS and BLEU in SemPOS_BLEU metric 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3106 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:47:04 +00:00
bojar
c954a507c6 fixed error message in zmert-moses.pl 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3105 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:46:49 +00:00
bojar
8e2e4eeecc bug fixed 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3104 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:46:34 +00:00
bojar
b8a6048e81 ZMERTSEMPOSSOURCE=factors working for SemPOS_BLEU 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3103 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:46:17 +00:00
bojar
866776810b typo fixed 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3102 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:45:59 +00:00
bojar
b87370aec2 few bugs fixed in ZMERTSEMPOSSOURCE=factors part 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3101 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:45:43 +00:00
bojar
19184783a9 fixed bug in --mert-verbose parameter 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3100 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:45:28 +00:00
bojar
9f3d2f427d fixed nbest-list conversion 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3098 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:44:41 +00:00
bojar
2fb22064a3 bug in passing nbest-lists to zmert 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3097 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:44:21 +00:00
bojar
cd64a02344 fixed bug - missing loop to copy one file to another 
Conflicts:

	scripts/training/zmert-moses.pl


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3096 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:43:55 +00:00
bojar
b276bacb72 bug in foreach 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3095 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:43:27 +00:00
bojar
94eb8a5c1b added default parameters for basic metrics 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3094 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:43:10 +00:00
bojar
2604a890d7 zmert training with new metric SemPOS_BLEU - linear combination of SemPOS and BLEU 
SemPOS_BLEU requires new TMT block ForSemPOSBLEUMetric.pm in TectoMT. Please,
update your TMT.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3093 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:42:49 +00:00
bojar
5b1541b96d Fixed order of feature scores passed to zmert. Evaluation using BLEU works. 
Zmert uses different order of features than Moses. It is necessary to reorder
them when passing to Zmert in nbest-lists.
Previous versions used wrong copy of moses.ini, which linked to phrase tables
that were already filtered for tuning. Thus, phrase tables for evaluation were
missing a lot of phrases and the reported scores were too low.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3092 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:42:20 +00:00
bojar
5a1c308673 Setting number of zmert iterations back to unlimited in zmert config. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3091 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:41:55 +00:00
bojar
671e7e2e4f Default size of nbest-list in zmert-moses.pl set back to 100. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3090 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:41:31 +00:00
bojar
1ce9a15483 Bug fixes im zmert-moses.pl and zmert.jar. Zmert works. 
Now it is possible to lauch Zmert with SemPOS metric.
It is possible to select a smaller model for McD parser by uncommenting line
with pdt20_train_autTag_golden_latin2_pruned_0.10.model in file zmert.tmt-scen.
Sentences are then analyzed faster (if you use TectoMT to get SemPOS tags).


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3089 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:41:11 +00:00
bojar
1b28cde2e8 small fixes of paths 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3088 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:40:29 +00:00
bojar
d47b82369f some small code modifications in zmert-moses.pl 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3087 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:39:47 +00:00
bojar
d34af9b769 updated zmert-moses.pl - zmert with sempos still not running 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3086 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:39:31 +00:00
bojar
49bb6b8882 Updated version of zmert training - still not finished SemPOS factor loading via TMT 
zmert-moses.pl - launches zmert training
zmert-decoder.pl - an is launched by zmert after each training iteration to compute scores
                   with updated lambdas
zmert.tmt-scen - TMT scenario to extract SemPOS factor for sentence without any factors


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3085 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:39:14 +00:00
bojar
90fecff3aa New mert training (Zmert) for Moses\n\nZmert jar includes SemPOS metric extension. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3084 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:38:53 +00:00
pjwilliams
b3f6e211fd Fix mistakes in previous commit (oh, and revert to my own svn username 
to prevent my shoddy check-ins from further sullying Hieu's good name).


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3083 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 14:02:58 +00:00
hieuhoang1972
c6d20e1f9f Update the training scripts to support the new format parameter for 
'ttable-file'


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3082 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 11:37:43 +00:00
sarst
7d3a22c8a7 Bugfixes for the new lexical reordering. Running without a reordering model, and with the reordering flag specyfying distance now works. Formatting of the extract.o file is now correct with 'f'-reordering models. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3054 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-06 12:08:04 +00:00