pjwilliams
05eb33d5ac
In train-model.perl, write an 'unknown-lhs' line out to the config file if
...
an unknown word label file has been generated. Also, disable this option
by default since it can greatly increase number of hypotheses generated
and hasn't been shown to help translation yet.
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3277 1f5c12ca-751b-0410-a591-d2e778427230
2010-05-27 15:11:16 +00:00
pjwilliams
7d2d79022a
Remove temporary file from scripts/released-files.
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3267 1f5c12ca-751b-0410-a591-d2e778427230
2010-05-24 15:25:32 +00:00
phkoehn
b271862d7c
various updates, mostly related to experiment.perl
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3262 1f5c12ca-751b-0410-a591-d2e778427230
2010-05-18 17:39:16 +00:00
phkoehn
c15fc6f104
minor bug fixes
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3247 1f5c12ca-751b-0410-a591-d2e778427230
2010-05-10 22:41:54 +00:00
phkoehn
524b1b12d2
added info for input phrase coverage
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3245 1f5c12ca-751b-0410-a591-d2e778427230
2010-05-10 03:05:35 +00:00
phkoehn
883d12d482
added info for input word coverage to analysis + fixes
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3244 1f5c12ca-751b-0410-a591-d2e778427230
2010-05-10 00:19:40 +00:00
bojar
dcb8aafca7
rename train-factored-....perl to train-model.perl also in the Makefile and releasing
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3242 1f5c12ca-751b-0410-a591-d2e778427230
2010-05-07 23:17:55 +00:00
phkoehn
447dccfc59
more analysis in experiment.perl
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3234 1f5c12ca-751b-0410-a591-d2e778427230
2010-05-07 11:28:55 +00:00
phkoehn
45ecfa72d2
minor changes to experiment.perl
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3225 1f5c12ca-751b-0410-a591-d2e778427230
2010-05-05 16:06:34 +00:00
phkoehn
2ed6804f12
official release of experiment.perl
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3224 1f5c12ca-751b-0410-a591-d2e778427230
2010-05-04 23:04:10 +00:00
bojar
4cc43c61f2
without parallelizer, -nbest-list must not be quoted
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3214 1f5c12ca-751b-0410-a591-d2e778427230
2010-04-29 20:25:36 +00:00
bojar
21de1e121f
Merge branch 'master' into moses-svn
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3192 1f5c12ca-751b-0410-a591-d2e778427230
2010-04-21 14:48:32 +00:00
bojar
db8e3357f5
Merge branch 'zmert' into moses-svn
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3152 1f5c12ca-751b-0410-a591-d2e778427230
2010-04-20 22:03:04 +00:00
bojar
aa6043a556
mert-moses must quote -n-best-list *if* passed through parallelizer (and must not otherwise)
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3151 1f5c12ca-751b-0410-a591-d2e778427230
2010-04-20 17:02:58 +00:00
bojar
8ea058bfc7
fixed handling of the lmodel section; has only 3 ints
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3150 1f5c12ca-751b-0410-a591-d2e778427230
2010-04-20 17:02:41 +00:00
bojar
a7677e7fa2
releasing zmert-moses as well
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3145 1f5c12ca-751b-0410-a591-d2e778427230
2010-04-20 11:11:49 +00:00
bojar
f36f347014
safer everything: tempdir, open, execution
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3144 1f5c12ca-751b-0410-a591-d2e778427230
2010-04-20 11:10:31 +00:00
pjwilliams
2edfc16912
Merge remaining script support for tree-based models from mt3_chart.
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3137 1f5c12ca-751b-0410-a591-d2e778427230
2010-04-16 09:45:51 +00:00
hieuhoang1972
a2233d0f8d
xcode
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3136 1f5c12ca-751b-0410-a591-d2e778427230
2010-04-14 16:53:39 +00:00
pjwilliams
264c9150d1
Use external consolidate' program in
train-factored-phrase-model.perl.'
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3135 1f5c12ca-751b-0410-a591-d2e778427230
2010-04-14 15:51:20 +00:00
pjwilliams
5faaedc0df
Copy in consolidate,'
consolidate-direct,' and the new version of
...
`score' from mt3_chart.
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3134 1f5c12ca-751b-0410-a591-d2e778427230
2010-04-14 15:50:17 +00:00
leven101
929bcf25fa
added traing/lexical-reordering subdir to makefile
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3133 1f5c12ca-751b-0410-a591-d2e778427230
2010-04-13 17:18:46 +00:00
hieuhoang1972
06ee9a3be3
vs
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3132 1f5c12ca-751b-0410-a591-d2e778427230
2010-04-13 16:50:44 +00:00
pjwilliams
53cb08efca
Use a generic version of the SAFE_GETLINE macro in scripts/phrase-extract
...
instead of defining one per source file.
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3131 1f5c12ca-751b-0410-a591-d2e778427230
2010-04-13 16:29:55 +00:00
hieuhoang1972
0440dfe079
xcode
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3130 1f5c12ca-751b-0410-a591-d2e778427230
2010-04-13 16:13:56 +00:00
pjwilliams
580acce9e2
Integrate rule extraction code from mt3_chart. There are now two extract
...
programs: `extract' for the phrase-based model and `extract-rules' for
tree-based models. They could be combined into a single program, but
they're probably sufficiently different that it isn't worthwhile.
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3129 1f5c12ca-751b-0410-a591-d2e778427230
2010-04-13 15:34:39 +00:00
pjwilliams
51ae927ede
Start merging in rule extraction code from mt3_chart branch.
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3126 1f5c12ca-751b-0410-a591-d2e778427230
2010-04-12 15:22:50 +00:00
pjwilliams
9c2536417f
Remove file limit option for phrase extraction.
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3122 1f5c12ca-751b-0410-a591-d2e778427230
2010-04-12 11:56:54 +00:00
pjwilliams
99f1c92edb
Remove redundant --ZipFiles option from extract.
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3120 1f5c12ca-751b-0410-a591-d2e778427230
2010-04-12 10:53:08 +00:00
pjwilliams
4c6c4b71cf
Remove redundant --ProperConditioning option from extract.
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3118 1f5c12ca-751b-0410-a591-d2e778427230
2010-04-12 10:41:32 +00:00
bojar
7172d05a43
fixed a bug, too eager check for preprocessing type; not needed in some cases
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3111 1f5c12ca-751b-0410-a591-d2e778427230
2010-04-09 23:21:13 +00:00
bojar
82d6cc714e
use qruncmd to parallelize srunblocks
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3110 1f5c12ca-751b-0410-a591-d2e778427230
2010-04-09 23:20:59 +00:00
bojar
c5f44a2abf
better verbosity level for srunblocks: emit some (most importantly fatal) msgs
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3109 1f5c12ca-751b-0410-a591-d2e778427230
2010-04-09 23:20:45 +00:00
bojar
390ee866d8
require TMT_ROOT only if TectoMT will be actually needed
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3108 1f5c12ca-751b-0410-a591-d2e778427230
2010-04-09 23:20:30 +00:00
bojar
9fe574f6ed
fixed weights for SemPOS_BLEU metric
...
usage: MERTFLAGS="--semposbleu-weights <sempos_weight>:<bleu_weight>"
e.g. --semposbleu-weights 2:1 to increase the weight of SemPOS
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3107 1f5c12ca-751b-0410-a591-d2e778427230
2010-04-09 22:47:22 +00:00
bojar
26d77d15b2
added option --semposbleu-weights to specify weight of SemPOS and BLEU in SemPOS_BLEU metric
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3106 1f5c12ca-751b-0410-a591-d2e778427230
2010-04-09 22:47:04 +00:00
bojar
c954a507c6
fixed error message in zmert-moses.pl
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3105 1f5c12ca-751b-0410-a591-d2e778427230
2010-04-09 22:46:49 +00:00
bojar
8e2e4eeecc
bug fixed
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3104 1f5c12ca-751b-0410-a591-d2e778427230
2010-04-09 22:46:34 +00:00
bojar
b8a6048e81
ZMERTSEMPOSSOURCE=factors working for SemPOS_BLEU
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3103 1f5c12ca-751b-0410-a591-d2e778427230
2010-04-09 22:46:17 +00:00
bojar
866776810b
typo fixed
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3102 1f5c12ca-751b-0410-a591-d2e778427230
2010-04-09 22:45:59 +00:00
bojar
b87370aec2
few bugs fixed in ZMERTSEMPOSSOURCE=factors part
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3101 1f5c12ca-751b-0410-a591-d2e778427230
2010-04-09 22:45:43 +00:00
bojar
19184783a9
fixed bug in --mert-verbose parameter
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3100 1f5c12ca-751b-0410-a591-d2e778427230
2010-04-09 22:45:28 +00:00
bojar
9f3d2f427d
fixed nbest-list conversion
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3098 1f5c12ca-751b-0410-a591-d2e778427230
2010-04-09 22:44:41 +00:00
bojar
2fb22064a3
bug in passing nbest-lists to zmert
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3097 1f5c12ca-751b-0410-a591-d2e778427230
2010-04-09 22:44:21 +00:00
bojar
cd64a02344
fixed bug - missing loop to copy one file to another
...
Conflicts:
scripts/training/zmert-moses.pl
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3096 1f5c12ca-751b-0410-a591-d2e778427230
2010-04-09 22:43:55 +00:00
bojar
b276bacb72
bug in foreach
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3095 1f5c12ca-751b-0410-a591-d2e778427230
2010-04-09 22:43:27 +00:00
bojar
94eb8a5c1b
added default parameters for basic metrics
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3094 1f5c12ca-751b-0410-a591-d2e778427230
2010-04-09 22:43:10 +00:00
bojar
2604a890d7
zmert training with new metric SemPOS_BLEU - linear combination of SemPOS and BLEU
...
SemPOS_BLEU requires new TMT block ForSemPOSBLEUMetric.pm in TectoMT. Please,
update your TMT.
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3093 1f5c12ca-751b-0410-a591-d2e778427230
2010-04-09 22:42:49 +00:00
bojar
5b1541b96d
Fixed order of feature scores passed to zmert. Evaluation using BLEU works.
...
Zmert uses different order of features than Moses. It is necessary to reorder
them when passing to Zmert in nbest-lists.
Previous versions used wrong copy of moses.ini, which linked to phrase tables
that were already filtered for tuning. Thus, phrase tables for evaluation were
missing a lot of phrases and the reported scores were too low.
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3092 1f5c12ca-751b-0410-a591-d2e778427230
2010-04-09 22:42:20 +00:00
bojar
5a1c308673
Setting number of zmert iterations back to unlimited in zmert config.
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3091 1f5c12ca-751b-0410-a591-d2e778427230
2010-04-09 22:41:55 +00:00