moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2025-01-06 19:49:41 +03:00
Code Issues Packages Projects Releases Wiki Activity
1,929 Commits 136 Branches 26 Tags 147 MiB
7929e4624e
Commit Graph

332 Commits

Author SHA1 Message Date
bojar
c5f44a2abf better verbosity level for srunblocks: emit some (most importantly fatal) msgs 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3109 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 23:20:45 +00:00
bojar
390ee866d8 require TMT_ROOT only if TectoMT will be actually needed 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3108 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 23:20:30 +00:00
bojar
9fe574f6ed fixed weights for SemPOS_BLEU metric 
usage: MERTFLAGS="--semposbleu-weights <sempos_weight>:<bleu_weight>"
e.g. --semposbleu-weights 2:1 to increase the weight of SemPOS


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3107 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:47:22 +00:00
bojar
26d77d15b2 added option --semposbleu-weights to specify weight of SemPOS and BLEU in SemPOS_BLEU metric 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3106 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:47:04 +00:00
bojar
c954a507c6 fixed error message in zmert-moses.pl 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3105 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:46:49 +00:00
bojar
8e2e4eeecc bug fixed 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3104 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:46:34 +00:00
bojar
b8a6048e81 ZMERTSEMPOSSOURCE=factors working for SemPOS_BLEU 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3103 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:46:17 +00:00
bojar
866776810b typo fixed 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3102 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:45:59 +00:00
bojar
b87370aec2 few bugs fixed in ZMERTSEMPOSSOURCE=factors part 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3101 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:45:43 +00:00
bojar
19184783a9 fixed bug in --mert-verbose parameter 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3100 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:45:28 +00:00
bojar
9f3d2f427d fixed nbest-list conversion 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3098 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:44:41 +00:00
bojar
2fb22064a3 bug in passing nbest-lists to zmert 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3097 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:44:21 +00:00
bojar
cd64a02344 fixed bug - missing loop to copy one file to another 
Conflicts:

	scripts/training/zmert-moses.pl


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3096 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:43:55 +00:00
bojar
b276bacb72 bug in foreach 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3095 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:43:27 +00:00
bojar
94eb8a5c1b added default parameters for basic metrics 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3094 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:43:10 +00:00
bojar
2604a890d7 zmert training with new metric SemPOS_BLEU - linear combination of SemPOS and BLEU 
SemPOS_BLEU requires new TMT block ForSemPOSBLEUMetric.pm in TectoMT. Please,
update your TMT.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3093 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:42:49 +00:00
bojar
5b1541b96d Fixed order of feature scores passed to zmert. Evaluation using BLEU works. 
Zmert uses different order of features than Moses. It is necessary to reorder
them when passing to Zmert in nbest-lists.
Previous versions used wrong copy of moses.ini, which linked to phrase tables
that were already filtered for tuning. Thus, phrase tables for evaluation were
missing a lot of phrases and the reported scores were too low.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3092 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:42:20 +00:00
bojar
5a1c308673 Setting number of zmert iterations back to unlimited in zmert config. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3091 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:41:55 +00:00
bojar
671e7e2e4f Default size of nbest-list in zmert-moses.pl set back to 100. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3090 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:41:31 +00:00
bojar
1ce9a15483 Bug fixes im zmert-moses.pl and zmert.jar. Zmert works. 
Now it is possible to lauch Zmert with SemPOS metric.
It is possible to select a smaller model for McD parser by uncommenting line
with pdt20_train_autTag_golden_latin2_pruned_0.10.model in file zmert.tmt-scen.
Sentences are then analyzed faster (if you use TectoMT to get SemPOS tags).


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3089 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:41:11 +00:00
bojar
1b28cde2e8 small fixes of paths 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3088 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:40:29 +00:00
bojar
d47b82369f some small code modifications in zmert-moses.pl 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3087 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:39:47 +00:00
bojar
d34af9b769 updated zmert-moses.pl - zmert with sempos still not running 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3086 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:39:31 +00:00
bojar
49bb6b8882 Updated version of zmert training - still not finished SemPOS factor loading via TMT 
zmert-moses.pl - launches zmert training
zmert-decoder.pl - an is launched by zmert after each training iteration to compute scores
                   with updated lambdas
zmert.tmt-scen - TMT scenario to extract SemPOS factor for sentence without any factors


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3085 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:39:14 +00:00
bojar
90fecff3aa New mert training (Zmert) for Moses\n\nZmert jar includes SemPOS metric extension. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3084 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:38:53 +00:00
pjwilliams
b3f6e211fd Fix mistakes in previous commit (oh, and revert to my own svn username 
to prevent my shoddy check-ins from further sullying Hieu's good name).


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3083 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 14:02:58 +00:00
hieuhoang1972
c6d20e1f9f Update the training scripts to support the new format parameter for 
'ttable-file'


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3082 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 11:37:43 +00:00
sarst
7d3a22c8a7 Bugfixes for the new lexical reordering. Running without a reordering model, and with the reordering flag specyfying distance now works. Formatting of the extract.o file is now correct with 'f'-reordering models. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3054 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-06 12:08:04 +00:00
bhaddow
bd9f392875 Fix for training with non-lexicalised reordering 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3016 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-01 14:05:58 +00:00
sarst
943275e331 Set the default debug mode to 0 in train-factored-phrase-model.perl (which was the case before merging the hierarchical reordering branch to trunk) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3009 1f5c12ca-751b-0410-a591-d2e778427230
 2010-03-30 12:09:19 +00:00
bhaddow
9573147e36 Fix --reordering-table option 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3004 1f5c12ca-751b-0410-a591-d2e778427230
 2010-03-26 09:32:15 +00:00
bhaddow
8060024cc1 Can now specify reordering table when executing step 9 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2997 1f5c12ca-751b-0410-a591-d2e778427230
 2010-03-23 18:55:48 +00:00
bhaddow
a9920a68e1 remove extra . from ini 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2991 1f5c12ca-751b-0410-a591-d2e778427230
 2010-03-20 21:18:55 +00:00
bhaddow
4b6d3dddd2 fix error in merge 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2990 1f5c12ca-751b-0410-a591-d2e778427230
 2010-03-19 18:35:48 +00:00
bhaddow
795224736b Merge revisions 2670-2988 from track. Passes all regression except lexicalised 
reordering


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2989 1f5c12ca-751b-0410-a591-d2e778427230
 2010-03-19 17:52:51 +00:00
bgottesman
872d50dd9b Fix a line that extracts the source FACTORLIST from a FACTORMAP (in the terms of the Moses manual) so that it succeeds even if the target FACTORLIST contains commas, i.e. contains multiple factors 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2965 1f5c12ca-751b-0410-a591-d2e778427230
 2010-03-10 18:03:53 +00:00
bojar
d7335c0de1 avoid reducing corpus file if it already contains the needed factors 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2945 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-24 16:14:07 +00:00
sarst
435539d4ed Added an option to choose exactly which steps to run, not only the first and the last (since we sometimes need to do step 5 and 7 to retrain lexical reordering) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2917 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-19 18:03:44 +00:00
sarst
b95cc2f556 Added the check from word-based models of the alignment points in the adjacent corners, to the more complex models. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2916 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-19 15:15:24 +00:00
sarst
bdfe47f99b added option to skip phrase-scoring, when we want to train new reordering models on old trainings 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2914 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-18 21:16:00 +00:00
sarst
03e24c9856 changed the maximum model config, to always print all models 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2912 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-18 13:45:15 +00:00
sarst
a86470c974 lexical reordering models: it is now possible to add the maximal reordering orientations to the extract file, and the collapsing information is no longer part of the filename. Also removed some old variables, that are not used anymore. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2911 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-18 13:07:32 +00:00
hieuhoang1972
ad3b032ebf roll back 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2910 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-18 11:00:40 +00:00
chardmeier
b10d8eb469 Added lex reo scorer to released-files and removed extra ||| from reordering table. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2894 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-12 16:23:32 +00:00
sarst
c65945b531 Cleaned up lescial reordering scoring, and sent vectors as references instead of copying them. Fixed bugs in extract: it used to choose the wrong orientation at end of sentences, and the hierarchical model typ is no longer dependent on the phrase-based model type. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2892 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-12 13:46:33 +00:00
sarst
e812b2a8e3 Bugfixes: printing the correct phrases in the table, and fixed misspelling of monotonicity. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2885 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-11 09:09:44 +00:00
sarst
92368ba490 Rewrote the lexical reordering model scoring in C++. Adapted train-factored-phrase-model.perl to that change. Minor fixes in other places, for compatibility 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2884 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-10 17:19:06 +00:00
mphi
05e21dc5e2 fixed compatibility of the --final-alignment-model 2 switch and mgiza (ibm2 is done in a single thread, hence no *.part* files) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2859 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-04 16:12:57 +00:00
bojar
a9c2069720 switching to bash, avoiding csh's >& redirection (not accepted by dash) 
making --robust take the maximum number of restarts


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2848 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-03 14:23:53 +00:00
bojar
011840bf65 adding --force-factored-filenames to avoid problems with eager removal of '0-0' 
Conflicts:

	scripts/training/train-factored-phrase-model.perl


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2847 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-03 14:23:35 +00:00
pasmargo
275c06d9e7 Kneser-Ney and modified Kneser-Ney smoothing implementation. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2837 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-02 18:14:01 +00:00
pasmargo
63bdf3b602 reverted to previous version due to a couple of mistakes 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2836 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-02 16:51:14 +00:00
pasmargo
71b15ae0d4 Kneser-Ney and modified Kneser-Ney smoothing implementation. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2835 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-02 16:28:56 +00:00
naditomeh
242d6c6ddd word-based, phrase-based and hierarchical reordering is implemented in the training 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2823 1f5c12ca-751b-0410-a591-d2e778427230
 2010-01-31 23:56:45 +00:00
sanmarf
c37f2dc14e 4th MT marathon - lexical decomposition 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2805 1f5c12ca-751b-0410-a591-d2e778427230
 2010-01-30 10:11:35 +00:00
sarst
0785c09d03 bugfix for fatored-training config 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2804 1f5c12ca-751b-0410-a591-d2e778427230
 2010-01-30 09:34:55 +00:00
sarst
f2a5678541 added new file hierarchical.h 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2803 1f5c12ca-751b-0410-a591-d2e778427230
 2010-01-30 09:16:27 +00:00
sarst
6a13b8d186 bugfixes in train-factored-phrase-model*.perl 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2801 1f5c12ca-751b-0410-a591-d2e778427230
 2010-01-30 00:09:45 +00:00
sarst
bf70dd4767 subimtted working scripts for hierarchical training (msd) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2796 1f5c12ca-751b-0410-a591-d2e778427230
 2010-01-29 22:38:18 +00:00
naditomeh
ad3b0760b2 adding extract.cpp 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2770 1f5c12ca-751b-0410-a591-d2e778427230
 2010-01-29 13:59:04 +00:00
naditomeh
03de8a99d8 adding extract.cpp 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2769 1f5c12ca-751b-0410-a591-d2e778427230
 2010-01-29 13:41:34 +00:00
sarst
4eec020d5b bugfixes to train-factored-phrase-model.perl 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2764 1f5c12ca-751b-0410-a591-d2e778427230
 2010-01-29 12:11:10 +00:00
sarst
a9ef19edf0 updated train-factored-phrase-model.perl to work with the new hierarchical reordering framework 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2759 1f5c12ca-751b-0410-a591-d2e778427230
 2010-01-29 11:29:39 +00:00
hieuhoang1972
53e54def0b indent 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2723 1f5c12ca-751b-0410-a591-d2e778427230
 2010-01-28 17:17:19 +00:00
hieuhoang1972
f5ebdbcec8 xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2721 1f5c12ca-751b-0410-a591-d2e778427230
 2010-01-28 16:43:36 +00:00
chardmeier
6317633148 Added memscore phrase scorer. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2653 1f5c12ca-751b-0410-a591-d2e778427230
 2010-01-18 14:52:34 +00:00
nicolabertoldi
3ad833d136 program to compute countings for phrase pairs 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2647 1f5c12ca-751b-0410-a591-d2e778427230
 2010-01-08 17:16:37 +00:00
nicolabertoldi
34d9feccc8 now it is possible to perform mert on a subset of features 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2646 1f5c12ca-751b-0410-a591-d2e778427230
 2010-01-08 15:56:45 +00:00
mphi
850e54f17d added a switch to the training script, which allows using different word alignment models: --final-alignment-model X, where X is either 1/2/3/4/5 (for the ibm models 1 to 5) or hmm; the latter is equivalent to using --hmm-align. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2644 1f5c12ca-751b-0410-a591-d2e778427230
 2010-01-08 12:00:28 +00:00
bhaddow
d8864fa6ad support for mgiza 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2640 1f5c12ca-751b-0410-a591-d2e778427230
 2009-12-15 17:36:06 +00:00
eisele
12dda84589 test, please ignore 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2598 1f5c12ca-751b-0410-a591-d2e778427230
 2009-10-23 15:45:35 +00:00
nicolabertoldi
5ad52827e3 minor change 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2576 1f5c12ca-751b-0410-a591-d2e778427230
 2009-10-09 07:47:14 +00:00
nicolabertoldi
7b77734e3c the ordered list of features names are now stored in a file after each step and re-load in the case of re-starting of the training 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2572 1f5c12ca-751b-0410-a591-d2e778427230
 2009-10-07 18:03:41 +00:00
nicolabertoldi
124f88e55a enabled the --continue option to re-start an interrupted mert from the last finished step 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2568 1f5c12ca-751b-0410-a591-d2e778427230
 2009-10-07 16:35:57 +00:00
nicolabertoldi
fa6a5bfc35 with this change, the usage of initial points for mert works properly 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2558 1f5c12ca-751b-0410-a591-d2e778427230
 2009-10-01 16:07:13 +00:00
nicolabertoldi
3484a1fd93 fixed minor bugs 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2550 1f5c12ca-751b-0410-a591-d2e778427230
 2009-10-01 07:44:30 +00:00
nicolabertoldi
f75e3993ac correction of parameter description 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2462 1f5c12ca-751b-0410-a591-d2e778427230
 2009-08-05 16:54:33 +00:00
nicolabertoldi
8384857f8c changes to mert-moses-new.pl to work with different reference length policies: shortest, average, closest (either "--shortest", "--average", or "--closest ) for BLEU and with case-sensitive/insensitive evaluation ( --nocase) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2461 1f5c12ca-751b-0410-a591-d2e778427230
 2009-08-05 16:39:06 +00:00
hieuhoang1972
568e973b2e make gcc amd make calls consistent for eric to use in ubuntu package 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2347 1f5c12ca-751b-0410-a591-d2e778427230
 2009-05-27 12:54:04 +00:00
phkoehn
8833098925 generalized n-best list reporting for feature functions, added experimental version of global lexical model 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2343 1f5c12ca-751b-0410-a591-d2e778427230
 2009-05-26 19:30:35 +00:00
bhaddow
981e440cc2 Fix detection of binarised reordering table 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2240 1f5c12ca-751b-0410-a591-d2e778427230
 2009-03-13 12:28:34 +00:00
bhaddow
e1d7bb986c Add option for predictable seeding 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2220 1f5c12ca-751b-0410-a591-d2e778427230
 2009-02-25 19:31:17 +00:00
jdschroeder
cc95706045 mert-moses.pl now supports multiple input weights for lattices and confusion networks, using the --inputweights argument. 
I'll leave it to someone who knows mert-moses-new.pl better to make the changes there.

"zcat" is now abstracted as a $ZCAT variable in these files, and is set to "gzip -cd" which should work on more platforms (notably on the mac, where zcat fails unless an archive name ends in ".Z").

 


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2082 1f5c12ca-751b-0410-a591-d2e778427230
 2009-02-05 17:39:36 +00:00
phkoehn
98381c0193 fixed xml removal 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1995 1f5c12ca-751b-0410-a591-d2e778427230
 2009-01-24 05:21:36 +00:00
nicolabertoldi
2075f9dda1 modification to mert script to allow the use of fewer nbest lists; features and scores are no more gzipped 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1965 1f5c12ca-751b-0410-a591-d2e778427230
 2008-12-30 17:33:16 +00:00
bojar
091c9ece28 raising line_max_length 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1953 1f5c12ca-751b-0410-a591-d2e778427230
 2008-12-05 10:01:05 +00:00
bojar
586d7e2f84 minor fix when handling gzipped corpora 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1952 1f5c12ca-751b-0410-a591-d2e778427230
 2008-12-04 17:49:59 +00:00
bojar
2c900c8bd7 uncompress input files for phrase extract, if needed 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1951 1f5c12ca-751b-0410-a591-d2e778427230
 2008-11-27 10:39:28 +00:00
bhaddow
1e13f6d2d6 Weights can sometimes be in exponential format. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1947 1f5c12ca-751b-0410-a591-d2e778427230
 2008-11-25 09:54:45 +00:00
hieuhoang1972
2807bc48ad absolute file name check, provided by Eric Kow 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1944 1f5c12ca-751b-0410-a591-d2e778427230
 2008-11-20 13:03:59 +00:00
hieuhoang1972
254284e57e patch to fix fiddly env variable and directory stuff, provided by Eric Kow@ 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1943 1f5c12ca-751b-0410-a591-d2e778427230
 2008-11-20 13:01:49 +00:00
phkoehn
abb2fc37b1 proper binarization of lexicalized reordering model 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1938 1f5c12ca-751b-0410-a591-d2e778427230
 2008-11-10 16:03:16 +00:00
phkoehn
bfbbefd710 bug fixes 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1917 1f5c12ca-751b-0410-a591-d2e778427230
 2008-10-26 03:32:29 +00:00
phkoehn
a09242ad16 bug fix with phrase table name in moses.ini, when using hmm alignment 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1913 1f5c12ca-751b-0410-a591-d2e778427230
 2008-10-20 21:37:57 +00:00
phkoehn
1c7b305152 bug fix 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1910 1f5c12ca-751b-0410-a591-d2e778427230
 2008-10-19 07:32:48 +00:00
phkoehn
1b5d99ad26 added headers for standard compliance (gcc 4.3 on 64 bit linux) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1905 1f5c12ca-751b-0410-a591-d2e778427230
 2008-10-16 21:14:38 +00:00
phkoehn
3a5981ce9d major improvements, see email to moses-support 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1904 1f5c12ca-751b-0410-a591-d2e778427230
 2008-10-15 23:25:14 +00:00
phkoehn
614876771d extended extract/score, to allow for one big file, not just parts 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1903 1f5c12ca-751b-0410-a591-d2e778427230
 2008-10-15 22:12:56 +00:00
jdschroeder
78534c1518 made all zcat calls through ZCAT variable. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1875 1f5c12ca-751b-0410-a591-d2e778427230
 2008-08-15 16:15:51 +00:00
jdschroeder
7a2ebedc20 minor bugfixes and error checking 
-added -rootdir option to enhanced-mert
	-fixed float regex in score-nbest.py and mert-moses.pl
	-allow for extra weights in constructing ini in mert-moses.pl
	-additional NFS bug checks in mert-moses.pl



git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1869 1f5c12ca-751b-0410-a591-d2e778427230
 2008-08-01 10:24:01 +00:00