moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2024-12-29 06:52:34 +03:00
Code Issues Packages Projects Releases Wiki Activity
1,775 Commits 136 Branches 26 Tags 147 MiB
1c615ea751
Commit Graph

1775 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
nicolabertoldi
1c615ea751 re-introduced the initialization and cleaning up into the constructor and destructor of Managers object. And I moved the check for LM cache cleanup into LanguageModelIRST 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3202 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-26 13:38:56 +00:00
hieuhoang1972
1a638746f3 SRI compile error in Ondrej's parallel backoff LM 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3201 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-26 12:02:28 +00:00
nicolabertoldi
cf468273bc Safer way to cleanup LM caches after a given number of processed sentences. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3197 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-24 00:06:49 +00:00
nicolabertoldi
476528c05f I added a new parameter (-clean-lm-cache <value>) in order to control how often the LMs clean up their caches (if available). 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3195 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-23 15:01:06 +00:00
chardmeier
ca353d3e68 - word lattice distortion calculation and distortion limit checks revisited 
- crash in a controlled way if a jump in a word lattice exceeds max-phrase-length
- check if a sufficient number of ttable-limits is specified in the configuration

Thanks to Arianna Bisazza for help with sorting out lattice distortion and
to Suzy Howlett for help with debugging the other issues.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3193 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-22 08:42:32 +00:00
bojar
21de1e121f Merge branch 'master' into moses-svn 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3192 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-21 14:48:32 +00:00
hieuhoang1972
44368da77d xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3153 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-21 11:27:45 +00:00
bojar
db8e3357f5 Merge branch 'zmert' into moses-svn 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3152 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-20 22:03:04 +00:00
bojar
aa6043a556 mert-moses must quote -n-best-list *if* passed through parallelizer (and must not otherwise) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3151 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-20 17:02:58 +00:00
bojar
8ea058bfc7 fixed handling of the lmodel section; has only 3 ints 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3150 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-20 17:02:41 +00:00
bojar
403bc99cfb Merge branch 'moses-svn' into temp3 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3148 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-20 15:27:42 +00:00
bojar
34a0e9b3a2 support for SRILM's factored language models, implemented by Michal Richter 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3147 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-20 15:25:52 +00:00
bojar
323d48a821 factored version of PhraseDictionaryDynSuffixArray; squashed cmt of following: 
commit b8a86018e1100efede28023c1b351a4e27da9b75
Author: Kamil Kos <kamil.kos@matfyz.cz>
Date:   Tue Apr 20 14:28:40 2010 +0200

    added comments and  minor bug fixes

    Conflicts:

    	moses/src/BilingualDynSuffixArray.cpp
    	moses/src/PhraseDictionary.cpp
    	moses/src/PhraseDictionaryDynSuffixArray.h

commit 7d09ec19bb49f516a98bf6e6e2cd86c19d428e73
Author: Kamil Kos <kamil.kos@matfyz.cz>
Date:   Thu Apr 15 18:29:08 2010 +0200

    factored version of PhraseDictionaryDynSuffixArray


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3146 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-20 14:09:53 +00:00
bojar
a7677e7fa2 releasing zmert-moses as well 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3145 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-20 11:11:49 +00:00
bojar
f36f347014 safer everything: tempdir, open, execution 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3144 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-20 11:10:31 +00:00
bojar
0dd5149925 allow changing FLAVOR using env. var. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3143 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-20 11:06:51 +00:00
hieuhoang1972
0e01ebaadb lex reordering bug 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3142 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-19 17:26:54 +00:00
bhaddow
ee1f6c9db9 Add serial mode to the server 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3141 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-19 17:04:25 +00:00
rafpayen
baa0a9df1c Check srilm's library (machine-type is not reliable enough) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3139 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-19 14:02:48 +00:00
bhaddow
b35ab24bb8 Fix mosesmt phrase table loading (it got squashed in the chart merge) 
Fix some of the unused warnings.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3138 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-19 13:52:56 +00:00
pjwilliams
2edfc16912 Merge remaining script support for tree-based models from mt3_chart. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3137 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-16 09:45:51 +00:00
hieuhoang1972
a2233d0f8d xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3136 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-14 16:53:39 +00:00
pjwilliams
264c9150d1 Use external consolidate' program in train-factored-phrase-model.perl.' 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3135 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-14 15:51:20 +00:00
pjwilliams
5faaedc0df Copy in consolidate,' consolidate-direct,' and the new version of 
`score' from mt3_chart.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3134 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-14 15:50:17 +00:00
leven101
929bcf25fa added traing/lexical-reordering subdir to makefile 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3133 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-13 17:18:46 +00:00
hieuhoang1972
06ee9a3be3 vs 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3132 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-13 16:50:44 +00:00
pjwilliams
53cb08efca Use a generic version of the SAFE_GETLINE macro in scripts/phrase-extract 
instead of defining one per source file.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3131 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-13 16:29:55 +00:00
hieuhoang1972
0440dfe079 xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3130 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-13 16:13:56 +00:00
pjwilliams
580acce9e2 Integrate rule extraction code from mt3_chart. There are now two extract 
programs: `extract' for the phrase-based model and `extract-rules' for
tree-based models.  They could be combined into a single program, but
they're probably sufficiently different that it isn't worthwhile.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3129 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-13 15:34:39 +00:00
bhaddow
a76b7e5bce Test for consensus decoding 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3128 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-12 17:01:51 +00:00
bhaddow
bf4534ec61 Fix server build 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3127 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-12 15:41:09 +00:00
pjwilliams
51ae927ede Start merging in rule extraction code from mt3_chart branch. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3126 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-12 15:22:50 +00:00
bhaddow
6e8856edc2 Prevent intersections going to infinity 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3125 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-12 14:31:10 +00:00
hieuhoang1972
b96bac6106 vs studio 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3124 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-12 13:50:11 +00:00
hieuhoang1972
3029aaccc3 delete using std namespace in .h files 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3123 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-12 12:29:37 +00:00
pjwilliams
9c2536417f Remove file limit option for phrase extraction. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3122 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-12 11:56:54 +00:00
bhaddow
5340c16dcf Fix threaded build. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3121 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-12 11:11:33 +00:00
pjwilliams
99f1c92edb Remove redundant --ZipFiles option from extract. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3120 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-12 10:53:08 +00:00
pjwilliams
e5d23d0ec9 Set svn:ignore to ignore binaries in scripts/training/cmert-0.5 and 
scripts/training/phrase-extract.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3119 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-12 10:49:44 +00:00
pjwilliams
4c6c4b71cf Remove redundant --ProperConditioning option from extract. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3118 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-12 10:41:32 +00:00
hieuhoang1972
f0933ad883 LGPL license 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3117 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-12 10:15:49 +00:00
hieuhoang1972
45ecd333f9 delete std namespace in .h files 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3116 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-12 10:04:35 +00:00
hieuhoang1972
49d3780bcc re-merge lattice mbr & consensus decoding 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3115 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-12 09:51:29 +00:00
bojar
7172d05a43 fixed a bug, too eager check for preprocessing type; not needed in some cases 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3111 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 23:21:13 +00:00
bojar
82d6cc714e use qruncmd to parallelize srunblocks 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3110 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 23:20:59 +00:00
bojar
c5f44a2abf better verbosity level for srunblocks: emit some (most importantly fatal) msgs 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3109 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 23:20:45 +00:00
bojar
390ee866d8 require TMT_ROOT only if TectoMT will be actually needed 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3108 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 23:20:30 +00:00
bojar
9fe574f6ed fixed weights for SemPOS_BLEU metric 
usage: MERTFLAGS="--semposbleu-weights <sempos_weight>:<bleu_weight>"
e.g. --semposbleu-weights 2:1 to increase the weight of SemPOS


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3107 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:47:22 +00:00
bojar
26d77d15b2 added option --semposbleu-weights to specify weight of SemPOS and BLEU in SemPOS_BLEU metric 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3106 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:47:04 +00:00
bojar
c954a507c6 fixed error message in zmert-moses.pl 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3105 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-09 22:46:49 +00:00