moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2025-01-06 03:33:37 +03:00
Code Issues Packages Projects Releases Wiki Activity
2,452 Commits 136 Branches 26 Tags 147 MiB
6b153c67f8
Commit Graph

2452 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
hieuhoang1972
12d16af0bb nothing important 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4056 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-30 17:55:16 +00:00
phkoehn
1c671787d4 minor & allows to specify a corpus for the generation model (-generation-corpus) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4055 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-30 16:00:18 +00:00
oliver-wilson
c8bf9bec0c Add autoconf bits for DMapLM. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4054 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-30 15:47:55 +00:00
oliver-wilson
e49144f49d Only include LanguageModelDMapLM.h if compiling with DMap. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4053 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-30 15:42:06 +00:00
oliver-wilson
fbe8f1467c Add new language model class for DMapLM but do not link it to the build. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4052 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-30 15:15:24 +00:00
hieuhoang1972
ed7ecd5ce2 compile on gcc 4.6 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4051 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-30 04:20:53 +00:00
pjwilliams
7e288fae98 moses_chart: reduce memory use for rule lookup by decreasing the amount 
of state information duplicated between CoveredChartSpan objects.

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4050 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-29 13:38:11 +00:00
bhaddow
7fe3143feb Improve debug 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4049 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-29 08:27:43 +00:00
hieuhoang1972
024b5f9100 vs.net build 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4048 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-28 19:38:57 +00:00
hieuhoang1972
b9ef46972c vs.net 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4047 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-28 19:26:12 +00:00
hieuhoang1972
f7d534bcdd xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4046 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-28 19:02:09 +00:00
heafield
025ab3f7f0 Sorry I used a GCC-only dynamically sized array 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4041 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-27 21:28:22 +00:00
heafield
3616cf09fb Fix accidental format change 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4040 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-27 21:20:42 +00:00
pjwilliams
2451371ca2 Changes to chart decoder cube pruning: create one cube per dotted rule 
instead of one per translation and do 'non-lazy' scoring, i.e. fully
score the corner and neighbor hypotheses inside the rule cube instead
of waiting until an item is popped.  The old behaviour -- faster but
with more search errors -- is available via the
cube-pruning-lazy-scoring option.

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4039 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-27 15:13:15 +00:00
phkoehn
c7cc79a20e output no dead end hypotheses in search graph, note recombination (chart decoder) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4038 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-27 00:38:43 +00:00
heafield
5e70e3bd40 Quantization. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4037 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-26 22:21:44 +00:00
pjwilliams
0c60dd7ef8 filter-rule-table: allow for non-integral rule counts. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4036 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-24 18:32:14 +00:00
pjwilliams
913f339dd0 Remove unused m_ngramScore and m_countInfo variables from TargetPhrase. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4035 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-24 18:29:50 +00:00
pjwilliams
c14723cc83 Oops, fix commit 4032: option is called --PhrasePairCount not --RuleCount. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4034 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-24 16:40:17 +00:00
pjwilliams
a1ca2722df Add --MinNonInitialRuleCount option to filter-model-given-input.pl. This 
prunes non-initial rules (i.e. rules with non-terminals) from the rule table
based on their frequency counts.  In Zollmann, Venugopal, Och, and Ponte (2008),
pruning hierarchical rules that occur only once was found to significantly
decrease rule table size without harming translation quality.

Also, add TUNING:filter-settings and EVALUATION[:<set>]:filter-settings
variables so that this can be enabled in the EMS.

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4033 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-24 16:36:27 +00:00
pjwilliams
108dc4d12e Add --PhrasePairCount option to score. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4032 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-24 16:24:33 +00:00
pjwilliams
0484d43a22 train-model.perl: don't write obsolete glue-rule-type option to config. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4031 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-24 09:38:53 +00:00
leven101
5acb99d76f ClearTransOptionCache() causes segfault when translating next sentence 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4030 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-24 08:12:50 +00:00
hieuhoang1972
13c1855e8f xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4029 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-23 14:33:43 +00:00
hieuhoang1972
565ef04057 run giza and friend on subsets of corpora. Not sure if mgiza etc does the same thing 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4028 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-23 08:09:00 +00:00
hieuhoang1972
debca7632b change order of arguments. Arguments for extract-parallel are simple extension of normal extract 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4027 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-23 02:25:49 +00:00
phkoehn
6acd6a8684 improvements to ems analysis 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4026 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-21 21:52:13 +00:00
hieuhoang1972
2cdc39f63f parallelize extract using perl fork. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4025 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-21 21:48:56 +00:00
hieuhoang1972
4b5c8aaf10 parallelize extract using perl fork. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4024 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-21 21:34:18 +00:00
hieuhoang1972
4689d33d0f parallelize extract using perl fork. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4023 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-21 21:31:47 +00:00
hieuhoang1972
9eb51e31fb parallelize extract using perl fork. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4022 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-21 16:47:22 +00:00
hieuhoang1972
62ddd6eb53 parallelize extract using perl fork. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4021 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-21 16:45:42 +00:00
hieuhoang1972
bd64e748ff parallelize extract using perl fork. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4020 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-21 16:38:15 +00:00
hieuhoang1972
56fc94c2a7 parallelize extract using perl fork. Not quite ready for prime time 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4019 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-21 16:20:22 +00:00
phkoehn
4285a8e236 various experiment.perl improvements: split filter and decode/tune; extensions to analysis.perl, especially precision by coverage graphs 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4018 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-16 23:43:29 +00:00
pjwilliams
ab3460591c Share AlignmentInfo objects instead of storing one per TargetPhrase. 
This can save a significant amount of memory used on rule table storage,
though may increase loading time slightly.

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4017 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-16 21:20:20 +00:00
rafpayen
cdc4179ce1 Add a space before double punctuation signs in French 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4016 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-16 17:24:25 +00:00
hieuhoang1972
85283f5bee vs.net build 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4015 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-16 00:55:45 +00:00
phkoehn
4a6fec7613 chart decoder recombination now based on lm state, not suffix anymore; same feature function handling as in the phrase decoder 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4014 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-15 21:31:27 +00:00
bhaddow
c7603add22 Option to specify language model type 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4013 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-13 12:13:55 +00:00
hieuhoang1972
e5955ef1b3 make sure each parameter in ini file is known 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4011 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-11 04:59:55 +00:00
hieuhoang1972
6f8f1adf3b remove unnecessary parameters from ini files 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4010 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-11 04:52:05 +00:00
hieuhoang1972
b8e517d167 remove unnecessary parameters from ini files 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4009 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-11 04:48:22 +00:00
hieuhoang1972
265b4451ad remove unnecessary parameters from ini files 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4008 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-11 04:41:55 +00:00
hieuhoang1972
1a29541243 don't use counts for desperation pruning 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4007 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-10 06:39:03 +00:00
hieuhoang1972
21df1feb26 xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4006 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-10 01:01:09 +00:00
leven101
894b49a5b2 added LM updates to mosesserver (only for LanguageModelORLM) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4005 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-09 17:27:48 +00:00
hieuhoang1972
4bf85266d8 dont process unknown words for 1st or last place. They're the <s> & </s> and should only be handled by the glue rules 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4004 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-08 16:22:56 +00:00
leven101
ec04285270 hash file 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4003 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-08 15:45:08 +00:00
leven101
4ea818f34a Added wrapper files for online randomised LM prototype 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4002 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-08 15:05:19 +00:00