moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2025-01-07 20:17:48 +03:00
Code Issues Packages Projects Releases Wiki Activity
2,839 Commits 136 Branches 26 Tags 147 MiB
98fc6d9917
Commit Graph

1265 Commits

Author SHA1 Message Date
hieuhoang1972
e7b97c1b1a vs build 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4109 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-04 04:53:21 +00:00
hieuhoang1972
a79651d239 fixed backoff phrase table. Allow backoff of unigrams 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4089 1f5c12ca-751b-0410-a591-d2e778427230
 2011-07-25 12:23:49 +00:00
hieuhoang1972
b0ec298ce2 vs.net build 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4088 1f5c12ca-751b-0410-a591-d2e778427230
 2011-07-23 23:52:34 +00:00
chesio
1b9d99a5ad BilingualDynSuffixArray corpus may now be loaded from gzipped file as well (removed needless call to seekg()). 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4083 1f5c12ca-751b-0410-a591-d2e778427230
 2011-07-21 23:29:11 +00:00
hieuhoang1972
06af5d40d4 Improved error message 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4081 1f5c12ca-751b-0410-a591-d2e778427230
 2011-07-21 02:41:23 +00:00
pjwilliams
113d0f24dd moses_chart: avoid doing some std::map retrievals during rule lookup 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4080 1f5c12ca-751b-0410-a591-d2e778427230
 2011-07-19 12:57:02 +00:00
hieuhoang1972
9c0d725cde visual studio 2010 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4079 1f5c12ca-751b-0410-a591-d2e778427230
 2011-07-19 03:07:15 +00:00
pjwilliams
beba4b475f moses_chart: merge DottedRule and CoveredChartSpan classes. This saves 
some memory for models that require a lot of lookup state (generally
grammars with lots of target categories).

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4078 1f5c12ca-751b-0410-a591-d2e778427230
 2011-07-18 21:44:27 +00:00
heafield
954dfd7d5e Optional compression for trie. Also, some better error handling. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4074 1f5c12ca-751b-0410-a591-d2e778427230
 2011-07-13 20:53:18 +00:00
leven101
cd96c02748 bug fixes 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4068 1f5c12ca-751b-0410-a591-d2e778427230
 2011-07-06 17:25:54 +00:00
leven101
52ce926901 added ClearWordInCache() to clear (nonfrequent) lexical word pair probs after suffix array updates 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4065 1f5c12ca-751b-0410-a591-d2e778427230
 2011-07-01 15:40:48 +00:00
leven101
e0174b413c added CacheFreqWords() to speed up decoding with suffix array PTs 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4064 1f5c12ca-751b-0410-a591-d2e778427230
 2011-07-01 14:36:28 +00:00
hieuhoang1972
3585d8bae2 xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4057 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-30 18:11:04 +00:00
hieuhoang1972
12d16af0bb nothing important 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4056 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-30 17:55:16 +00:00
oliver-wilson
e49144f49d Only include LanguageModelDMapLM.h if compiling with DMap. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4053 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-30 15:42:06 +00:00
oliver-wilson
fbe8f1467c Add new language model class for DMapLM but do not link it to the build. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4052 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-30 15:15:24 +00:00
hieuhoang1972
ed7ecd5ce2 compile on gcc 4.6 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4051 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-30 04:20:53 +00:00
pjwilliams
7e288fae98 moses_chart: reduce memory use for rule lookup by decreasing the amount 
of state information duplicated between CoveredChartSpan objects.

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4050 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-29 13:38:11 +00:00
bhaddow
7fe3143feb Improve debug 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4049 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-29 08:27:43 +00:00
hieuhoang1972
024b5f9100 vs.net build 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4048 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-28 19:38:57 +00:00
hieuhoang1972
b9ef46972c vs.net 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4047 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-28 19:26:12 +00:00
hieuhoang1972
f7d534bcdd xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4046 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-28 19:02:09 +00:00
pjwilliams
2451371ca2 Changes to chart decoder cube pruning: create one cube per dotted rule 
instead of one per translation and do 'non-lazy' scoring, i.e. fully
score the corner and neighbor hypotheses inside the rule cube instead
of waiting until an item is popped.  The old behaviour -- faster but
with more search errors -- is available via the
cube-pruning-lazy-scoring option.

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4039 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-27 15:13:15 +00:00
phkoehn
c7cc79a20e output no dead end hypotheses in search graph, note recombination (chart decoder) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4038 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-27 00:38:43 +00:00
heafield
5e70e3bd40 Quantization. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4037 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-26 22:21:44 +00:00
pjwilliams
913f339dd0 Remove unused m_ngramScore and m_countInfo variables from TargetPhrase. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4035 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-24 18:29:50 +00:00
leven101
5acb99d76f ClearTransOptionCache() causes segfault when translating next sentence 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4030 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-24 08:12:50 +00:00
hieuhoang1972
13c1855e8f xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4029 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-23 14:33:43 +00:00
pjwilliams
ab3460591c Share AlignmentInfo objects instead of storing one per TargetPhrase. 
This can save a significant amount of memory used on rule table storage,
though may increase loading time slightly.

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4017 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-16 21:20:20 +00:00
hieuhoang1972
85283f5bee vs.net build 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4015 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-16 00:55:45 +00:00
phkoehn
4a6fec7613 chart decoder recombination now based on lm state, not suffix anymore; same feature function handling as in the phrase decoder 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4014 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-15 21:31:27 +00:00
hieuhoang1972
e5955ef1b3 make sure each parameter in ini file is known 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4011 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-11 04:59:55 +00:00
hieuhoang1972
1a29541243 don't use counts for desperation pruning 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4007 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-10 06:39:03 +00:00
hieuhoang1972
21df1feb26 xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4006 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-10 01:01:09 +00:00
leven101
894b49a5b2 added LM updates to mosesserver (only for LanguageModelORLM) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4005 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-09 17:27:48 +00:00
hieuhoang1972
4bf85266d8 dont process unknown words for 1st or last place. They're the <s> & </s> and should only be handled by the glue rules 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4004 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-08 16:22:56 +00:00
leven101
ec04285270 hash file 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4003 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-08 15:45:08 +00:00
leven101
4ea818f34a Added wrapper files for online randomised LM prototype 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4002 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-08 15:05:19 +00:00
hieuhoang1972
0a04cb11bf added std::size_t for gcc 4.6 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4001 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-08 05:49:47 +00:00
hieuhoang1972
46c4dd441e loosen assert in case of rounding errors 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4000 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-06 16:06:51 +00:00
leven101
bcbc0c6e09 bug fix 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3998 1f5c12ca-751b-0410-a591-d2e778427230
 2011-05-31 12:04:40 +00:00
leven101
547a6cbf62 added ClearTransOptCache() to StaticData class and call it when updating suffix PTs 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3997 1f5c12ca-751b-0410-a591-d2e778427230
 2011-05-31 09:43:17 +00:00
leven101
f64885599d added ClearTransOptCache() to StaticData class and call it when updating suffix PTs 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3996 1f5c12ca-751b-0410-a591-d2e778427230
 2011-05-31 09:42:27 +00:00
dowobeha
d12dd2b4d7 Added additional level 3 logging to help verify that Syntactic LM is being loaded and used 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3992 1f5c12ca-751b-0410-a591-d2e778427230
 2011-05-24 15:47:47 +00:00
dowobeha
e04e155986 Syntactic LM can now be loaded properly by StaticData. This change should have been merged earlier, but was omitted. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3991 1f5c12ca-751b-0410-a591-d2e778427230
 2011-05-24 15:47:35 +00:00
heafield
c74f5446c8 Unbreak the build. Sorry about that. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3981 1f5c12ca-751b-0410-a591-d2e778427230
 2011-05-17 17:02:10 +00:00
naditomeh
5019e98abe A minor issue when trying to output the search graph with one of these 
flags: output-search-graph[-extended].
An empty input sentence results in a segfault.
Fix:
after the line
const Hypothesis *prevHypo = hypo->GetPrevHypo();
test if prevHypo is null before using it.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3978 1f5c12ca-751b-0410-a591-d2e778427230
 2011-05-16 13:41:15 +00:00
dowobeha
cd017c1448 Added missing syntactic LM function declaration. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3976 1f5c12ca-751b-0410-a591-d2e778427230
 2011-05-13 20:58:03 +00:00
dowobeha
9581b6aae2 Updated syntactic classes (cherry-picking from commit bf7a7bb8e7) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3973 1f5c12ca-751b-0410-a591-d2e778427230
 2011-05-13 19:28:23 +00:00
dowobeha
9375aa8846 Reverting changes. Revision 3971 was a bad commit. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3972 1f5c12ca-751b-0410-a591-d2e778427230
 2011-05-13 18:27:30 +00:00
dowobeha
bb941c01f6 Merge branch 'master' into local-trunk 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3971 1f5c12ca-751b-0410-a591-d2e778427230
 2011-05-13 18:07:21 +00:00
pjwilliams
70d67ffd46 Chart decoder: include child hypothesis costs in estimates when 
pruning the ChartTranslationOptionList.

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3943 1f5c12ca-751b-0410-a591-d2e778427230
 2011-04-04 20:43:02 +00:00
hieuhoang1972
8c4692e581 warnings 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3941 1f5c12ca-751b-0410-a591-d2e778427230
 2011-04-01 23:37:20 +00:00
hieuhoang1972
6acbdcdd28 derivation of n-best list output 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3939 1f5c12ca-751b-0410-a591-d2e778427230
 2011-03-30 21:31:15 +00:00
hieuhoang1972
adc2ac2c6a xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3938 1f5c12ca-751b-0410-a591-d2e778427230
 2011-03-30 20:31:09 +00:00
bhaddow
d9d1c9440a Fix prehistoric memory leak. This was affecting the loading of 
alignment info from binarised phrase tables.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3936 1f5c12ca-751b-0410-a591-d2e778427230
 2011-03-25 17:42:21 +00:00
hieuhoang1972
6cb1993541 xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3932 1f5c12ca-751b-0410-a591-d2e778427230
 2011-03-21 06:32:06 +00:00
oliver-wilson
e168875599 Fix broken refactoring. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3931 1f5c12ca-751b-0410-a591-d2e778427230
 2011-03-18 15:58:12 +00:00
oliver-wilson
888e551aae Init thread specific data in Load() so the LM is useable in the main 
thread.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3929 1f5c12ca-751b-0410-a591-d2e778427230
 2011-03-16 21:04:34 +00:00
hieuhoang1972
a4d22f7714 vs.net 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3928 1f5c12ca-751b-0410-a591-d2e778427230
 2011-03-11 19:26:01 +00:00
bhaddow
4a652200e4 Fix compile error in boost version 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3927 1f5c12ca-751b-0410-a591-d2e778427230
 2011-03-11 17:35:31 +00:00
hieuhoang1972
77c7a27f13 xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3926 1f5c12ca-751b-0410-a591-d2e778427230
 2011-03-11 17:05:05 +00:00
phkoehn
ff8a624344 re-organized syntax code for clarity - part II 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3925 1f5c12ca-751b-0410-a591-d2e778427230
 2011-03-11 16:28:36 +00:00
phkoehn
1cbe245919 re-organized syntax code for clarity - part I 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3924 1f5c12ca-751b-0410-a591-d2e778427230
 2011-03-11 13:08:43 +00:00
pjwilliams
26d125d3cd documentation of chart decoder 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3923 1f5c12ca-751b-0410-a591-d2e778427230
 2011-03-10 12:58:49 +00:00
phkoehn
ad140593b0 documentation of chart decoder 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3920 1f5c12ca-751b-0410-a591-d2e778427230
 2011-03-09 17:55:13 +00:00
heafield
9d4d2a413e Have language models also return whether the word was OOV or not. This isn't used yet, but will make it into a feature. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3916 1f5c12ca-751b-0410-a591-d2e778427230
 2011-03-07 23:21:09 +00:00
bhaddow
3c9112bc74 remove useless debug 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3912 1f5c12ca-751b-0410-a591-d2e778427230
 2011-03-03 17:05:28 +00:00
phkoehn
4ee7e5f673 added some comments 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3911 1f5c12ca-751b-0410-a591-d2e778427230
 2011-03-02 19:02:07 +00:00
oliver-wilson
106c4e0fc2 Init thread specific data on RandLM before sentence processing. Needed 
for multithreaded RandLM.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3910 1f5c12ca-751b-0410-a591-d2e778427230
 2011-03-02 16:07:20 +00:00
pjwilliams
75709a6c87 Memory efficiency: make the reserveSize argument non-optional in 
Moses::Phrase's constructor.  The default used to be ARRAY_SIZE_INCR = 10,
which will be excessive in many cases.  Where the default was used, I've
set the exact size where that was obvious and explicitly used ARRAY_SIZE_INCR 
otherwise.  If you know the code involved, it's probably worth reviewing.



git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3908 1f5c12ca-751b-0410-a591-d2e778427230
 2011-02-28 11:41:08 +00:00
leven101
a5a860ad1f optimized some parameters 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3907 1f5c12ca-751b-0410-a591-d2e778427230
 2011-02-25 14:53:34 +00:00
hieuhoang1972
251011df9e irstlm variable 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3906 1f5c12ca-751b-0410-a591-d2e778427230
 2011-02-25 02:26:05 +00:00
phkoehn
93ed0ca1a9 get rid of "unused variable" warnings 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3903 1f5c12ca-751b-0410-a591-d2e778427230
 2011-02-24 16:17:38 +00:00
hieuhoang1972
a0b6abdfd3 run beautify.perl. Consistent formatting for .h & .cpp files 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3901 1f5c12ca-751b-0410-a591-d2e778427230
 2011-02-24 13:14:42 +00:00
oliver-wilson
06df9d1770 Changed white space to test effect on svn blame. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3892 1f5c12ca-751b-0410-a591-d2e778427230
 2011-02-22 16:48:51 +00:00
bhaddow
e7a8ba3b7f check for correct number of factors 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3891 1f5c12ca-751b-0410-a591-d2e778427230
 2011-02-22 13:06:16 +00:00
maurocettolo
4c6dfbddc3 minor changes to make Moses compliant with IRSTLM toolkit (release 5.60.01) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3878 1f5c12ca-751b-0410-a591-d2e778427230
 2011-02-11 11:32:35 +00:00
ales-t
e922c159b6 Alignment points are also created for unknown source words. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3876 1f5c12ca-751b-0410-a591-d2e778427230
 2011-02-08 18:04:09 +00:00
pjwilliams
d4359f9875 If Boost is available, use per-sentence object pools to allocate ProcessedRule 
and WordConsumed objects (which are used to store rule table lookup state).
Large numbers of these objects are used during decoding and this can
significantly improve performance, especially for multithreaded decoding,
though at the cost of increased total memory use.

The ./configure option --disable-boost-pool can be used to disable this
feature if memory is tight.  This currently only affects moses_chart with
in-memory rule tables.



git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3873 1f5c12ca-751b-0410-a591-d2e778427230
 2011-02-07 15:43:19 +00:00
bhaddow
6221d2a558 Patch to add covered to osgx from Dennis Mehay 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3872 1f5c12ca-751b-0410-a591-d2e778427230
 2011-02-04 16:21:57 +00:00
maurocettolo
8fcd76f2fc made handling of chunk LM compatible with recent efficiency updates to IRSTLM toolkit by Nicola 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3870 1f5c12ca-751b-0410-a591-d2e778427230
 2011-02-04 07:42:56 +00:00
phkoehn
4e72cd91be added decoding-graph-backoff, still experimenting with it 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3869 1f5c12ca-751b-0410-a591-d2e778427230
 2011-02-03 13:41:44 +00:00
bojar
0bc0ece594 Ales Tamchyna's printing of alignments (-print-alignment-info did nothing) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3867 1f5c12ca-751b-0410-a591-d2e778427230
 2011-02-03 09:08:42 +00:00
hieuhoang1972
0eed5716b7 get rid of linked trans opt 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3864 1f5c12ca-751b-0410-a591-d2e778427230
 2011-02-02 11:24:19 +00:00
hieuhoang1972
e087e78df9 get rid of linked trans opt 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3863 1f5c12ca-751b-0410-a591-d2e778427230
 2011-02-02 11:06:19 +00:00
hieuhoang1972
abacb9166a xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3857 1f5c12ca-751b-0410-a591-d2e778427230
 2011-01-28 14:57:55 +00:00
heafield
87f15593da Remove vestigial len parameter from language model calls 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3856 1f5c12ca-751b-0410-a591-d2e778427230
 2011-01-27 19:01:45 +00:00
pjwilliams
967b7be213 Support for multithreading in moses_chart (-threads option). This hasn't 
been thoroughly tested yet, so don't be surprised if it breaks.  Verbose
output will be scrambled.



git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3853 1f5c12ca-751b-0410-a591-d2e778427230
 2011-01-26 13:15:13 +00:00
redpony
eddb28e0ce facilitate programmatic creation of word lattices 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3848 1f5c12ca-751b-0410-a591-d2e778427230
 2011-01-25 20:08:29 +00:00
pjwilliams
8051c5ad35 Use TranslationTask objects to perform sentence decoding in moses-chart. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3846 1f5c12ca-751b-0410-a591-d2e778427230
 2011-01-25 17:15:30 +00:00
pjwilliams
99bbfe938b Use OutputCollector to write moses-chart output. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3843 1f5c12ca-751b-0410-a591-d2e778427230
 2011-01-25 15:17:17 +00:00
pjwilliams
67b30ea0c7 Move sentence-specific rule lookup state out of PhraseDictionarySCFG and 
PhraseDictionaryOnDisk and into ChartRuleLookupManager.



git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3842 1f5c12ca-751b-0410-a591-d2e778427230
 2011-01-24 19:14:19 +00:00
bhaddow
7b6503680a Shortcut when trans opts cache is size 0. Avoids potential uninitialised read. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3825 1f5c12ca-751b-0410-a591-d2e778427230
 2011-01-21 21:25:14 +00:00
hieuhoang1972
96bd3a164d vs.net 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3802 1f5c12ca-751b-0410-a591-d2e778427230
 2011-01-17 16:19:33 +00:00
bhaddow
a9cd71628a Change of boost macros - please make sure you favourite configuration still works 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3799 1f5c12ca-751b-0410-a591-d2e778427230
 2011-01-13 23:38:48 +00:00
pjwilliams
d20667a46d Faster lookup for rules with source and/or target syntax labels (in-memory rule 
table only).


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3797 1f5c12ca-751b-0410-a591-d2e778427230
 2011-01-13 00:25:10 +00:00
hieuhoang1972
41c5b3a1c2 xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3769 1f5c12ca-751b-0410-a591-d2e778427230
 2010-12-10 13:04:06 +00:00
heafield
5e9df58a3c Respect -v 0 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3768 1f5c12ca-751b-0410-a591-d2e778427230
 2010-12-09 22:13:09 +00:00
bhaddow
50f0e6c07d Add a show-weights option. It prints out the moses features and exits. May 
load tables as a side-effect.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3744 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-29 16:44:28 +00:00
bhaddow
6255216b6a Remove gnu-specific typeof 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3725 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-22 10:05:17 +00:00
bhaddow
a7e0977eea Fix compile error by using correct macro. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3720 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-18 10:27:30 +00:00
chardmeier
837a667a95 Cleaned up language modelling code by disentangling the decoder's LM feature 
function from the LM toolkit abstraction layer. There are two different groups
of classes now:
- LanguageModel, which inherits from StatefulFeatureFunction and contains
  the n-gram model feature function.
- LanguageModelImplementation, which is the base class of the individual
  LM implementations (SRI, IRST, RandLM, KenLM) and provides methods to
  query LM probabilities and states.
Each LanguageModel controls a LanguageModelImplementation. Implementations can
be shared by more than one LanguageModel.
This should make it easier to use the LM libraries as a backend for other
feature functions while retaining the flexibility to use different LM toolkits.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3719 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-17 14:06:21 +00:00
chardmeier
d18ff948f5 Bugfixes in srilm adaptor. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3718 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-17 13:23:44 +00:00
leven101
84d83480b6 function name changes 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3711 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-15 11:32:02 +00:00
leven101
5251a2823a separated source and target vocab in suffixarrays to support unequal factors 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3710 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-15 11:28:27 +00:00
bojar
2ea140062b don't warn about probs outside [0,1] in -verbose 0 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3698 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-10 11:51:26 +00:00
heafield
2784923899 Rename a bunch of kenlm files. A ./regenerate-makefiles.sh is required. 
Make loading with MAP_POPULATE on Linux and read on other OSes the default.
Use LM #9 for lazy loading, as recommended by other devs.  
Slightly faster trie.  



git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3688 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-06 00:40:16 +00:00
leven101
34b45c0480 removed debug messages from BilingualDynSuffixArray.cpp 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3687 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-04 18:41:04 +00:00
bhaddow
3aee6fab5d Use correct conditional compilation flag for threaded moses 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3686 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-03 18:43:18 +00:00
heafield
c12c2c59d2 Autodetect model from binary format. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3675 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-28 01:05:04 +00:00
hieuhoang1972
735d5b682f xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3673 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-27 18:54:50 +00:00
heafield
614d6002a6 Integrate heafield-refactorlm. Faster kenlm with new binary format. Stateful language model 
framework.  



git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3671 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-27 17:50:40 +00:00
nicolabertoldi
bb08dcb5b6 made code compliant with the enhanced IRSTLM library; IRSTLM release 5.50.01 is needed; back compatibility is not assured; 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3662 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-26 16:06:01 +00:00
nicolabertoldi
3d7f391ae5 remove inefficient function calls 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3638 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-22 13:30:50 +00:00
hieuhoang1972
564a9ad371 word alignment info in binary pt, code provided by Christof Pintaske 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3637 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-22 10:37:19 +00:00
hieuhoang1972
ac875ffc81 mem leak 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3617 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-12 14:47:58 +00:00
hieuhoang1972
620d92c68a bug in on-disk pt due to refactoring 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3616 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-12 14:10:19 +00:00
rsennrich
6e032b6ab5 XML spans parameter now uses '-' as token segmentation everywhere, as documented on the website. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3609 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-07 08:35:08 +00:00
pjwilliams
be8dbd20e9 Remove unused Phrase::m_arity member variable. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3600 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-01 10:37:10 +00:00
hieuhoang1972
e504b797b2 xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3599 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-01 00:21:27 +00:00
pjwilliams
2761762434 If Boost is available, use boost::unordered_map instead of std::map for 
storing child nodes in PhraseDictionaryNodeSCFG.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3598 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-30 21:28:30 +00:00
heafield
770df2a92d Unbodge kenlm by moving compilation to kenlm/ instead of kenlm/lm. Changing the headers every 
time I copied to Moses was getting annoying.  



git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3587 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-28 16:26:55 +00:00
heafield
d99066e4e9 Hieu wanted GetLmID 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3585 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-28 15:21:50 +00:00
hieuhoang1972
8fa18b50a7 xcode. And don't invoke internal LM when sri is specified, even if sri isn't compiled 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3583 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-28 11:02:58 +00:00
heafield
c2b49c9d97 Now returning the same probabilities as SRI. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3582 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-28 00:22:33 +00:00
heafield
331283f040 LanguageModelKen now without segfaults. The load function was missing some undocumented initialization. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3581 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-27 21:52:15 +00:00
pjwilliams
34767c4d0a Oops, don't prune rule table if ttable-limit=0. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3580 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-27 20:49:42 +00:00
heafield
5b74b38527 Remove vestigial dub parameter. Surrender to tab-based whitespace. More passive-aggressive message about nGramOrder. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3575 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-27 16:01:58 +00:00
heafield
cac7600662 Make --with-kenlm=$PWD/kenlm compile mine instead of IRST. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3574 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-27 15:07:28 +00:00
hieuhoang1972
375ed9b7fe ken lm integration 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3573 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-27 10:46:39 +00:00
heafield
e6184ae947 Updates to kenlm: 
Kludged and slow interface requested by Hieu because apparently Moses can't store language model state.  
Separate files for ARPA reading, vocabulary, and weights.  
Remove build shell scripts that won't work after Hieu changed the header file layout.  



git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3572 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-27 03:46:44 +00:00
hieuhoang1972
559a5e3ece ken lm integration 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3570 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-26 22:29:01 +00:00
pjwilliams
f5a096e2fa In ChartTranslationOptionList::Add(), always consider every target phrase as 
the TargetPhraseCollection won't be ordered.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3567 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-26 12:09:29 +00:00
pjwilliams
81372704de Enforce ttable-limit for memory-based rule table. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3566 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-26 11:38:09 +00:00
hieuhoang1972
031617a8db delete unused function 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3563 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-25 20:30:19 +00:00
hieuhoang1972
ea04dc5209 vs.net 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3562 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-24 21:01:11 +00:00
hieuhoang1972
465d1cef0a make class name consistent for chart & pb decoders 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3561 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-23 17:39:32 +00:00
hieuhoang1972
4d34e5155e make class name consistent for chart & pb decoders 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3560 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-23 16:54:32 +00:00
hieuhoang1972
0b6a29327c make class name consistent for chart & pb decoders 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3559 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-23 16:32:44 +00:00
hieuhoang1972
cafc5046c7 make class name consistent for chart & pb decoders 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3558 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-23 15:48:33 +00:00
hieuhoang1972
73ece4f859 make class name consistent for chart & pb decoders 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3557 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-23 15:18:11 +00:00
hieuhoang1972
1920fc007c makefile for ken lm 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3556 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-23 14:37:06 +00:00
hieuhoang1972
af228d7e93 make class name consistent for chart & pb decoders 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3555 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-23 12:34:35 +00:00
hieuhoang1972
34873a1ba4 make class name consistent for chart & pb decoders 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3554 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-23 12:12:49 +00:00
hieuhoang1972
b83fc49a1f make class name consistent for chart & pb decoders 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3553 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-23 11:05:38 +00:00
hieuhoang1972
989eb55449 ken lm integration 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3544 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-21 23:10:15 +00:00
hieuhoang1972
32d3565b04 ken lm integration 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3543 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-21 22:43:29 +00:00
hieuhoang1972
e9b355ed8c get rid of weights in translation model 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3471 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-15 16:49:49 +00:00
hieuhoang1972
d0aecca9fe get rid of weight in LM class 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3466 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-15 16:02:41 +00:00
pjwilliams
130ae457dd In PhraseDictionarySCFG, only add non-leaf nodes to the running node lists. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3424 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-10 15:19:25 +00:00
hieuhoang1972
bdfa2beab8 delete eclipse build 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3422 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-10 09:16:50 +00:00
bhaddow
d3680b4255 Make sure distortion feature are adjacent 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3418 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-07 10:54:04 +00:00
pjwilliams
2deb68af84 In PhraseDictionaryNodeSCFG, use separate maps for children with terminal and non-terminal keys. This removes the need to look up source terminals twice. The chart decoder spends a *lot* of time in PhraseDictionaryNodeSCFG::GetChild() (approx 38% of post-startup decoding time in my target syntax test, according to callgrind), so this makes a significant difference. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3409 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-17 13:41:46 +00:00
pjwilliams
98383c1393 Pare down PhraseDictionaryNodeSCFG. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3408 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-17 11:01:03 +00:00
hieuhoang1972
ef09298824 move function calls with side effects out of asserts 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3403 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-12 12:54:55 +00:00
hieuhoang1972
8616a2bdee visual studio 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3396 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-10 13:51:20 +00:00
hieuhoang1972
8fc72ee74a xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3395 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-10 13:35:08 +00:00
bhaddow
904133fcb7 Merge in the multiple models branch. These changes allow the moses server 
to support multiple translation, language and generation models within the
same process. The main design change is the introduction of a TranslationSystem
object to manage the models, which have been moved out of StaticData.
The changes should have no effect on existing systems.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3394 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-10 13:12:00 +00:00
hieuhoang1972
277ba483e1 alignment info in the decoder 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3381 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-02 16:32:36 +00:00
phkoehn
56447ed42c bug fix for nested zones 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3379 1f5c12ca-751b-0410-a591-d2e778427230
 2010-07-30 22:17:08 +00:00
hieuhoang1972
7221bf2dd4 alignment info, for chart decoding, updated regression 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3368 1f5c12ca-751b-0410-a591-d2e778427230
 2010-07-28 09:53:21 +00:00
hieuhoang1972
7c6007f018 alignment info, new format 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3365 1f5c12ca-751b-0410-a591-d2e778427230
 2010-07-27 11:21:12 +00:00
hieuhoang1972
fc56e031d4 alignment info, new format 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3364 1f5c12ca-751b-0410-a591-d2e778427230
 2010-07-27 11:10:13 +00:00
hieuhoang1972
b9339bdf0e svn properties 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3360 1f5c12ca-751b-0410-a591-d2e778427230
 2010-07-17 23:23:09 +00:00
hieuhoang1972
dd7d3d1b56 vs.net 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3359 1f5c12ca-751b-0410-a591-d2e778427230
 2010-07-17 23:11:55 +00:00
hieuhoang1972
31930eb6fc alignment info in pt 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3358 1f5c12ca-751b-0410-a591-d2e778427230
 2010-07-17 22:29:06 +00:00
hieuhoang1972
a21c9bff68 debug output 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3334 1f5c12ca-751b-0410-a591-d2e778427230
 2010-06-13 21:06:34 +00:00
hieuhoang1972
f24fb6449e delete pragma once when using #define 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3324 1f5c12ca-751b-0410-a591-d2e778427230
 2010-06-10 15:36:08 +00:00
nicolabertoldi
6e67edd11f sorted lif of headers and sources 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3323 1f5c12ca-751b-0410-a591-d2e778427230
 2010-06-10 15:32:33 +00:00
nicolabertoldi
e6d39bf83a minor fix 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3322 1f5c12ca-751b-0410-a591-d2e778427230
 2010-06-10 15:19:07 +00:00
nicolabertoldi
79d91a572d memory-unmap moved into the destructor 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3316 1f5c12ca-751b-0410-a591-d2e778427230
 2010-06-09 14:34:15 +00:00
nicolabertoldi
f38d220b67 moving compilation of LanguageModelParallelBackoff in the SRILM-dependent region 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3315 1f5c12ca-751b-0410-a591-d2e778427230
 2010-06-09 14:08:55 +00:00
hieuhoang1972
7db72471ce xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3313 1f5c12ca-751b-0410-a591-d2e778427230
 2010-06-09 13:00:54 +00:00
hieuhoang1972
0d6d78b5a9 xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3308 1f5c12ca-751b-0410-a591-d2e778427230
 2010-06-08 11:42:28 +00:00
hieuhoang1972
c620d1a5f7 xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3307 1f5c12ca-751b-0410-a591-d2e778427230
 2010-06-08 01:26:34 +00:00
hieuhoang1972
fdfd4907f5 xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3306 1f5c12ca-751b-0410-a591-d2e778427230
 2010-06-08 00:59:48 +00:00
hieuhoang1972
dee9be66f8 don't check for files exists in Parameter class. May be a folder for OnDisk format 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3304 1f5c12ca-751b-0410-a591-d2e778427230
 2010-06-07 17:19:51 +00:00
hieuhoang1972
c9eb463fe6 vs.net 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3301 1f5c12ca-751b-0410-a591-d2e778427230
 2010-06-05 13:10:26 +00:00
leven101
69d92bfec3 git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3281 1f5c12ca-751b-0410-a591-d2e778427230 2010-05-27 18:53:39 +00:00
leven101
da9814fcf9 fixed bug 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3278 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-27 15:44:23 +00:00
bhaddow
f62d9a9be1 Remove trailing space on GetStringRep() 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3273 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-27 08:14:21 +00:00
leven101
4338889c71 removed debug statments 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3270 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-25 13:11:42 +00:00
leven101
142a05c64d suffix array operates on single thread. bugs in dynamic suffix array 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3269 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-25 13:06:56 +00:00
bhaddow
4874f711ca Update mosesserver so that it can return the search graph and/or 
translation options. Also add a sample client to demonstrate
these new features.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3265 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-20 16:52:46 +00:00
bhaddow
c9baabe2ea Merging of moses mains. mosesmt now does single and multi-threeaded 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3264 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-19 16:42:18 +00:00
leven101
c2f35f614b fixed bug 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3263 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-19 16:37:57 +00:00
chardmeier
531598684b Removed PhraseDictionaryTree's dependency on StaticData to facilitate using the object outside the decoder. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3258 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-16 19:13:32 +00:00
bhaddow
bb2431331c Fix to consensus decoding to stop it discarding hypos with zero score. 
Move hypothesis id management into Manager to make it thread-safe.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3257 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-14 09:33:03 +00:00
leven101
ff6d734b60 small fix for regex use 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3256 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-13 16:05:10 +00:00
leven101
f0e8f86501 fixed double free bug 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3255 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-13 16:03:56 +00:00
chardmeier
6f8d9fa0fb Added limited backwards compatibility with old ini file format for phrase table specs. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3253 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-13 14:12:05 +00:00
hieuhoang1972
f36759478b don't spew out param list if there's a genuine error 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3250 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-11 21:44:35 +00:00
phkoehn
f52a2cc55d code cleanup & enable trace ("-t") for MBR decoding ("-mbr") 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3246 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-10 21:18:47 +00:00
pjwilliams
2a5b262503 Write translation details to a named file instead of cerr. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3243 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-08 15:51:59 +00:00
leven101
ed0a9a369b git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3241 1f5c12ca-751b-0410-a591-d2e778427230 2010-05-07 21:25:53 +00:00
hieuhoang1972
e67670ad5b add alignment info for -T output 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3240 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-07 19:33:39 +00:00
pjwilliams
01f4ca892a Remove the lmstats parameter since it doesn't have any effect except as 
a synonym for translation-details.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3237 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-07 17:38:20 +00:00
leven101
f4696695f8 git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3236 1f5c12ca-751b-0410-a591-d2e778427230 2010-05-07 16:55:48 +00:00
leven101
ad554c7d6d git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3235 1f5c12ca-751b-0410-a591-d2e778427230 2010-05-07 16:54:25 +00:00