moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2024-12-29 15:04:05 +03:00
Code Issues Packages Projects Releases Wiki Activity
2,625 Commits 136 Branches 26 Tags 147 MiB
1192e6f2b0
Commit Graph

1223 Commits

Author SHA1 Message Date
heafield
07e611ebcb Organize language models into an LM directory. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4350 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-13 14:27:01 +00:00
heafield
a95e791056 Back to using StringPiece 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4349 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-13 13:32:14 +00:00
heafield
f084248405 Cut the middle men out of the language model interface. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4348 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-13 12:33:05 +00:00
heafield
7d9bc523a6 Remove unused code 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4347 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-13 09:44:51 +00:00
heafield
541f776198 Remove unused calls 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4346 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-12 20:04:02 +00:00
heafield
e5d15a537e KenLM-specific Evaluate function 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4345 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-12 19:49:27 +00:00
heafield
cd19f14826 Faster CalcScore implementation for KenLM 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4339 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-12 13:04:12 +00:00
heafield
81acd0ffa2 Dear Hieu, a StringPiece is not necessairly null-terminated. When loading ARPA files directly, it was copying the ARPA file as 
part of the vocabulary word and breaking everything.  



git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4338 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-12 11:45:46 +00:00
heafield
c3f2ef7b25 Fix bhaddow's oovCount. Should be all words, not just the first in the phrase. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4337 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-12 10:22:45 +00:00
heafield
15adb17e35 Move EnumerateVocab to namespace lm 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4335 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-12 10:18:23 +00:00
hieuhoang1972
a65efa5a60 relax overly harsh assert 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4334 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-12 10:12:49 +00:00
heafield
ba41862d37 Source files are not executables. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4330 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-11 16:07:16 +00:00
heafield
16e37adbe0 Move phrase scoring from LanguageModel to LanguageModelImplementation. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4324 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-11 13:50:44 +00:00
hieuhoang1972
b0e5d6c005 delete align info flag in target phrase. Not used 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4322 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-11 08:55:19 +00:00
heafield
8f0c841d28 Move ChartHypothesis stuff to LanguageModelImplementation. Ran the 
regression tests. . . the passes and fails are the same FWIW.  



git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4319 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-10 16:25:56 +00:00
heafield
5a0d84da9a Move LanguageModelChartState into LanguageModelImplementation in preparation for moving responsibility for boundary word tracking 
from ChartHypothesis to LanguageModelChartState.  



git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4316 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-10 11:15:13 +00:00
hieuhoang1972
96c8ff4b15 last checkin was for the oldest bug found in moses! Goes back to svn version 4, and prob earlier, but svn can't diff that far. 
Should have crapped out whenever there is a blank line in the ini file, which is basically every ini file. Only visual studio 2010 complained, and only recently. Very strange, and a bit worrying. Horaay anyway

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4304 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-06 15:06:59 +00:00
heafield
9ba5460e53 Apparently we wanted a sequential id after all. . . get one in a thread-safe way from the manager. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4302 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-06 10:31:09 +00:00
hieuhoang1972
6f22c2ae29 bug reading over string size 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4301 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-06 09:16:21 +00:00
bhaddow
84d73700af Implementation of Lattice sampling (Chatterjee and Cancedda, emnlp 2010) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4295 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-04 15:46:24 +00:00
nicolabertoldi
47e452a076 made LM interface compliant with IRSTLM 5.70.02; fixed a bug related to word encoding 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4292 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-03 16:02:02 +00:00
hieuhoang1972
1ea3acde3d compile error due to last commit 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4291 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-03 08:31:05 +00:00
hieuhoang1972
3dafb3589c visual studio 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4290 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-03 07:18:14 +00:00
nicolabertoldi
2838970fc0 changed the interface towards IRSTLM according to the recent changes in FactorCollection 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4280 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-28 18:21:55 +00:00
hieuhoang1972
7538f5406a xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4278 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-28 00:30:18 +00:00
pjwilliams
ea272dc198 Move SCFG rule table loading code out of PhraseDictionarySCFG and into a 
separate RuleTableLoader class.  Start adding support for a faster-loading
rule table format.

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4273 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-27 00:34:46 +00:00
hieuhoang1972
9762a65bd8 xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4264 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-25 10:44:47 +00:00
heafield
8edf53dcf3 StringPiece. Does a body good. And avoids string copying. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4263 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-24 15:58:23 +00:00
heafield
d40ba48da3 Replace hypothesis ids with pointers to ChartHypothesis. This avoids locking. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4261 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-24 14:48:52 +00:00
pjwilliams
efddd0f390 Remove race condition from ChartHypothesis ID allocation in moses_chart. 
Thanks to Kenneth Heafield for pointing that out.

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4259 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-24 10:06:21 +00:00
heafield
19879a8b1e -threads all option 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4252 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-22 22:29:56 +00:00
heafield
ddbfee788c Sorry about the compiler error. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4251 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-22 08:55:54 +00:00
hieuhoang1972
d2245390e0 visual studio build 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4250 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-22 05:39:32 +00:00
heafield
daf0e37175 Barry wanted to insure Factor is non-copyable. But STL containers want it to be copyable. Here's some friend magic to meet both demands. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4249 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-21 20:35:12 +00:00
heafield
3402bdfe7a Merge mtm_lm into trunk. 
There's a fair number of files with no change that somebody must have touched in the branch so metadata is being recorded. 
Updates kenlm binary file format, sorry. 
It looks like OOV isn't being computed in EvaluateChart anyway, just phrasal.  
  


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4247 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-21 16:06:48 +00:00
hieuhoang1972
659e34735d xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4245 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-21 11:01:19 +00:00
heafield
fdc6c37ca4 Store the string in the factor itself. Use MurmurHash because it beats Boost's hash. Also, apparently nobody calls Exists. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4244 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-21 10:26:04 +00:00
heafield
4c8552b16f Cruft removal 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4243 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-20 19:29:24 +00:00
heafield
b9e433977d Redo FactorCollection including better locking 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4242 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-20 19:08:42 +00:00
heafield
ed30a602af Remove deprecated, and uncalled LoadVocab function 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4241 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-20 17:12:27 +00:00
nicolabertoldi
c2de897cd2 added support for a new class-based lm type provided by IRSTLM 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4240 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-20 14:46:50 +00:00
phkoehn
7927439ce5 option to get unpruned search graph for chart decoder 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4228 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-16 11:58:53 +00:00
leven101
3824ca3ea2 fixed operator [] adding bug 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4216 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-12 18:04:56 +00:00
bhaddow
e95c004145 Implement LM OOV feature 
- still have to run regression
 - doesn't work for moses_chart (awaiting H & K LM state merge)


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4211 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-09 18:03:00 +00:00
bhaddow
ca5c0f19b7 Multi-threading of mert, for random restarts. 
Fix mert tests.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4182 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-07 08:08:35 +00:00
heafield
c4d6471c3c Print exceptions because some peopls' machines don't. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4175 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-06 09:47:08 +00:00
leven101
762c47d8c9 added checks for loopy data in dynamic suffix array 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4169 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-01 09:57:16 +00:00
nicolabertoldi
75edc2eddd change to print the corrett name of the features with InputScores 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4168 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-30 12:25:50 +00:00
hieuhoang1972
33ced5538a option to sort word alignment info, as suggested by arianna bisazza 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4165 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-26 02:38:48 +00:00
hieuhoang1972
5449839d75 option to sort word alignment info, as suggested by arianna bisazza 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4164 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-26 02:37:52 +00:00
heafield
56824c07e5 Should have made these return const * as well. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4163 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-24 16:44:59 +00:00
bhaddow
4d8f9a0716 Remove excessive debug 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4162 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-24 14:00:21 +00:00
mlegendr
3f0d83531f Part 3 of n-gram thing: added LanguageKenLM.h to public library headers 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4161 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-24 11:26:41 +00:00
heafield
6f391a7dbd Part 2 of Marc LEGENDRE's changes to expose n-gram length. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4160 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-24 10:45:41 +00:00
heafield
b3c06822ed Fix memory leak reported by Marc LEGENDRE. Also make the FFState for begin and null context const. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4158 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-24 10:15:29 +00:00
oliver-wilson
96417949c2 Keep track of the order at which the last ngram request succeeded and 
use it to inform the next request.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4151 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-18 12:36:02 +00:00
theleopardess
d7752b44fc I tested check-in by adding a test line in moses/src/StaticData.cpp, producing a trivial moses revision 4122. Now I have removed that line. Everything ok but sorry for the confusion. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4129 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-05 18:57:09 +00:00
theleopardess
f8a99e5d6d yanggao-softdep-v0 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4122 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-05 15:11:43 +00:00
hieuhoang1972
e7b97c1b1a vs build 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4109 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-04 04:53:21 +00:00
hieuhoang1972
a79651d239 fixed backoff phrase table. Allow backoff of unigrams 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4089 1f5c12ca-751b-0410-a591-d2e778427230
 2011-07-25 12:23:49 +00:00
hieuhoang1972
b0ec298ce2 vs.net build 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4088 1f5c12ca-751b-0410-a591-d2e778427230
 2011-07-23 23:52:34 +00:00
chesio
1b9d99a5ad BilingualDynSuffixArray corpus may now be loaded from gzipped file as well (removed needless call to seekg()). 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4083 1f5c12ca-751b-0410-a591-d2e778427230
 2011-07-21 23:29:11 +00:00
hieuhoang1972
06af5d40d4 Improved error message 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4081 1f5c12ca-751b-0410-a591-d2e778427230
 2011-07-21 02:41:23 +00:00
pjwilliams
113d0f24dd moses_chart: avoid doing some std::map retrievals during rule lookup 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4080 1f5c12ca-751b-0410-a591-d2e778427230
 2011-07-19 12:57:02 +00:00
hieuhoang1972
9c0d725cde visual studio 2010 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4079 1f5c12ca-751b-0410-a591-d2e778427230
 2011-07-19 03:07:15 +00:00
pjwilliams
beba4b475f moses_chart: merge DottedRule and CoveredChartSpan classes. This saves 
some memory for models that require a lot of lookup state (generally
grammars with lots of target categories).

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4078 1f5c12ca-751b-0410-a591-d2e778427230
 2011-07-18 21:44:27 +00:00
heafield
954dfd7d5e Optional compression for trie. Also, some better error handling. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4074 1f5c12ca-751b-0410-a591-d2e778427230
 2011-07-13 20:53:18 +00:00
leven101
cd96c02748 bug fixes 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4068 1f5c12ca-751b-0410-a591-d2e778427230
 2011-07-06 17:25:54 +00:00
leven101
52ce926901 added ClearWordInCache() to clear (nonfrequent) lexical word pair probs after suffix array updates 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4065 1f5c12ca-751b-0410-a591-d2e778427230
 2011-07-01 15:40:48 +00:00
leven101
e0174b413c added CacheFreqWords() to speed up decoding with suffix array PTs 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4064 1f5c12ca-751b-0410-a591-d2e778427230
 2011-07-01 14:36:28 +00:00
hieuhoang1972
3585d8bae2 xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4057 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-30 18:11:04 +00:00
hieuhoang1972
12d16af0bb nothing important 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4056 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-30 17:55:16 +00:00
oliver-wilson
e49144f49d Only include LanguageModelDMapLM.h if compiling with DMap. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4053 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-30 15:42:06 +00:00
oliver-wilson
fbe8f1467c Add new language model class for DMapLM but do not link it to the build. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4052 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-30 15:15:24 +00:00
hieuhoang1972
ed7ecd5ce2 compile on gcc 4.6 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4051 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-30 04:20:53 +00:00
pjwilliams
7e288fae98 moses_chart: reduce memory use for rule lookup by decreasing the amount 
of state information duplicated between CoveredChartSpan objects.

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4050 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-29 13:38:11 +00:00
bhaddow
7fe3143feb Improve debug 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4049 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-29 08:27:43 +00:00
hieuhoang1972
024b5f9100 vs.net build 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4048 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-28 19:38:57 +00:00
hieuhoang1972
b9ef46972c vs.net 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4047 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-28 19:26:12 +00:00
hieuhoang1972
f7d534bcdd xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4046 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-28 19:02:09 +00:00
pjwilliams
2451371ca2 Changes to chart decoder cube pruning: create one cube per dotted rule 
instead of one per translation and do 'non-lazy' scoring, i.e. fully
score the corner and neighbor hypotheses inside the rule cube instead
of waiting until an item is popped.  The old behaviour -- faster but
with more search errors -- is available via the
cube-pruning-lazy-scoring option.

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4039 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-27 15:13:15 +00:00
phkoehn
c7cc79a20e output no dead end hypotheses in search graph, note recombination (chart decoder) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4038 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-27 00:38:43 +00:00
heafield
5e70e3bd40 Quantization. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4037 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-26 22:21:44 +00:00
pjwilliams
913f339dd0 Remove unused m_ngramScore and m_countInfo variables from TargetPhrase. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4035 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-24 18:29:50 +00:00
leven101
5acb99d76f ClearTransOptionCache() causes segfault when translating next sentence 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4030 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-24 08:12:50 +00:00
hieuhoang1972
13c1855e8f xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4029 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-23 14:33:43 +00:00
pjwilliams
ab3460591c Share AlignmentInfo objects instead of storing one per TargetPhrase. 
This can save a significant amount of memory used on rule table storage,
though may increase loading time slightly.

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4017 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-16 21:20:20 +00:00
hieuhoang1972
85283f5bee vs.net build 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4015 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-16 00:55:45 +00:00
phkoehn
4a6fec7613 chart decoder recombination now based on lm state, not suffix anymore; same feature function handling as in the phrase decoder 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4014 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-15 21:31:27 +00:00
hieuhoang1972
e5955ef1b3 make sure each parameter in ini file is known 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4011 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-11 04:59:55 +00:00
hieuhoang1972
1a29541243 don't use counts for desperation pruning 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4007 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-10 06:39:03 +00:00
hieuhoang1972
21df1feb26 xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4006 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-10 01:01:09 +00:00
leven101
894b49a5b2 added LM updates to mosesserver (only for LanguageModelORLM) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4005 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-09 17:27:48 +00:00
hieuhoang1972
4bf85266d8 dont process unknown words for 1st or last place. They're the <s> & </s> and should only be handled by the glue rules 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4004 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-08 16:22:56 +00:00
leven101
ec04285270 hash file 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4003 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-08 15:45:08 +00:00
leven101
4ea818f34a Added wrapper files for online randomised LM prototype 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4002 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-08 15:05:19 +00:00
hieuhoang1972
0a04cb11bf added std::size_t for gcc 4.6 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4001 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-08 05:49:47 +00:00
hieuhoang1972
46c4dd441e loosen assert in case of rounding errors 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4000 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-06 16:06:51 +00:00
leven101
bcbc0c6e09 bug fix 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3998 1f5c12ca-751b-0410-a591-d2e778427230
 2011-05-31 12:04:40 +00:00
leven101
547a6cbf62 added ClearTransOptCache() to StaticData class and call it when updating suffix PTs 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3997 1f5c12ca-751b-0410-a591-d2e778427230
 2011-05-31 09:43:17 +00:00