moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2024-09-20 15:48:05 +03:00
Code Issues Packages Projects Releases Wiki Activity
1,943 Commits 136 Branches 26 Tags 147 MiB
d7a61c751f
Commit Graph

1943 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
pjwilliams
d7a61c751f Don't bother popping QueueEntrys off the priority queue when destructing 
MosesChart::Cube.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3564 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-26 10:30:28 +00:00
hieuhoang1972
031617a8db delete unused function 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3563 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-25 20:30:19 +00:00
hieuhoang1972
ea04dc5209 vs.net 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3562 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-24 21:01:11 +00:00
hieuhoang1972
465d1cef0a make class name consistent for chart & pb decoders 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3561 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-23 17:39:32 +00:00
hieuhoang1972
4d34e5155e make class name consistent for chart & pb decoders 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3560 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-23 16:54:32 +00:00
hieuhoang1972
0b6a29327c make class name consistent for chart & pb decoders 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3559 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-23 16:32:44 +00:00
hieuhoang1972
cafc5046c7 make class name consistent for chart & pb decoders 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3558 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-23 15:48:33 +00:00
hieuhoang1972
73ece4f859 make class name consistent for chart & pb decoders 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3557 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-23 15:18:11 +00:00
hieuhoang1972
1920fc007c makefile for ken lm 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3556 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-23 14:37:06 +00:00
hieuhoang1972
af228d7e93 make class name consistent for chart & pb decoders 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3555 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-23 12:34:35 +00:00
hieuhoang1972
34873a1ba4 make class name consistent for chart & pb decoders 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3554 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-23 12:12:49 +00:00
hieuhoang1972
b83fc49a1f make class name consistent for chart & pb decoders 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3553 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-23 11:05:38 +00:00
hieuhoang1972
9f5d530910 make class name consistent for chart & pb decoders 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3552 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-23 10:43:31 +00:00
hieuhoang1972
52c823fec4 ken lm integration 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3551 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-23 10:21:29 +00:00
rsennrich
7929e4624e more informative error message when hierarchical phrase extraction fails. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3550 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-22 12:56:11 +00:00
rosasjolu
8746482d04 Change data files location 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3549 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-22 11:03:36 +00:00
rosasjolu
16302a45a8 git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3548 1f5c12ca-751b-0410-a591-d2e778427230 2010-09-22 11:02:29 +00:00
rosasjolu
d2fd75ac49 Change data files location 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3547 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-22 10:27:07 +00:00
rosasjolu
ad62e27c90 git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3546 1f5c12ca-751b-0410-a591-d2e778427230 2010-09-22 10:24:52 +00:00
hieuhoang1972
ee842f578c delete data files 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3545 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-22 10:05:11 +00:00
hieuhoang1972
989eb55449 ken lm integration 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3544 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-21 23:10:15 +00:00
hieuhoang1972
32d3565b04 ken lm integration 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3543 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-21 22:43:29 +00:00
phkoehn
f34b37bad3 added hierarchical alignment view to web analysis tool 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3514 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-17 13:28:04 +00:00
heafield
a02268a7c1 Fix memory corruption with exceptions. 
Fix compilation with -m64 in murmur_hash.  
Extract most mmap calls.  



git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3494 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-16 19:53:33 +00:00
hieuhoang1972
a582d483cc add lowercase script 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3475 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-16 07:32:32 +00:00
hieuhoang1972
e9b355ed8c get rid of weights in translation model 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3471 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-15 16:49:49 +00:00
phkoehn
fb8b0eb180 new prefix files for tokenizer 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3467 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-15 16:06:04 +00:00
hieuhoang1972
d0aecca9fe get rid of weight in LM class 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3466 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-15 16:02:41 +00:00
heafield
d00c788760 kenlm update 
mmap works; utility to build binary format included.  
Configuration struct (including unknown handling options). 
config option to build a binary format while loading an ARPA.  
Doesn't require Boost or ICU. 
Works on 32 and 64 bit. 
query appends </s>. 
Reduced memory consumption: 12 bytes per 5-gram instead of 16 bytes on 64-bit machines.  
Reduced memory consumption: vocabulary takes 8 bytes/word instead of 12 bytes/word if sorted is 
used. 
Removed some cruft that wasn't needed by this code.  
Compiles on Mac OS X.  
Add script to run tests; these depend on Boost.  
SRI wrapper works again, is slightly faster, no longer depends on Boost, and has a test.
Debugging code only appears with -DDEBUG, so the default is fast.  



git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3447 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-14 21:33:11 +00:00
rosasjolu
128a885406 git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3446 1f5c12ca-751b-0410-a591-d2e778427230 2010-09-14 19:27:15 +00:00
pjwilliams
130ae457dd In PhraseDictionarySCFG, only add non-leaf nodes to the running node lists. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3424 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-10 15:19:25 +00:00
hieuhoang1972
c73dddfb5b delete eclipse build 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3423 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-10 09:34:17 +00:00
hieuhoang1972
bdfa2beab8 delete eclipse build 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3422 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-10 09:16:50 +00:00
hieuhoang1972
473e0e3e96 Ken's LM 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3421 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-10 00:36:07 +00:00
suzyh
fa4eca6ccb Added loop to check_if_crashed in EMS experiment.perl to wait in case the .STDERR file is slow in appearing after the step has completed. Reinstated --old-sge and --filterfile command-line arguments to mert-moses.pl. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3420 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-09 11:40:40 +00:00
bhaddow
fd7997dbf5 Fix for mert script from Yu Chen, to make sure it reads the correct 
feature order from the nbest list.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3419 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-08 08:17:05 +00:00
bhaddow
d3680b4255 Make sure distortion feature are adjacent 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3418 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-07 10:54:04 +00:00
hieuhoang1972
e53aeb903c need non-empty arg in mert-moses.pl otherwise it crashes. The wonders of a non-typechecked, arg checking scripting language 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3417 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-04 00:16:26 +00:00
phkoehn
4a85fd95ce srilm as setting in interpolation 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3416 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-03 12:57:16 +00:00
hieuhoang1972
51b99ede7a delete old qsub args 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3415 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-02 20:42:55 +00:00
bhaddow
12269b062c Only add queue-flags if non-empty. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3414 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-02 09:01:39 +00:00
bhaddow
7efec1a087 Set default max mert iterations to 25. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3413 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-01 19:57:55 +00:00
bhaddow
5f2f345165 max iterations option 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3412 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-01 17:02:12 +00:00
hieuhoang1972
888cc09d34 xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3411 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-27 10:27:51 +00:00
bgottesman
e409b6827c add --max-word-length option to cleaning script, with default value 1000; any segment with a word (or factor) exceeding this length in chars is discarded; motivated by symal.cpp, which has its own such parameter (hardcoded to 1000) and crashes if it encounters a word that exceeds it 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3410 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-23 16:35:14 +00:00
pjwilliams
2deb68af84 In PhraseDictionaryNodeSCFG, use separate maps for children with terminal and non-terminal keys. This removes the need to look up source terminals twice. The chart decoder spends a *lot* of time in PhraseDictionaryNodeSCFG::GetChild() (approx 38% of post-startup decoding time in my target syntax test, according to callgrind), so this makes a significant difference. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3409 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-17 13:41:46 +00:00
pjwilliams
98383c1393 Pare down PhraseDictionaryNodeSCFG. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3408 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-17 11:01:03 +00:00
redpony
570183461c plf checker 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3407 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-16 01:03:39 +00:00
hieuhoang1972
083a9af215 delete alignment info for terminals 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3405 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-13 10:03:13 +00:00
rafpayen
b431f951c5 git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3404 1f5c12ca-751b-0410-a591-d2e778427230 2010-08-13 09:58:17 +00:00