moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2024-12-29 06:52:34 +03:00
Code Issues Packages Projects Releases Wiki Activity
1,978 Commits 136 Branches 26 Tags 147 MiB
620d92c68a
Commit Graph

998 Commits

Author SHA1 Message Date
hieuhoang1972
620d92c68a bug in on-disk pt due to refactoring 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3616 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-12 14:10:19 +00:00
rsennrich
6e032b6ab5 XML spans parameter now uses '-' as token segmentation everywhere, as documented on the website. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3609 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-07 08:35:08 +00:00
pjwilliams
be8dbd20e9 Remove unused Phrase::m_arity member variable. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3600 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-01 10:37:10 +00:00
hieuhoang1972
e504b797b2 xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3599 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-01 00:21:27 +00:00
pjwilliams
2761762434 If Boost is available, use boost::unordered_map instead of std::map for 
storing child nodes in PhraseDictionaryNodeSCFG.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3598 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-30 21:28:30 +00:00
heafield
770df2a92d Unbodge kenlm by moving compilation to kenlm/ instead of kenlm/lm. Changing the headers every 
time I copied to Moses was getting annoying.  



git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3587 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-28 16:26:55 +00:00
heafield
d99066e4e9 Hieu wanted GetLmID 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3585 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-28 15:21:50 +00:00
hieuhoang1972
8fa18b50a7 xcode. And don't invoke internal LM when sri is specified, even if sri isn't compiled 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3583 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-28 11:02:58 +00:00
heafield
c2b49c9d97 Now returning the same probabilities as SRI. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3582 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-28 00:22:33 +00:00
heafield
331283f040 LanguageModelKen now without segfaults. The load function was missing some undocumented initialization. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3581 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-27 21:52:15 +00:00
pjwilliams
34767c4d0a Oops, don't prune rule table if ttable-limit=0. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3580 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-27 20:49:42 +00:00
heafield
5b74b38527 Remove vestigial dub parameter. Surrender to tab-based whitespace. More passive-aggressive message about nGramOrder. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3575 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-27 16:01:58 +00:00
heafield
cac7600662 Make --with-kenlm=$PWD/kenlm compile mine instead of IRST. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3574 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-27 15:07:28 +00:00
hieuhoang1972
375ed9b7fe ken lm integration 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3573 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-27 10:46:39 +00:00
heafield
e6184ae947 Updates to kenlm: 
Kludged and slow interface requested by Hieu because apparently Moses can't store language model state.  
Separate files for ARPA reading, vocabulary, and weights.  
Remove build shell scripts that won't work after Hieu changed the header file layout.  



git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3572 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-27 03:46:44 +00:00
hieuhoang1972
559a5e3ece ken lm integration 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3570 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-26 22:29:01 +00:00
pjwilliams
f5a096e2fa In ChartTranslationOptionList::Add(), always consider every target phrase as 
the TargetPhraseCollection won't be ordered.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3567 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-26 12:09:29 +00:00
pjwilliams
81372704de Enforce ttable-limit for memory-based rule table. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3566 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-26 11:38:09 +00:00
hieuhoang1972
031617a8db delete unused function 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3563 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-25 20:30:19 +00:00
hieuhoang1972
ea04dc5209 vs.net 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3562 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-24 21:01:11 +00:00
hieuhoang1972
465d1cef0a make class name consistent for chart & pb decoders 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3561 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-23 17:39:32 +00:00
hieuhoang1972
4d34e5155e make class name consistent for chart & pb decoders 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3560 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-23 16:54:32 +00:00
hieuhoang1972
0b6a29327c make class name consistent for chart & pb decoders 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3559 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-23 16:32:44 +00:00
hieuhoang1972
cafc5046c7 make class name consistent for chart & pb decoders 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3558 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-23 15:48:33 +00:00
hieuhoang1972
73ece4f859 make class name consistent for chart & pb decoders 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3557 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-23 15:18:11 +00:00
hieuhoang1972
1920fc007c makefile for ken lm 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3556 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-23 14:37:06 +00:00
hieuhoang1972
af228d7e93 make class name consistent for chart & pb decoders 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3555 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-23 12:34:35 +00:00
hieuhoang1972
34873a1ba4 make class name consistent for chart & pb decoders 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3554 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-23 12:12:49 +00:00
hieuhoang1972
b83fc49a1f make class name consistent for chart & pb decoders 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3553 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-23 11:05:38 +00:00
hieuhoang1972
989eb55449 ken lm integration 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3544 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-21 23:10:15 +00:00
hieuhoang1972
32d3565b04 ken lm integration 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3543 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-21 22:43:29 +00:00
hieuhoang1972
e9b355ed8c get rid of weights in translation model 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3471 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-15 16:49:49 +00:00
hieuhoang1972
d0aecca9fe get rid of weight in LM class 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3466 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-15 16:02:41 +00:00
pjwilliams
130ae457dd In PhraseDictionarySCFG, only add non-leaf nodes to the running node lists. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3424 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-10 15:19:25 +00:00
hieuhoang1972
bdfa2beab8 delete eclipse build 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3422 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-10 09:16:50 +00:00
bhaddow
d3680b4255 Make sure distortion feature are adjacent 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3418 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-07 10:54:04 +00:00
pjwilliams
2deb68af84 In PhraseDictionaryNodeSCFG, use separate maps for children with terminal and non-terminal keys. This removes the need to look up source terminals twice. The chart decoder spends a *lot* of time in PhraseDictionaryNodeSCFG::GetChild() (approx 38% of post-startup decoding time in my target syntax test, according to callgrind), so this makes a significant difference. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3409 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-17 13:41:46 +00:00
pjwilliams
98383c1393 Pare down PhraseDictionaryNodeSCFG. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3408 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-17 11:01:03 +00:00
hieuhoang1972
ef09298824 move function calls with side effects out of asserts 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3403 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-12 12:54:55 +00:00
hieuhoang1972
8616a2bdee visual studio 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3396 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-10 13:51:20 +00:00
hieuhoang1972
8fc72ee74a xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3395 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-10 13:35:08 +00:00
bhaddow
904133fcb7 Merge in the multiple models branch. These changes allow the moses server 
to support multiple translation, language and generation models within the
same process. The main design change is the introduction of a TranslationSystem
object to manage the models, which have been moved out of StaticData.
The changes should have no effect on existing systems.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3394 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-10 13:12:00 +00:00
hieuhoang1972
277ba483e1 alignment info in the decoder 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3381 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-02 16:32:36 +00:00
phkoehn
56447ed42c bug fix for nested zones 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3379 1f5c12ca-751b-0410-a591-d2e778427230
 2010-07-30 22:17:08 +00:00
hieuhoang1972
7221bf2dd4 alignment info, for chart decoding, updated regression 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3368 1f5c12ca-751b-0410-a591-d2e778427230
 2010-07-28 09:53:21 +00:00
hieuhoang1972
7c6007f018 alignment info, new format 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3365 1f5c12ca-751b-0410-a591-d2e778427230
 2010-07-27 11:21:12 +00:00
hieuhoang1972
fc56e031d4 alignment info, new format 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3364 1f5c12ca-751b-0410-a591-d2e778427230
 2010-07-27 11:10:13 +00:00
hieuhoang1972
b9339bdf0e svn properties 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3360 1f5c12ca-751b-0410-a591-d2e778427230
 2010-07-17 23:23:09 +00:00
hieuhoang1972
dd7d3d1b56 vs.net 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3359 1f5c12ca-751b-0410-a591-d2e778427230
 2010-07-17 23:11:55 +00:00
hieuhoang1972
31930eb6fc alignment info in pt 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3358 1f5c12ca-751b-0410-a591-d2e778427230
 2010-07-17 22:29:06 +00:00