moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2024-11-10 10:59:21 +03:00
Code Issues Packages Projects Releases Wiki Activity
2,144 Commits 136 Branches 26 Tags 147 MiB
1cbe245919
Commit Graph

2144 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
hieuhoang1972
564a9ad371 word alignment info in binary pt, code provided by Christof Pintaske 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3637 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-22 10:37:19 +00:00
phkoehn
ace33d16dd bug fix 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3636 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-22 07:30:52 +00:00
phkoehn
3b880bbdda added biconcor to make 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3634 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-21 10:04:38 +00:00
phkoehn
85a5a13e4c improvements to web analysis, fixes to syntax wrappers 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3633 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-21 09:49:27 +00:00
bhaddow
88eaf49c5e remove detokeniser 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3632 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-21 08:42:28 +00:00
heafield
e65ecd0632 Put official website in README 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3631 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-19 15:55:49 +00:00
sarst
0594b13c61 Added nonbreaking_prefix.sv for Swedish 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3630 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-19 12:45:49 +00:00
hieuhoang1972
5f6baa9021 xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3627 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-18 05:47:07 +00:00
bhaddow
2dc951b062 More informative error messages 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3625 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-15 09:00:18 +00:00
rafpayen
a1ab166692 reset file handle between opens, so as to have an error if no file is given 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3623 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-14 18:23:05 +00:00
hieuhoang1972
e5edb4b971 delete duplicate detokenizer 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3622 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-13 16:39:46 +00:00
hieuhoang1972
08739d8f49 add from josh's script 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3621 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-13 10:14:13 +00:00
hieuhoang1972
677f0f47a8 copyright notice & svn properties 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3620 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-12 16:17:41 +00:00
hieuhoang1972
a875ef8361 regression test for ondisk 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3619 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-12 15:07:43 +00:00
hieuhoang1972
2d06b12a2c regression test for ondisk 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3618 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-12 15:04:31 +00:00
hieuhoang1972
ac875ffc81 mem leak 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3617 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-12 14:47:58 +00:00
hieuhoang1972
620d92c68a bug in on-disk pt due to refactoring 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3616 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-12 14:10:19 +00:00
hieuhoang1972
7463257be5 gcc 3.4 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3613 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-11 13:36:40 +00:00
hieuhoang1972
eedef63277 keep perl scripts with Unix line endings 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3612 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-11 11:32:27 +00:00
hieuhoang1972
105e83df82 beautify 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3610 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-07 19:08:44 +00:00
rsennrich
6e032b6ab5 XML spans parameter now uses '-' as token segmentation everywhere, as documented on the website. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3609 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-07 08:35:08 +00:00
hieuhoang1972
60db2a2971 Time logging info 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3601 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-01 18:08:10 +00:00
pjwilliams
be8dbd20e9 Remove unused Phrase::m_arity member variable. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3600 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-01 10:37:10 +00:00
hieuhoang1972
e504b797b2 xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3599 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-01 00:21:27 +00:00
pjwilliams
2761762434 If Boost is available, use boost::unordered_map instead of std::map for 
storing child nodes in PhraseDictionaryNodeSCFG.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3598 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-30 21:28:30 +00:00
pjwilliams
687ddf2d10 Bump the minimum Boost version to 1.36 to get functional/hash. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3597 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-30 21:27:20 +00:00
pjwilliams
dd39c734e8 If Boost is available, use boost::unordered_set instead of std::set to store 
MosesChart::Cube's set of unique QueueEntry object values.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3596 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-30 21:10:25 +00:00
pjwilliams
79a61c5526 Add --enable-boost option to ./configure to allow the Boost library to be used 
in Moses (if not already required by --enable-threads).  This is off by default.
The --with-boost option can only be used if --enable-boost or --enable-threads
is given, since the AX_BOOST_BASE macro is only conditionally called.  This is
to avoid making Boost a requirement by default, but is a bit clunky.  Probably
easy to fix if your Autotools knowledge exceeds nowt...



git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3595 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-30 21:07:01 +00:00
suzyh
d071296cde Fix to reuse-weights.perl to copy weights containing an exponent 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3590 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-29 04:04:12 +00:00
heafield
193109e200 Link kenlm by default (it was already being built). Allow --without-kenlm to avoid linking. Add myself to the build instructions. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3588 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-28 18:02:08 +00:00
heafield
770df2a92d Unbodge kenlm by moving compilation to kenlm/ instead of kenlm/lm. Changing the headers every 
time I copied to Moses was getting annoying.  



git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3587 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-28 16:26:55 +00:00
heafield
d99066e4e9 Hieu wanted GetLmID 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3585 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-28 15:21:50 +00:00
hieuhoang1972
8fa18b50a7 xcode. And don't invoke internal LM when sri is specified, even if sri isn't compiled 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3583 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-28 11:02:58 +00:00
heafield
c2b49c9d97 Now returning the same probabilities as SRI. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3582 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-28 00:22:33 +00:00
heafield
331283f040 LanguageModelKen now without segfaults. The load function was missing some undocumented initialization. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3581 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-27 21:52:15 +00:00
pjwilliams
34767c4d0a Oops, don't prune rule table if ttable-limit=0. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3580 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-27 20:49:42 +00:00
heafield
73565501f8 Make ltmain.sh executable so svn doesn't whine when the status is changed under its feet. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3579 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-27 20:41:39 +00:00
heafield
d1a7c636ac Jon Clark complained that IRSTLM puts 0.0 backoff for n-grams of longest order and that I throw 
an error.  



git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3578 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-27 18:38:21 +00:00
hieuhoang1972
1ea4970325 svn properties 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3577 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-27 18:16:10 +00:00
hieuhoang1972
1c342ffefd Deal with updated file format. Thanks to Christof Pintaske 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3576 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-27 18:03:22 +00:00
heafield
5b74b38527 Remove vestigial dub parameter. Surrender to tab-based whitespace. More passive-aggressive message about nGramOrder. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3575 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-27 16:01:58 +00:00
heafield
cac7600662 Make --with-kenlm=$PWD/kenlm compile mine instead of IRST. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3574 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-27 15:07:28 +00:00
hieuhoang1972
375ed9b7fe ken lm integration 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3573 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-27 10:46:39 +00:00
heafield
e6184ae947 Updates to kenlm: 
Kludged and slow interface requested by Hieu because apparently Moses can't store language model state.  
Separate files for ARPA reading, vocabulary, and weights.  
Remove build shell scripts that won't work after Hieu changed the header file layout.  



git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3572 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-27 03:46:44 +00:00
heafield
61f5472f1c Avoid some unused parameter complaints and force automake dependencies to fix parallel make. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3571 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-27 00:57:11 +00:00
hieuhoang1972
559a5e3ece ken lm integration 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3570 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-26 22:29:01 +00:00
hieuhoang1972
a82c2d5531 ken lm integration 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3569 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-26 17:02:53 +00:00
hieuhoang1972
f7ceddd0a9 ken lm integration 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3568 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-26 16:49:07 +00:00
pjwilliams
f5a096e2fa In ChartTranslationOptionList::Add(), always consider every target phrase as 
the TargetPhraseCollection won't be ordered.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3567 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-26 12:09:29 +00:00
pjwilliams
81372704de Enforce ttable-limit for memory-based rule table. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3566 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-26 11:38:09 +00:00