moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2024-09-21 08:07:14 +03:00
Code Issues Packages Projects Releases Wiki Activity
6,071 Commits 136 Branches 26 Tags 147 MiB
ffcd03dd51
Commit Graph

1132 Commits

Author SHA1 Message Date
ehasler
ffcd03dd51 change max-sentence-update to scale weights instead of clipping them, fix mpi commands 
git-svn-id: http://svn.statmt.org/repository/mira@3823 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:34 +02:00
ehasler
75565119d2 introduce parameters --max-sentence-update, --check-updates, --decr-learning-rate, change some default values, code cleanup 
git-svn-id: http://svn.statmt.org/repository/mira@3819 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:34 +02:00
bhaddow
3348dcd03e Configurable factor id for target bigram 
git-svn-id: http://svn.statmt.org/repository/mira@3812 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:33 +02:00
ehasler
7ceae4581a return unaltered state if no new words are translated 
git-svn-id: http://svn.statmt.org/repository/mira@3809 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:33 +02:00
ehasler
0993dde7ce fix precision 
git-svn-id: http://svn.statmt.org/repository/mira@3808 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:33 +02:00
ehasler
2f54b8f61f remove smoothing from function CalculateBleuOfCorpus 
git-svn-id: http://svn.statmt.org/repository/mira@3805 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:33 +02:00
ehasler
5312e8fc22 compute bleu on oracles translations of dev set, introduce params --min-weight-change, --msf-step, --msf-min, --slack-step, --slack-max, --decoder-settings 
git-svn-id: http://svn.statmt.org/repository/mira@3804 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:33 +02:00
ehasler
0db01446b7 fix bug introduced in last CL 
git-svn-id: http://svn.statmt.org/repository/mira@3802 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:33 +02:00
ehasler
df66bf2816 extend lexical reordering component to allow for more weights 
git-svn-id: http://svn.statmt.org/repository/mira@3801 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:33 +02:00
ehasler
606355deba Enable taking logs of feature values 
git-svn-id: http://svn.statmt.org/repository/mira@3798 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:33 +02:00
bhaddow
a2730c445d Merge up to 3791 from trunk. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3792 1f5c12ca-751b-0410-a591-d2e778427230
 2011-01-05 13:49:44 +00:00
evahasler
b8a4100ecd introduce parameters --log-feature-values and --base-of-log 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3785 1f5c12ca-751b-0410-a591-d2e778427230
 2010-12-16 20:01:36 +00:00
evahasler
64ccd4bb1a clipping, ignore certain updates 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3784 1f5c12ca-751b-0410-a591-d2e778427230
 2010-12-15 19:51:21 +00:00
evahasler
dd976678d3 clipping, ignore certain updates 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3783 1f5c12ca-751b-0410-a591-d2e778427230
 2010-12-15 19:48:45 +00:00
evahasler
2f1b959302 change bleu smoothing, change handling of multiple oracles, parameter for increasing BP 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3770 1f5c12ca-751b-0410-a591-d2e778427230
 2010-12-10 16:34:43 +00:00
hieuhoang1972
41c5b3a1c2 xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3769 1f5c12ca-751b-0410-a591-d2e778427230
 2010-12-10 13:04:06 +00:00
heafield
5e9df58a3c Respect -v 0 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3768 1f5c12ca-751b-0410-a591-d2e778427230
 2010-12-09 22:13:09 +00:00
evahasler
47d91e8b93 Introduce stopping criterion based on changes in average total weights 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3760 1f5c12ca-751b-0410-a591-d2e778427230
 2010-12-06 15:28:51 +00:00
evahasler
406f55584e introduce parameter --history-smoothing, add mpi_finalize statement 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3752 1f5c12ca-751b-0410-a591-d2e778427230
 2010-12-01 18:09:49 +00:00
evahasler
13dd3b282c increase BP 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3748 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-29 21:12:46 +00:00
bhaddow
50f0e6c07d Add a show-weights option. It prints out the moses features and exits. May 
load tables as a side-effect.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3744 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-29 16:44:28 +00:00
evahasler
4af767af60 Introduce parameter --increase-BP 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3742 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-29 14:40:15 +00:00
evahasler
1de7aa4f81 enable batch optimisation, batch history updating, fix bug in Decoder.cpp (new: staticData.ReLoadBleuScoreFeatureParameter()) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3734 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-24 17:06:54 +00:00
evahasler
4a6027f7c7 introduce parameter --scale-by-input-length 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3731 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-23 15:59:36 +00:00
evahasler
904329798d introduce arguments --accumulate-weights and --use-scaled-reference 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3727 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-22 20:36:14 +00:00
bhaddow
6255216b6a Remove gnu-specific typeof 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3725 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-22 10:05:17 +00:00
bhaddow
a7e0977eea Fix compile error by using correct macro. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3720 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-18 10:27:30 +00:00
chardmeier
837a667a95 Cleaned up language modelling code by disentangling the decoder's LM feature 
function from the LM toolkit abstraction layer. There are two different groups
of classes now:
- LanguageModel, which inherits from StatefulFeatureFunction and contains
  the n-gram model feature function.
- LanguageModelImplementation, which is the base class of the individual
  LM implementations (SRI, IRST, RandLM, KenLM) and provides methods to
  query LM probabilities and states.
Each LanguageModel controls a LanguageModelImplementation. Implementations can
be shared by more than one LanguageModel.
This should make it easier to use the LM libraries as a backend for other
feature functions while retaining the flexibility to use different LM toolkits.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3719 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-17 14:06:21 +00:00
chardmeier
d18ff948f5 Bugfixes in srilm adaptor. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3718 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-17 13:23:44 +00:00
evahasler
57b0e64cda Introduce margin scale factor, change printing of feature vectors 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3715 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-16 11:44:44 +00:00
leven101
84d83480b6 function name changes 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3711 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-15 11:32:02 +00:00
leven101
5251a2823a separated source and target vocab in suffixarrays to support unequal factors 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3710 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-15 11:28:27 +00:00
bojar
2ea140062b don't warn about probs outside [0,1] in -verbose 0 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3698 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-10 11:51:26 +00:00
heafield
2784923899 Rename a bunch of kenlm files. A ./regenerate-makefiles.sh is required. 
Make loading with MAP_POPULATE on Linux and read on other OSes the default.
Use LM #9 for lazy loading, as recommended by other devs.  
Slightly faster trie.  



git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3688 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-06 00:40:16 +00:00
leven101
34b45c0480 removed debug messages from BilingualDynSuffixArray.cpp 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3687 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-04 18:41:04 +00:00
bhaddow
3aee6fab5d Use correct conditional compilation flag for threaded moses 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3686 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-03 18:43:18 +00:00
bhaddow
68d5f4469d weight dumping 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3677 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-28 12:41:33 +00:00
heafield
c12c2c59d2 Autodetect model from binary format. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3675 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-28 01:05:04 +00:00
hieuhoang1972
735d5b682f xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3673 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-27 18:54:50 +00:00
heafield
614d6002a6 Integrate heafield-refactorlm. Faster kenlm with new binary format. Stateful language model 
framework.  



git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3671 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-27 17:50:40 +00:00
nicolabertoldi
bb08dcb5b6 made code compliant with the enhanced IRSTLM library; IRSTLM release 5.50.01 is needed; back compatibility is not assured; 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3662 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-26 16:06:01 +00:00
evahasler
0e68ce45e4 Add smoothing, geometric mean, source length history 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3641 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-22 16:33:36 +00:00
nicolabertoldi
3d7f391ae5 remove inefficient function calls 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3638 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-22 13:30:50 +00:00
hieuhoang1972
564a9ad371 word alignment info in binary pt, code provided by Christof Pintaske 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3637 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-22 10:37:19 +00:00
bhaddow
9b438d9849 correction to error message 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3635 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-21 13:23:53 +00:00
bhaddow
a12386918b correction to score producer description 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3629 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-19 09:47:36 +00:00
bhaddow
0ab6122c8e correct short arg for weight-file 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3628 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-19 09:32:04 +00:00
bhaddow
0f09365a99 Enable usage of target bigram feature in decoding. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3626 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-15 15:19:17 +00:00
bhaddow
4126c19f91 Implementation and testing of target bigram feature 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3624 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-14 21:52:35 +00:00
hieuhoang1972
ac875ffc81 mem leak 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3617 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-12 14:47:58 +00:00