moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2024-09-20 07:42:21 +03:00
Code Issues Packages Projects Releases Wiki Activity
6,913 Commits 136 Branches 26 Tags 147 MiB
a65cec2a21
Commit Graph

275 Commits

Author SHA1 Message Date
ehasler
1da0544d77 enable 1best history for perceptron/analytical update 
git-svn-id: http://svn.statmt.org/repository/mira@3894 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:37 +02:00
ehasler
86dba9f743 default: stop when weights converge 
git-svn-id: http://svn.statmt.org/repository/mira@3892 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:37 +02:00
ehasler
a1d2252529 enable perceptron update with only 1 hope and fear translation each 
git-svn-id: http://svn.statmt.org/repository/mira@3891 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:37 +02:00
ehasler
a58dd23b03 remove stop-optimal parameter, default for precision is 1.0, fix flexible slack, inable analytical update 
git-svn-id: http://svn.statmt.org/repository/mira@3890 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:37 +02:00
ehasler
bc2a2078ea add 1best decoding after optimization for --updates-per-epoch option 
git-svn-id: http://svn.statmt.org/repository/mira@3889 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:36 +02:00
ehasler
46cf77fa79 include precision option for constraint checking, fix some logs, ignore update when constraint change == 0 and error increases 
git-svn-id: http://svn.statmt.org/repository/mira@3888 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:36 +02:00
ehasler
921735593d print out epoch for ignored updates 
git-svn-id: http://svn.statmt.org/repository/mira@3887 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:36 +02:00
ehasler
ecfb8465d4 introduce parameter --nbest-first 
git-svn-id: http://svn.statmt.org/repository/mira@3886 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:36 +02:00
ehasler
1f6f8b4abb update training script 
git-svn-id: http://svn.statmt.org/repository/mira@3885 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:36 +02:00
ehasler
479ba8d160 fix bug in options --past-and-current-constraints, --accumulate-most-violated-constraints 
git-svn-id: http://svn.statmt.org/repository/mira@3884 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:36 +02:00
ehasler
bea8af99ff refactor handling of accumulated constraints, constraint checking, introduce burn-in for bleu history, sentence-level bleu, bleu score weight 
git-svn-id: http://svn.statmt.org/repository/mira@3883 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:36 +02:00
ehasler
1964eaf98a refactor handling of accumulated constraints, constraint checking, introduce burn-in for bleu history, sentence-level bleu, bleu score weight 
git-svn-id: http://svn.statmt.org/repository/mira@3882 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:36 +02:00
ehasler
53cbedfd28 update hildreth code for linear classifier to work 
git-svn-id: http://svn.statmt.org/repository/mira@3879 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:36 +02:00
ehasler
10abc4fd62 update weighted loss, adapt accumulate-most-violated-constraints and --past-and-current-constraints to batching, accumulate 1 constraint per sentence in batch, additional option for accumulating only 1 constraint per batch --one-per-batch 
git-svn-id: http://svn.statmt.org/repository/mira@3878 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:36 +02:00
ehasler
bc7f8655d6 implement option --one-constraint that uses only the best model translation and the oracle and computes the update analytically 
git-svn-id: http://svn.statmt.org/repository/mira@3877 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:36 +02:00
ehasler
e784b23eac fix case of mixing and weight dump frequency being 0 
git-svn-id: http://svn.statmt.org/repository/mira@3876 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:36 +02:00
ehasler
1338ccb718 monitor changes in bleu feature history 
git-svn-id: http://svn.statmt.org/repository/mira@3875 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:36 +02:00
ehasler
b23fcf8143 fix comparison of old vs. new model bleu for batches 
git-svn-id: http://svn.statmt.org/repository/mira@3874 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:36 +02:00
bhaddow
e664b4a4b3 Merge 3791-3842 from trunk 
git-svn-id: http://svn.statmt.org/repository/mira@3873 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:36 +02:00
ehasler
ec6272b3fc bleu before and after opt. for batch, print error of constraints 
git-svn-id: http://svn.statmt.org/repository/mira@3872 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:36 +02:00
ehasler
f29d58e3f3 bleu before and after opt. for batch, print error of constraints 
git-svn-id: http://svn.statmt.org/repository/mira@3871 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:36 +02:00
ehasler
6b4a0b442b track the number of violated constraints and constraint changes 
git-svn-id: http://svn.statmt.org/repository/mira@3870 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:36 +02:00
ehasler
16edd0b9bc track the number of violated constraints and constraint changes 
git-svn-id: http://svn.statmt.org/repository/mira@3869 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:36 +02:00
ehasler
ac307d9e8c fix weight dumping for batching 
git-svn-id: http://svn.statmt.org/repository/mira@3868 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:36 +02:00
ehasler
03c5aa5c36 sanity check for updates-per-epoch, stopping criterion 'sum of updates this epoch' 
git-svn-id: http://svn.statmt.org/repository/mira@3867 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:36 +02:00
ehasler
864ade99a3 sanity check for updates-per-epoch, stopping criterion 'sum of updates this epoch' 
git-svn-id: http://svn.statmt.org/repository/mira@3866 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:35 +02:00
ehasler
58eb0d0052 sanity check for updates-per-epoch, stopping criterion 'sum of updates this epoch' 
git-svn-id: http://svn.statmt.org/repository/mira@3865 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:35 +02:00
ehasler
4187e0f691 rename parameter --average-weights 
git-svn-id: http://svn.statmt.org/repository/mira@3864 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:35 +02:00
ehasler
0cbc5b06f4 forgot to check in .h file 
git-svn-id: http://svn.statmt.org/repository/mira@3863 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:35 +02:00
ehasler
cefddaea88 change default values: --control-updates --> true, --accumulate-weights --> false, --distinct-nbest --> true 
git-svn-id: http://svn.statmt.org/repository/mira@3862 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:35 +02:00
ehasler
9ee3edef82 add Hildreth unit tests, fix consstraint comparison in MiraOptimiser, some code clean-up 
git-svn-id: http://svn.statmt.org/repository/mira@3861 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:35 +02:00
ehasler
0198fc5e41 add Hildreth unit tests, fix consstraint comparison in MiraOptimiser, some code clean-up 
git-svn-id: http://svn.statmt.org/repository/mira@3860 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:35 +02:00
ehasler
ac8b86f0c8 show cache usage and weights 
git-svn-id: http://svn.statmt.org/repository/mira@3859 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:35 +02:00
ehasler
56e97a851f change default value for averaging weights to false 
git-svn-id: http://svn.statmt.org/repository/mira@3858 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:35 +02:00
ehasler
61a5d7f511 compare best model score and bleu before and after weight change 
git-svn-id: http://svn.statmt.org/repository/mira@3857 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:35 +02:00
ehasler
2dfe7163b8 fix average moses weights 
git-svn-id: http://svn.statmt.org/repository/mira@3856 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:35 +02:00
ehasler
3b760065ec update 
git-svn-id: http://svn.statmt.org/repository/mira@3855 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:35 +02:00
ehasler
fa661939dd update training script 
git-svn-id: http://svn.statmt.org/repository/mira@3853 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:35 +02:00
ehasler
269f1018c3 introduce option to use average weights for pruning 
git-svn-id: http://svn.statmt.org/repository/mira@3852 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:35 +02:00
ehasler
f6483df41c average approx bleu of best model score translation 
git-svn-id: http://svn.statmt.org/repository/mira@3851 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:35 +02:00
ehasler
62b5fb7f4d introduce parameter --update-after 
git-svn-id: http://svn.statmt.org/repository/mira@3848 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:35 +02:00
ehasler
1630603dce code cleanup 
git-svn-id: http://svn.statmt.org/repository/mira@3847 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:35 +02:00
ehasler
dbd2146188 introduce parameter --update-after-epoch 
git-svn-id: http://svn.statmt.org/repository/mira@3846 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:35 +02:00
ehasler
d04066ee0e introduce parameter --update-after-epoch 
git-svn-id: http://svn.statmt.org/repository/mira@3845 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:35 +02:00
ehasler
41d2d28408 update training script to compute BLEU of dev set, update computation of dev bleu (on best model score translation), check best model scores after weight update, new 
git-svn-id: http://svn.statmt.org/repository/mira@3844 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:35 +02:00
ehasler
f95a9d5d26 update training script to compute BLEU of dev set, update computation of dev bleu (on best model score translation), check best model scores after weight update 
git-svn-id: http://svn.statmt.org/repository/mira@3843 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:35 +02:00
ehasler
9f057c039f adapt BPfactor by 0.05, introduce params for minimum learning rate and minimum sentence update 
git-svn-id: http://svn.statmt.org/repository/mira@3841 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:34 +02:00
ehasler
c41202f2ff fix case when learning rate/max sentence update is 0 
git-svn-id: http://svn.statmt.org/repository/mira@3838 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:34 +02:00
ehasler
b7971689cb adapt BP factor 
git-svn-id: http://svn.statmt.org/repository/mira@3836 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:34 +02:00
ehasler
8139f70df7 introduce parameter --decr-sentence-update 
git-svn-id: http://svn.statmt.org/repository/mira@3835 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:34 +02:00
ehasler
6642a0437c fix reduce.. 
git-svn-id: http://svn.statmt.org/repository/mira@3834 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:34 +02:00
ehasler
ad1812f63a fix weight dumping 
git-svn-id: http://svn.statmt.org/repository/mira@3833 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:34 +02:00
ehasler
7e181ffefd fix weight dumping 
git-svn-id: http://svn.statmt.org/repository/mira@3832 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:34 +02:00
ehasler
8d95753d0c back to dumping cumulative weights, fix learning rate and max-sentence-update 
git-svn-id: http://svn.statmt.org/repository/mira@3831 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:34 +02:00
ehasler
aae83df416 correct file names 
git-svn-id: http://svn.statmt.org/repository/mira@3830 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:34 +02:00
ehasler
6f3506b46b rename parameter --mixing-frequency 
git-svn-id: http://svn.statmt.org/repository/mira@3829 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:34 +02:00
ehasler
9eb2cdee12 rename dev set oracles 
git-svn-id: http://svn.statmt.org/repository/mira@3828 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:34 +02:00
ehasler
377bd27b36 use total weights for convergence check, dump total weights after every epoch 
git-svn-id: http://svn.statmt.org/repository/mira@3827 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:34 +02:00
ehasler
3906510611 closing bracket.. 
git-svn-id: http://svn.statmt.org/repository/mira@3826 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:34 +02:00
ehasler
fecfe64d52 Dump weights always after mixing, dump final weights after averaging cumulative weights, stopping criterion related to weights after every epoch 
git-svn-id: http://svn.statmt.org/repository/mira@3825 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:34 +02:00
ehasler
8bfb25f703 change default for --accumulate-weights to true 
git-svn-id: http://svn.statmt.org/repository/mira@3824 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:34 +02:00
ehasler
ffcd03dd51 change max-sentence-update to scale weights instead of clipping them, fix mpi commands 
git-svn-id: http://svn.statmt.org/repository/mira@3823 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:34 +02:00
ehasler
505eba736e introduce parameters --stop-dev-bleu, --stop-approx-dev-bleu 
git-svn-id: http://svn.statmt.org/repository/mira@3822 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:34 +02:00
ehasler
a26a38533b code cleanup 
git-svn-id: http://svn.statmt.org/repository/mira@3821 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:34 +02:00
ehasler
512674dacb change default value for slack to 0.01 
git-svn-id: http://svn.statmt.org/repository/mira@3820 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:34 +02:00
ehasler
75565119d2 introduce parameters --max-sentence-update, --check-updates, --decr-learning-rate, change some default values, code cleanup 
git-svn-id: http://svn.statmt.org/repository/mira@3819 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:34 +02:00
ehasler
691d1bcb48 update training script not to do normalisation 
git-svn-id: http://svn.statmt.org/repository/mira@3818 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:34 +02:00
ehasler
e81a7e6da1 optional normalisation at start 
git-svn-id: http://svn.statmt.org/repository/mira@3817 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:33 +02:00
ehasler
a049734fd6 print oracles for rank0 when stopping 
git-svn-id: http://svn.statmt.org/repository/mira@3816 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:33 +02:00
ehasler
ca7b28155c update of printing dev oracles, parameter for printing feature values 
git-svn-id: http://svn.statmt.org/repository/mira@3815 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:33 +02:00
ehasler
b44f1447cd write oracle translation to file 
git-svn-id: http://svn.statmt.org/repository/mira@3814 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:33 +02:00
ehasler
0722b78c0d introduce parameter --normalise 
git-svn-id: http://svn.statmt.org/repository/mira@3813 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:33 +02:00
ehasler
9b3c9a6fd6 fix handling of large memory jobs 
git-svn-id: http://svn.statmt.org/repository/mira@3811 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:33 +02:00
ehasler
eba945b61c print rank with BLEU 
git-svn-id: http://svn.statmt.org/repository/mira@3810 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:33 +02:00
ehasler
eeba3547cd change default: --dev-bleu 1 
git-svn-id: http://svn.statmt.org/repository/mira@3807 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:33 +02:00
ehasler
fbcc9e1401 update script to pass on decoder settings 
git-svn-id: http://svn.statmt.org/repository/mira@3806 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:33 +02:00
ehasler
5312e8fc22 compute bleu on oracles translations of dev set, introduce params --min-weight-change, --msf-step, --msf-min, --slack-step, --slack-max, --decoder-settings 
git-svn-id: http://svn.statmt.org/repository/mira@3804 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:33 +02:00
ehasler
c4acd339fd enable second language model 
git-svn-id: http://svn.statmt.org/repository/mira@3803 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:33 +02:00
ehasler
c911b44ac3 comment printing of weights, change mpi quitting, several changes to training script 
git-svn-id: http://svn.statmt.org/repository/mira@3800 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:33 +02:00
ehasler
391c6806a3 print information separately for each process ('rank 0: ...') 
git-svn-id: http://svn.statmt.org/repository/mira@3799 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:33 +02:00
ehasler
606355deba Enable taking logs of feature values 
git-svn-id: http://svn.statmt.org/repository/mira@3798 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:33 +02:00
ehasler
8309fa8dd7 update scripts training-expt.perl, extractInformation.perl, mapWeightsToIni.perl 
git-svn-id: http://svn.statmt.org/repository/mira@3797 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:33 +02:00
ehasler
804033bd2c Revert to revision 3774 of public svn + add 0 to weight files < 10 and write 'stoppinDg' file 
git-svn-id: http://svn.statmt.org/repository/mira@3795 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:55:38 +02:00
evahasler
b8a4100ecd introduce parameters --log-feature-values and --base-of-log 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3785 1f5c12ca-751b-0410-a591-d2e778427230
 2010-12-16 20:01:36 +00:00
evahasler
64ccd4bb1a clipping, ignore certain updates 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3784 1f5c12ca-751b-0410-a591-d2e778427230
 2010-12-15 19:51:21 +00:00
evahasler
dd976678d3 clipping, ignore certain updates 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3783 1f5c12ca-751b-0410-a591-d2e778427230
 2010-12-15 19:48:45 +00:00
evahasler
3f8a00f0e4 revert clipping 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3780 1f5c12ca-751b-0410-a591-d2e778427230
 2010-12-15 12:58:32 +00:00
evahasler
6e7c6d530e revert clipping 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3779 1f5c12ca-751b-0410-a591-d2e778427230
 2010-12-15 12:54:19 +00:00
evahasler
aa2e988132 revert clipping 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3778 1f5c12ca-751b-0410-a591-d2e778427230
 2010-12-15 12:53:49 +00:00
evahasler
4024b5ef79 fix unknown word penalty parameter 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3776 1f5c12ca-751b-0410-a591-d2e778427230
 2010-12-14 18:27:25 +00:00
evahasler
4839fee8cb add parameter --ignore-weird-updates 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3774 1f5c12ca-751b-0410-a591-d2e778427230
 2010-12-14 15:31:20 +00:00
evahasler
54824d132f add parameter --ignore-u-weight 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3773 1f5c12ca-751b-0410-a591-d2e778427230
 2010-12-14 14:51:04 +00:00
evahasler
fbc8910102 reintroduce clipping 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3772 1f5c12ca-751b-0410-a591-d2e778427230
 2010-12-14 12:17:17 +00:00
evahasler
da95cb2560 compute error only for rank 0 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3771 1f5c12ca-751b-0410-a591-d2e778427230
 2010-12-13 18:10:17 +00:00
evahasler
2f1b959302 change bleu smoothing, change handling of multiple oracles, parameter for increasing BP 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3770 1f5c12ca-751b-0410-a591-d2e778427230
 2010-12-10 16:34:43 +00:00
evahasler
cd62ffc021 mpi abort, compute error over fixed set of ex., add params --suppress-convergence and --past-and-current-constraints 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3766 1f5c12ca-751b-0410-a591-d2e778427230
 2010-12-09 18:55:52 +00:00
evahasler
7c238b6b44 add parameters --accumulate-oracles and --accumulate-most-violated-constraints 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3765 1f5c12ca-751b-0410-a591-d2e778427230
 2010-12-09 11:49:52 +00:00
evahasler
0c70c1a4ef introduce weighted loss function 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3761 1f5c12ca-751b-0410-a591-d2e778427230
 2010-12-06 19:32:59 +00:00
evahasler
47d91e8b93 Introduce stopping criterion based on changes in average total weights 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3760 1f5c12ca-751b-0410-a591-d2e778427230
 2010-12-06 15:28:51 +00:00
evahasler
87141f9f89 debug code 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3758 1f5c12ca-751b-0410-a591-d2e778427230
 2010-12-02 18:27:21 +00:00
evahasler
b852fb6763 add scripts 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3757 1f5c12ca-751b-0410-a591-d2e778427230
 2010-12-02 10:22:50 +00:00
evahasler
5c96b2d07d test 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3756 1f5c12ca-751b-0410-a591-d2e778427230
 2010-12-02 10:22:12 +00:00
evahasler
fc64a41495 update training script 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3755 1f5c12ca-751b-0410-a591-d2e778427230
 2010-12-01 18:51:19 +00:00
evahasler
2ee9e26104 typo 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3754 1f5c12ca-751b-0410-a591-d2e778427230
 2010-12-01 18:21:58 +00:00
evahasler
cd12923603 typo 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3753 1f5c12ca-751b-0410-a591-d2e778427230
 2010-12-01 18:13:52 +00:00
evahasler
406f55584e introduce parameter --history-smoothing, add mpi_finalize statement 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3752 1f5c12ca-751b-0410-a591-d2e778427230
 2010-12-01 18:09:49 +00:00
evahasler
da712f0eeb change return type from size_t to int 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3751 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-30 17:26:34 +00:00
evahasler
859483640f fix batching 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3747 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-29 19:09:34 +00:00
evahasler
628c2380a9 add parameter --regularise-hildreth-updates 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3746 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-29 18:42:18 +00:00
evahasler
e442c513dd sanity check number of violated constraint/overall change in objective 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3743 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-29 15:07:38 +00:00
evahasler
4af767af60 Introduce parameter --increase-BP 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3742 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-29 14:40:15 +00:00
evahasler
2feabd94fc Introduce parameter --distinct-nbest 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3741 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-29 14:13:45 +00:00
evahasler
360c03b3bf Introduce parameter --distinct-nbest 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3740 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-29 14:11:19 +00:00
evahasler
e7d3ddb358 disable mpi finalize (not working) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3735 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-24 17:28:08 +00:00
evahasler
1de7aa4f81 enable batch optimisation, batch history updating, fix bug in Decoder.cpp (new: staticData.ReLoadBleuScoreFeatureParameter()) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3734 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-24 17:06:54 +00:00
evahasler
97b7e1401e delete fear translations 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3733 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-23 16:20:26 +00:00
evahasler
8cd6d2c34a delete fear translations 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3732 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-23 16:17:35 +00:00
evahasler
4a6027f7c7 introduce parameter --scale-by-input-length 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3731 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-23 15:59:36 +00:00
evahasler
8de8773b39 fix averaging of cumulative weights 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3730 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-23 15:11:32 +00:00
evahasler
904329798d introduce arguments --accumulate-weights and --use-scaled-reference 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3727 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-22 20:36:14 +00:00
evahasler
a25bbecd76 update smo 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3722 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-19 11:35:16 +00:00
evahasler
53b61425d5 add fixed clipping for smo 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3721 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-18 16:24:51 +00:00
evahasler
93ff481caa add parameter for size of nbest lists and for adding constraints 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3717 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-16 18:52:08 +00:00
evahasler
81088c8f30 Introduce margin scale factor, change printing of feature vectors 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3716 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-16 12:11:00 +00:00
evahasler
57b0e64cda Introduce margin scale factor, change printing of feature vectors 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3715 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-16 11:44:44 +00:00
evahasler
ffc37e64b3 normalize weights after mixing 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3707 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-12 15:40:53 +00:00
evahasler
590ad17713 normalize weights after mixing 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3706 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-12 15:35:10 +00:00
evahasler
7bc6318270 normalize weights after mixing 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3705 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-12 15:22:28 +00:00
evahasler
74cd65c24a update 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3693 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-09 17:37:10 +00:00
evahasler
7af072ace0 remove clipping from updateWeights(), update Main 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3692 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-09 17:31:30 +00:00
evahasler
9697b152be all to cerr 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3684 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-01 16:56:10 +00:00
evahasler
f17a6c6809 fix hildreth 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3683 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-01 12:30:49 +00:00
evahasler
acc6ac21dc fix loop 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3681 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-29 17:36:16 +00:00
evahasler
08981b87f0 add Hildreth algorithm 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3680 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-29 15:42:50 +00:00
evahasler
8f00469259 add Hildreth algorithm 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3679 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-29 15:41:37 +00:00
bhaddow
179572c83a experiment script 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3678 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-28 16:20:57 +00:00
bhaddow
68d5f4469d weight dumping 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3677 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-28 12:41:33 +00:00
evahasler
17358fd834 sanity check for margin 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3676 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-28 10:00:35 +00:00
evahasler
144d750e33 Fix memory leak in Main.cpp 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3654 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-25 15:16:34 +00:00
evahasler
85a71793a6 Update mira optimatization code and merge Main.cpp 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3652 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-25 12:22:35 +00:00
evahasler
1bd263c4ec Return Bleu scores instead of mixed scores (to be used in loss function) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3642 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-22 16:50:42 +00:00
bhaddow
7e72ceea22 Goodbye ScoreIndexManager. 
Compiles ok, but haven't dared to run regression yet.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3608 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-06 22:06:49 +00:00
bhaddow
a4b4ba15d7 Enable parallelisation 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3584 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-28 15:13:50 +00:00
hieuhoang1972
03419ac02a it runs\!\! 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3542 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-17 20:27:26 +00:00
starsplash
1e8d54a0d2 Add the mira criterion 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3541 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-17 17:52:33 +00:00
hieuhoang1972
c28168538a bugs 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3540 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-17 17:38:25 +00:00
hieuhoang1972
104c6a34fd bug 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3539 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-17 17:25:51 +00:00
starsplash
15810d7899 again my fault 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3538 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-17 17:07:03 +00:00
starsplash
b020c5945b sry for the typo 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3537 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-17 16:38:28 +00:00
bhaddow
44f14c776d history update 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3536 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-17 16:34:48 +00:00