Commit Graph

6108 Commits

Author SHA1 Message Date
ehasler
0198fc5e41 add Hildreth unit tests, fix consstraint comparison in MiraOptimiser, some code clean-up
git-svn-id: http://svn.statmt.org/repository/mira@3860 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:35 +02:00
ehasler
ac8b86f0c8 show cache usage and weights
git-svn-id: http://svn.statmt.org/repository/mira@3859 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:35 +02:00
ehasler
56e97a851f change default value for averaging weights to false
git-svn-id: http://svn.statmt.org/repository/mira@3858 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:35 +02:00
ehasler
61a5d7f511 compare best model score and bleu before and after weight change
git-svn-id: http://svn.statmt.org/repository/mira@3857 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:35 +02:00
ehasler
2dfe7163b8 fix average moses weights
git-svn-id: http://svn.statmt.org/repository/mira@3856 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:35 +02:00
ehasler
3b760065ec update
git-svn-id: http://svn.statmt.org/repository/mira@3855 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:35 +02:00
ehasler
4bf98c9169 dont print name of feature in vector
git-svn-id: http://svn.statmt.org/repository/mira@3854 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:35 +02:00
ehasler
fa661939dd update training script
git-svn-id: http://svn.statmt.org/repository/mira@3853 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:35 +02:00
ehasler
269f1018c3 introduce option to use average weights for pruning
git-svn-id: http://svn.statmt.org/repository/mira@3852 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:35 +02:00
ehasler
f6483df41c average approx bleu of best model score translation
git-svn-id: http://svn.statmt.org/repository/mira@3851 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:35 +02:00
bhaddow
f7b4ef79d5 missed from previous checkin
git-svn-id: http://svn.statmt.org/repository/mira@3850 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:35 +02:00
bhaddow
c8e7c5da05 initialisation of source phrase member variable in TargetPhrase
git-svn-id: http://svn.statmt.org/repository/mira@3849 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:35 +02:00
ehasler
62b5fb7f4d introduce parameter --update-after
git-svn-id: http://svn.statmt.org/repository/mira@3848 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:35 +02:00
ehasler
1630603dce code cleanup
git-svn-id: http://svn.statmt.org/repository/mira@3847 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:35 +02:00
ehasler
dbd2146188 introduce parameter --update-after-epoch
git-svn-id: http://svn.statmt.org/repository/mira@3846 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:35 +02:00
ehasler
d04066ee0e introduce parameter --update-after-epoch
git-svn-id: http://svn.statmt.org/repository/mira@3845 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:35 +02:00
ehasler
41d2d28408 update training script to compute BLEU of dev set, update computation of dev bleu (on best model score translation), check best model scores after weight update, new
git-svn-id: http://svn.statmt.org/repository/mira@3844 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:35 +02:00
ehasler
f95a9d5d26 update training script to compute BLEU of dev set, update computation of dev bleu (on best model score translation), check best model scores after weight update
git-svn-id: http://svn.statmt.org/repository/mira@3843 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:35 +02:00
bhaddow
367ac683a3 Implementation of phrase pair feature
git-svn-id: http://svn.statmt.org/repository/mira@3842 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:34 +02:00
ehasler
9f057c039f adapt BPfactor by 0.05, introduce params for minimum learning rate and minimum sentence update
git-svn-id: http://svn.statmt.org/repository/mira@3841 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:34 +02:00
bhaddow
5f4ac522e8 Print names as well as values in feature vector printout
git-svn-id: http://svn.statmt.org/repository/mira@3840 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:34 +02:00
bhaddow
9545a2e068 don't try to get feature list for sparse features
git-svn-id: http://svn.statmt.org/repository/mira@3839 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:34 +02:00
ehasler
c41202f2ff fix case when learning rate/max sentence update is 0
git-svn-id: http://svn.statmt.org/repository/mira@3838 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:34 +02:00
bhaddow
14b35072bc Avoid bad_alloc for variable length features
git-svn-id: http://svn.statmt.org/repository/mira@3837 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:34 +02:00
ehasler
b7971689cb adapt BP factor
git-svn-id: http://svn.statmt.org/repository/mira@3836 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:34 +02:00
ehasler
8139f70df7 introduce parameter --decr-sentence-update
git-svn-id: http://svn.statmt.org/repository/mira@3835 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:34 +02:00
ehasler
6642a0437c fix reduce..
git-svn-id: http://svn.statmt.org/repository/mira@3834 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:34 +02:00
ehasler
ad1812f63a fix weight dumping
git-svn-id: http://svn.statmt.org/repository/mira@3833 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:34 +02:00
ehasler
7e181ffefd fix weight dumping
git-svn-id: http://svn.statmt.org/repository/mira@3832 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:34 +02:00
ehasler
8d95753d0c back to dumping cumulative weights, fix learning rate and max-sentence-update
git-svn-id: http://svn.statmt.org/repository/mira@3831 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:34 +02:00
ehasler
aae83df416 correct file names
git-svn-id: http://svn.statmt.org/repository/mira@3830 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:34 +02:00
ehasler
6f3506b46b rename parameter --mixing-frequency
git-svn-id: http://svn.statmt.org/repository/mira@3829 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:34 +02:00
ehasler
9eb2cdee12 rename dev set oracles
git-svn-id: http://svn.statmt.org/repository/mira@3828 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:34 +02:00
ehasler
377bd27b36 use total weights for convergence check, dump total weights after every epoch
git-svn-id: http://svn.statmt.org/repository/mira@3827 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:34 +02:00
ehasler
3906510611 closing bracket..
git-svn-id: http://svn.statmt.org/repository/mira@3826 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:34 +02:00
ehasler
fecfe64d52 Dump weights always after mixing, dump final weights after averaging cumulative weights, stopping criterion related to weights after every epoch
git-svn-id: http://svn.statmt.org/repository/mira@3825 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:34 +02:00
ehasler
8bfb25f703 change default for --accumulate-weights to true
git-svn-id: http://svn.statmt.org/repository/mira@3824 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:34 +02:00
ehasler
ffcd03dd51 change max-sentence-update to scale weights instead of clipping them, fix mpi commands
git-svn-id: http://svn.statmt.org/repository/mira@3823 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:34 +02:00
ehasler
505eba736e introduce parameters --stop-dev-bleu, --stop-approx-dev-bleu
git-svn-id: http://svn.statmt.org/repository/mira@3822 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:34 +02:00
ehasler
a26a38533b code cleanup
git-svn-id: http://svn.statmt.org/repository/mira@3821 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:34 +02:00
ehasler
512674dacb change default value for slack to 0.01
git-svn-id: http://svn.statmt.org/repository/mira@3820 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:34 +02:00
ehasler
75565119d2 introduce parameters --max-sentence-update, --check-updates, --decr-learning-rate, change some default values, code cleanup
git-svn-id: http://svn.statmt.org/repository/mira@3819 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:34 +02:00
ehasler
691d1bcb48 update training script not to do normalisation
git-svn-id: http://svn.statmt.org/repository/mira@3818 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:34 +02:00
ehasler
e81a7e6da1 optional normalisation at start
git-svn-id: http://svn.statmt.org/repository/mira@3817 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:33 +02:00
ehasler
a049734fd6 print oracles for rank0 when stopping
git-svn-id: http://svn.statmt.org/repository/mira@3816 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:33 +02:00
ehasler
ca7b28155c update of printing dev oracles, parameter for printing feature values
git-svn-id: http://svn.statmt.org/repository/mira@3815 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:33 +02:00
ehasler
b44f1447cd write oracle translation to file
git-svn-id: http://svn.statmt.org/repository/mira@3814 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:33 +02:00
ehasler
0722b78c0d introduce parameter --normalise
git-svn-id: http://svn.statmt.org/repository/mira@3813 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:33 +02:00
bhaddow
3348dcd03e Configurable factor id for target bigram
git-svn-id: http://svn.statmt.org/repository/mira@3812 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:33 +02:00
ehasler
9b3c9a6fd6 fix handling of large memory jobs
git-svn-id: http://svn.statmt.org/repository/mira@3811 cc96ff50-19ce-11e0-b349-13d7f0bd23df
2011-08-18 12:59:33 +02:00