nicolabertoldi
|
820e3dfe9c
|
adding regression tests for mert
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2551 1f5c12ca-751b-0410-a591-d2e778427230
|
2009-10-01 12:53:44 +00:00 |
|
nicolabertoldi
|
0393183eb4
|
mert software now works with different reference length policies: shortest, average, closest (default) and with case information (default is preserving case). Pay attention that both defaults are different from the previous version (which were shortest reflen and case-insensitive).
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2459 1f5c12ca-751b-0410-a591-d2e778427230
|
2009-08-05 15:38:35 +00:00 |
|
bhaddow
|
1ef0f3844d
|
Connect new mert to the moses build system
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2221 1f5c12ca-751b-0410-a591-d2e778427230
|
2009-02-26 11:17:12 +00:00 |
|
bhaddow
|
bb9e2f427d
|
Fix random restarts so that they are actually random, using the system clock
for seeding.
Previously, new mert followed old mert in using the same set of "random" points
for every iteration of the outer loop.
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2126 1f5c12ca-751b-0410-a591-d2e778427230
|
2009-02-12 12:58:27 +00:00 |
|
bhaddow
|
6c8c8e9dc4
|
initial weights for toy example
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1989 1f5c12ca-751b-0410-a591-d2e778427230
|
2009-01-22 10:25:14 +00:00 |
|
nicolabertoldi
|
830d9f3404
|
small change to reduce few useless computations
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1970 1f5c12ca-751b-0410-a591-d2e778427230
|
2009-01-07 13:47:38 +00:00 |
|
nicolabertoldi
|
4b4c1b3973
|
imported utilities for timing from Moses
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1969 1f5c12ca-751b-0410-a591-d2e778427230
|
2009-01-07 13:30:06 +00:00 |
|
nicolabertoldi
|
32029561da
|
mert can now load more data files
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1942 1f5c12ca-751b-0410-a591-d2e778427230
|
2008-11-18 18:51:02 +00:00 |
|
phkoehn
|
1b5d99ad26
|
added headers for standard compliance (gcc 4.3 on 64 bit linux)
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1905 1f5c12ca-751b-0410-a591-d2e778427230
|
2008-10-16 21:14:38 +00:00 |
|
bhaddow
|
cd28f119c6
|
mert tests
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1881 1f5c12ca-751b-0410-a591-d2e778427230
|
2008-09-01 17:18:51 +00:00 |
|
bhaddow
|
83f234cf17
|
Implementation of Cer et al mert regularisation. Use with argument such
as --scconfig regtype:min,regwin:3 in extractor and mert. Only tested
on toy example so far.
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1860 1f5c12ca-751b-0410-a591-d2e778427230
|
2008-06-24 19:27:18 +00:00 |
|
nicolabertoldi
|
e94834012d
|
added facilities to read and write score statistics in binary format
moved facilities for feature names in FeatureData object
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1824 1f5c12ca-751b-0410-a591-d2e778427230
|
2008-06-05 17:03:54 +00:00 |
|
nicolabertoldi
|
8e96e68476
|
overall change of a variable name: array_ instead of array2_
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1823 1f5c12ca-751b-0410-a591-d2e778427230
|
2008-06-05 11:42:00 +00:00 |
|
nicolabertoldi
|
930e67c5e3
|
fixed another bug related to the handling of feature names
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1822 1f5c12ca-751b-0410-a591-d2e778427230
|
2008-06-05 11:30:37 +00:00 |
|
nicolabertoldi
|
44d7e0e0f7
|
fixed a bug related to the handling of feature names
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1821 1f5c12ca-751b-0410-a591-d2e778427230
|
2008-06-05 11:26:54 +00:00 |
|
bhaddow
|
37cf805139
|
Fix bug in output of scorestats to text file
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1820 1f5c12ca-751b-0410-a591-d2e778427230
|
2008-06-05 11:08:29 +00:00 |
|
nicolabertoldi
|
281bf610b8
|
added binary read/load facility for feature data
added names of features in the header
added methods to access the features by name
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1819 1f5c12ca-751b-0410-a591-d2e778427230
|
2008-06-05 07:23:34 +00:00 |
|
jfouet
|
f849154a73
|
fix FeatureStats::savetxt
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1815 1f5c12ca-751b-0410-a591-d2e778427230
|
2008-06-04 08:33:37 +00:00 |
|
jfouet
|
8e54557e9f
|
fix crash in getNextPound
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1814 1f5c12ca-751b-0410-a591-d2e778427230
|
2008-06-04 08:32:19 +00:00 |
|
nicolabertoldi
|
1402f3036b
|
added ScorerFactory.h
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1811 1f5c12ca-751b-0410-a591-d2e778427230
|
2008-06-03 08:56:37 +00:00 |
|
nicolabertoldi
|
af585bc492
|
nbest can be read from stdin, too
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1797 1f5c12ca-751b-0410-a591-d2e778427230
|
2008-05-27 17:20:01 +00:00 |
|
nicolabertoldi
|
291260abf7
|
- made output more compliant with old version
- added PerSCorer.h and BleuScorer.h
- stored feature names
- fixed bug about output of best Point
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1796 1f5c12ca-751b-0410-a591-d2e778427230
|
2008-05-27 16:50:52 +00:00 |
|
jfouet
|
89194be5eb
|
safer(hopefully) insertion of threshold in LineOptimize
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1795 1f5c12ca-751b-0410-a591-d2e778427230
|
2008-05-23 12:02:14 +00:00 |
|
nicolabertoldi
|
c9593648bb
|
change from int to unsigned where needed
add some debugging output (to remove later)
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1794 1f5c12ca-751b-0410-a591-d2e778427230
|
2008-05-23 11:48:16 +00:00 |
|
bhaddow
|
e8d26f4f5c
|
Fixes to optimiser to stop it crashing and failing assert
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1793 1f5c12ca-751b-0410-a591-d2e778427230
|
2008-05-22 17:19:41 +00:00 |
|
nicolabertoldi
|
09b5a0a282
|
modified log outputs to e comliant with old mert
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1792 1f5c12ca-751b-0410-a591-d2e778427230
|
2008-05-21 16:52:00 +00:00 |
|
bhaddow
|
f1fa67d15e
|
Fixed small bug in initialisation of 1bests
removed failing assert
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1791 1f5c12ca-751b-0410-a591-d2e778427230
|
2008-05-21 16:22:43 +00:00 |
|
nicolabertoldi
|
f87b4261cf
|
I fixed a small bug when reading parameters
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1789 1f5c12ca-751b-0410-a591-d2e778427230
|
2008-05-21 10:03:48 +00:00 |
|
nicolabertoldi
|
b611428b2b
|
added missing file for managing gzipped files
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1788 1f5c12ca-751b-0410-a591-d2e778427230
|
2008-05-21 09:33:19 +00:00 |
|
nicolabertoldi
|
8cf59edcdc
|
remove loadnbest from FeatureData and Scoredata; change test_scorer accordingly;
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1787 1f5c12ca-751b-0410-a591-d2e778427230
|
2008-05-20 14:33:47 +00:00 |
|
nicolabertoldi
|
8a594fc254
|
reading from textual gzipped file is now possible
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1786 1f5c12ca-751b-0410-a591-d2e778427230
|
2008-05-20 14:15:30 +00:00 |
|
jfouet
|
f30000b875
|
add type threshold
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1785 1f5c12ca-751b-0410-a591-d2e778427230
|
2008-05-20 12:57:32 +00:00 |
|
jfouet
|
6ee4084797
|
simplification of LineOptimizer code (same behaviour)
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1784 1f5c12ca-751b-0410-a591-d2e778427230
|
2008-05-20 12:42:11 +00:00 |
|
bhaddow
|
39cf94984c
|
Fix bug in finding of leftmost, higher line in optimizer
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1783 1f5c12ca-751b-0410-a591-d2e778427230
|
2008-05-20 10:21:20 +00:00 |
|
bhaddow
|
86772c27e2
|
Fixed return of non-initialised variable
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1782 1f5c12ca-751b-0410-a591-d2e778427230
|
2008-05-17 13:24:09 +00:00 |
|
bhaddow
|
74fb532e00
|
Option to use random optimiser
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1781 1f5c12ca-751b-0410-a591-d2e778427230
|
2008-05-17 12:19:02 +00:00 |
|
bhaddow
|
83300050f7
|
Remove extraneous debug messages
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1766 1f5c12ca-751b-0410-a591-d2e778427230
|
2008-05-16 20:24:00 +00:00 |
|
bhaddow
|
4ec448b3f2
|
Command line arg tidy up in extractor
Remove buffers from feature
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1761 1f5c12ca-751b-0410-a591-d2e778427230
|
2008-05-16 19:57:01 +00:00 |
|
bhaddow
|
597c960933
|
Remove extra debug
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1752 1f5c12ca-751b-0410-a591-d2e778427230
|
2008-05-16 16:58:15 +00:00 |
|
bhaddow
|
1512abb7bd
|
Save correct weight set!
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1750 1f5c12ca-751b-0410-a591-d2e778427230
|
2008-05-16 16:51:39 +00:00 |
|
bhaddow
|
61e7ca1437
|
Fixes to optimiser: correct calculation of intersection, choice of bestx and prevent it
from deleting best point
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1749 1f5c12ca-751b-0410-a591-d2e778427230
|
2008-05-16 16:47:01 +00:00 |
|
jfouet
|
2b07443683
|
bugfixes
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1740 1f5c12ca-751b-0410-a591-d2e778427230
|
2008-05-16 14:21:24 +00:00 |
|
jfouet
|
044a43b512
|
add init.opt file
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1731 1f5c12ca-751b-0410-a591-d2e778427230
|
2008-05-16 12:35:17 +00:00 |
|
jfouet
|
e9df80a05c
|
bugfixes
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1726 1f5c12ca-751b-0410-a591-d2e778427230
|
2008-05-16 10:57:24 +00:00 |
|
jfouet
|
bb42cb0dd5
|
bugfixes+ introduction of verbose level
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1723 1f5c12ca-751b-0410-a591-d2e778427230
|
2008-05-16 07:09:15 +00:00 |
|
jfouet
|
ad79af41df
|
bug fixes
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1719 1f5c12ca-751b-0410-a591-d2e778427230
|
2008-05-15 20:49:49 +00:00 |
|
bhaddow
|
165f7d049f
|
pass by reference
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1717 1f5c12ca-751b-0410-a591-d2e778427230
|
2008-05-15 20:44:24 +00:00 |
|
bhaddow
|
1a6dcf5e36
|
remove buffers
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1715 1f5c12ca-751b-0410-a591-d2e778427230
|
2008-05-15 20:32:37 +00:00 |
|
bhaddow
|
6112215974
|
Fix crash in parameter reading
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1714 1f5c12ca-751b-0410-a591-d2e778427230
|
2008-05-15 19:26:35 +00:00 |
|
jfouet
|
b231ffc8b1
|
add Types.h to unify the typedefs
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1713 1f5c12ca-751b-0410-a591-d2e778427230
|
2008-05-15 19:09:01 +00:00 |
|