moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2025-01-01 16:33:16 +03:00
Code Issues Packages Projects Releases Wiki Activity
3,102 Commits 136 Branches 26 Tags 147 MiB
6c356958b0
Commit Graph

292 Commits

Author SHA1 Message Date
Tetsuo Kiso
8f0ba037f3 Add comments. 2011-11-13 21:13:44 +09:00
Tetsuo Kiso
3d70b2e1a5 Small change: modify initialization of the Data class. 2011-11-12 22:04:22 +09:00
Tetsuo Kiso
eecfb171da Fix usage of whitespaces for Optimizers. 2011-11-12 18:47:31 +09:00
Tetsuo Kiso
6c845323ac Fix the order of initialization when the constructor will be called. 2011-11-12 18:17:31 +09:00
Tetsuo Kiso
20e27aab92 Refactoring: constructors and copying objects. 2011-11-12 18:12:07 +09:00
Tetsuo Kiso
87a5c16b55 Replace macros with const variables and inline functions. 2011-11-12 17:30:33 +09:00
Tetsuo Kiso
ae9701ff77 Fix initialization of score_, and add destructor. 2011-11-12 16:50:18 +09:00
Tetsuo Kiso
df0874df89 Make Point::score a private by defining accessor/mutator. 2011-11-12 16:39:57 +09:00
Tetsuo Kiso
fdcd148cac Replace macros with const char arrays used in XXXArray classes. 2011-11-12 12:13:35 +09:00
Tetsuo Kiso
03abf54d21 Add a utility function to make sure the filename with ".gz" extension. 2011-11-12 11:50:39 +09:00
Tetsuo Kiso
ee0345b9b6 Move filestream wrapper classes to FileStream.{h,cpp} 2011-11-12 11:44:39 +09:00
Tetsuo Kiso
8e7693076c Add const to stream wrapper classes. 2011-11-12 11:35:20 +09:00
Tetsuo Kiso
fb3b0f9f62 Add const to Data and Feature class members. 2011-11-12 11:26:13 +09:00
Tetsuo Kiso
54b3b846c7 Add const member functions in Scorer classes. 2011-11-12 10:58:14 +09:00
Tetsuo Kiso
00b8c6d768 Use const Scorer::calculateScore(). 2011-11-12 10:40:54 +09:00
Tetsuo Kiso
1ec58e7edf Fix constructors and initializing const members. 2011-11-12 10:28:08 +09:00
Tetsuo Kiso
d776281b8b Simple refactoring of BLEU scorer. 2011-11-12 10:21:08 +09:00
Tetsuo Kiso
43beb88df5 Fix constructors of scorer classes and optimizer classes. 
Using public const members is not good idea.
It should be initialized in private by constructors.
 2011-11-12 10:16:31 +09:00
Tetsuo Kiso
fdb83b0f6d Put constructor and destructor into private for Factory class 2011-11-12 09:56:21 +09:00
Tetsuo Kiso
dfb714296f Add 'explicit' for constructors with one argument. 2011-11-12 09:51:27 +09:00
Tetsuo Kiso
ce9a628ed0 Remove unnecessary semicolons used in end of member functions. 2011-11-12 09:40:01 +09:00
Tetsuo Kiso
664ffe0130 Fix indentation. 2011-11-12 09:24:19 +09:00
Tetsuo Kiso
0fc56ae869 Add a comment to mark the end of #define guard. 2011-11-12 09:16:05 +09:00
Tetsuo Kiso
68315d6407 Fix class, function, and implementation comments format. 
Functions comments should be placed in their declarations.
 2011-11-12 08:58:23 +09:00
Tetsuo Kiso
4f6d022fe7 Add comments to mark the end of #define guards. 2011-11-12 07:59:50 +09:00
Tetsuo Kiso
888c47d921 Fix splitting strings from a string. 2011-11-11 22:00:30 +09:00
Tetsuo Kiso
c2121695c2 Fix memory leaks in mert. 2011-11-11 20:40:59 +09:00
Tetsuo Kiso
087756b8c3 Fix memory leaks in extractor. 2011-11-11 20:02:26 +09:00
Kenneth Heafield
1c4be5675c Lines are better when they end 2011-11-07 17:08:19 +00:00
bhaddow
7c0d9c34da Fix formatting of pro input file 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4360 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-14 14:01:15 +00:00
bhaddow
718c55d3aa Reducing memory usage for sparse features 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4355 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-14 07:40:53 +00:00
servan
eef0f213e9 A mert/MergeScorer.h 
A    mert/MergeScorer.cpp


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4303 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-06 11:00:48 +00:00
servan
f223f5a276 M mert/TerScorer.cpp 
M    mert/BleuScorer.h
M    mert/ScorerFactory.h
M    mert/Scorer.h
M    mert/PerScorer.h
M    mert/TerScorer.h
M    mert/Makefile.am
AM   scripts/training/mert-moses-multi.pl


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4299 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-05 13:36:17 +00:00
bhaddow
fc695c38a7 Implementation of sharding and resampling in mert. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4226 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-15 17:45:35 +00:00
servan
c55b03dd66 M TerScorer.cpp 
M    TerScorer.h


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4206 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-09 13:26:45 +00:00
bhaddow
0955474dea remove old tests 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4195 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-08 18:04:13 +00:00
bhaddow
ed8d85acc1 moving mert tests 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4193 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-08 17:51:09 +00:00
phkoehn
41a1849437 support for sparse feature functions (mert support only when using PRO) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4184 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-07 16:37:33 +00:00
bhaddow
9fee4a97f2 clean-up 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4183 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-07 08:36:08 +00:00
bhaddow
ca5c0f19b7 Multi-threading of mert, for random restarts. 
Fix mert tests.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4182 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-07 08:08:35 +00:00
bhaddow
37ebe7a62e Build common files into archive so we don't have to compile them 3 times 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4179 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-06 12:29:18 +00:00
machacekmatous
1daacfbb72 TER Scorer sources cleaned, compilable without boost now. TER Scorer added back to the default built. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4173 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-05 22:26:18 +00:00
bhaddow
1786d33e0f Make ter scoring optional 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4172 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-05 16:23:46 +00:00
machacekmatous
642e8dce95 Added evaluator to MERT directory. This tool computes a metric score for given candidate and reference files: 
evaluator --sctype PER --reference ref.file --candidate cand.file

usage: evaluator [options] --reference ref1[,ref2[,ref3...]] --candidate cand1[,cand2[,cand3...]]
[--sctype|-s] the scorer type (default BLEU)
[--scconfig|-c] configuration string passed to scorer
        This is of the form NAME1:VAL1,NAME2:VAL2 etc
[--reference|-R] comma separated list of reference files
[--candidate|-C] comma separated list of candidate files
[--bootstrap|-b] number of booststraped samples (default 0 - no bootstraping)
[--rseed|-r] the random seed for bootstraping (defaults to system clock)
[--help|-h] print this message and exit


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4153 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-20 15:25:19 +00:00
machacekmatous
63fd490a51 Added CDER metric to use in MERT. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4152 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-18 21:35:16 +00:00
bojar
998b86f639 addind a TODO list for anyone, esp. Matous Machacek 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4150 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-17 10:01:14 +00:00
machacekmatous
3ef02eb7e6 merged in TER Scorer from mert-other_metrics (at r4140) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4146 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-16 16:21:31 +00:00
phkoehn
36db0ffe48 added pairwise ranked optimization (PRO) as proposd by [Hopkins&May,2011], just use switch --pairwise-ranked 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4106 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-03 17:00:17 +00:00
phkoehn
1bd74fc87f added random directions [Cer&al.,2008] and historic best as starting points [Foster&Kuhn,2009] to MERT 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4086 1f5c12ca-751b-0410-a591-d2e778427230
 2011-07-23 00:24:45 +00:00
bojar
1ba2de3c02 - cmert: added support for passing min and max values for weights 
(used to be in old cmert but not in new cmert, i.e. moses/mert/)
- modified mert-moses.pl accordingly, esp. set min&max to 0&1 as it used to be
  hardwired in the new cmert
- adding mert-moses-ondrej.pl, a simplification of mert-moses.pl, please test it


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4066 1f5c12ca-751b-0410-a591-d2e778427230
 2011-07-03 21:01:16 +00:00
hieuhoang1972
148c1e8305 run beautify.perl. Consistent formatting for .h & .cpp files 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3899 1f5c12ca-751b-0410-a591-d2e778427230
 2011-02-24 12:42:19 +00:00
rsennrich
ec00f9a916 fix to MERT: disable normaliziation when optimizing subset of features. 
before, active features were normalized to 1; optimizing one feature would always set it to 1, preventing any real optimization.

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3845 1f5c12ca-751b-0410-a591-d2e778427230
 2011-01-25 16:10:47 +00:00
bhaddow
2e77dce57e improvement to error message 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3763 1f5c12ca-751b-0410-a591-d2e778427230
 2010-12-08 10:13:19 +00:00
bhaddow
6e8856edc2 Prevent intersections going to infinity 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3125 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-12 14:31:10 +00:00
jfouet
79a8f0f133 restore trace by default 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2950 1f5c12ca-751b-0410-a591-d2e778427230
 2010-03-01 11:32:15 +00:00
jfouet
e14dc33dc8 enable trace with a configure options and fix compilation error when trace are disabled 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2938 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-24 10:37:49 +00:00
nicolabertoldi
e0ae1ad605 now it is possible to perform mert on a subset of features 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2645 1f5c12ca-751b-0410-a591-d2e778427230
 2010-01-08 15:12:28 +00:00
nicolabertoldi
53ab10d620 minor changes to extractor-bin regr-test to make it compliant with both mac and linux 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2556 1f5c12ca-751b-0410-a591-d2e778427230
 2009-10-01 14:33:08 +00:00
nicolabertoldi
1a90b6a915 changing the randomize function -from rand() to random()- to have the same behavior of linux machine also on mac; changing the regression test accordingly 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2554 1f5c12ca-751b-0410-a591-d2e778427230
 2009-10-01 14:08:22 +00:00
nicolabertoldi
fa59f147b9 minor changes to extractor-bin regr-test to be compliant with both mac and linux 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2553 1f5c12ca-751b-0410-a591-d2e778427230
 2009-10-01 14:05:01 +00:00
nicolabertoldi
820e3dfe9c adding regression tests for mert 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2551 1f5c12ca-751b-0410-a591-d2e778427230
 2009-10-01 12:53:44 +00:00
nicolabertoldi
0393183eb4 mert software now works with different reference length policies: shortest, average, closest (default) and with case information (default is preserving case). Pay attention that both defaults are different from the previous version (which were shortest reflen and case-insensitive). 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2459 1f5c12ca-751b-0410-a591-d2e778427230
 2009-08-05 15:38:35 +00:00
bhaddow
1ef0f3844d Connect new mert to the moses build system 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2221 1f5c12ca-751b-0410-a591-d2e778427230
 2009-02-26 11:17:12 +00:00
bhaddow
bb9e2f427d Fix random restarts so that they are actually random, using the system clock 
for seeding.

Previously, new mert followed old mert in using the same set of "random" points 
for every iteration of the outer loop.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2126 1f5c12ca-751b-0410-a591-d2e778427230
 2009-02-12 12:58:27 +00:00
bhaddow
6c8c8e9dc4 initial weights for toy example 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1989 1f5c12ca-751b-0410-a591-d2e778427230
 2009-01-22 10:25:14 +00:00
nicolabertoldi
830d9f3404 small change to reduce few useless computations 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1970 1f5c12ca-751b-0410-a591-d2e778427230
 2009-01-07 13:47:38 +00:00
nicolabertoldi
4b4c1b3973 imported utilities for timing from Moses 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1969 1f5c12ca-751b-0410-a591-d2e778427230
 2009-01-07 13:30:06 +00:00
nicolabertoldi
32029561da mert can now load more data files 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1942 1f5c12ca-751b-0410-a591-d2e778427230
 2008-11-18 18:51:02 +00:00
phkoehn
1b5d99ad26 added headers for standard compliance (gcc 4.3 on 64 bit linux) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1905 1f5c12ca-751b-0410-a591-d2e778427230
 2008-10-16 21:14:38 +00:00
bhaddow
cd28f119c6 mert tests 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1881 1f5c12ca-751b-0410-a591-d2e778427230
 2008-09-01 17:18:51 +00:00
bhaddow
83f234cf17 Implementation of Cer et al mert regularisation. Use with argument such 
as --scconfig regtype:min,regwin:3 in extractor and mert. Only tested
on toy example so far.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1860 1f5c12ca-751b-0410-a591-d2e778427230
 2008-06-24 19:27:18 +00:00
nicolabertoldi
e94834012d added facilities to read and write score statistics in binary format 
moved facilities for feature names in FeatureData object


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1824 1f5c12ca-751b-0410-a591-d2e778427230
 2008-06-05 17:03:54 +00:00
nicolabertoldi
8e96e68476 overall change of a variable name: array_ instead of array2_ 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1823 1f5c12ca-751b-0410-a591-d2e778427230
 2008-06-05 11:42:00 +00:00
nicolabertoldi
930e67c5e3 fixed another bug related to the handling of feature names 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1822 1f5c12ca-751b-0410-a591-d2e778427230
 2008-06-05 11:30:37 +00:00
nicolabertoldi
44d7e0e0f7 fixed a bug related to the handling of feature names 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1821 1f5c12ca-751b-0410-a591-d2e778427230
 2008-06-05 11:26:54 +00:00
bhaddow
37cf805139 Fix bug in output of scorestats to text file 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1820 1f5c12ca-751b-0410-a591-d2e778427230
 2008-06-05 11:08:29 +00:00
nicolabertoldi
281bf610b8 added binary read/load facility for feature data 
added names of features in the header
added methods to access the features by name


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1819 1f5c12ca-751b-0410-a591-d2e778427230
 2008-06-05 07:23:34 +00:00
jfouet
f849154a73 fix FeatureStats::savetxt 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1815 1f5c12ca-751b-0410-a591-d2e778427230
 2008-06-04 08:33:37 +00:00
jfouet
8e54557e9f fix crash in getNextPound 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1814 1f5c12ca-751b-0410-a591-d2e778427230
 2008-06-04 08:32:19 +00:00
nicolabertoldi
1402f3036b added ScorerFactory.h 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1811 1f5c12ca-751b-0410-a591-d2e778427230
 2008-06-03 08:56:37 +00:00
nicolabertoldi
af585bc492 nbest can be read from stdin, too 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1797 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-27 17:20:01 +00:00
nicolabertoldi
291260abf7 - made output more compliant with old version 
- added PerSCorer.h and BleuScorer.h
- stored feature names
- fixed bug about output of best Point


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1796 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-27 16:50:52 +00:00
jfouet
89194be5eb safer(hopefully) insertion of threshold in LineOptimize 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1795 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-23 12:02:14 +00:00
nicolabertoldi
c9593648bb change from int to unsigned where needed 
add some debugging output (to remove later)

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1794 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-23 11:48:16 +00:00
bhaddow
e8d26f4f5c Fixes to optimiser to stop it crashing and failing assert 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1793 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-22 17:19:41 +00:00
nicolabertoldi
09b5a0a282 modified log outputs to e comliant with old mert 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1792 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-21 16:52:00 +00:00
bhaddow
f1fa67d15e Fixed small bug in initialisation of 1bests 
removed failing assert


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1791 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-21 16:22:43 +00:00
nicolabertoldi
f87b4261cf I fixed a small bug when reading parameters 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1789 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-21 10:03:48 +00:00
nicolabertoldi
b611428b2b added missing file for managing gzipped files 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1788 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-21 09:33:19 +00:00
nicolabertoldi
8cf59edcdc remove loadnbest from FeatureData and Scoredata; change test_scorer accordingly; 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1787 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-20 14:33:47 +00:00
nicolabertoldi
8a594fc254 reading from textual gzipped file is now possible 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1786 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-20 14:15:30 +00:00
jfouet
f30000b875 add type threshold 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1785 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-20 12:57:32 +00:00
jfouet
6ee4084797 simplification of LineOptimizer code (same behaviour) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1784 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-20 12:42:11 +00:00
bhaddow
39cf94984c Fix bug in finding of leftmost, higher line in optimizer 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1783 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-20 10:21:20 +00:00
bhaddow
86772c27e2 Fixed return of non-initialised variable 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1782 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-17 13:24:09 +00:00
bhaddow
74fb532e00 Option to use random optimiser 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1781 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-17 12:19:02 +00:00
bhaddow
83300050f7 Remove extraneous debug messages 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1766 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-16 20:24:00 +00:00
bhaddow
4ec448b3f2 Command line arg tidy up in extractor 
Remove buffers from feature


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1761 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-16 19:57:01 +00:00
bhaddow
597c960933 Remove extra debug 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1752 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-16 16:58:15 +00:00
bhaddow
1512abb7bd Save correct weight set! 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1750 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-16 16:51:39 +00:00
bhaddow
61e7ca1437 Fixes to optimiser: correct calculation of intersection, choice of bestx and prevent it 
from deleting best point


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1749 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-16 16:47:01 +00:00
jfouet
2b07443683 bugfixes 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1740 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-16 14:21:24 +00:00
jfouet
044a43b512 add init.opt file 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1731 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-16 12:35:17 +00:00
jfouet
e9df80a05c bugfixes 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1726 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-16 10:57:24 +00:00
jfouet
bb42cb0dd5 bugfixes+ introduction of verbose level 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1723 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-16 07:09:15 +00:00
jfouet
ad79af41df bug fixes 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1719 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-15 20:49:49 +00:00
bhaddow
165f7d049f pass by reference 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1717 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-15 20:44:24 +00:00
bhaddow
1a6dcf5e36 remove buffers 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1715 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-15 20:32:37 +00:00
bhaddow
6112215974 Fix crash in parameter reading 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1714 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-15 19:26:35 +00:00
jfouet
b231ffc8b1 add Types.h to unify the typedefs 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1713 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-15 19:09:01 +00:00
bhaddow
ad2ee87603 Fix makefile 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1711 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-15 17:16:46 +00:00
bhaddow
933400a503 extractor uses scorer factory 
remove feature_extractor


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1710 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-15 17:14:11 +00:00
nicolabertoldi
cc1d95272a add READMEs 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1707 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-15 16:20:23 +00:00
bhaddow
c0643d47f2 Add scorer factory. Fix compile error in Optimizer 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1706 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-15 16:03:49 +00:00
bhaddow
f320cf5174 Refactor PerScorer and BleuScorer to remove common code 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1704 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-15 14:48:11 +00:00
jfouet
b0ee845d7e various small fixes 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1702 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-15 14:17:34 +00:00
bhaddow
777ea5f754 Finish per scorer 
Command line args for extractor


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1700 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-15 14:13:32 +00:00
jfouet
c66aec9e83 implementation of the main 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1698 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-15 14:04:42 +00:00
jfouet
94d47101ca add (almost empty) main 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1696 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-15 11:06:32 +00:00
jfouet
bfe3661110 implementation of optimization on a subset of the parameters + debug mode in the makefile 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1695 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-15 10:57:20 +00:00
nicolabertoldi
2ef6f3e2d8 main command for managing feature and error statistics is ready; small example fortesting is available in directory example 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1692 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-15 08:35:56 +00:00
bhaddow
1f84c9eab7 Get rid of warnings about lack of newlines 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1691 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-15 06:44:36 +00:00
bhaddow
f98de25e70 Stub out per scorer. some refactoring to make this possible 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1681 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-14 20:36:11 +00:00
bhaddow
e2921af063 Incremental interface for scorer 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1678 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-14 19:47:34 +00:00
bhaddow
69c6164b82 normalisation script 
normalisation of test data


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1673 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-14 17:17:44 +00:00
bhaddow
48533d1021 Fixes to bleu scorer 
add method get ref length


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1669 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-14 16:31:22 +00:00
jfouet
cff8d4702b add some sanity checks 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1668 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-14 16:19:00 +00:00
jfouet
13f6a427c5 buxfixes on lineOptimize 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1665 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-14 15:44:59 +00:00
jfouet
cb5305ab46 implementation of LineOptimizer 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1659 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-14 14:25:07 +00:00
nicolabertoldi
9202738527 support for mergin; other small changes 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1657 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-14 14:16:05 +00:00
bhaddow
87efff7d66 bleu scorer 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1653 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-14 13:36:55 +00:00
jfouet
558c8bd9b7 fix compilation of Optimizer 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1651 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-14 12:49:45 +00:00
bhaddow
b672d9f6df Stubbing out of Scorer. 
Scorer test data.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1646 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-14 12:23:58 +00:00
jfouet
43ba7835bb class to inplement a set of lambda parameter and their stat score 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1642 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-14 11:32:17 +00:00
nicolabertoldi
36b25323c9 small changes 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1641 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-14 11:28:54 +00:00
jfouet
40c93618fc add const get accesor to FeatureDataand Array 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1639 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-14 11:11:55 +00:00
nicolabertoldi
b68b41f7e2 add Parameter 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1638 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-14 11:05:32 +00:00
jfouet
f9448b76a0 objet code for the Optimzer algorithm 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1637 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-14 11:00:59 +00:00
nicolabertoldi
eb2d6e971d changes 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1636 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-14 10:51:42 +00:00
bhaddow
d794418121 First version of scorer 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1634 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-14 08:46:15 +00:00
nicolabertoldi
9c12330785 I am adding new object for handling statistics forerror measures 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1633 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-14 08:14:13 +00:00
nicolabertoldi
49891adeed I am starting the improvement of Minimum Error Training scritps 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1632 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-14 07:57:45 +00:00