moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2024-12-27 05:55:02 +03:00
Code Issues Packages Projects Releases Wiki Activity
3,005 Commits 136 Branches 26 Tags 147 MiB
e8a94a7bd2
Commit Graph

231 Commits

Author SHA1 Message Date
Matous Machacek
e8a94a7bd2 Added interpolated scorer 
example: to interpolate BLEU and CDER use --sctype=BLEU,CDER
to specify weights use --scconfig=weights:0.3+0.7

This scorer should replace MergeScorer (which requires mert-moses-multi.pl) soon.
Interpolated scorer is more universal and is used in the same way as other scorers.
 2012-02-26 18:53:08 +01:00
Tetsuo Kiso
c26e83fd09 Remove obsolete and unused logging statements. 2012-02-26 02:19:40 +09:00
Tetsuo Kiso
224c654fa5 Don't repeat calling functions many times. 
Consider using constants the result if it is possible.
 2012-02-26 02:12:59 +09:00
Tetsuo Kiso
669b9d9c7a Minor change the logging utility for n-gram counts. 
Use std::ostream instead of directly using std::cerr.
 2012-02-26 02:01:03 +09:00
Tetsuo Kiso
8e0a61d0d7 Clean up calculation effective reference length. 2012-02-26 01:54:51 +09:00
Tetsuo Kiso
c4fa8a3865 Add a more efficient member to set up ScoreStats. 
- Remove unnecessary conversions.

- Add 'const' to local variables.
 2012-02-26 01:41:17 +09:00
Tetsuo Kiso
2c2bd63bbd Replace string objects with const char[]. 2012-02-26 01:18:08 +09:00
Tetsuo Kiso
17f06a3250 Hide the implementation details of Ngram counts from the header. 2012-02-26 01:11:56 +09:00
Tetsuo Kiso
0c9023abc6 Clean up commented out code snippets for debugging purposes. 2012-02-25 18:14:00 +09:00
Matous Machacek
16376eabcc Fixed quadratic time when adding ScoreStats to ScoreData 2012-02-21 10:39:04 +01:00
Tetsuo Kiso
aefa6e1000 Fix a memory leak. 2012-02-20 11:04:21 +09:00
Tetsuo Kiso
c2ef7093ed Add 'virtual' to destructors. 2012-02-20 10:23:59 +09:00
Tetsuo Kiso
47ac8a474d Change the naming conventions for the guard macros; Rename TER directory. 
This change might be useful to avoid duplicating the names.
The reason is that although MERT programs are standalone
applications, some header files such as data.h and
point.h have common guard macro names like "DATA_H" and
"POINT_H", and this is not good naming conventions
when you want to include external headers.
Some files actually include headers in Moses and KenLM's util.
 2012-02-20 09:46:08 +09:00
Tetsuo Kiso
82da44b030 Fix typo. 2012-02-20 08:29:53 +09:00
Tetsuo Kiso
ce7b136994 Add comments; remove unused macros. 2012-02-20 08:20:44 +09:00
Tetsuo Kiso
a70925317e Put global variables in mert/util.cpp in anonymous space. 
We do not allow clients to access the following variables.
Instead, use the APIs which we provide for that.

Also, remove the unused function, and fix smoke tests.
 2012-02-20 08:02:23 +09:00
Tetsuo Kiso
5d1cfa0ebb Bug fix: tokenizer used in mert; add unit tests for that. 
When tokenizing a string delimited by spaces (say, "9 9 8 7 ")
with Tokenize(), resulting a sequence of strings are
{"9", "9", "8", "7", "" }, which is different
from we have expected. We are not interested in empty strings.

This commit fix this issue, and add unit tests for
the tokenize functions.
 2012-02-20 07:39:24 +09:00
Tetsuo Kiso
a7666735b5 Add error checking to setup 'to_optimize'. 
mert will check whether the dimension and the number of
fetures are equal.
 2012-02-17 09:16:10 +09:00
Tetsuo Kiso
6c003e544a Bug fix mert: when you want to optimize fewer features. 
This commit is a temporary bug fix.
 2012-02-17 08:25:18 +09:00
Tetsuo Kiso
819dc9e0f9 Add a utility function to FeatureData for debugging. 2012-02-17 07:27:07 +09:00
Tetsuo Kiso
c1b85b480c Delete mert/sample/README; Add smoke tests. 
Replace README with a bunch of shell script
for smoke testing of MERT.

The README file was not a typical README file.
It was like a sample script to run mert and
extractor, so I renamed it as smoke tests stuff.
 2012-02-17 03:53:52 +09:00
Barry Haddow
7091555cd6 Merge branch 'master' of github.com:moses-smt/mosesdecoder 2012-02-08 17:48:19 +00:00
Barry Haddow
fa6753b0f3 Really simple sharding test 2012-02-08 17:47:54 +00:00
Barry Haddow
62d7d034bb Fix sharding bug 2012-02-08 17:11:56 +00:00
Tetsuo Kiso
905f959d83 Move funcions defined in a header into .cpp file. 2012-02-01 21:44:37 +09:00
Tetsuo Kiso
b2987337d8 Remove virtual keyword from whoami() function. 
The funtion is neither inherited from Scorer nor
StatisticsBasedScorer.
 2012-02-01 21:36:25 +09:00
Tetsuo Kiso
923efa0a4c Move funcionts used in StatisticsBasedScorer into anonymous namespace. 
And add 'inline' keyword to the functions.
 2012-02-01 21:30:50 +09:00
Tetsuo Kiso
5cd5b90d0d Create a initialize function. 2012-02-01 21:26:47 +09:00
Tetsuo Kiso
17e864e446 Create private class to encapssulate encoding process. 
Instead of using typedefs inside a class only,
it might be better to create a private class to do same things.
 2012-02-01 21:19:25 +09:00
Tetsuo Kiso
a351a74c18 Move regularizaion type into StatisticsBasedScorer. 
The type is used as internal purpose.
 2012-02-01 20:58:49 +09:00
Tetsuo Kiso
b19e7777ce Add prefix 'm_' to private and protected members in Scorer classes. 2012-02-01 20:54:20 +09:00
Tetsuo Kiso
33cea0f1dc Delete unused enums. 2012-02-01 20:26:25 +09:00
Tetsuo Kiso
30fa97e404 Move reference length type into a private member of BleuScorer. 
The reason is that the type is used as internal purpose.
 2012-02-01 20:24:48 +09:00
Tetsuo Kiso
d742341176 Bug fix: memory leak issues when using MergeScorer. 2012-02-01 18:13:32 +09:00
Tetsuo Kiso
3ef03a77c4 Change casts to C++ style casts. 2012-02-01 18:13:00 +09:00
Tetsuo Kiso
142342f8be Change casts to C++ style casts, and delete unnecessary casts. 2012-02-01 17:17:58 +09:00
Tetsuo Kiso
3c04b7e826 Create a function to save final weights to a file. 2012-02-01 16:59:48 +09:00
Tetsuo Kiso
1bafc99ea2 Use a constant instead of hard-coding. 2012-02-01 16:51:25 +09:00
Tetsuo Kiso
8e1bf6d7d1 Pass std::string objects by const reference instead of by value. 2012-02-01 16:46:36 +09:00
Tetsuo Kiso
b399dfc906 Rename too short variables. 
'D' and 'O' is hard to find when you might want to
search by command line tools such as grep.
 2012-02-01 16:39:38 +09:00
Tetsuo Kiso
8007150db4 Add constans to prevent avoid hard coding many times. 
Default filenames in usage() were differen from the actual ones.
 2012-02-01 16:33:23 +09:00
Tetsuo Kiso
c3fbe5081e Add whitespaces. 2012-02-01 16:15:22 +09:00
Tetsuo Kiso
e8823d5015 Add 'virtual' keyword for functions in OptimizationTask. 
Adding 'virtual' keyword for an inherited virtual function is good
practice because it will let the users know where the function
comes from.
 2012-02-01 15:49:33 +09:00
Tetsuo Kiso
c1c9c1c8bb Create a struct for command line options in mert. 2012-02-01 15:21:49 +09:00
Tetsuo Kiso
616b014554 Delete unused variables. 2012-02-01 12:37:41 +09:00
Tetsuo Kiso
bb9a6b9184 Introduce anonymous namespace. 2012-02-01 12:35:50 +09:00
Tetsuo Kiso
c94b1f8ff0 Add whitespaces. 2012-02-01 12:29:45 +09:00
Tetsuo Kiso
cad03f7a03 Create a struct for command line options in extractor. 2012-02-01 12:23:15 +09:00
Tetsuo Kiso
4d3fd9fd4b Create a wrapper function to init seed. 
Move g_bootstrap from a global variable to
a member of struct ProgramOption.
 2012-02-01 11:49:26 +09:00
Tetsuo Kiso
1452f88ed5 Create a struct for command line options. 
Add a wrapper function to parse the options, too.
 2012-02-01 11:27:17 +09:00