- Add a high resolution timing function to measure the
wall-clock time by gettimeofday().
- Now the Timer class use getrusage() to measure the elapsed
CPU time as KenLM does.
- Revive Timer::restart().
- Add Timer::ToString() for reporting the detail statistics
as well as for debugging.
- Add a simple unit test for Timer.
example: to interpolate BLEU and CDER use --sctype=BLEU,CDER
to specify weights use --scconfig=weights:0.3+0.7
This scorer should replace MergeScorer (which requires mert-moses-multi.pl) soon.
Interpolated scorer is more universal and is used in the same way as other scorers.
This change might be useful to avoid duplicating the names.
The reason is that although MERT programs are standalone
applications, some header files such as data.h and
point.h have common guard macro names like "DATA_H" and
"POINT_H", and this is not good naming conventions
when you want to include external headers.
Some files actually include headers in Moses and KenLM's util.
When tokenizing a string delimited by spaces (say, "9 9 8 7 ")
with Tokenize(), resulting a sequence of strings are
{"9", "9", "8", "7", "" }, which is different
from we have expected. We are not interested in empty strings.
This commit fix this issue, and add unit tests for
the tokenize functions.
