2011-08-17 14:01:14 +04:00
|
|
|
- check that mert-moses.pl emits devset score after every iteration
|
|
|
|
|
- correctly for whichever metric we are optimizing
|
|
|
|
|
- even when using --pairwise-ranked (PRO)
|
|
|
|
|
- this may make use of 'evaluator', soon to be added by Matous Machacek
|
|
|
|
|
|
|
|
|
|
- check that --pairwise-ranked is compatible with all optimization metrics
|
2012-04-08 17:45:04 +04:00
|
|
|
|
|
|
|
|
- Replace the standard rand() currently used in MERT and PRO with better
|
|
|
|
|
random generators such as Boost's random generators (e.g., boost::mt19937).
|
|
|
|
|
- create a Random class to hide the details, i.e., how to generate
|
|
|
|
|
random numbers, which allows us to use custom random generators more
|
|
|
|
|
easily.
|
|
|
|
|
|
|
|
|
|
Pros:
|
|
|
|
|
- In MERT, you might want to use the random restarting technique to avoid
|
|
|
|
|
local optima.
|
|
|
|
|
- PRO uses a sampling technique to choose candidate translation pairs
|
2012-04-08 17:47:20 +04:00
|
|
|
from N-best lists, which means the choice of random generators seems to
|
2012-04-08 17:45:04 +04:00
|
|
|
be important.
|
|
|
|
|
|
|
|
|
|
Cons:
|
|
|
|
|
- This change will require us to re-create the truth results for regression
|
|
|
|
|
testing related to MERT and PRO because the new random generator will
|
|
|
|
|
generate different numbers from the current generator does.
|
