moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2024-08-16 06:50:32 +03:00
Code Issues Packages Projects Releases Wiki Activity
17,477 Commits 136 Branches 26 Tags 147 MiB
master
Commit Graph

52 Commits

Author SHA1 Message Date
Prashant Mathur
e315438bea Make CHRFscorer compile 2018-05-18 16:18:47 +02:00
Marcin Junczys-Dowmunt
73ffe51bd8 Added Grammatical Error Correction specific scorer (M^2) and features 2016-06-01 18:36:43 +02:00
Rico Sennrich
bec950cf72 support factors in InternalTree 2015-07-20 10:39:01 +01:00
Ulrich Germann
5807ca4714 Mert programs won't link with shared linking without -lboost_filesystem. 2015-05-21 02:56:03 +00:00
Barry Haddow
89b364bbf0 move hg decoding test to reg tests 2015-05-13 11:21:50 +01:00
Matthias Huck
34d1d3a904 sentence-bleu-nbest 2015-04-30 19:44:29 +01:00
Barry Haddow
217f389230 Attempt at fixing sparse features for hgmira 2015-04-03 15:46:59 +01:00
Christophe SERVAN
87a4f19546 Memory leak correction in TER algorithm 2015-02-16 19:02:46 +01:00
Rico Sennrich
f40bb2c53c HWCM for MERT 2014-09-22 10:49:20 +01:00
Barry Haddow
efee2695c3 Merge 08811deb17337356cd8dae9c59c0160590679a35 from joshua 2014-07-21 11:04:43 +01:00
Michael Denkowski
bb201bbde8 Meteor scoring for Moses MERT 2014-03-06 16:07:01 -05:00
Sara Stymne
b2eb42ed12 added document level Bleu scoring to mert 2013-07-03 14:03:58 +02:00
hieu
1dfbe1113c delete MergeScorer in mert/ 2013-01-03 15:01:30 +00:00
Ales Tamchyna
598d65bcfd adding a simple command-line utility for computing sentence-level BLEU (+1) 2012-12-10 13:12:34 +01:00
Hieu Hoang
f96b33de83 only include moses root when compiling 2012-11-14 13:43:04 +00:00
Hieu Hoang
5e3ef23cef move moses/src/* to moses/ 2012-11-12 19:56:18 +00:00
Kenneth Heafield
e9eb7dd021 More shared build fixes 2012-11-07 23:28:42 +01:00
Kenneth Heafield
d7ecd0be1a Remove bleu_lib target. 2012-11-07 23:21:59 +01:00
Barry Haddow
2b4e61d826 Merge branch 'trunk' into miramerge 
Compiles, not tested.

Conflicts:
	Jamroot
	OnDiskPt/PhraseNode.h
	OnDiskPt/TargetPhrase.cpp
	OnDiskPt/TargetPhrase.h
	OnDiskPt/TargetPhraseCollection.cpp
	mert/BleuScorer.cpp
	mert/Data.cpp
	mert/FeatureData.cpp
	moses-chart-cmd/src/Main.cpp
	moses/src/AlignmentInfo.h
	moses/src/ChartManager.cpp
	moses/src/LM/Ken.cpp
	moses/src/LM/Ken.h
	moses/src/LMList.h
	moses/src/LexicalReordering.h
	moses/src/PhraseDictionaryTree.h
	moses/src/ScoreIndexManager.h
	moses/src/StaticData.h
	moses/src/TargetPhrase.h
	moses/src/Word.cpp
	scripts/ems/experiment.meta
	scripts/ems/experiment.perl
	scripts/training/train-model.perl
 2012-07-17 13:36:50 +01:00
Eva Hasler
027a20730e merge Jamfiles 2012-07-04 11:49:07 +01:00
Hieu Hoang
0fd0adc1f6 merge Lexi Birch's LRScore from mert_mtm5 branch. Compiles and run. Hack, must double check with barry or lexi 2012-06-23 22:58:18 -04:00
Hieu Hoang
b1ca36387f mert now compiles with PermScorer. However, didn't implement score() - assert(false). Update Jamfile 2012-06-22 21:07:05 -04:00
Eva Hasler
e1c1a5343c merge 2012-06-07 11:16:52 +01:00
Eva Hasler
6a6a35c65e fix start weights in experiment.perl, add hypothesis queue for picking hope and fear translations, add variations to 1slack formulation 2012-06-01 01:49:42 +01:00
Hieu Hoang
465c5cbf97 move all executables into bin 2012-05-31 12:55:05 +01:00
Colin Cherry
fd577d7a65 Batch k-best MIRA is written and integrated into mert-moses.pl 
Regression tests all check out, and kbmira seems to work fine
on a Hansard French->English task.

HypPackEnumerator class may be of interest to pro.cpp and future
optimizers, as it abstracts a lot of the boilerplate involved in
enumerating multiple k-best lists.

MiraWeightVector is not really mira-specific - just a weight vector
that enables efficient averaging. Could be useful to a perceptron
as well. Same goes for MiraFeatureVector.

Interaction with sparse features is written, but untested.
 2012-05-29 13:38:57 -04:00
Eva Hasler
30deedde9f changed permission, everything changed.. 2012-05-10 18:54:24 +01:00
Matous Machacek
440650bd6e Added support for external unix filters to preprocess sentences in mert and evaluator 2012-05-09 19:21:41 +02:00
Eva
6f39ad0b3e test 2012-04-28 23:11:30 -07:00
Tetsuo Kiso
49a4be2899 Reduce the number of calling the numeric_limits constructor. 
- Create a test module for Point.
- Add test cases.
 2012-04-12 05:19:11 +09:00
Tetsuo Kiso
5e5f500254 Remove unnecessary headers; prefix private members with "m_". 
- Add const.
- Add a virtual destructor to abstract class.
- Add some TODOs to some constant values.

Create files for "overlapping" classes.
 2012-03-24 02:39:04 +09:00
Matous Machacek
3319805102 Added SemposScorer to mert and evaluator 2012-03-23 14:39:24 +01:00
Tetsuo Kiso
0a852b60d1 Move the GzFileBuf implementation into .cpp file. 
Add error check when gzopen() fails.
 2012-03-21 00:11:54 +09:00
Tetsuo Kiso
2fdb47fe67 Add an unit test for BleuScorer. 
The test contains only ngram counting. More tests for calculating
BLEU score are reuiqred.
 2012-03-20 05:52:35 +09:00
Tetsuo Kiso
8987fed667 Add thread unsafe Singleton class. 
- Add Vocabulary factory and the unit test.
- Remove Scorer::ClearVocabulary().
 2012-03-20 05:49:10 +09:00
Tetsuo Kiso
525f06452c Change the Encoder class to Vocabulary. 
- Introduce the namespace to avoid naming collisions. The class name
  is used in KenLM.
- Add the unit test.
 2012-03-20 03:43:04 +09:00
Tetsuo Kiso
54233d3270 Create files for OptimizerFactory class; add the unit test. 2012-03-18 13:08:38 +09:00
Tetsuo Kiso
6b95a19eda Create Reference class to clean up BleuScorer. 
- Add an unit test for Reference.
- Move functions to calculate the reference length from
  BleuScorer to Reference.
 2012-03-18 05:58:40 +09:00
Tetsuo Kiso
fba01c7cdf Create a header file for NgramCounts class. 
The reason is that we want to add the unit test.
 2012-03-14 22:14:11 +09:00
Tetsuo Kiso
d6c1abe6bb Rewrite FeatureData::setFeatureMap(); add the unit test. 2012-03-07 06:32:38 +09:00
Tetsuo Kiso
04a717be2b Merge branch 'master' of github.com:moses-smt/mosesdecoder 2012-02-27 08:35:09 +09:00
Tetsuo Kiso
7093d2e2cd Change mert/Timer. 
- Add a high resolution timing function to measure the
  wall-clock time by gettimeofday().

- Now the Timer class use getrusage() to measure the elapsed
  CPU time as KenLM does.

- Revive Timer::restart().

- Add Timer::ToString() for reporting the detail statistics
  as well as for debugging.

- Add a simple unit test for Timer.
 2012-02-27 08:34:51 +09:00
Matous Machacek
e8a94a7bd2 Added interpolated scorer 
example: to interpolate BLEU and CDER use --sctype=BLEU,CDER
to specify weights use --scconfig=weights:0.3+0.7

This scorer should replace MergeScorer (which requires mert-moses-multi.pl) soon.
Interpolated scorer is more universal and is used in the same way as other scorers.
 2012-02-26 18:53:08 +01:00
Tetsuo Kiso
47ac8a474d Change the naming conventions for the guard macros; Rename TER directory. 
This change might be useful to avoid duplicating the names.
The reason is that although MERT programs are standalone
applications, some header files such as data.h and
point.h have common guard macro names like "DATA_H" and
"POINT_H", and this is not good naming conventions
when you want to include external headers.
Some files actually include headers in Moses and KenLM's util.
 2012-02-20 09:46:08 +09:00
Tetsuo Kiso
5d1cfa0ebb Bug fix: tokenizer used in mert; add unit tests for that. 
When tokenizing a string delimited by spaces (say, "9 9 8 7 ")
with Tokenize(), resulting a sequence of strings are
{"9", "9", "8", "7", "" }, which is different
from we have expected. We are not interested in empty strings.

This commit fix this issue, and add unit tests for
the tokenize functions.
 2012-02-20 07:39:24 +09:00
Barry Haddow
fa6753b0f3 Really simple sharding test 2012-02-08 17:47:54 +00:00
Kenneth Heafield
f564876e5e Avoid naming conflict with glibc's libutil 2011-11-28 09:19:45 +00:00
Kenneth Heafield
58054214b7 Avoid naming conflict with glibc's libutil 2011-11-28 09:19:45 +00:00
Kenneth Heafield
a7222a322d Regression tests expect mert in mert/ 2011-11-25 22:14:04 +00:00
Kenneth Heafield
6edf5fe9cc Regression tests expect mert in mert/ 2011-11-25 22:14:04 +00:00