moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2024-08-17 07:20:48 +03:00
Code Issues Packages Projects Releases Wiki Activity
17,477 Commits 136 Branches 26 Tags 147 MiB
master
Commit Graph

51 Commits

Author SHA1 Message Date
Rico Sennrich
d39cbca0b9 (optionally) use n-best file for evaluator/return-best-dev 
this adds support for metrics that rely on alignment / trees
 2014-09-22 10:49:20 +01:00
Hieu Hoang
6249432407 beautify 2013-05-29 18:16:15 +01:00
Tetsuo Kiso
38e145e556 Use util::TokenIter to tokenize n-best lists. 
Reduce creating std::string objects, too. In both ScoreArray
and FeatureArray classes, the private members to track sentence
indices (namely, "m_index") were unnecessarily declared as
std::string, but it's better to directly declare them as 'int'.
 2012-12-07 01:39:22 +09:00
Barry Haddow
2b4e61d826 Merge branch 'trunk' into miramerge 
Compiles, not tested.

Conflicts:
	Jamroot
	OnDiskPt/PhraseNode.h
	OnDiskPt/TargetPhrase.cpp
	OnDiskPt/TargetPhrase.h
	OnDiskPt/TargetPhraseCollection.cpp
	mert/BleuScorer.cpp
	mert/Data.cpp
	mert/FeatureData.cpp
	moses-chart-cmd/src/Main.cpp
	moses/src/AlignmentInfo.h
	moses/src/ChartManager.cpp
	moses/src/LM/Ken.cpp
	moses/src/LM/Ken.h
	moses/src/LMList.h
	moses/src/LexicalReordering.h
	moses/src/PhraseDictionaryTree.h
	moses/src/ScoreIndexManager.h
	moses/src/StaticData.h
	moses/src/TargetPhrase.h
	moses/src/Word.cpp
	scripts/ems/experiment.meta
	scripts/ems/experiment.perl
	scripts/training/train-model.perl
 2012-07-17 13:36:50 +01:00
Hieu Hoang
e3dd3a8d2c namespace all classes in mert directory 2012-06-30 20:23:45 +01:00
Barry Haddow
c397d2068b Merge branch 'trunk' into miramerge. Still to fix build. 
Conflicts:
	Jamroot
	mert/Data.cpp
	mert/Data.h
	mert/FeatureArray.cpp
	mert/FeatureArray.h
	mert/FeatureData.cpp
	mert/FeatureData.h
	mert/FeatureStats.cpp
	mert/FeatureStats.h
	mert/mert.cpp
	moses-chart-cmd/src/IOWrapper.h
	moses-chart-cmd/src/Main.cpp
	moses-cmd/src/IOWrapper.cpp
	moses-cmd/src/IOWrapper.h
	moses-cmd/src/Main.cpp
	moses/src/GlobalLexicalModel.cpp
	moses/src/Jamfile
	moses/src/Parameter.cpp
	moses/src/PhraseDictionary.cpp
	moses/src/ScoreIndexManager.h
	moses/src/TargetPhrase.h
	regression-testing/tests/phrase.lexicalized-reordering-bin/truth/results.txt
	regression-testing/tests/phrase.lexicalized-reordering-cn/truth/results.txt
	regression-testing/tests/phrase.lexicalized-reordering/truth/results.txt
	regression-testing/tests/phrase.multiple-translation-system-lr/truth/results.txt
	regression-testing/tests/phrase.show-weights.lex-reorder/truth/results.txt
	regression-testing/tests/phrase.show-weights/truth/results.txt
	scripts/ems/experiment.meta
	scripts/ems/experiment.perl
	scripts/training/filter-model-given-input.pl
	scripts/training/mert-moses.pl
 2012-05-24 21:11:35 +01:00
Tetsuo Kiso
dbfe766f2c Fix using directive refers to implicitly-defined namespace 'std'. 2012-05-06 05:27:04 +09:00
Tetsuo Kiso
df4586740d Fix using directive refers to implicitly-defined namespace 'std'. 2012-05-06 05:27:04 +09:00
Tetsuo Kiso
27515f5de1 Add a function to check whether a string ends with a suffix. 
- Use the function in Data::InitFeatureMap().
- Add an unit test for InitFeatureMap().
- Move helper functions for Data::loadnbest() to public for unit testing.
 2012-04-04 22:04:51 +09:00
Tetsuo Kiso
1ade69a546 Add a function to check whether a string ends with a suffix. 
- Use the function in Data::InitFeatureMap().
- Add an unit test for InitFeatureMap().
- Move helper functions for Data::loadnbest() to public for unit testing.
 2012-04-04 22:04:51 +09:00
Tetsuo Kiso
3ce46da4cd Clean up Data; add TODOs. 2012-03-10 17:47:01 +09:00
Tetsuo Kiso
81309bdb2d Clean up Data; add TODOs. 2012-03-10 17:47:01 +09:00
Tetsuo Kiso
b5bcf48b17 Pass by pointers to Scorer instead of references. 2012-03-10 17:28:38 +09:00
Tetsuo Kiso
a1ab79c7fc Pass by pointers to Scorer instead of references. 2012-03-10 17:28:38 +09:00
Tetsuo Kiso
e7a2483b22 mert: Prefix private members with "m_" except TER. 
Squashed commit of the following:

- Clean up PRO.
- Clean up ScoreStats.
- Clean up ScoreData.
- Clean up ScoreArray.
- Remove unnecessary headers.
- Clean up ScopedVector.
- Clean up Point.
- Clean up PerScorer.
- Clean up Optimizer.
- Clean up MergeScorer.
- Clean up InterpolatedScorer.
- Clean up FileStream.
- Clean up FeatureStats.
- Remove inefficient string concatenation.
- Clean up FeatureData.
- Clean up FeatureArray.
- Clean up Data.
 2012-03-10 17:12:34 +09:00
Tetsuo Kiso
eb2c9ee5e3 mert: Prefix private members with "m_" except TER. 
Squashed commit of the following:

- Clean up PRO.
- Clean up ScoreStats.
- Clean up ScoreData.
- Clean up ScoreArray.
- Remove unnecessary headers.
- Clean up ScopedVector.
- Clean up Point.
- Clean up PerScorer.
- Clean up Optimizer.
- Clean up MergeScorer.
- Clean up InterpolatedScorer.
- Clean up FileStream.
- Clean up FeatureStats.
- Remove inefficient string concatenation.
- Clean up FeatureData.
- Clean up FeatureArray.
- Clean up Data.
 2012-03-10 17:12:34 +09:00
Tetsuo Kiso
6b1dfa3434 Clean up Data::loadnbest(). 
Add helper functions.
 2012-03-07 07:01:28 +09:00
Tetsuo Kiso
2bdeee9caa Clean up Data::loadnbest(). 
Add helper functions.
 2012-03-07 07:01:28 +09:00
Tetsuo Kiso
47ac8a474d Change the naming conventions for the guard macros; Rename TER directory. 
This change might be useful to avoid duplicating the names.
The reason is that although MERT programs are standalone
applications, some header files such as data.h and
point.h have common guard macro names like "DATA_H" and
"POINT_H", and this is not good naming conventions
when you want to include external headers.
Some files actually include headers in Moses and KenLM's util.
 2012-02-20 09:46:08 +09:00
Tetsuo Kiso
8c3b82e596 Change the naming conventions for the guard macros; Rename TER directory. 
This change might be useful to avoid duplicating the names.
The reason is that although MERT programs are standalone
applications, some header files such as data.h and
point.h have common guard macro names like "DATA_H" and
"POINT_H", and this is not good naming conventions
when you want to include external headers.
Some files actually include headers in Moses and KenLM's util.
 2012-02-20 09:46:08 +09:00
Tetsuo Kiso
82da44b030 Fix typo. 2012-02-20 08:29:53 +09:00
Tetsuo Kiso
94888b258d Fix typo. 2012-02-20 08:29:53 +09:00
Barry Haddow
62d7d034bb Fix sharding bug 2012-02-08 17:11:56 +00:00
Barry Haddow
752724594e Fix sharding bug 2012-02-08 17:11:56 +00:00
Barry Haddow
ced24a881d Implementation of feature-merging for pro-mert 2012-01-13 16:52:15 +00:00
Hieu Hoang
753eebd959 revert 2011-12-12 20:48:42 +07:00
Hieu Hoang
21009b5d1e revert 2011-12-12 20:48:42 +07:00
Barry Haddow
ba5633c7b5 Merge remote branch 'origin/master' into pro 
Conflicts:
	kenlm
	mert/Data.cpp
	mert/Data.h
	mert/Makefile.am
	mert/mert.cpp
 2011-11-17 15:58:56 +00:00
Barry Haddow
71c777f01d Remove old PRO. Fix running of PRO from mert script. 2011-11-16 17:43:54 +00:00
Tetsuo Kiso
be1506e759 Use passing objects by const references not passing by their values. 2011-11-14 14:00:47 +09:00
Tetsuo Kiso
625fe118e0 Fix bugs, private members were not initialized when the instances are created. 
When the objects of Data and FeatureData classes are created,
the primitive private members were not initialized properly.
 2011-11-14 12:20:04 +09:00
Tetsuo Kiso
3d70b2e1a5 Small change: modify initialization of the Data class. 2011-11-12 22:04:22 +09:00
Tetsuo Kiso
fb3b0f9f62 Add const to Data and Feature class members. 2011-11-12 11:26:13 +09:00
Tetsuo Kiso
dfb714296f Add 'explicit' for constructors with one argument. 2011-11-12 09:51:27 +09:00
Tetsuo Kiso
ce9a628ed0 Remove unnecessary semicolons used in end of member functions. 2011-11-12 09:40:01 +09:00
Tetsuo Kiso
664ffe0130 Fix indentation. 2011-11-12 09:24:19 +09:00
Tetsuo Kiso
68315d6407 Fix class, function, and implementation comments format. 
Functions comments should be placed in their declarations.
 2011-11-12 08:58:23 +09:00
Tetsuo Kiso
4f6d022fe7 Add comments to mark the end of #define guards. 2011-11-12 07:59:50 +09:00
Tetsuo Kiso
087756b8c3 Fix memory leaks in extractor. 2011-11-11 20:02:26 +09:00
bhaddow
fc695c38a7 Implementation of sharding and resampling in mert. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4226 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-15 17:45:35 +00:00
phkoehn
41a1849437 support for sparse feature functions (mert support only when using PRO) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4184 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-07 16:37:33 +00:00
phkoehn
36db0ffe48 added pairwise ranked optimization (PRO) as proposd by [Hopkins&May,2011], just use switch --pairwise-ranked 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4106 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-03 17:00:17 +00:00
hieuhoang1972
148c1e8305 run beautify.perl. Consistent formatting for .h & .cpp files 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3899 1f5c12ca-751b-0410-a591-d2e778427230
 2011-02-24 12:42:19 +00:00
nicolabertoldi
e94834012d added facilities to read and write score statistics in binary format 
moved facilities for feature names in FeatureData object


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1824 1f5c12ca-751b-0410-a591-d2e778427230
 2008-06-05 17:03:54 +00:00
nicolabertoldi
930e67c5e3 fixed another bug related to the handling of feature names 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1822 1f5c12ca-751b-0410-a591-d2e778427230
 2008-06-05 11:30:37 +00:00
nicolabertoldi
44d7e0e0f7 fixed a bug related to the handling of feature names 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1821 1f5c12ca-751b-0410-a591-d2e778427230
 2008-06-05 11:26:54 +00:00
nicolabertoldi
281bf610b8 added binary read/load facility for feature data 
added names of features in the header
added methods to access the features by name


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1819 1f5c12ca-751b-0410-a591-d2e778427230
 2008-06-05 07:23:34 +00:00
nicolabertoldi
af585bc492 nbest can be read from stdin, too 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1797 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-27 17:20:01 +00:00
nicolabertoldi
291260abf7 - made output more compliant with old version 
- added PerSCorer.h and BleuScorer.h
- stored feature names
- fixed bug about output of best Point


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1796 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-27 16:50:52 +00:00
nicolabertoldi
8cf59edcdc remove loadnbest from FeatureData and Scoredata; change test_scorer accordingly; 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1787 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-20 14:33:47 +00:00