moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2024-12-27 05:55:02 +03:00
Code Issues Packages Projects Releases Wiki Activity
11,223 Commits 136 Branches 26 Tags 147 MiB
cedc815c88
Commit Graph

586 Commits

Author SHA1 Message Date
Hieu Hoang
cedc815c88 return default even if compiling with non-gcc 2013-11-25 16:38:19 +00:00
Hieu Hoang
905ab6de82 stop warning about incomplete data type 2013-11-25 16:24:51 +00:00
Hieu Hoang
d9be81596e replace CHECK with UTIL_THROW_IF in mert 2013-11-18 18:13:10 +00:00
Hieu Hoang
d6d0877ea3 add comment for future work on making fdstream more portable /Jeroen Vermeulen 2013-11-15 11:56:00 +00:00
Hieu Hoang
17887a2796 replace nth_element() with macro that execute sort() instead for gcc 4.8.1 & 4.8.2 2013-11-15 10:55:38 +00:00
Hieu Hoang
3d37a8ffda more compile errors, with clang 2013-11-14 19:30:45 +00:00
Hieu Hoang
862e1ad4ae more gcc compile errors 2013-11-14 19:15:53 +00:00
Hieu Hoang
dfbad35abd minor compile errors on gcc now 2013-11-14 18:59:36 +00:00
Hieu Hoang
0b1bb6a443 mert compiles under Mac OSX Mavericks. #ifdef PreProcessFilter.cpp and accomplices 2013-11-14 17:58:59 +00:00
Hieu Hoang
f35750bc08 beautify 2013-07-04 20:19:51 +01:00
Hieu Hoang
b10159a29f mac compile 2013-07-03 20:19:45 +01:00
Sara Stymne
b2eb42ed12 added document level Bleu scoring to mert 2013-07-03 14:03:58 +02:00
Hieu Hoang
abe6bb7c22 refactor parsing of feature functiona args 2013-06-10 18:11:55 +01:00
Hieu Hoang
b9f54b195a implement GenerationDictionary.Load() 2013-06-05 13:42:56 +01:00
Matous Machacek
3055be8837 fixed two bugs in CderScorer.cpp 2013-06-03 16:59:13 +02:00
Hieu Hoang
6249432407 beautify 2013-05-29 18:16:15 +01:00
phikoehn
4cdffc8a89 fixes for sparse feature handling 2013-05-17 08:37:29 +01:00
Hieu Hoang
fd4e954322 merge 2013-03-24 09:57:36 +00:00
Matous Machacek
7b9c5c1194 fixed bug in InterpolatedScorer 2013-03-19 23:08:28 +01:00
Hieu Hoang
5c10a2889e Merge github.com:moses-smt/mosesdecoder into weight-new 2013-02-20 17:02:34 +00:00
Barry Haddow
9ca364fb22 Implement brevity penalty smoothing for PRO 
As in Nakov et al (Coling 2012)
 2013-02-18 11:11:20 +00:00
hieu
01243e415a Merge branch 'master' into weight-new 2013-01-03 17:17:35 +00:00
hieu
1dfbe1113c delete MergeScorer in mert/ 2013-01-03 15:01:30 +00:00
Hieu Hoang
4f0d3c2032 Merge branch 'master' into weight-new 2012-12-20 20:27:50 +00:00
Tetsuo Kiso
ce1b650b53 Fix memory leak. 
The object was allocated with new, but it was not deleted.
This may not be a serious problem because the program mostly runs
a short time. However, it is not a good practice.
 2012-12-21 03:06:41 +09:00
Hieu Hoang
bc615bdac8 Merge branch 'master' into weight-new 2012-12-18 12:46:00 +00:00
hieu
05045d574c don't display unknown weight penalty when showing weight, don't usually tune. Also, change delimiter in mert extractor from : to = 2012-12-16 18:29:53 +00:00
Ales Tamchyna
598d65bcfd adding a simple command-line utility for computing sentence-level BLEU (+1) 2012-12-10 13:12:34 +01:00
Tetsuo Kiso
2a3c9fc679 Further optimization for extractor. 
Fixes inefficient updating N-gram counts.

NOTE: Using '--binary' option (this option is not enabled by default yet)
for saving outputs would lead to significant speed up.
 2012-12-07 08:45:47 +09:00
Tetsuo Kiso
8fdec9bf30 Use boost::unordered_map instead of std::map. 
For storing the word vocabulary used in computation of
BLEU scores. This change will reduce the running time
of extractor about 2-3 seconds (9% reduction).
 2012-12-07 05:12:24 +09:00
Tetsuo Kiso
6c04c4ad9c Add more tests to the Data class. 2012-12-07 02:46:59 +09:00
Tetsuo Kiso
c7f6e38326 Use FilePiece to load N-best lists. 
Since FilePiece is friendly with StringPiece.
 2012-12-07 02:39:02 +09:00
Tetsuo Kiso
38e145e556 Use util::TokenIter to tokenize n-best lists. 
Reduce creating std::string objects, too. In both ScoreArray
and FeatureArray classes, the private members to track sentence
indices (namely, "m_index") were unnecessarily declared as
std::string, but it's better to directly declare them as 'int'.
 2012-12-07 01:39:22 +09:00
Tetsuo Kiso
cd3fb3b831 Untabify. 2012-12-06 23:46:22 +09:00
Tetsuo Kiso
ac045a11c1 Speed up N-gram counts when running extractor. 
By replacing std::map with boost::unordered_map.

Runtime of extractor on 100-best lists of 2679 sentences:

Before:
real    0m35.314s
user    0m34.030s
sys     0m1.280s

Ater:
real    0m26.729s
user    0m25.420s
sys     0m1.310s
 2012-12-06 22:08:33 +09:00
Hieu Hoang
f96b33de83 only include moses root when compiling 2012-11-14 13:43:04 +00:00
Hieu Hoang
f8438f80cc move moses/src/* to moses/ 2012-11-12 20:30:39 +00:00
Hieu Hoang
5e3ef23cef move moses/src/* to moses/ 2012-11-12 19:56:18 +00:00
Kenneth Heafield
e9eb7dd021 More shared build fixes 2012-11-07 23:28:42 +01:00
Kenneth Heafield
d7ecd0be1a Remove bleu_lib target. 2012-11-07 23:21:59 +01:00
Tetsuo Kiso
96f7b42eb9 Move implementation details from the header to .cpp file. 
Also add const to variables that we don't want to change.
 2012-11-05 01:24:16 +09:00
Tetsuo Kiso
cccfb9a0c9 Using namespace std in a header file pollutes the global namespace. 
Using directives should be put into the implementation files.
 2012-11-05 00:43:36 +09:00
Kenneth Heafield
dc31857bbc Reduce header pollution 2012-10-30 20:25:05 +00:00
Barry Haddow
848aafb644 Merge remote branch 'github/master' into miramerge 
Conflicts:
	moses/src/AlignmentInfo.cpp
	moses/src/AlignmentInfo.h
	moses/src/ChartHypothesis.cpp
	moses/src/ChartTrellisNode.cpp
	moses/src/LM/Implementation.cpp
	moses/src/LM/Ken.cpp
	moses/src/TargetPhrase.cpp
	moses/src/TargetPhrase.h
 2012-10-08 17:54:59 +01:00
Arianna Bisazza
c78d571561 fixed bug in permutation scores due to scientific notation 2012-10-08 16:15:59 +01:00
Barry Haddow
0a950ee9f4 Merge remote branch 'github/master' into miramerge 
Compiles, but not tested. Had to disable relent filter. Strangely, it seems to contain the
whole of moses-cmd.

Conflicts:
	Jamroot
	OnDiskPt/TargetPhrase.cpp
	moses-cmd/src/Main.cpp
	moses/src/AlignmentInfo.cpp
	moses/src/AlignmentInfo.h
	moses/src/ChartTranslationOptionCollection.cpp
	moses/src/ChartTranslationOptionCollection.h
	moses/src/GenerationDictionary.cpp
	moses/src/Jamfile
	moses/src/Parameter.cpp
	moses/src/PhraseDictionary.cpp
	moses/src/StaticData.cpp
	moses/src/StaticData.h
	moses/src/TargetPhrase.h
	moses/src/TranslationSystem.cpp
	moses/src/TranslationSystem.h
	moses/src/Word.cpp
	phrase-extract/score.cpp
	regression-testing/Jamfile
	scripts/ems/experiment.meta
	scripts/ems/experiment.perl
	scripts/training/train-model.perl
 2012-09-26 22:49:33 +01:00
Arianna Bisazza
ff276e9911 Fixed several bugs in LRscore-MERT. Namely, solved a float-to-int conversion; added hypothesis counter to the scores file to enable later computation of average reordering score; fixed special case of 1-word hypothesis; enabled reading of word-based alignments from n-best-list. 2012-09-24 15:40:18 +02:00
Colin Cherry
ae6ac1c2ae Merge branch 'master' of https://github.com/moses-smt/mosesdecoder 2012-09-14 14:07:53 -04:00
Colin Cherry
3fa95c022b Addad a "--safe-hope" option to kbmira. 
This will limit the influence of model score on oracle (hope) selection.
Good for cases whith extremely large feature values. May make it the defult.
 2012-09-14 13:58:28 -04:00
Barry Haddow
021f5702a7 remove obsolete file 2012-09-13 22:21:31 +01:00