moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2024-09-20 23:58:15 +03:00
Code Issues Packages Projects Releases Wiki Activity
1,021 Commits 136 Branches 26 Tags 147 MiB
e7821e84ef
Commit Graph

1021 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
bojar
e7821e84ef Added support for Ondrej Bojar's scoring of nbestlist, faster than python and 
does not rescore previous iterations. The scoring tool is however not included
in the scripts distribution.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1329 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-26 09:26:05 +00:00
bojar
55ea5d6f94 Adding simple Czech rules to detokenizer. Making detokenizer 'released'. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1328 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-26 06:08:13 +00:00
bojar
58bf2089af Adding detokenizer from WMT07 shared scripts.tgz, hoping there are no copyright 
problems. Please withdraw if necessary.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1327 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-26 05:46:50 +00:00
bojar
3d288d81e4 Proper unicode-based lower and uppercasing. 
Added language option to recase.perl, English remains the default.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1326 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-26 05:44:27 +00:00
bojar
b8a0761af3 Allow a workaround flag -feed-moses-via-stdin (the default to use -input-file 
is not changed).
Correctly prepare qsub arguments for -old-sge workaround.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1325 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-25 22:58:06 +00:00
bojar
c7b78dfc2a Automagically fix moses.ini if it points to a non-gzipped version of a file but 
only the gzipped exists.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1324 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-25 22:55:31 +00:00
bojar
9fe45b747a Not everyone has 'pawd'. Using pawd as the default but reverting to pwd to get 
the current directory.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1323 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-25 05:46:05 +00:00
lexi_birch
d20ae825f8 Alternate graph code: 
Now can specify an alternate path in the config file for experiment.perl
All you need to do is change the line: decoding-steps
Every : you add will mean that the next steps are in an alternate graph
Ie. 
	decoding-steps = " t0 , g1 , t1 "
will produce a single graph but
	decoding-steps = " t0 , g1 : t1 "
will produce two alternate graphs

The only change it actually makes is to the create configuration file.

This is also a bugfix!
There was no ability to have multiple factors in the target
for the two filering scripts. They just ignored those lines
as if they were not valid table descriptions.




git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1322 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-22 12:42:23 +00:00
bojar
02bb5540cb Changed file format back from DOS to unix. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1321 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-21 00:53:28 +00:00
bojar
d4b1103bd7 Fixed the case when the intersection of two alignments is empty. Used to throw 
std::out_of_range at basic_string::replace, now emits an empty line.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1320 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-21 00:52:28 +00:00
bojar
2f210366f6 Flushing stdout for intersect and union alignments. 
(No explicit flushing leads to truncated output! C++ is crap!)
Also report the number of sentences processed.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1319 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-21 00:31:53 +00:00
hieuhoang1972
9394f571e6 eclipse project file 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1318 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-17 14:26:53 +00:00
maurocettolo
92788cadb5 added -V option to qsub call in qsub-wrapper.pl. This allows to make 
available to the submitted job the value of variable PYTHONPATH, set to
$pythonpath in mert-moses.pl through %ENV (no more need of using
setenv/export)



git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1317 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-16 13:20:27 +00:00
nicolabertoldi
9912815c53 csh requires setenv instead of export. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1316 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-15 18:00:57 +00:00
nicolabertoldi
ffa520a2a9 Robust (forced and not interactive) copy, move and remove of files. 
Add usage infos.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1315 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-15 17:54:00 +00:00
konrad_rawlik
710099be91 queryLexicalTable 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1314 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-15 12:43:27 +00:00
hieuhoang1972
f57a36af1b add queryLexicalTable.cpp placeholder 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1313 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-15 12:01:53 +00:00
hieuhoang1972
a611a78de7 change unsigned in to size_t 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1312 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-15 10:11:28 +00:00
hieuhoang1972
17e732b644 set svn properties 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1310 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-14 22:44:36 +00:00
hieuhoang1972
4b0ea463c8 add svn id comments to start of file 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1308 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-14 22:30:25 +00:00
hieuhoang1972
3c07c5df4d add svn id comments to start of file 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1307 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-14 22:22:36 +00:00
hieuhoang1972
a184354ff3 get rid of idx arg in GetScoreProducerDescription() 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1306 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-14 20:07:58 +00:00
hieuhoang1972
8566962ecc output empty line even if no best hypothesis found 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1305 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-14 19:47:51 +00:00
jorcisai
a83fe593c2 startup ini file is backed up in order to preserve the pathnames to the original models, so that the final moses.ini file doesn't point to the filtered models 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1304 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-14 18:31:53 +00:00
hieuhoang1972
05a45bd0f9 lexicalised reodering working with visual studio 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1302 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-14 13:02:00 +00:00
konrad_rawlik
770d8e08c3 Added lexical reordering changes 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1301 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-14 12:42:35 +00:00
hieuhoang1972
b2a5a65522 set executable property 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1300 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-13 23:28:06 +00:00
hieuhoang1972
71833f3bee merge from hieu-async branch 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1299 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-13 23:03:53 +00:00
bojar
87b168cd1b Handles gzipped input. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1296 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-12 02:24:34 +00:00
phkoehn
41ee7f69a2 adapted mert to work with multiple decoding paths 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1293 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-09 17:58:05 +00:00
hieuhoang1972
6900160a0a change DecodeStep* to const DecodeStep* 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1287 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-07 22:15:03 +00:00
hieuhoang1972
93593b891d make naming of hypo stacks classes consistent 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1282 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-07 15:37:38 +00:00
hieuhoang1972
025f2f3e03 gcc compile error 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1276 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-06 15:57:30 +00:00
hieuhoang1972
86e6d99d76 add comments and Reset() 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1275 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-06 12:15:51 +00:00
hieuhoang1972
075681d6fc visual studio proj file 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1267 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-04 09:01:50 +00:00
hieuhoang1972
5f22fb13d3 make output of decimal places consistent by not format anywhere but in Main.cpp 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1266 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-04 00:43:45 +00:00
hieuhoang1972
2c9f1a13fe make output of decimal places consistent by not format anywhere but in Main.cpp 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1265 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-04 00:17:54 +00:00
hieuhoang1972
c40d07d0a1 code cleanup 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1257 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-02 13:37:12 +00:00
jdschroeder
19129713a2 added filter-and-binarize training script to released-files 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1256 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-01 14:44:46 +00:00
maurocettolo
5439a7796d Fixed a minor bug in mert-moses.pl regarding sanity checks for specified lambda triples 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1254 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-01 13:02:22 +00:00
hieuhoang1972
e6b7866f4a get rid of warning message in srilm class 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1251 1f5c12ca-751b-0410-a591-d2e778427230
 2007-02-28 11:59:32 +00:00
phkoehn
f1d2bd0eb5 added option -include-alignment-in-n-best to include the word alignment for each sentence in the n-best list file 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1246 1f5c12ca-751b-0410-a591-d2e778427230
 2007-02-26 20:59:41 +00:00
hieuhoang1972
3413bf7046 visual studio output paths 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1245 1f5c12ca-751b-0410-a591-d2e778427230
 2007-02-26 16:49:32 +00:00
jorcisai
ad28cee802 distortion filename was incorrectly written into moses.ini file in step 9 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1243 1f5c12ca-751b-0410-a591-d2e778427230
 2007-02-26 14:36:53 +00:00
phkoehn
a89acb34ae minor bug fix to recaser training 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1242 1f5c12ca-751b-0410-a591-d2e778427230
 2007-02-26 12:19:06 +00:00
hieuhoang1972
62b4741de0 move calling InitializeBeforeSentenceProcessing() & CleanUpAfterSentenceProcessing() in Manager class 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1241 1f5c12ca-751b-0410-a591-d2e778427230
 2007-02-24 20:32:32 +00:00
jorcisai
b361817067 In old_sge mode: sync script name is now prefixed by ${jobscript} to be able to run several moses_parallel.pl in parallel. Also a new function check_translation_old_sge was added, this function is derived from the former check_translation function 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1239 1f5c12ca-751b-0410-a591-d2e778427230
 2007-02-23 18:08:47 +00:00
jorcisai
d5b4565f23 language model parser for --lm option is now again able to parse $type, but it is backward compatible 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1238 1f5c12ca-751b-0410-a591-d2e778427230
 2007-02-23 17:32:51 +00:00
jorcisai
c69bd4079b reordering model was left in the local directory instead of model directory 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1236 1f5c12ca-751b-0410-a591-d2e778427230
 2007-02-23 12:27:50 +00:00
jorcisai
872f2d3612 Trying to parse $type in --lm option, but not available. So we just need to parse three tokens. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1235 1f5c12ca-751b-0410-a591-d2e778427230
 2007-02-23 11:49:54 +00:00