moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2024-09-20 15:48:05 +03:00
Code Issues Packages Projects Releases Wiki Activity
1,712 Commits 136 Branches 26 Tags 147 MiB
5a1c308673
Commit Graph

375 Commits

Author SHA1 Message Date
hieuhoang1972
b9d2288c22 compileable with visual studio 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1349 1f5c12ca-751b-0410-a591-d2e778427230
 2007-04-11 11:35:36 +00:00
hieuhoang1972
e868986885 compileable with visual studio 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1348 1f5c12ca-751b-0410-a591-d2e778427230
 2007-04-11 11:18:37 +00:00
jdschroeder
84d12552c2 added additional numbered count to output phrase table names so multiple phrase tables can be filtered and used at the same time 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1337 1f5c12ca-751b-0410-a591-d2e778427230
 2007-04-04 17:54:12 +00:00
jdschroeder
667e85264a added LC_ALL=C call and temp directory specification to sort command, hoping to minimize failed sorts crashing processPhraseTable 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1336 1f5c12ca-751b-0410-a591-d2e778427230
 2007-04-04 17:07:16 +00:00
jdschroeder
04ae9361d2 added "-v 0" moses flag to decoder call to minimize log output. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1335 1f5c12ca-751b-0410-a591-d2e778427230
 2007-04-04 17:04:50 +00:00
hieuhoang1972
c17316495c use pawd instead of pwd whenever we can 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1332 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-26 21:00:20 +00:00
hieuhoang1972
502093573b set svn:eol property 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1331 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-26 20:06:44 +00:00
bojar
e7821e84ef Added support for Ondrej Bojar's scoring of nbestlist, faster than python and 
does not rescore previous iterations. The scoring tool is however not included
in the scripts distribution.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1329 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-26 09:26:05 +00:00
bojar
55ea5d6f94 Adding simple Czech rules to detokenizer. Making detokenizer 'released'. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1328 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-26 06:08:13 +00:00
bojar
58bf2089af Adding detokenizer from WMT07 shared scripts.tgz, hoping there are no copyright 
problems. Please withdraw if necessary.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1327 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-26 05:46:50 +00:00
bojar
3d288d81e4 Proper unicode-based lower and uppercasing. 
Added language option to recase.perl, English remains the default.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1326 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-26 05:44:27 +00:00
bojar
b8a0761af3 Allow a workaround flag -feed-moses-via-stdin (the default to use -input-file 
is not changed).
Correctly prepare qsub arguments for -old-sge workaround.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1325 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-25 22:58:06 +00:00
bojar
c7b78dfc2a Automagically fix moses.ini if it points to a non-gzipped version of a file but 
only the gzipped exists.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1324 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-25 22:55:31 +00:00
bojar
9fe45b747a Not everyone has 'pawd'. Using pawd as the default but reverting to pwd to get 
the current directory.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1323 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-25 05:46:05 +00:00
lexi_birch
d20ae825f8 Alternate graph code: 
Now can specify an alternate path in the config file for experiment.perl
All you need to do is change the line: decoding-steps
Every : you add will mean that the next steps are in an alternate graph
Ie. 
	decoding-steps = " t0 , g1 , t1 "
will produce a single graph but
	decoding-steps = " t0 , g1 : t1 "
will produce two alternate graphs

The only change it actually makes is to the create configuration file.

This is also a bugfix!
There was no ability to have multiple factors in the target
for the two filering scripts. They just ignored those lines
as if they were not valid table descriptions.




git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1322 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-22 12:42:23 +00:00
bojar
02bb5540cb Changed file format back from DOS to unix. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1321 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-21 00:53:28 +00:00
bojar
d4b1103bd7 Fixed the case when the intersection of two alignments is empty. Used to throw 
std::out_of_range at basic_string::replace, now emits an empty line.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1320 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-21 00:52:28 +00:00
bojar
2f210366f6 Flushing stdout for intersect and union alignments. 
(No explicit flushing leads to truncated output! C++ is crap!)
Also report the number of sentences processed.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1319 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-21 00:31:53 +00:00
maurocettolo
92788cadb5 added -V option to qsub call in qsub-wrapper.pl. This allows to make 
available to the submitted job the value of variable PYTHONPATH, set to
$pythonpath in mert-moses.pl through %ENV (no more need of using
setenv/export)



git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1317 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-16 13:20:27 +00:00
nicolabertoldi
9912815c53 csh requires setenv instead of export. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1316 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-15 18:00:57 +00:00
nicolabertoldi
ffa520a2a9 Robust (forced and not interactive) copy, move and remove of files. 
Add usage infos.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1315 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-15 17:54:00 +00:00
hieuhoang1972
4b0ea463c8 add svn id comments to start of file 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1308 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-14 22:30:25 +00:00
hieuhoang1972
3c07c5df4d add svn id comments to start of file 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1307 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-14 22:22:36 +00:00
jorcisai
a83fe593c2 startup ini file is backed up in order to preserve the pathnames to the original models, so that the final moses.ini file doesn't point to the filtered models 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1304 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-14 18:31:53 +00:00
hieuhoang1972
71833f3bee merge from hieu-async branch 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1299 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-13 23:03:53 +00:00
bojar
87b168cd1b Handles gzipped input. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1296 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-12 02:24:34 +00:00
phkoehn
41ee7f69a2 adapted mert to work with multiple decoding paths 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1293 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-09 17:58:05 +00:00
jdschroeder
19129713a2 added filter-and-binarize training script to released-files 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1256 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-01 14:44:46 +00:00
maurocettolo
5439a7796d Fixed a minor bug in mert-moses.pl regarding sanity checks for specified lambda triples 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1254 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-01 13:02:22 +00:00
phkoehn
f1d2bd0eb5 added option -include-alignment-in-n-best to include the word alignment for each sentence in the n-best list file 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1246 1f5c12ca-751b-0410-a591-d2e778427230
 2007-02-26 20:59:41 +00:00
jorcisai
ad28cee802 distortion filename was incorrectly written into moses.ini file in step 9 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1243 1f5c12ca-751b-0410-a591-d2e778427230
 2007-02-26 14:36:53 +00:00
phkoehn
a89acb34ae minor bug fix to recaser training 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1242 1f5c12ca-751b-0410-a591-d2e778427230
 2007-02-26 12:19:06 +00:00
jorcisai
b361817067 In old_sge mode: sync script name is now prefixed by ${jobscript} to be able to run several moses_parallel.pl in parallel. Also a new function check_translation_old_sge was added, this function is derived from the former check_translation function 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1239 1f5c12ca-751b-0410-a591-d2e778427230
 2007-02-23 18:08:47 +00:00
jorcisai
d5b4565f23 language model parser for --lm option is now again able to parse $type, but it is backward compatible 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1238 1f5c12ca-751b-0410-a591-d2e778427230
 2007-02-23 17:32:51 +00:00
jorcisai
c69bd4079b reordering model was left in the local directory instead of model directory 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1236 1f5c12ca-751b-0410-a591-d2e778427230
 2007-02-23 12:27:50 +00:00
jorcisai
872f2d3612 Trying to parse $type in --lm option, but not available. So we just need to parse three tokens. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1235 1f5c12ca-751b-0410-a591-d2e778427230
 2007-02-23 11:49:54 +00:00
phkoehn
6c5cb3a6ec changes to fit with edinburgh setup, added switch -generation-type: "single" only produces one probability, not both 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1231 1f5c12ca-751b-0410-a591-d2e778427230
 2007-02-22 19:37:11 +00:00
jdschroeder
9576345394 added recaser scripts to released-files 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1226 1f5c12ca-751b-0410-a591-d2e778427230
 2007-02-21 15:47:13 +00:00
phkoehn
9f227aa26b minor bug fix with config file 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1224 1f5c12ca-751b-0410-a591-d2e778427230
 2007-02-19 18:35:51 +00:00
phkoehn
14839768c8 a large number of changes. besides little tweaks: 
* training script now has proper default behaviour for single-factor models, 
* mert script has better handling of default lambda parameters that now
  works with lexicalized reordering models, and also with multiple 
  models files (e.g. multiple language models)
* parallel mert script is more robust when single jobs fail: detects it
  and resubmits the crashed (or killed) jobs
* recaser added that builds on moses
* filtering script added that also binarizes filtered model files
  (this will be eventually replaced when the lexicalized reordering
  model also uses the binary format)


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1210 1f5c12ca-751b-0410-a591-d2e778427230
 2007-02-13 19:22:35 +00:00
hieuhoang1972
970af347e4 Bug fixed for distortion model proposed by Tim Murray 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1111 1f5c12ca-751b-0410-a591-d2e778427230
 2007-01-07 12:36:18 +00:00
hieuhoang1972
0aba61ca8b don't insist on using python 2.3 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1104 1f5c12ca-751b-0410-a591-d2e778427230
 2007-01-02 12:54:00 +00:00
lexi_birch
93937b529d Making remaining scripts os independent re pawd/pwd 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1103 1f5c12ca-751b-0410-a591-d2e778427230
 2006-12-29 13:45:21 +00:00
nicolabertoldi
239e57c16c managing of pwd/pawd 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1102 1f5c12ca-751b-0410-a591-d2e778427230
 2006-12-29 13:19:36 +00:00
nicolabertoldi
26aff6ead9 managing of pwd/pawd 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1101 1f5c12ca-751b-0410-a591-d2e778427230
 2006-12-29 13:19:21 +00:00
nicolabertoldi
51d74a3941 remove obsolete code 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1100 1f5c12ca-751b-0410-a591-d2e778427230
 2006-12-29 12:13:23 +00:00
nicolabertoldi
92eadd7c0c - change from pawd to pwd, because pawd is not available on some Linux distribution 
- moses-parallel.pl: new way of passing parameters to decoder with parameter  -decoder-parameters
- moses-parallel.pl: possibility of saving decoder logs (parameter -logfile)



git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1099 1f5c12ca-751b-0410-a591-d2e778427230
 2006-12-29 10:48:41 +00:00
hieuhoang1972
ddd2fdeb20 Fix automount partition bug 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1098 1f5c12ca-751b-0410-a591-d2e778427230
 2006-12-29 00:59:59 +00:00
hieuhoang1972
566491237a Fix automount partition bug 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1097 1f5c12ca-751b-0410-a591-d2e778427230
 2006-12-29 00:51:11 +00:00
lexi_birch
dee506806f Fix for mount bug using pwd on terabyte 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1096 1f5c12ca-751b-0410-a591-d2e778427230
 2006-12-28 22:17:06 +00:00
hieuhoang1972
e701f57f07 halcion days of the jhu workshop are over and grim reality has taken hold. 
default qsub not to use workshop specific queue

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1079 1f5c12ca-751b-0410-a591-d2e778427230
 2006-12-16 22:33:55 +00:00
bojar
c1370484a2 - Fixed bug with brevity penalty: 
If two reference translations of a sentence were equally "close" to the
    hypothesis, the *first* one was taken into account, given the order of
    references.
    Now the *shorter* is used, making brevity independent on the order of
    references. (Papineni etal are not specific about this, either).
    (Consider the case where the hypothesis is 30 words and there are two
    references, one of 28 and one of 32 words.)
- Fixed usage-behaviour inconsistency:
    usage said that ref.0, ref.1, .. are loaded but it loaded only
      ref.1, ref.2,...


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1071 1f5c12ca-751b-0410-a591-d2e778427230
 2006-12-15 05:31:47 +00:00
bojar
72ff1f8450 added yet another combiner for factored corpora 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1026 1f5c12ca-751b-0410-a591-d2e778427230
 2006-11-30 06:17:45 +00:00
bojar
412f04737c allows reducing factors from stdin 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1025 1f5c12ca-751b-0410-a591-d2e778427230
 2006-11-30 03:46:21 +00:00
phkoehn
0a088dbb38 fixed error in filtering for lexicalized reordering tables 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@998 1f5c12ca-751b-0410-a591-d2e778427230
 2006-11-22 15:50:20 +00:00
hieuhoang1972
d2a56a1ca1 comments/better example in Makefile 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@983 1f5c12ca-751b-0410-a591-d2e778427230
 2006-11-16 20:01:23 +00:00
phkoehn
28ca9b57fd minor bug fixes for training and using lexicalized reordering 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@978 1f5c12ca-751b-0410-a591-d2e778427230
 2006-11-15 17:04:19 +00:00
nicolabertoldi
5c17fe6505 Fixed bug about nbest generation when a sentence is not translated. 
Now, one fictitious "empty translation" with score 0 is added.
Before, problems happened with MERT due to a misalignement.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@972 1f5c12ca-751b-0410-a591-d2e778427230
 2006-11-10 10:36:23 +00:00
bojar
e2518b7799 - support for sun grid engine prior to v.6.0 in qsubwrapper and mert-moses 
- changed temporary scripts to csh (because my sge runs them in csh regardless of my wishes)
- added a two tests + sample data for the full chain: train-mert-decode-eval
  (a parallel and a serial version)
- cleanup of other tests
- Makefile rules for running single tests in foreground or background


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@899 1f5c12ca-751b-0410-a591-d2e778427230
 2006-10-18 10:36:39 +00:00
bojar
0cd79a9b7d fixed order of 'configure' and 'make clean' in validate_revision 
scripts/Makefile now do not always clean, but 'make clean' has been added


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@895 1f5c12ca-751b-0410-a591-d2e778427230
 2006-10-17 14:47:28 +00:00
nicolabertoldi
605e47d978 psyco library is maintained only for 386-compatible processors. 
I modified score-nbest.py to import psyco only if $MACHTYPE is equal to "i386"
If MACHTYPE does not matchpsyco library is not imported,
but script works properly.
I do not know if the control is efffective under Windows


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@879 1f5c12ca-751b-0410-a591-d2e778427230
 2006-10-12 09:49:09 +00:00
nicolabertoldi
24d3ed8a37 Changes to mert-moses.pl: 
- added a flag (--no-filter-phrase-table) to disallow filtering of phrase tables (useful if binary hrase tables are used)
- added a flag to compute bleu score without text normalization (--nonorm) (default is with normalization)
- added a flag to compute bleu score with the "closest reference length" (--closest), which is 
   alternative to "average reference length" (--average) or "shortest reference length" (default)
- added a parameter (--inputype=[0|1]) to manage different input types (0 for text, 1 for confusion network, default is 0)
Changes to moses-parallel.pl:
- corrected a typos


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@878 1f5c12ca-751b-0410-a591-d2e778427230
 2006-10-11 16:58:30 +00:00
nicolabertoldi
0646dc6472 Temporary bash files generated and used by moses-parallel.pl 
and qsub-wrapper.pl are transformed in readable and executable files.
qsub call them as binary files (see option -b yes)


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@877 1f5c12ca-751b-0410-a591-d2e778427230
 2006-10-11 09:27:43 +00:00
bojar
d90b1d348e reuses lexical translation 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@876 1f5c12ca-751b-0410-a591-d2e778427230
 2006-10-10 13:53:45 +00:00
bojar
2eb05906aa skips giza if older output reusable 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@872 1f5c12ca-751b-0410-a591-d2e778427230
 2006-10-09 15:01:27 +00:00
bojar
998a8216ba skips mkcls and some other steps, if already finished 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@870 1f5c12ca-751b-0410-a591-d2e778427230
 2006-10-06 21:19:29 +00:00
bojar
33e7d3a8c4 fixed a typo in Makefile and check-dependencies checks for mkcls 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@868 1f5c12ca-751b-0410-a591-d2e778427230
 2006-10-06 16:51:31 +00:00
nicolabertoldi
a73c412b88 added clean to some Makefiles 
use of "make clean" in scripts/Makefile



git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@867 1f5c12ca-751b-0410-a591-d2e778427230
 2006-10-06 15:07:30 +00:00
bojar
c8f5e2aeba fixed an error message 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@866 1f5c12ca-751b-0410-a591-d2e778427230
 2006-10-06 10:47:16 +00:00
phkoehn
a71f247596 bugfix: option rootdir misnamed roodir 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@835 1f5c12ca-751b-0410-a591-d2e778427230
 2006-09-28 19:25:08 +00:00
mfederico
ef42ad791e symal.cpp: just a minor change 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@833 1f5c12ca-751b-0410-a591-d2e778427230
 2006-09-28 16:14:21 +00:00
bojar
c6c02a83c6 Just a short description added. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@823 1f5c12ca-751b-0410-a591-d2e778427230
 2006-09-21 12:23:42 +00:00
bojar
271b78d94c Just checking if I can commit. Added my name. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@820 1f5c12ca-751b-0410-a591-d2e778427230
 2006-09-20 14:20:17 +00:00
redpony
9582bcecff turn on O3 optimization for symal 
increase MAX_WORD in symal.cpp (I was hitting this limit in a chinese corpus that had some tokenization errors)



git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@816 1f5c12ca-751b-0410-a591-d2e778427230
 2006-09-18 15:36:04 +00:00
redpony
da7fed9e7e add --corpus-compression [gz|bz2] to allow corpora to be compressed 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@814 1f5c12ca-751b-0410-a591-d2e778427230
 2006-09-15 12:38:13 +00:00
redpony
7d50d155dc fix compilation error on gcc 4.1, fix warnings in mert 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@813 1f5c12ca-751b-0410-a591-d2e778427230
 2006-09-12 19:46:16 +00:00
redpony
c69cfaf33e Allow the factor delimiter, that is, the string that separates the factors in a 'word' to be specified to moses and to train-factored-phrase-model.perl. The default is still to use '|'. Multi-character delimiters are allowed (for example, '+++'). Added a regression test for multi-character delimiters. 
Remove JHU dependencies on make release.  It now looks for GIZA++ and sets the BINDIR inside train-factored-phrase-model.perl at installation time (note: because of this, this script MUST BE released before it can be run now).



git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@812 1f5c12ca-751b-0410-a591-d2e778427230
 2006-09-12 15:53:50 +00:00
phkoehn
572c577ef7 initial release 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@806 1f5c12ca-751b-0410-a591-d2e778427230
 2006-09-01 18:02:56 +00:00
nicolabertoldi
041e6ed3c5 Changes to compilation scripts: 
- irstlm/src/Makefile.am did not install some files
- irstlm/mkinstalldirs needed by OSX
- irstlm/regenerate-makefiles.sh substitutes 
  explicit calls of aclocal, autoconf and automake

Changes to scoring script used by MERT
- added the option ("-e") to compute BLEU wrt the
  "closest" reference length like in multi-bleu.perl
- now multi-bleu.perl manages 0 counts for ngram-statistics




git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@805 1f5c12ca-751b-0410-a591-d2e778427230
 2006-09-01 14:54:41 +00:00
eherbst
c646717009 trying to fix caching 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@775 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-17 12:31:52 +00:00
eherbst
9c7ffb1fbb thought I had added this before 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@772 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-16 16:38:19 +00:00
eherbst
24cd2f3441 updating docs 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@771 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-16 16:37:11 +00:00
eherbst
674c609fcd adding show-phrases-used 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@768 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-16 14:51:04 +00:00
eherbst
c34aca3053 modified sentence-by-sentence to handle multiple outputs; 
edited cache handling in newsmtgui (should increase speed and decrease errors)


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@767 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-16 14:49:10 +00:00
eherbst
486f88157f add formatting for sentence strings to make token comparison more accurate 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@761 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-15 20:17:41 +00:00
eherbst
25767cd5b0 fixed background-color HTML 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@757 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-15 18:20:05 +00:00
bojar
53bbbbfa22 --continue now also attempts to step one extra step back if necessary moses output is not found 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@754 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-15 15:16:28 +00:00
bojar
568cff8e34 fixed serious stupid bug: value ranges were ignored and min. and max were set to the starting value 
this bug occurred only if lambdas were supplied on command line, not with the default lambdas and ranges


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@753 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-15 15:04:19 +00:00
eherbst
1374aefc6d - fixed caching behavior of Corpus to remove gibberish and cache everything 
- fixed javascript sorting in sentence-by-sentence


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@735 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-14 22:18:54 +00:00
bojar
5c2d19a156 reversed exit codes of symal and added safesystem to call symal 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@730 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-14 17:54:11 +00:00
bojar
7735bc6b6d the python compiled files should not be in the cvs 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@729 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-14 17:53:23 +00:00
mfederico
f211a2a738 New version with c++ module (symal) performing step (3). 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@728 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-14 17:43:17 +00:00
bojar
0241f2fc5f better explanation in README, fixed test preparation in tests/train-factored-test-step3.test 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@727 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-14 17:36:36 +00:00
mfederico
f0a5eb167e Added a test to check step 3 of train-factored-models 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@726 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-14 16:20:40 +00:00
eherbst
87056b15a7 added my script to the docs 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@724 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-14 16:13:29 +00:00
eherbst
20f49a1ded fixed legend display 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@723 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-14 16:09:21 +00:00
mfederico
a1944e1985 Added symal stuff 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@719 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-14 15:18:58 +00:00
mfederico
6d6ac5c1e4 New version with faster computation of word alignments. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@718 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-14 15:18:12 +00:00
mfederico
c3ea1ef545 Filter to make GIZA++ alignment files more readable. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@717 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-14 15:06:58 +00:00
mfederico
e72010d6ce A tool to compute symmetric alignments from GIZA++ alignments. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@716 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-14 15:03:49 +00:00
bojar
840441dc1a die if phrase mismatch discovered 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@688 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-13 05:35:02 +00:00
bojar
f246845489 utf8 output 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@686 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-13 02:44:28 +00:00
bojar
6fc349f75f gives nice overview of model complexity (in terms of ambiguity in translation and generation tables) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@670 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-11 22:33:48 +00:00
bojar
e6914693a1 reports also the top N words 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@668 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-11 21:48:39 +00:00
bojar
8f504a1d9b a handy script to count words that passed through the decoder unchanged (mostly because they're unknown); can exclude numbers and punctuation 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@667 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-11 21:26:24 +00:00
callison-burch
fce87ded03 Removed the .pyc files that were preventing the command 'make release' from executing properly. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@658 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-11 18:16:01 +00:00
bojar
e1936af681 marking finished_step also after last iteration finished 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@655 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-11 17:05:04 +00:00
bojar
75194c441d just a typo 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@647 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-11 15:50:23 +00:00
bojar
68ef1413cd allows arbitrary mixing of 'kept' and 'added' factors in output 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@627 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-10 22:00:02 +00:00
bojar
b65eafacc6 die if no refs found, report also number of refs and sents used 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@622 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-10 18:53:12 +00:00
bojar
15566bb58a utf8, support for printing source, too 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@618 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-10 14:35:09 +00:00
bojar
9b23b6d9c8 die in safesystem on child's death 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@612 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-10 01:42:49 +00:00
bojar
af1be61259 die when there are no phrases in input 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@611 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-10 01:37:29 +00:00
bojar
3deea84ccb adding cvsignore to ignore python-compiled files 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@609 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-10 00:27:34 +00:00
swadey
683435e058 - updated bleu and score-nbest to allow optional bypass of NIST-style normalization 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@608 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-10 00:24:19 +00:00
phkoehn
0595062d7d fixed error message on scripts root dir 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@607 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-09 23:52:50 +00:00
redpony
0ea85deef7 fix off-by-one error in tables-score (prevents null characters from being inserted) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@595 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-09 19:37:14 +00:00
eherbst
cf8c271469 minor, and moved stuff around 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@588 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-08 23:38:45 +00:00
bojar
e97b542717 added --debug mode to training script to keep all intermediate files 
exit status of extract and score are 1 on error, not zero


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@585 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-08 22:28:26 +00:00
redpony
523527fa17 get rid of profiling 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@573 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-08 20:16:06 +00:00
redpony
db5a6bd11e fix bug that prevents | and _ from being tokenized properly. 
fix bug in --parallel


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@572 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-08 20:13:57 +00:00
bojar
81ddb0e4f9 added train-factored... to releases, added dependency on our copy of phrase-extract 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@569 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-08 19:22:53 +00:00
bojar
303f411387 simplified Makefile, removed duplicit implementation of tokenize() 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@568 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-08 19:04:59 +00:00
phkoehn
b83fc72dd2 initial version of phrase-extract and phrase-score used by training script 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@567 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-08 18:54:28 +00:00
bojar
264f045a6b fixing ensure_absolute 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@556 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-08 03:41:50 +00:00
bojar
0541ce3689 just cleanup of variable initialization 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@555 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-08 01:54:50 +00:00
bojar
5290653a4d Added reduce_combine to release 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@554 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-08 01:26:08 +00:00
eherbst
384f8ccb07 adding sentence-by-sentence.pl: display all sentences in a corpus, system output vs. reference 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@552 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-08 00:09:40 +00:00
bojar
64ec2e5ca4 checking in multi-bleu.perl 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@551 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-07 23:44:35 +00:00
bojar
ab5bb31797 allowing to override default paths 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@547 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-07 23:04:33 +00:00
bojar
26ce21f29b fixed unintended structure-sharing bug 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@541 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-07 22:17:02 +00:00
bojar
a41a4e95d6 now expects 3 numbers on [generation-file] lines before the pathname 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@538 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-07 21:55:06 +00:00
eherbst
0d91864621 adding scripts to extract POSs from LOPAR output and to extract arbitrary sets of factors from a corpus 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@530 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-07 17:35:16 +00:00
eherbst
8420ecf516 added statistical testing, both to compare different outputs and to get a confidence measure for a single output 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@529 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-07 17:22:39 +00:00
bojar
2d7cf749a6 Allowing scores in 'scientific' float format from moses. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@514 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-05 00:53:48 +00:00
bojar
10a0e23801 checking in reduce_combine 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@510 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-05 00:11:57 +00:00
bojar
a5c122dfc8 added mert to list of released files, make rules to release moses (personally or publicly) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@505 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-04 22:25:38 +00:00
bojar
12f50a5f26 Added labelling of scores in nbestlist and fixed mert to understand that. 
Before release, these have to be checked:
- train-factored-phrase-model.perl (the whole process)
- mert on newly generated moses.ini with 2 weights for generation


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@492 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-04 04:45:48 +00:00
redpony
7d0e0f5698 fix 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@491 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-03 21:10:18 +00:00
redpony
7b11b66b6d enable --parallel in tfpm.perl 
add a script to build a generation table from a monolingual corpus.
add a script to post-process the german morpho tagger output


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@490 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-03 21:05:55 +00:00
bojar
18dac34fe2 checking in the current version of cmert we're using 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@489 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-03 20:00:00 +00:00
bojar
232727e0e4 removed the dependence on external lowercaser, lowercasing internally 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@488 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-03 18:07:10 +00:00
bojar
c2fdfae2c1 modifying Makefile and released-files so that clean-n-corpus is properly released 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@487 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-03 17:55:56 +00:00
bojar
4d49e12bc4 checking the latest version from /export/bin to cvs 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@486 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-03 17:53:13 +00:00
nicolabertoldi
8b459e004a check in qsub-wrpper.pl with temporary log dir in the working dir 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@472 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-02 20:51:06 +00:00
nicolabertoldi
fac860e205 check in moses-parallel.pl with strict requirement 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@470 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-02 20:19:08 +00:00
bojar
32e73c3785 yet another clarification of messages 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@467 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-02 20:06:04 +00:00
bojar
763bb72642 clarification 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@460 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-02 12:21:17 +00:00
nicolabertoldi
8568c3beda Check in moses-parallel.pl with several bugs corrected 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@458 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-01 22:24:33 +00:00
bojar
60f9301ab7 Fixed matching of lambdas. (Back to the hardwired order.) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@457 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-01 22:14:12 +00:00
phkoehn
63a86828ba Added setting "distortion-limit=6" to moses.ini 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@455 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-01 21:11:17 +00:00
bojar
9304d71469 improved passing (and checking) of command-line options 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@446 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-01 15:18:49 +00:00
callison-burch
c0968b9041 Updated the script so that it correctly passes the qflags argument along to the qsub_wrapper script. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@445 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-01 14:29:08 +00:00
bojar
e59035efca Default to use only our team's queue. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@437 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-01 02:43:36 +00:00
bojar
5f3965de12 various tiny bugfixes 
added basic testcases
moved qsub-wrapper to generic


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@434 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-01 01:18:13 +00:00
nicolabertoldi
7910b65cdf Check in generic/moses-parallel.pl 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@431 1f5c12ca-751b-0410-a591-d2e778427230
 2006-07-31 23:01:02 +00:00
eherbst
54ab89deab seems this script does not have the same functionality as Ondrej's, and his are meant for training and this for analysis 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@430 1f5c12ca-751b-0410-a591-d2e778427230
 2006-07-31 22:14:08 +00:00
phkoehn
6f80f8c12a Speed-up of lexical translation table training, old code was crap 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@429 1f5c12ca-751b-0410-a591-d2e778427230
 2006-07-31 22:10:34 +00:00
eherbst
3b46c17ace believe Ondrej has a script w/same functionality; will investigate 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@428 1f5c12ca-751b-0410-a591-d2e778427230
 2006-07-31 22:07:34 +00:00
eherbst
5cce8336c0 add CGI-based tool for calculating and displaying various error measures 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@427 1f5c12ca-751b-0410-a591-d2e778427230
 2006-07-31 22:05:11 +00:00
bojar
75a5f9e935 clearer error message 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@425 1f5c12ca-751b-0410-a591-d2e778427230
 2006-07-31 21:33:54 +00:00
bojar
540aadea2b Allowing to optimize unknown lambdas, release methodology 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@421 1f5c12ca-751b-0410-a591-d2e778427230
 2006-07-31 21:01:07 +00:00
bojar
1c2cd47881 checking in the current version of clone_moses_model 
working on a single scripts directory


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@416 1f5c12ca-751b-0410-a591-d2e778427230
 2006-07-31 20:13:25 +00:00
nicolabertoldi
ba76013a5c *** empty log message *** 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@411 1f5c12ca-751b-0410-a591-d2e778427230
 2006-07-31 18:26:44 +00:00
bojar
a325df6380 renamed pythonpath variable, correctly passing --jobs, checking for blank moses.ini 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@410 1f5c12ca-751b-0410-a591-d2e778427230
 2006-07-31 17:00:22 +00:00
bojar
51ad454a39 checking in this useful script 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@409 1f5c12ca-751b-0410-a591-d2e778427230
 2006-07-31 16:53:40 +00:00
bojar
32853150fc added a placeholder 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@408 1f5c12ca-751b-0410-a591-d2e778427230
 2006-07-31 16:39:33 +00:00
bojar
57bcad0c5f the cleanup of mert-moses seems to be finished 
added first simple 'make release' goal


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@405 1f5c12ca-751b-0410-a591-d2e778427230
 2006-07-31 14:17:43 +00:00
bojar
54c6554d09 Removed the 'run-moses' functionality, so that the script is now usable by various variants of moses. (parallel and non parallel, mainly, but also by mert-moses.pl and others). 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@378 1f5c12ca-751b-0410-a591-d2e778427230
 2006-07-29 01:47:33 +00:00
bojar
9f4178e36e Added 
-rwxrwxr-x  1 pkoehn ws06osmt 5769 Jul 19 15:47 run-filtered-moses.perl
under a new name. Just for diffing purposes.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@377 1f5c12ca-751b-0410-a591-d2e778427230
 2006-07-29 01:45:14 +00:00
bojar
46ea34ab20 Merged the parallel and non-parallel copies of this script. 
Changed the command line and added some options.
Added extensive checking of validity of input files and options.
Still not ready for deployment due to the following bugs:
- the generation of output moses.ini was not tested
- the --start-step option does not work (not critical)


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@376 1f5c12ca-751b-0410-a591-d2e778427230
 2006-07-29 01:41:52 +00:00
bojar
05b0a07892 Checking in the last version of 
-rwxr-xr-x  1 nbertoldi ws06osmt 12430 Jul 28 00:01 mert-moses-parallel.perl-2006-07-27
Just for diff purposes.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@374 1f5c12ca-751b-0410-a591-d2e778427230
 2006-07-29 01:35:46 +00:00
bojar
6272fa6ecf Added the change in giza default options as done by ccb. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@373 1f5c12ca-751b-0410-a591-d2e778427230
 2006-07-29 01:18:58 +00:00
bojar
9061c682eb Checking in the version: 
-rwxrwxr-x  29 obojar ws06osmt 51861 Jul 24 18:15 train-factored-phrase-model.perl


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@372 1f5c12ca-751b-0410-a591-d2e778427230
 2006-07-29 01:13:30 +00:00
bojar
6188fa338d basis for a cleaner way of handling with our scripts 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@369 1f5c12ca-751b-0410-a591-d2e778427230
 2006-07-28 23:59:33 +00:00