moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2025-01-06 19:49:41 +03:00
Code Issues Packages Projects Releases Wiki Activity
1,445 Commits 136 Branches 26 Tags 147 MiB
616842f278
Commit Graph

198 Commits

Author SHA1 Message Date
nicolabertoldi
2075f9dda1 modification to mert script to allow the use of fewer nbest lists; features and scores are no more gzipped 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1965 1f5c12ca-751b-0410-a591-d2e778427230
 2008-12-30 17:33:16 +00:00
bojar
091c9ece28 raising line_max_length 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1953 1f5c12ca-751b-0410-a591-d2e778427230
 2008-12-05 10:01:05 +00:00
bojar
586d7e2f84 minor fix when handling gzipped corpora 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1952 1f5c12ca-751b-0410-a591-d2e778427230
 2008-12-04 17:49:59 +00:00
bojar
2c900c8bd7 uncompress input files for phrase extract, if needed 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1951 1f5c12ca-751b-0410-a591-d2e778427230
 2008-11-27 10:39:28 +00:00
bhaddow
1e13f6d2d6 Weights can sometimes be in exponential format. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1947 1f5c12ca-751b-0410-a591-d2e778427230
 2008-11-25 09:54:45 +00:00
hieuhoang1972
2807bc48ad absolute file name check, provided by Eric Kow 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1944 1f5c12ca-751b-0410-a591-d2e778427230
 2008-11-20 13:03:59 +00:00
hieuhoang1972
254284e57e patch to fix fiddly env variable and directory stuff, provided by Eric Kow@ 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1943 1f5c12ca-751b-0410-a591-d2e778427230
 2008-11-20 13:01:49 +00:00
phkoehn
abb2fc37b1 proper binarization of lexicalized reordering model 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1938 1f5c12ca-751b-0410-a591-d2e778427230
 2008-11-10 16:03:16 +00:00
phkoehn
bfbbefd710 bug fixes 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1917 1f5c12ca-751b-0410-a591-d2e778427230
 2008-10-26 03:32:29 +00:00
phkoehn
a09242ad16 bug fix with phrase table name in moses.ini, when using hmm alignment 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1913 1f5c12ca-751b-0410-a591-d2e778427230
 2008-10-20 21:37:57 +00:00
phkoehn
1c7b305152 bug fix 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1910 1f5c12ca-751b-0410-a591-d2e778427230
 2008-10-19 07:32:48 +00:00
phkoehn
1b5d99ad26 added headers for standard compliance (gcc 4.3 on 64 bit linux) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1905 1f5c12ca-751b-0410-a591-d2e778427230
 2008-10-16 21:14:38 +00:00
phkoehn
3a5981ce9d major improvements, see email to moses-support 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1904 1f5c12ca-751b-0410-a591-d2e778427230
 2008-10-15 23:25:14 +00:00
phkoehn
614876771d extended extract/score, to allow for one big file, not just parts 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1903 1f5c12ca-751b-0410-a591-d2e778427230
 2008-10-15 22:12:56 +00:00
jdschroeder
78534c1518 made all zcat calls through ZCAT variable. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1875 1f5c12ca-751b-0410-a591-d2e778427230
 2008-08-15 16:15:51 +00:00
jdschroeder
7a2ebedc20 minor bugfixes and error checking 
-added -rootdir option to enhanced-mert
	-fixed float regex in score-nbest.py and mert-moses.pl
	-allow for extra weights in constructing ini in mert-moses.pl
	-additional NFS bug checks in mert-moses.pl



git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1869 1f5c12ca-751b-0410-a591-d2e778427230
 2008-08-01 10:24:01 +00:00
bojar
6a087d59c4 removed SCRIPTS_ROOTDIR from this 'my' declaration, it was obscuring previous 
declaration!
lines wrapped


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1865 1f5c12ca-751b-0410-a591-d2e778427230
 2008-07-14 16:24:18 +00:00
bojar
c20e682f18 Avoid NFS race condition: 
explicitly remove old cmert output files (hoping that they will be correctly
  replaced by a 'mv' in the shell script submitted to SGE by qsubwrapper
  occasionally reveals a race condition in NFS => weights seem unchanged =>
  mert finishes too early)


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1862 1f5c12ca-751b-0410-a591-d2e778427230
 2008-07-10 11:47:55 +00:00
bhaddow
83f234cf17 Implementation of Cer et al mert regularisation. Use with argument such 
as --scconfig regtype:min,regwin:3 in extractor and mert. Only tested
on toy example so far.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1860 1f5c12ca-751b-0410-a591-d2e778427230
 2008-06-24 19:27:18 +00:00
hieuhoang1972
52c2843e6c perl regexpr bug, submitted by German Sanchis Trilles 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1855 1f5c12ca-751b-0410-a591-d2e778427230
 2008-06-19 21:57:29 +00:00
bhaddow
4195b70247 First cut of new mert outer loop 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1842 1f5c12ca-751b-0410-a591-d2e778427230
 2008-06-10 09:07:20 +00:00
hieuhoang1972
1b44c7c445 most popular alignment outputted, finally 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1818 1f5c12ca-751b-0410-a591-d2e778427230
 2008-06-04 14:49:56 +00:00
hieuhoang1972
8554a7c89d most popular alignment outputted, finally 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1817 1f5c12ca-751b-0410-a591-d2e778427230
 2008-06-04 14:42:51 +00:00
hieuhoang1972
3832f68fed most popular alignment outputted, finally 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1816 1f5c12ca-751b-0410-a591-d2e778427230
 2008-06-04 14:40:04 +00:00
hieuhoang1972
bf34eb891d don't output alignment if inverse 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1813 1f5c12ca-751b-0410-a591-d2e778427230
 2008-06-03 12:25:37 +00:00
hieuhoang1972
b48ce341e9 output most aligned instead of merged 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1798 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-28 13:49:04 +00:00
phkoehn
7498f469ab get scripts rootdir by FindBin 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1745 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-16 15:54:02 +00:00
hieuhoang1972
a2a3d33103 explicitly use bash 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1693 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-15 08:50:22 +00:00
hieuhoang1972
3fc0b8ddb4 git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1603 1f5c12ca-751b-0410-a591-d2e778427230 2008-05-04 12:52:52 +00:00
hieuhoang1972
a822d61d8f prevent -inf in lex re-ordering. Code contributed by Christian Hardmeier 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1596 1f5c12ca-751b-0410-a591-d2e778427230
 2008-04-18 09:04:38 +00:00
nicolabertoldi
1aff3d2382 correct handling of binary phrase tables 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1579 1f5c12ca-751b-0410-a591-d2e778427230
 2008-02-28 14:33:15 +00:00
nicolabertoldi
def0fff5cd changes to handle lattice input format 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1578 1f5c12ca-751b-0410-a591-d2e778427230
 2008-02-28 08:55:40 +00:00
hieuhoang1972
0bb92c2e79 merge properly 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1577 1f5c12ca-751b-0410-a591-d2e778427230
 2008-02-27 19:01:38 +00:00
hieuhoang1972
cb1f0e56dc optional output what lines are retained 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1576 1f5c12ca-751b-0410-a591-d2e778427230
 2008-02-27 18:38:31 +00:00
bojar
f056bdbfde fixed to correctly handle models in [distortion-file] section 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1572 1f5c12ca-751b-0410-a591-d2e778427230
 2008-02-26 10:24:53 +00:00
bojar
3957dc6b4c default to reordering factors of 0-0 even if decoding steps are set (users 
might have explicitly said e.g. t0-0!)


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1571 1f5c12ca-751b-0410-a591-d2e778427230
 2008-02-26 10:20:09 +00:00
bojar
f7a1fb5b9c corpus compression correctly used even for generation step 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1568 1f5c12ca-751b-0410-a591-d2e778427230
 2008-02-22 16:14:30 +00:00
bojar
8b3d44b2e2 SAFE_GETLINE made safer: will exit if the line does not fit into the buffer 
instead of just going on and getting the src/tgt/alignment files out of sync


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1565 1f5c12ca-751b-0410-a591-d2e778427230
 2008-02-22 14:42:01 +00:00
redpony
25750c6555 if giza returns sentences that have different lengths in different directions (due to truncation or other errors), don't silenty fail. print a blank line instead. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1562 1f5c12ca-751b-0410-a591-d2e778427230
 2008-02-19 20:48:14 +00:00
bojar
fa31d83421 even factors that are being added can be gzipped 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1561 1f5c12ca-751b-0410-a591-d2e778427230
 2008-02-19 17:32:51 +00:00
bojar
eec1bdb623 added support to open gzipped files 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1560 1f5c12ca-751b-0410-a591-d2e778427230
 2008-02-19 16:05:11 +00:00
nicolabertoldi
ae319da62b revert to /bin/sh for enhanced-mert; use of setenv (instead of export) in the csh scripts created by qsub-wrapper.pl 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1515 1f5c12ca-751b-0410-a591-d2e778427230
 2007-11-21 14:31:21 +00:00
nicolabertoldi
0176c5f8ec use fo csh instead of sh 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1514 1f5c12ca-751b-0410-a591-d2e778427230
 2007-11-21 07:59:17 +00:00
bojar
89ea9828ba added ttable iterator to this script, too 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1498 1f5c12ca-751b-0410-a591-d2e778427230
 2007-11-06 03:33:41 +00:00
nicolabertoldi
568f92b310 bug fixed 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1491 1f5c12ca-751b-0410-a591-d2e778427230
 2007-10-30 15:42:26 +00:00
jdschroeder
e52040bc12 added str length check to stop std::out_of_range error in a few more spots - similar bug to one corrected in v. 1319 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1488 1f5c12ca-751b-0410-a591-d2e778427230
 2007-10-26 12:42:55 +00:00
nicolabertoldi
fd3ecd4334 bug fixed 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1487 1f5c12ca-751b-0410-a591-d2e778427230
 2007-10-25 14:34:49 +00:00
nicolabertoldi
1b0576ba6c small bug fixed: temporary concatenated sorted file is now deleted only at the end 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1486 1f5c12ca-751b-0410-a591-d2e778427230
 2007-10-24 07:50:51 +00:00
nicolabertoldi
8710cc9bc9 features can be activated using a comma- or blank-separated list 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1485 1f5c12ca-751b-0410-a591-d2e778427230
 2007-10-23 16:55:02 +00:00
nicolabertoldi
9e70b5ffd8 Features are activated using their names 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1484 1f5c12ca-751b-0410-a591-d2e778427230
 2007-10-23 16:30:02 +00:00