moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2024-09-20 23:58:15 +03:00
Code Issues Packages Projects Releases Wiki Activity
1,557 Commits 136 Branches 26 Tags 147 MiB
55e3ee4a30
Commit Graph

320 Commits

Author SHA1 Message Date
bojar
2c900c8bd7 uncompress input files for phrase extract, if needed 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1951 1f5c12ca-751b-0410-a591-d2e778427230
 2008-11-27 10:39:28 +00:00
bhaddow
1e13f6d2d6 Weights can sometimes be in exponential format. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1947 1f5c12ca-751b-0410-a591-d2e778427230
 2008-11-25 09:54:45 +00:00
hieuhoang1972
2807bc48ad absolute file name check, provided by Eric Kow 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1944 1f5c12ca-751b-0410-a591-d2e778427230
 2008-11-20 13:03:59 +00:00
hieuhoang1972
254284e57e patch to fix fiddly env variable and directory stuff, provided by Eric Kow@ 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1943 1f5c12ca-751b-0410-a591-d2e778427230
 2008-11-20 13:01:49 +00:00
phkoehn
abb2fc37b1 proper binarization of lexicalized reordering model 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1938 1f5c12ca-751b-0410-a591-d2e778427230
 2008-11-10 16:03:16 +00:00
phkoehn
bfbbefd710 bug fixes 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1917 1f5c12ca-751b-0410-a591-d2e778427230
 2008-10-26 03:32:29 +00:00
mphi
8a4c6a2c63 pus significance test into proper location 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1915 1f5c12ca-751b-0410-a591-d2e778427230
 2008-10-23 09:16:33 +00:00
mphi
88d3b775ce altered the bootstrap significance script algorithm according to (Riezler and Maxwell 2005 @ MTSE'05) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1914 1f5c12ca-751b-0410-a591-d2e778427230
 2008-10-23 09:03:41 +00:00
phkoehn
a09242ad16 bug fix with phrase table name in moses.ini, when using hmm alignment 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1913 1f5c12ca-751b-0410-a591-d2e778427230
 2008-10-20 21:37:57 +00:00
mphi
f033e32979 Added implementation of Koehn's 2004 EMNLP paired bootsrap resampling 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1911 1f5c12ca-751b-0410-a591-d2e778427230
 2008-10-20 11:55:12 +00:00
phkoehn
1c7b305152 bug fix 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1910 1f5c12ca-751b-0410-a591-d2e778427230
 2008-10-19 07:32:48 +00:00
phkoehn
1b5d99ad26 added headers for standard compliance (gcc 4.3 on 64 bit linux) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1905 1f5c12ca-751b-0410-a591-d2e778427230
 2008-10-16 21:14:38 +00:00
phkoehn
3a5981ce9d major improvements, see email to moses-support 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1904 1f5c12ca-751b-0410-a591-d2e778427230
 2008-10-15 23:25:14 +00:00
phkoehn
614876771d extended extract/score, to allow for one big file, not just parts 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1903 1f5c12ca-751b-0410-a591-d2e778427230
 2008-10-15 22:12:56 +00:00
jdschroeder
78534c1518 made all zcat calls through ZCAT variable. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1875 1f5c12ca-751b-0410-a591-d2e778427230
 2008-08-15 16:15:51 +00:00
jdschroeder
7a2ebedc20 minor bugfixes and error checking 
-added -rootdir option to enhanced-mert
	-fixed float regex in score-nbest.py and mert-moses.pl
	-allow for extra weights in constructing ini in mert-moses.pl
	-additional NFS bug checks in mert-moses.pl



git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1869 1f5c12ca-751b-0410-a591-d2e778427230
 2008-08-01 10:24:01 +00:00
bojar
6a087d59c4 removed SCRIPTS_ROOTDIR from this 'my' declaration, it was obscuring previous 
declaration!
lines wrapped


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1865 1f5c12ca-751b-0410-a591-d2e778427230
 2008-07-14 16:24:18 +00:00
bojar
2afe9e0357 avoid coredump files in parallel moses (usually just kills NFS for a while), 
debug on a smaller scale, if needed


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1864 1f5c12ca-751b-0410-a591-d2e778427230
 2008-07-14 13:58:42 +00:00
bojar
c20e682f18 Avoid NFS race condition: 
explicitly remove old cmert output files (hoping that they will be correctly
  replaced by a 'mv' in the shell script submitted to SGE by qsubwrapper
  occasionally reveals a race condition in NFS => weights seem unchanged =>
  mert finishes too early)


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1862 1f5c12ca-751b-0410-a591-d2e778427230
 2008-07-10 11:47:55 +00:00
bhaddow
83f234cf17 Implementation of Cer et al mert regularisation. Use with argument such 
as --scconfig regtype:min,regwin:3 in extractor and mert. Only tested
on toy example so far.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1860 1f5c12ca-751b-0410-a591-d2e778427230
 2008-06-24 19:27:18 +00:00
hieuhoang1972
52c2843e6c perl regexpr bug, submitted by German Sanchis Trilles 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1855 1f5c12ca-751b-0410-a591-d2e778427230
 2008-06-19 21:57:29 +00:00
bhaddow
4195b70247 First cut of new mert outer loop 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1842 1f5c12ca-751b-0410-a591-d2e778427230
 2008-06-10 09:07:20 +00:00
hieuhoang1972
1b44c7c445 most popular alignment outputted, finally 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1818 1f5c12ca-751b-0410-a591-d2e778427230
 2008-06-04 14:49:56 +00:00
hieuhoang1972
8554a7c89d most popular alignment outputted, finally 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1817 1f5c12ca-751b-0410-a591-d2e778427230
 2008-06-04 14:42:51 +00:00
hieuhoang1972
3832f68fed most popular alignment outputted, finally 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1816 1f5c12ca-751b-0410-a591-d2e778427230
 2008-06-04 14:40:04 +00:00
hieuhoang1972
bf34eb891d don't output alignment if inverse 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1813 1f5c12ca-751b-0410-a591-d2e778427230
 2008-06-03 12:25:37 +00:00
hieuhoang1972
b48ce341e9 output most aligned instead of merged 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1798 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-28 13:49:04 +00:00
phkoehn
7498f469ab get scripts rootdir by FindBin 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1745 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-16 15:54:02 +00:00
hieuhoang1972
a2a3d33103 explicitly use bash 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1693 1f5c12ca-751b-0410-a591-d2e778427230
 2008-05-15 08:50:22 +00:00
hieuhoang1972
3fc0b8ddb4 git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1603 1f5c12ca-751b-0410-a591-d2e778427230 2008-05-04 12:52:52 +00:00
nicolabertoldi
def5f419c2 - handling of word graph generation in the parallel environment (-output-word-graph) 
- handling of '-' (i.e. /dev/stdout) for word graphs
- if either translations, nbests, searchgraphs or wordgraphs are output to stdout
  they are concatenated in this order
  BUT I STRONGLY RECOMMENT NO TO DO THAT


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1598 1f5c12ca-751b-0410-a591-d2e778427230
 2008-04-23 10:03:45 +00:00
nicolabertoldi
d514c277df - handling of search graph generation in the parallel environment (-output-search-graph) 
- modification of the parameter for nbest generation in the parallel environment:
  I make it similar to moses parameter (-n-best-list)
- handling of '-' (i.e. /dev/stdout) for nbest and search graphs


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1597 1f5c12ca-751b-0410-a591-d2e778427230
 2008-04-23 08:37:44 +00:00
hieuhoang1972
a822d61d8f prevent -inf in lex re-ordering. Code contributed by Christian Hardmeier 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1596 1f5c12ca-751b-0410-a591-d2e778427230
 2008-04-18 09:04:38 +00:00
nicolabertoldi
1aff3d2382 correct handling of binary phrase tables 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1579 1f5c12ca-751b-0410-a591-d2e778427230
 2008-02-28 14:33:15 +00:00
nicolabertoldi
def0fff5cd changes to handle lattice input format 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1578 1f5c12ca-751b-0410-a591-d2e778427230
 2008-02-28 08:55:40 +00:00
hieuhoang1972
0bb92c2e79 merge properly 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1577 1f5c12ca-751b-0410-a591-d2e778427230
 2008-02-27 19:01:38 +00:00
hieuhoang1972
cb1f0e56dc optional output what lines are retained 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1576 1f5c12ca-751b-0410-a591-d2e778427230
 2008-02-27 18:38:31 +00:00
bojar
f056bdbfde fixed to correctly handle models in [distortion-file] section 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1572 1f5c12ca-751b-0410-a591-d2e778427230
 2008-02-26 10:24:53 +00:00
bojar
3957dc6b4c default to reordering factors of 0-0 even if decoding steps are set (users 
might have explicitly said e.g. t0-0!)


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1571 1f5c12ca-751b-0410-a591-d2e778427230
 2008-02-26 10:20:09 +00:00
hieuhoang1972
cced54cf7d win32 fix provided by jc read 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1569 1f5c12ca-751b-0410-a591-d2e778427230
 2008-02-22 21:47:32 +00:00
bojar
f7a1fb5b9c corpus compression correctly used even for generation step 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1568 1f5c12ca-751b-0410-a591-d2e778427230
 2008-02-22 16:14:30 +00:00
bojar
7f3e34207a added some heuristics for Czech quotation marks 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1567 1f5c12ca-751b-0410-a591-d2e778427230
 2008-02-22 15:07:46 +00:00
bojar
6af3140978 added optional sentence uppercasing (use -u) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1566 1f5c12ca-751b-0410-a591-d2e778427230
 2008-02-22 14:50:43 +00:00
bojar
8b3d44b2e2 SAFE_GETLINE made safer: will exit if the line does not fit into the buffer 
instead of just going on and getting the src/tgt/alignment files out of sync


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1565 1f5c12ca-751b-0410-a591-d2e778427230
 2008-02-22 14:42:01 +00:00
bojar
f89ab590ec added Nicola's enhancedmert to released files 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1564 1f5c12ca-751b-0410-a591-d2e778427230
 2008-02-22 13:30:05 +00:00
redpony
25750c6555 if giza returns sentences that have different lengths in different directions (due to truncation or other errors), don't silenty fail. print a blank line instead. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1562 1f5c12ca-751b-0410-a591-d2e778427230
 2008-02-19 20:48:14 +00:00
bojar
fa31d83421 even factors that are being added can be gzipped 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1561 1f5c12ca-751b-0410-a591-d2e778427230
 2008-02-19 17:32:51 +00:00
bojar
eec1bdb623 added support to open gzipped files 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1560 1f5c12ca-751b-0410-a591-d2e778427230
 2008-02-19 16:05:11 +00:00
nicolabertoldi
ae319da62b revert to /bin/sh for enhanced-mert; use of setenv (instead of export) in the csh scripts created by qsub-wrapper.pl 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1515 1f5c12ca-751b-0410-a591-d2e778427230
 2007-11-21 14:31:21 +00:00
nicolabertoldi
0176c5f8ec use fo csh instead of sh 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1514 1f5c12ca-751b-0410-a591-d2e778427230
 2007-11-21 07:59:17 +00:00
bojar
89ea9828ba added ttable iterator to this script, too 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1498 1f5c12ca-751b-0410-a591-d2e778427230
 2007-11-06 03:33:41 +00:00
bojar
09d8b5e657 improving documentation and allowing environment variables to override the 
default paths


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1497 1f5c12ca-751b-0410-a591-d2e778427230
 2007-11-06 03:16:17 +00:00
nicolabertoldi
568f92b310 bug fixed 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1491 1f5c12ca-751b-0410-a591-d2e778427230
 2007-10-30 15:42:26 +00:00
jdschroeder
e52040bc12 added str length check to stop std::out_of_range error in a few more spots - similar bug to one corrected in v. 1319 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1488 1f5c12ca-751b-0410-a591-d2e778427230
 2007-10-26 12:42:55 +00:00
nicolabertoldi
fd3ecd4334 bug fixed 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1487 1f5c12ca-751b-0410-a591-d2e778427230
 2007-10-25 14:34:49 +00:00
nicolabertoldi
1b0576ba6c small bug fixed: temporary concatenated sorted file is now deleted only at the end 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1486 1f5c12ca-751b-0410-a591-d2e778427230
 2007-10-24 07:50:51 +00:00
nicolabertoldi
8710cc9bc9 features can be activated using a comma- or blank-separated list 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1485 1f5c12ca-751b-0410-a591-d2e778427230
 2007-10-23 16:55:02 +00:00
nicolabertoldi
9e70b5ffd8 Features are activated using their names 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1484 1f5c12ca-751b-0410-a591-d2e778427230
 2007-10-23 16:30:02 +00:00
nicolabertoldi
8fe62f2b95 some small bugs fixed and clean up 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1483 1f5c12ca-751b-0410-a591-d2e778427230
 2007-10-23 14:15:19 +00:00
nicolabertoldi
4720d1cb9f bug fixed 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1482 1f5c12ca-751b-0410-a591-d2e778427230
 2007-10-22 17:01:53 +00:00
nicolabertoldi
918dae011a bug fixed 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1481 1f5c12ca-751b-0410-a591-d2e778427230
 2007-10-22 14:00:10 +00:00
nicolabertoldi
e7ac20d4d6 bug fixed in the name of a temporary file 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1480 1f5c12ca-751b-0410-a591-d2e778427230
 2007-10-22 13:34:12 +00:00
nicolabertoldi
db9d0fc539 Added a more time-efficient (but more memory-consumptive) method to rescore nbest list 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1479 1f5c12ca-751b-0410-a591-d2e778427230
 2007-10-22 10:08:07 +00:00
nicolabertoldi
b827d51870 changes to cope with the new mert suite (enhanced-mert) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1478 1f5c12ca-751b-0410-a591-d2e778427230
 2007-10-22 09:45:19 +00:00
jdschroeder
a969197e16 Fixed passing decoder parameters when tuning on single machine. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1477 1f5c12ca-751b-0410-a591-d2e778427230
 2007-10-22 09:27:09 +00:00
nicolabertoldi
5759005857 Suite of scripts to perform MERT on a subset of fetures. 
Look at the directory example to learn about its use.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1476 1f5c12ca-751b-0410-a591-d2e778427230
 2007-10-22 09:04:03 +00:00
redpony
0cf583e249 add --hmm-align option. Allows using Giza++'s HMM word alignment model as the underlying word alignment. It is much faster than Model 4 alignment and not much worse. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1474 1f5c12ca-751b-0410-a591-d2e778427230
 2007-10-19 03:44:05 +00:00
nicolabertoldi
901823d83a explicit export of PYTHONPATH variable 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1473 1f5c12ca-751b-0410-a591-d2e778427230
 2007-10-18 15:33:04 +00:00
nicolabertoldi
81b439d728 minor changes in passing parameters to moses-parallel 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1472 1f5c12ca-751b-0410-a591-d2e778427230
 2007-10-18 12:36:49 +00:00
hieuhoang1972
4e1cad4bbe fixed sync/async bug 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1471 1f5c12ca-751b-0410-a591-d2e778427230
 2007-10-04 17:31:31 +00:00
redpony
57dcaa8e80 performance fixes for scorer 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1470 1f5c12ca-751b-0410-a591-d2e778427230
 2007-10-02 21:43:54 +00:00
hieuhoang1972
9cbc2922b4 separate word penalty for each decode step for async 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1469 1f5c12ca-751b-0410-a591-d2e778427230
 2007-10-02 12:52:00 +00:00
hieuhoang1972
d2d03c33e7 fixed bug which prevented mert working when phrase table NOT filtered or binarised 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1449 1f5c12ca-751b-0410-a591-d2e778427230
 2007-08-10 15:48:58 +00:00
hieuhoang1972
53fa2cb18a async - don't use binarising or filtering 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1443 1f5c12ca-751b-0410-a591-d2e778427230
 2007-08-05 20:29:46 +00:00
hieuhoang1972
9eba034662 turn off debugging 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1435 1f5c12ca-751b-0410-a591-d2e778427230
 2007-07-25 10:05:34 +00:00
hieuhoang1972
2beb0c44e9 mkdir before doing generation 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1425 1f5c12ca-751b-0410-a591-d2e778427230
 2007-07-14 09:54:05 +00:00
nicolabertoldi
75afdf04a5 I corrected direction of alignment 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1421 1f5c12ca-751b-0410-a591-d2e778427230
 2007-06-27 17:57:55 +00:00
nicolabertoldi
ac91cb78cc two additional (and simpler) ways of extracting alignments: source-to-target and target-to-source 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1420 1f5c12ca-751b-0410-a591-d2e778427230
 2007-06-27 16:34:14 +00:00
nicolabertoldi
7f9c2856c2 changes to reduce disk memory consumption during training 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1419 1f5c12ca-751b-0410-a591-d2e778427230
 2007-06-27 16:30:20 +00:00
phkoehn
960bebdd4a fixed clean script to handle '|'s 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1416 1f5c12ca-751b-0410-a591-d2e778427230
 2007-06-18 15:50:04 +00:00
redpony
c747cdd505 fix dumb error 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1414 1f5c12ca-751b-0410-a591-d2e778427230
 2007-06-01 16:27:56 +00:00
redpony
1f050e198a fix compile error, enable optimizations 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1413 1f5c12ca-751b-0410-a591-d2e778427230
 2007-06-01 16:26:26 +00:00
redpony
564bb5a64e make scorer use compiler optimization 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1412 1f5c12ca-751b-0410-a591-d2e778427230
 2007-06-01 16:22:30 +00:00
hieuhoang1972
aa25c7341d fixed bug with non-ascii data, recieved from Jaakko Väyrynen 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1392 1f5c12ca-751b-0410-a591-d2e778427230
 2007-05-21 13:06:40 +00:00
bojar
74954cb0ae prefer hardlinking, dropped dependency on a proprietary script 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1381 1f5c12ca-751b-0410-a591-d2e778427230
 2007-05-09 00:54:07 +00:00
bojar
31def05428 - added a comment where the binarizer is 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1380 1f5c12ca-751b-0410-a591-d2e778427230
 2007-05-07 07:19:02 +00:00
abarun
ba90a05233 Added script to perform Minimum Bayes Risk reranking 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1372 1f5c12ca-751b-0410-a591-d2e778427230
 2007-05-02 17:26:01 +00:00
hieuhoang1972
13e07cef5f multiple distance based distortion for async decoder 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1370 1f5c12ca-751b-0410-a591-d2e778427230
 2007-04-23 19:17:38 +00:00
redpony
485bda2db5 andreas zollman's changes to write span information 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1367 1f5c12ca-751b-0410-a591-d2e778427230
 2007-04-20 14:53:46 +00:00
jdschroeder
3e1aabc487 Removed a few errant svn diff lines that found their way into the file. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1366 1f5c12ca-751b-0410-a591-d2e778427230
 2007-04-19 10:52:17 +00:00
jdschroeder
752d148c6e Changed initial setting of number of distortion weights from 0 to 1. For models with lexicalized reordering, this script was generating one too few weights in moses.ini 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1365 1f5c12ca-751b-0410-a591-d2e778427230
 2007-04-18 19:38:14 +00:00
redpony
c80d8b8d47 Support for the decoding of arbitrary word lattices. Must be given in the form of a "plf" file, which is a little tricky. I'll add documentation at some point; for now, refer to the example plf file in the "lattice-surface" regression test. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1359 1f5c12ca-751b-0410-a591-d2e778427230
 2007-04-18 14:08:46 +00:00
hieuhoang1972
45dde20c54 comment out psyco library 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1354 1f5c12ca-751b-0410-a591-d2e778427230
 2007-04-12 17:48:48 +00:00
hieuhoang1972
75c20e7609 Add alignment info to phrase table 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1352 1f5c12ca-751b-0410-a591-d2e778427230
 2007-04-11 19:58:38 +00:00
hieuhoang1972
b84191c9d3 Add alignment info to phrase table 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1351 1f5c12ca-751b-0410-a591-d2e778427230
 2007-04-11 19:56:43 +00:00
hieuhoang1972
b9d2288c22 compileable with visual studio 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1349 1f5c12ca-751b-0410-a591-d2e778427230
 2007-04-11 11:35:36 +00:00
hieuhoang1972
e868986885 compileable with visual studio 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1348 1f5c12ca-751b-0410-a591-d2e778427230
 2007-04-11 11:18:37 +00:00
jdschroeder
84d12552c2 added additional numbered count to output phrase table names so multiple phrase tables can be filtered and used at the same time 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1337 1f5c12ca-751b-0410-a591-d2e778427230
 2007-04-04 17:54:12 +00:00
jdschroeder
667e85264a added LC_ALL=C call and temp directory specification to sort command, hoping to minimize failed sorts crashing processPhraseTable 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1336 1f5c12ca-751b-0410-a591-d2e778427230
 2007-04-04 17:07:16 +00:00
jdschroeder
04ae9361d2 added "-v 0" moses flag to decoder call to minimize log output. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1335 1f5c12ca-751b-0410-a591-d2e778427230
 2007-04-04 17:04:50 +00:00