moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2024-10-27 03:49:57 +03:00
Code Issues Packages Projects Releases Wiki Activity
1,460 Commits 136 Branches 26 Tags 147 MiB
cc95706045
Commit Graph

1460 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
jdschroeder
cc95706045 mert-moses.pl now supports multiple input weights for lattices and confusion networks, using the --inputweights argument. 
I'll leave it to someone who knows mert-moses-new.pl better to make the changes there.

"zcat" is now abstracted as a $ZCAT variable in these files, and is set to "gzip -cd" which should work on more platforms (notably on the mac, where zcat fails unless an archive name ends in ".Z").

 


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2082 1f5c12ca-751b-0410-a591-d2e778427230
 2009-02-05 17:39:36 +00:00
jdschroeder
e53ab5da6d Support for multiple input feature scores on confusion networks and lattices. 
Use "link-param-count" to tell Moses how many to expect in the input.
If weight-i (I) is one more than link-param-count, a feature for non-null word count will be added (this has actually always been there, but only for the 1 param, 2 weights scenario).
Input feature scores are now preserved for unknown words.

Unknown word penalty weight is now tunable with -weight-u (u), default is 1, as was hard-coded before.

Changes to mert-moses.pl will be checked in shortly.




git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2081 1f5c12ca-751b-0410-a591-d2e778427230
 2009-02-05 17:37:09 +00:00
bhaddow
8fc1c1b95e Fix loading of gzipped phrase tables 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2073 1f5c12ca-751b-0410-a591-d2e778427230
 2009-02-04 11:09:38 +00:00
dowobeha
324393afe7 Allow constraint file without tabs 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1999 1f5c12ca-751b-0410-a591-d2e778427230
 2009-01-26 16:14:38 +00:00
phkoehn
98381c0193 fixed xml removal 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1995 1f5c12ca-751b-0410-a591-d2e778427230
 2009-01-24 05:21:36 +00:00
redpony
e61b9da9f7 better example 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1994 1f5c12ca-751b-0410-a591-d2e778427230
 2009-01-22 21:32:39 +00:00
redpony
3f7f12f4ad add client for remote language model 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1993 1f5c12ca-751b-0410-a591-d2e778427230
 2009-01-22 21:31:17 +00:00
redpony
e923c82cf5 add another example 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1992 1f5c12ca-751b-0410-a591-d2e778427230
 2009-01-22 17:58:45 +00:00
redpony
f067a6cf1d add missing file 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1991 1f5c12ca-751b-0410-a591-d2e778427230
 2009-01-22 17:52:22 +00:00
redpony
3172abca21 check in code for remote LM-server 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1990 1f5c12ca-751b-0410-a591-d2e778427230
 2009-01-22 17:50:51 +00:00
bhaddow
6c8c8e9dc4 initial weights for toy example 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1989 1f5c12ca-751b-0410-a591-d2e778427230
 2009-01-22 10:25:14 +00:00
hieuhoang1972
f076b03c10 conf net fix 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1981 1f5c12ca-751b-0410-a591-d2e778427230
 2009-01-15 18:12:53 +00:00
hieuhoang1972
5161b380d5 regress 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1980 1f5c12ca-751b-0410-a591-d2e778427230
 2009-01-14 13:55:54 +00:00
saintamh
aeb93ec23e added the tokenizer scripts that were distributed for the Marathon last year - translate.cgi needs them and it simplifies distribution to have them here 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1979 1f5c12ca-751b-0410-a591-d2e778427230
 2009-01-12 22:36:56 +00:00
phkoehn
f9be34dd35 fixed bug in zones 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1972 1f5c12ca-751b-0410-a591-d2e778427230
 2009-01-08 17:46:29 +00:00
phkoehn
616842f278 fixed multi-bleu documentation 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1971 1f5c12ca-751b-0410-a591-d2e778427230
 2009-01-08 00:47:10 +00:00
nicolabertoldi
830d9f3404 small change to reduce few useless computations 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1970 1f5c12ca-751b-0410-a591-d2e778427230
 2009-01-07 13:47:38 +00:00
nicolabertoldi
4b4c1b3973 imported utilities for timing from Moses 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1969 1f5c12ca-751b-0410-a591-d2e778427230
 2009-01-07 13:30:06 +00:00
phkoehn
4ff372a6d6 fixed compile problem on i686-64 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1967 1f5c12ca-751b-0410-a591-d2e778427230
 2009-01-01 18:25:06 +00:00
phkoehn
7507373b84 enable use of both edt and sd; reduction of translation option cache 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1966 1f5c12ca-751b-0410-a591-d2e778427230
 2009-01-01 18:16:54 +00:00
nicolabertoldi
2075f9dda1 modification to mert script to allow the use of fewer nbest lists; features and scores are no more gzipped 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1965 1f5c12ca-751b-0410-a591-d2e778427230
 2008-12-30 17:33:16 +00:00
phkoehn
5e66cd48f0 faster early discarding, code cleanup and documentation 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1964 1f5c12ca-751b-0410-a591-d2e778427230
 2008-12-20 20:22:35 +00:00
jdschroeder
83cc7f6476 added SquareMatrix to Makefile.am 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1963 1f5c12ca-751b-0410-a591-d2e778427230
 2008-12-17 16:33:59 +00:00
hieuhoang1972
beaf1856cd vs build 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1962 1f5c12ca-751b-0410-a591-d2e778427230
 2008-12-17 14:07:17 +00:00
phkoehn
813fb5cf81 big fix in early discarding, requiring moving CalcFutureCost into 
SquareMatrix


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1961 1f5c12ca-751b-0410-a591-d2e778427230
 2008-12-17 12:32:52 +00:00
phkoehn
a360b71426 initial version of reordering zones and walls, may work 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1960 1f5c12ca-751b-0410-a591-d2e778427230
 2008-12-15 12:52:38 +00:00
phkoehn
e13e45dc63 improved early discarding, ifverbose(2) time tracking 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1959 1f5c12ca-751b-0410-a591-d2e778427230
 2008-12-14 13:23:33 +00:00
jdschroeder
3aadf33bef Fixed XML bug where TranslationOptions generated by XML had null for source phrases. TODO: drop <linked> XML tags feature? 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1958 1f5c12ca-751b-0410-a591-d2e778427230
 2008-12-13 13:12:54 +00:00
phkoehn
8caf6053da ooops 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1957 1f5c12ca-751b-0410-a591-d2e778427230
 2008-12-13 12:18:45 +00:00
phkoehn
11139a364d improvements to pruning, working version passed regression 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1956 1f5c12ca-751b-0410-a591-d2e778427230
 2008-12-13 12:08:55 +00:00
abarun
3cc8305efa Modified xcode project to make it work with both sri and irst lms 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1955 1f5c12ca-751b-0410-a591-d2e778427230
 2008-12-11 19:27:59 +00:00
hieuhoang1972
69b2dc441d eclipse files 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1954 1f5c12ca-751b-0410-a591-d2e778427230
 2008-12-10 14:20:10 +00:00
bojar
091c9ece28 raising line_max_length 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1953 1f5c12ca-751b-0410-a591-d2e778427230
 2008-12-05 10:01:05 +00:00
bojar
586d7e2f84 minor fix when handling gzipped corpora 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1952 1f5c12ca-751b-0410-a591-d2e778427230
 2008-12-04 17:49:59 +00:00
bojar
2c900c8bd7 uncompress input files for phrase extract, if needed 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1951 1f5c12ca-751b-0410-a591-d2e778427230
 2008-11-27 10:39:28 +00:00
xandfraser
7268b31292 Added include so ReorderingConstraint.h compiles 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1950 1f5c12ca-751b-0410-a591-d2e778427230
 2008-11-26 19:07:39 +00:00
hieuhoang1972
8ef9fcc9f8 allow no LM 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1949 1f5c12ca-751b-0410-a591-d2e778427230
 2008-11-25 20:20:40 +00:00
phkoehn
94d18dd219 needed for -monotone-at-punctuation 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1948 1f5c12ca-751b-0410-a591-d2e778427230
 2008-11-25 19:29:47 +00:00
bhaddow
1e13f6d2d6 Weights can sometimes be in exponential format. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1947 1f5c12ca-751b-0410-a591-d2e778427230
 2008-11-25 09:54:45 +00:00
phkoehn
b2818633f4 that one was missing 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1946 1f5c12ca-751b-0410-a591-d2e778427230
 2008-11-24 17:31:26 +00:00
phkoehn
a0edbf1522 added -monotone-at-punctuation option 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1945 1f5c12ca-751b-0410-a591-d2e778427230
 2008-11-24 17:30:37 +00:00
hieuhoang1972
2807bc48ad absolute file name check, provided by Eric Kow 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1944 1f5c12ca-751b-0410-a591-d2e778427230
 2008-11-20 13:03:59 +00:00
hieuhoang1972
254284e57e patch to fix fiddly env variable and directory stuff, provided by Eric Kow@ 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1943 1f5c12ca-751b-0410-a591-d2e778427230
 2008-11-20 13:01:49 +00:00
nicolabertoldi
32029561da mert can now load more data files 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1942 1f5c12ca-751b-0410-a591-d2e778427230
 2008-11-18 18:51:02 +00:00
jdschroeder
6eab281ce6 CreateAlignmentInfo is now only called when UseAlignmentInfo is true. Shrinks phrase table memory size when extra data is not needed. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1941 1f5c12ca-751b-0410-a591-d2e778427230
 2008-11-13 17:43:15 +00:00
hieuhoang1972
102856bf84 init m_sourceStartPosMattersForRecombination 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1940 1f5c12ca-751b-0410-a591-d2e778427230
 2008-11-12 20:26:02 +00:00
hieuhoang1972
d342a5e6d8 add interal language model into makefile build 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1939 1f5c12ca-751b-0410-a591-d2e778427230
 2008-11-12 19:57:28 +00:00
phkoehn
abb2fc37b1 proper binarization of lexicalized reordering model 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1938 1f5c12ca-751b-0410-a591-d2e778427230
 2008-11-10 16:03:16 +00:00
bhaddow
ec68b5235f Fix minor bugs in randlm and irstlm config 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1937 1f5c12ca-751b-0410-a591-d2e778427230
 2008-11-05 18:06:14 +00:00
hieuhoang1972
789d6d96d1 intergrate randlm 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1935 1f5c12ca-751b-0410-a591-d2e778427230
 2008-11-04 18:03:03 +00:00