moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2025-01-06 19:49:41 +03:00
Code Issues Packages Projects Releases Wiki Activity
1,494 Commits 136 Branches 26 Tags 147 MiB
9b18ec4a29
Commit Graph

1494 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
hieuhoang1972
9b18ec4a29 add release files 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2527 1f5c12ca-751b-0410-a591-d2e778427230
 2009-08-25 10:13:47 +00:00
hieuhoang1972
ef3ed4ec19 xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2506 1f5c12ca-751b-0410-a591-d2e778427230
 2009-08-16 19:41:34 +00:00
bhaddow
4337446631 missing includes 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2486 1f5c12ca-751b-0410-a591-d2e778427230
 2009-08-11 09:37:29 +00:00
bhaddow
c5d39f039f Merge moses-server branch (includes mt moses) into trunk. 
Plain (single-thread) moses should configure and build as before.
Multi-thread and server only available if appropriate options are selected
at configure/compile time.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2477 1f5c12ca-751b-0410-a591-d2e778427230
 2009-08-07 16:47:54 +00:00
nicolabertoldi
f75e3993ac correction of parameter description 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2462 1f5c12ca-751b-0410-a591-d2e778427230
 2009-08-05 16:54:33 +00:00
nicolabertoldi
8384857f8c changes to mert-moses-new.pl to work with different reference length policies: shortest, average, closest (either "--shortest", "--average", or "--closest ) for BLEU and with case-sensitive/insensitive evaluation ( --nocase) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2461 1f5c12ca-751b-0410-a591-d2e778427230
 2009-08-05 16:39:06 +00:00
nicolabertoldi
0393183eb4 mert software now works with different reference length policies: shortest, average, closest (default) and with case information (default is preserving case). Pay attention that both defaults are different from the previous version (which were shortest reflen and case-insensitive). 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2459 1f5c12ca-751b-0410-a591-d2e778427230
 2009-08-05 15:38:35 +00:00
hieuhoang1972
c50596d1e1 includes suggested by Tom Hoar's mate 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2399 1f5c12ca-751b-0410-a591-d2e778427230
 2009-07-23 10:29:30 +00:00
chardmeier
c5de015ce0 Fixed format of -v 3 output. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2390 1f5c12ca-751b-0410-a591-d2e778427230
 2009-07-20 15:18:46 +00:00
chardmeier
a0219c2160 Fixed crash with word lattices and verbosity level 3. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2389 1f5c12ca-751b-0410-a591-d2e778427230
 2009-07-20 15:14:51 +00:00
chardmeier
5d785ef86f Fixed language model computation for unigram models. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2386 1f5c12ca-751b-0410-a591-d2e778427230
 2009-07-20 13:56:37 +00:00
hieuhoang1972
efc3bcb392 gcc 4.4 (fedora 11) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2379 1f5c12ca-751b-0410-a591-d2e778427230
 2009-07-14 17:16:48 +00:00
hieuhoang1972
6f84048261 gui 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2352 1f5c12ca-751b-0410-a591-d2e778427230
 2009-05-31 20:32:18 +00:00
hieuhoang1972
7d17d823ae xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2351 1f5c12ca-751b-0410-a591-d2e778427230
 2009-05-31 15:56:45 +00:00
hieuhoang1972
568e973b2e make gcc amd make calls consistent for eric to use in ubuntu package 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2347 1f5c12ca-751b-0410-a591-d2e778427230
 2009-05-27 12:54:04 +00:00
phkoehn
a48931dc65 experimental version of global lexical model 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2344 1f5c12ca-751b-0410-a591-d2e778427230
 2009-05-26 19:31:32 +00:00
phkoehn
8833098925 generalized n-best list reporting for feature functions, added experimental version of global lexical model 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2343 1f5c12ca-751b-0410-a591-d2e778427230
 2009-05-26 19:30:35 +00:00
mphi
17c3cfffac added unpaired significance evaluation 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2328 1f5c12ca-751b-0410-a591-d2e778427230
 2009-05-12 18:56:01 +00:00
hieuhoang1972
625d442f6d xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2326 1f5c12ca-751b-0410-a591-d2e778427230
 2009-05-12 16:54:31 +00:00
bhaddow
981e440cc2 Fix detection of binarised reordering table 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2240 1f5c12ca-751b-0410-a591-d2e778427230
 2009-03-13 12:28:34 +00:00
redpony
558c723c7c fix for one round of problems- this moses will work with g++ > 4.3 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2236 1f5c12ca-751b-0410-a591-d2e778427230
 2009-03-12 16:52:27 +00:00
hieuhoang1972
a8dbe9bf56 gcc runtime error 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2229 1f5c12ca-751b-0410-a591-d2e778427230
 2009-03-04 11:45:34 +00:00
redpony
17fc97f6dd fix bug for monotone decoding 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2226 1f5c12ca-751b-0410-a591-d2e778427230
 2009-02-26 20:12:09 +00:00
redpony
4d72e4243a fix reordering bug 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2224 1f5c12ca-751b-0410-a591-d2e778427230
 2009-02-26 16:05:41 +00:00
bhaddow
1ef0f3844d Connect new mert to the moses build system 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2221 1f5c12ca-751b-0410-a591-d2e778427230
 2009-02-26 11:17:12 +00:00
bhaddow
e1d7bb986c Add option for predictable seeding 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2220 1f5c12ca-751b-0410-a591-d2e778427230
 2009-02-25 19:31:17 +00:00
redpony
652408ccfb more recommendations 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2219 1f5c12ca-751b-0410-a591-d2e778427230
 2009-02-25 17:28:38 +00:00
redpony
bef4b1b08c fixes for sigtest filter 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2217 1f5c12ca-751b-0410-a591-d2e778427230
 2009-02-25 15:48:29 +00:00
hieuhoang1972
833f3b9c25 print score indices 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2181 1f5c12ca-751b-0410-a591-d2e778427230
 2009-02-18 11:46:20 +00:00
hieuhoang1972
c2da0faf05 vs build 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2180 1f5c12ca-751b-0410-a591-d2e778427230
 2009-02-18 11:35:41 +00:00
bhaddow
bb9e2f427d Fix random restarts so that they are actually random, using the system clock 
for seeding.

Previously, new mert followed old mert in using the same set of "random" points 
for every iteration of the outer loop.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2126 1f5c12ca-751b-0410-a591-d2e778427230
 2009-02-12 12:58:27 +00:00
phkoehn
8d5aef137b bug fix 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2113 1f5c12ca-751b-0410-a591-d2e778427230
 2009-02-09 16:00:35 +00:00
phkoehn
a62f8ee316 added truecaser 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2112 1f5c12ca-751b-0410-a591-d2e778427230
 2009-02-09 15:32:34 +00:00
redpony
63effe85b5 Feature function overhaul. Each feature function is computed in one of three ways: 
1) Stateless feature functions from the phrase table/generation table: these are computed when
   the TranslationOption is created.  They become part of the ScoreBreakdown object contained in
   the TranslationOption and are added to the feature value vector when a hypothesis is extended.
2) Stateless feature functions that are computed during state exploration. Currently, only
   WordPenalty falls into this category, but these functions implement a method Evaluate which
   do does not receive a Hypothesis or any contextual information.
3) Stateful feature functions: these features receive the arc information (translation option),
   compute some value and then return some context information.  The context information created
   by a particular feature function is passed back to it as the previous context when a hypothesis
   originating at the node where the previous edge terminates is created.  States in the search
   space may be recombined if the context information is identical.  The context information must
   be stored in an object implementing the FFState interface.

TODO:
1) the command line interface / MERT interface needs to go to named parameters that are otherwise opaque
2) StatefulFeatureFunction's Evaluate method should just take a TranslationOption and a context object.  It is not good that it takes a hypothesis, because then people may be tempted to access information about the "previous" hypothesis without "declaring" this dependency.
3) Future cost estimates should be handled using feature functions.  All stateful feature functions need some kind of future cost estimate.
4) Philipp's poor-man's cube pruning is broken.



git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2087 1f5c12ca-751b-0410-a591-d2e778427230
 2009-02-06 15:43:06 +00:00
jdschroeder
cc95706045 mert-moses.pl now supports multiple input weights for lattices and confusion networks, using the --inputweights argument. 
I'll leave it to someone who knows mert-moses-new.pl better to make the changes there.

"zcat" is now abstracted as a $ZCAT variable in these files, and is set to "gzip -cd" which should work on more platforms (notably on the mac, where zcat fails unless an archive name ends in ".Z").

 


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2082 1f5c12ca-751b-0410-a591-d2e778427230
 2009-02-05 17:39:36 +00:00
jdschroeder
e53ab5da6d Support for multiple input feature scores on confusion networks and lattices. 
Use "link-param-count" to tell Moses how many to expect in the input.
If weight-i (I) is one more than link-param-count, a feature for non-null word count will be added (this has actually always been there, but only for the 1 param, 2 weights scenario).
Input feature scores are now preserved for unknown words.

Unknown word penalty weight is now tunable with -weight-u (u), default is 1, as was hard-coded before.

Changes to mert-moses.pl will be checked in shortly.




git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2081 1f5c12ca-751b-0410-a591-d2e778427230
 2009-02-05 17:37:09 +00:00
bhaddow
8fc1c1b95e Fix loading of gzipped phrase tables 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2073 1f5c12ca-751b-0410-a591-d2e778427230
 2009-02-04 11:09:38 +00:00
dowobeha
324393afe7 Allow constraint file without tabs 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1999 1f5c12ca-751b-0410-a591-d2e778427230
 2009-01-26 16:14:38 +00:00
phkoehn
98381c0193 fixed xml removal 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1995 1f5c12ca-751b-0410-a591-d2e778427230
 2009-01-24 05:21:36 +00:00
redpony
e61b9da9f7 better example 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1994 1f5c12ca-751b-0410-a591-d2e778427230
 2009-01-22 21:32:39 +00:00
redpony
3f7f12f4ad add client for remote language model 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1993 1f5c12ca-751b-0410-a591-d2e778427230
 2009-01-22 21:31:17 +00:00
redpony
e923c82cf5 add another example 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1992 1f5c12ca-751b-0410-a591-d2e778427230
 2009-01-22 17:58:45 +00:00
redpony
f067a6cf1d add missing file 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1991 1f5c12ca-751b-0410-a591-d2e778427230
 2009-01-22 17:52:22 +00:00
redpony
3172abca21 check in code for remote LM-server 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1990 1f5c12ca-751b-0410-a591-d2e778427230
 2009-01-22 17:50:51 +00:00
bhaddow
6c8c8e9dc4 initial weights for toy example 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1989 1f5c12ca-751b-0410-a591-d2e778427230
 2009-01-22 10:25:14 +00:00
hieuhoang1972
f076b03c10 conf net fix 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1981 1f5c12ca-751b-0410-a591-d2e778427230
 2009-01-15 18:12:53 +00:00
hieuhoang1972
5161b380d5 regress 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1980 1f5c12ca-751b-0410-a591-d2e778427230
 2009-01-14 13:55:54 +00:00
saintamh
aeb93ec23e added the tokenizer scripts that were distributed for the Marathon last year - translate.cgi needs them and it simplifies distribution to have them here 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1979 1f5c12ca-751b-0410-a591-d2e778427230
 2009-01-12 22:36:56 +00:00
phkoehn
f9be34dd35 fixed bug in zones 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1972 1f5c12ca-751b-0410-a591-d2e778427230
 2009-01-08 17:46:29 +00:00
phkoehn
616842f278 fixed multi-bleu documentation 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1971 1f5c12ca-751b-0410-a591-d2e778427230
 2009-01-08 00:47:10 +00:00