moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2024-09-21 08:07:14 +03:00
Code Issues Packages Projects Releases Wiki Activity
1,980 Commits 136 Branches 26 Tags 147 MiB
2d06b12a2c
Commit Graph

73 Commits

Author SHA1 Message Date
hieuhoang1972
2d06b12a2c regression test for ondisk 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3618 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-12 15:04:31 +00:00
hieuhoang1972
ac875ffc81 mem leak 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3617 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-12 14:47:58 +00:00
hieuhoang1972
bdfa2beab8 delete eclipse build 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3422 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-10 09:16:50 +00:00
bhaddow
904133fcb7 Merge in the multiple models branch. These changes allow the moses server 
to support multiple translation, language and generation models within the
same process. The main design change is the introduction of a TranslationSystem
object to manage the models, which have been moved out of StaticData.
The changes should have no effect on existing systems.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3394 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-10 13:12:00 +00:00
rafpayen
1896cc7fff better messages 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3383 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-03 15:34:34 +00:00
hieuhoang1972
7221bf2dd4 alignment info, for chart decoding, updated regression 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3368 1f5c12ca-751b-0410-a591-d2e778427230
 2010-07-28 09:53:21 +00:00
hieuhoang1972
e8ab51f594 regression test 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3314 1f5c12ca-751b-0410-a591-d2e778427230
 2010-06-09 13:48:44 +00:00
hieuhoang1972
65194ba505 drop unknown option for chart decoding, xcode, minor adjustments to regression test formatting 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3312 1f5c12ca-751b-0410-a591-d2e778427230
 2010-06-09 12:56:31 +00:00
hieuhoang1972
f028e1d9bd add space @ end of line after each trans 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3251 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-11 23:07:15 +00:00
bojar
0118733a45 fixing traditional distortion penalty with -continue-partial-translation 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3222 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-03 21:41:28 +00:00
bojar
b7a48d3962 adding a simple regression test for continue-partial-translation 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3220 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-03 21:40:43 +00:00
hieuhoang1972
e9cd6d0fde regression 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3209 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-26 18:41:53 +00:00
hieuhoang1972
b73293f3a8 regression 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3208 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-26 18:39:05 +00:00
hieuhoang1972
a8f75cb789 regression 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3207 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-26 17:48:15 +00:00
hieuhoang1972
87c3becbf6 regression 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3206 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-26 16:21:19 +00:00
hieuhoang1972
d5a0eb2423 regression 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3205 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-26 16:16:44 +00:00
hieuhoang1972
430446f404 regression 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3204 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-26 16:03:35 +00:00
chardmeier
ca353d3e68 - word lattice distortion calculation and distortion limit checks revisited 
- crash in a controlled way if a jump in a word lattice exceeds max-phrase-length
- check if a sufficient number of ttable-limits is specified in the configuration

Thanks to Arianna Bisazza for help with sorting out lattice distortion and
to Suzy Howlett for help with debugging the other issues.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3193 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-22 08:42:32 +00:00
bhaddow
a76b7e5bce Test for consensus decoding 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3128 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-12 17:01:51 +00:00
hieuhoang1972
c117ef7c17 Copy in changes from the chart_merge branch (doing it manually because the 
server doesn't seem to support subversion's --reintegrate option).


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3078 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-08 17:16:10 +00:00
bhaddow
407dd68aec Update lexicalised reordering test truths to take account of new, improved scores. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@3001 1f5c12ca-751b-0410-a591-d2e778427230
 2010-03-25 11:04:17 +00:00
hieuhoang1972
cbb329b754 minor 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2955 1f5c12ca-751b-0410-a591-d2e778427230
 2010-03-07 19:39:07 +00:00
chardmeier
917737d1eb Fixed regression test for lattice distortion because implementation is now right and regression test isn't. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2652 1f5c12ca-751b-0410-a591-d2e778427230
 2010-01-18 11:13:39 +00:00
phkoehn
8833098925 generalized n-best list reporting for feature functions, added experimental version of global lexical model 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2343 1f5c12ca-751b-0410-a591-d2e778427230
 2009-05-26 19:30:35 +00:00
redpony
558c723c7c fix for one round of problems- this moses will work with g++ > 4.3 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2236 1f5c12ca-751b-0410-a591-d2e778427230
 2009-03-12 16:52:27 +00:00
redpony
63effe85b5 Feature function overhaul. Each feature function is computed in one of three ways: 
1) Stateless feature functions from the phrase table/generation table: these are computed when
   the TranslationOption is created.  They become part of the ScoreBreakdown object contained in
   the TranslationOption and are added to the feature value vector when a hypothesis is extended.
2) Stateless feature functions that are computed during state exploration. Currently, only
   WordPenalty falls into this category, but these functions implement a method Evaluate which
   do does not receive a Hypothesis or any contextual information.
3) Stateful feature functions: these features receive the arc information (translation option),
   compute some value and then return some context information.  The context information created
   by a particular feature function is passed back to it as the previous context when a hypothesis
   originating at the node where the previous edge terminates is created.  States in the search
   space may be recombined if the context information is identical.  The context information must
   be stored in an object implementing the FFState interface.

TODO:
1) the command line interface / MERT interface needs to go to named parameters that are otherwise opaque
2) StatefulFeatureFunction's Evaluate method should just take a TranslationOption and a context object.  It is not good that it takes a hypothesis, because then people may be tempted to access information about the "previous" hypothesis without "declaring" this dependency.
3) Future cost estimates should be handled using feature functions.  All stateful feature functions need some kind of future cost estimate.
4) Philipp's poor-man's cube pruning is broken.



git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2087 1f5c12ca-751b-0410-a591-d2e778427230
 2009-02-06 15:43:06 +00:00
redpony
02067b2be0 fix some nasty edge cases in lattice decoding that arise when decoding lattices with complex topologies. also fix lattice regression tests. work done jointly with j schroeder. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1853 1f5c12ca-751b-0410-a591-d2e778427230
 2008-06-17 17:32:19 +00:00
hieuhoang1972
4f84e3ccfe don't run 2 tests which don't pass 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1850 1f5c12ca-751b-0410-a591-d2e778427230
 2008-06-11 13:42:23 +00:00
redpony
aed1758623 fix bug in word lattice decoding that prevented some viable hypotheses from being considered. 
also fix XML-markup test failures.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1563 1f5c12ca-751b-0410-a591-d2e778427230
 2008-02-19 22:03:05 +00:00
redpony
9521e78fbf add functionalty to support adding direct translation (ie, passthrouh) even when there are other translation options available for the span. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1510 1f5c12ca-751b-0410-a591-d2e778427230
 2007-11-10 23:19:41 +00:00
redpony
180d9bac5d add support for unary XML options. refer to regression-testing/tests/xml-markup/to-translate.txt for examples. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1508 1f5c12ca-751b-0410-a591-d2e778427230
 2007-11-10 04:40:18 +00:00
redpony
3b008f67a9 add support for multiple xml options for the same span, start refactoring to add support for overlapping xml options. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1507 1f5c12ca-751b-0410-a591-d2e778427230
 2007-11-09 23:56:57 +00:00
redpony
7dbcef925d Implemented -r (-recover-input-path) to recover the actual path through the CN/word lattice that was used in the best translation. Also cleaned up handling of source Phrases in the context of a particular hypothesis, which may improve performance translating with lexicalized reordering models. 
Removed PhraseReference since the concept was broken for CN/lattice inputs.



git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1468 1f5c12ca-751b-0410-a591-d2e778427230
 2007-09-28 16:43:33 +00:00
redpony
17884106d4 is this the last of the word lattice distortion errors? maybe. i hope so. the nbest list is at least in the right order now. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1466 1f5c12ca-751b-0410-a591-d2e778427230
 2007-09-27 23:22:58 +00:00
redpony
541a486c0c stricter checks on word lattice distortion 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1464 1f5c12ca-751b-0410-a591-d2e778427230
 2007-09-27 12:14:42 +00:00
redpony
251aa600fa fixed a subtle bug in the word lattice decoder. it now acts sane. This experience also gave me more evidence for thinking that numbering the words in a phrase is not good-- we should be numbering the spaces between them. this would fix oh so many nasty things we've got (like the fact that we represent word drops with [n,n-1] etc). 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1463 1f5c12ca-751b-0410-a591-d2e778427230
 2007-09-27 05:03:06 +00:00
redpony
81c0642e36 fix distortion limiting functionality for WordLattices. Now, the shortest path to a another node is used to determine whether the threshold has been exceeded. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1460 1f5c12ca-751b-0410-a591-d2e778427230
 2007-09-17 02:49:08 +00:00
redpony
63da638077 The primary change here is that the distortion distance in a word lattice is now defined to be the number of words on the shortest path between the previous phrase's ending span and the next phrase's starting span. Originally, the distance was defined to be the difference in node number, which could be extremely large even for words that are, in some sense, adjacent. Only the distortion scoring has been fixed for word lattices, but the distortion limit is still broken. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1458 1f5c12ca-751b-0410-a591-d2e778427230
 2007-09-14 07:31:56 +00:00
redpony
d8536d15b7 add lattice test, not just CN test 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1422 1f5c12ca-751b-0410-a591-d2e778427230
 2007-07-03 15:40:26 +00:00
redpony
bf756e12fb add test 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1363 1f5c12ca-751b-0410-a591-d2e778427230
 2007-04-18 17:03:38 +00:00
redpony
945c77e3f2 fix bug with unknown words/CNs/lexical reordering 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1362 1f5c12ca-751b-0410-a591-d2e778427230
 2007-04-18 16:49:51 +00:00
redpony
c80d8b8d47 Support for the decoding of arbitrary word lattices. Must be given in the form of a "plf" file, which is a little tricky. I'll add documentation at some point; for now, refer to the example plf file in the "lattice-surface" regression test. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1359 1f5c12ca-751b-0410-a591-d2e778427230
 2007-04-18 14:08:46 +00:00
hieuhoang1972
f4a02f840d changed lex reordering regression test to fit with philipp's change to staticData 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1346 1f5c12ca-751b-0410-a591-d2e778427230
 2007-04-08 22:15:14 +00:00
hieuhoang1972
d48e71cb57 cleanup regression files - consistent file extension, delete old ini files 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1344 1f5c12ca-751b-0410-a591-d2e778427230
 2007-04-06 17:07:54 +00:00
hieuhoang1972
b6945f91bc change user message for downloading data 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1343 1f5c12ca-751b-0410-a591-d2e778427230
 2007-04-06 14:06:39 +00:00
hieuhoang1972
10ee6bedf5 update regression data 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1342 1f5c12ca-751b-0410-a591-d2e778427230
 2007-04-06 13:55:45 +00:00
hieuhoang1972
fa0231aea9 add lexicalized reordering test 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1341 1f5c12ca-751b-0410-a591-d2e778427230
 2007-04-06 13:25:20 +00:00
hieuhoang1972
3c07c5df4d add svn id comments to start of file 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1307 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-14 22:22:36 +00:00
hieuhoang1972
5f22fb13d3 make output of decimal places consistent by not format anywhere but in Main.cpp 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1266 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-04 00:43:45 +00:00
hieuhoang1972
6b611279d5 minor gcc compile error. 
also, no longer use IRSTLM as a subsitutute for SRILM, and vice versa. They don't give identitcal results - avoids confusion.

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1229 1f5c12ca-751b-0410-a591-d2e778427230
 2007-02-21 20:05:27 +00:00