moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2024-10-06 00:08:52 +03:00
Code Issues Packages Projects Releases Wiki Activity
17,097 Commits 136 Branches 26 Tags 147 MiB
54c2eae4b4
Commit Graph

191 Commits

Author SHA1 Message Date
redpony
7dbcef925d Implemented -r (-recover-input-path) to recover the actual path through the CN/word lattice that was used in the best translation. Also cleaned up handling of source Phrases in the context of a particular hypothesis, which may improve performance translating with lexicalized reordering models. 
Removed PhraseReference since the concept was broken for CN/lattice inputs.



git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1468 1f5c12ca-751b-0410-a591-d2e778427230
 2007-09-28 16:43:33 +00:00
redpony
17884106d4 is this the last of the word lattice distortion errors? maybe. i hope so. the nbest list is at least in the right order now. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1466 1f5c12ca-751b-0410-a591-d2e778427230
 2007-09-27 23:22:58 +00:00
redpony
541a486c0c stricter checks on word lattice distortion 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1464 1f5c12ca-751b-0410-a591-d2e778427230
 2007-09-27 12:14:42 +00:00
redpony
251aa600fa fixed a subtle bug in the word lattice decoder. it now acts sane. This experience also gave me more evidence for thinking that numbering the words in a phrase is not good-- we should be numbering the spaces between them. this would fix oh so many nasty things we've got (like the fact that we represent word drops with [n,n-1] etc). 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1463 1f5c12ca-751b-0410-a591-d2e778427230
 2007-09-27 05:03:06 +00:00
redpony
81c0642e36 fix distortion limiting functionality for WordLattices. Now, the shortest path to a another node is used to determine whether the threshold has been exceeded. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1460 1f5c12ca-751b-0410-a591-d2e778427230
 2007-09-17 02:49:08 +00:00
redpony
63da638077 The primary change here is that the distortion distance in a word lattice is now defined to be the number of words on the shortest path between the previous phrase's ending span and the next phrase's starting span. Originally, the distance was defined to be the difference in node number, which could be extremely large even for words that are, in some sense, adjacent. Only the distortion scoring has been fixed for word lattices, but the distortion limit is still broken. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1458 1f5c12ca-751b-0410-a591-d2e778427230
 2007-09-14 07:31:56 +00:00
redpony
d8536d15b7 add lattice test, not just CN test 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1422 1f5c12ca-751b-0410-a591-d2e778427230
 2007-07-03 15:40:26 +00:00
redpony
bf756e12fb add test 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1363 1f5c12ca-751b-0410-a591-d2e778427230
 2007-04-18 17:03:38 +00:00
redpony
945c77e3f2 fix bug with unknown words/CNs/lexical reordering 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1362 1f5c12ca-751b-0410-a591-d2e778427230
 2007-04-18 16:49:51 +00:00
redpony
c80d8b8d47 Support for the decoding of arbitrary word lattices. Must be given in the form of a "plf" file, which is a little tricky. I'll add documentation at some point; for now, refer to the example plf file in the "lattice-surface" regression test. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1359 1f5c12ca-751b-0410-a591-d2e778427230
 2007-04-18 14:08:46 +00:00
hieuhoang1972
f4a02f840d changed lex reordering regression test to fit with philipp's change to staticData 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1346 1f5c12ca-751b-0410-a591-d2e778427230
 2007-04-08 22:15:14 +00:00
hieuhoang1972
d48e71cb57 cleanup regression files - consistent file extension, delete old ini files 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1344 1f5c12ca-751b-0410-a591-d2e778427230
 2007-04-06 17:07:54 +00:00
hieuhoang1972
b6945f91bc change user message for downloading data 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1343 1f5c12ca-751b-0410-a591-d2e778427230
 2007-04-06 14:06:39 +00:00
hieuhoang1972
10ee6bedf5 update regression data 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1342 1f5c12ca-751b-0410-a591-d2e778427230
 2007-04-06 13:55:45 +00:00
hieuhoang1972
fa0231aea9 add lexicalized reordering test 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1341 1f5c12ca-751b-0410-a591-d2e778427230
 2007-04-06 13:25:20 +00:00
hieuhoang1972
3c07c5df4d add svn id comments to start of file 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1307 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-14 22:22:36 +00:00
hieuhoang1972
5f22fb13d3 make output of decimal places consistent by not format anywhere but in Main.cpp 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1266 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-04 00:43:45 +00:00
hieuhoang1972
6b611279d5 minor gcc compile error. 
also, no longer use IRSTLM as a subsitutute for SRILM, and vice versa. They don't give identitcal results - avoids confusion.

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1229 1f5c12ca-751b-0410-a591-d2e778427230
 2007-02-21 20:05:27 +00:00
hieuhoang1972
b62dda41ed change unknown word processing to be closer to the way pharaoh does it - create unknown word whenever single word is not in translation table but penalise hypothesis for using it. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1228 1f5c12ca-751b-0410-a591-d2e778427230
 2007-02-21 19:51:17 +00:00
hieuhoang1972
7ecb0ce66e change unknown word processing to be closer to the way pharaoh does it - create unknown word whenever single word is not in translation table but penalise hypothesis for using it. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1227 1f5c12ca-751b-0410-a591-d2e778427230
 2007-02-21 19:48:53 +00:00
hieuhoang1972
e247f1da6f fixed regression test failing. Number of features for generation models MUST be specified in ini file, no backward compatability hack 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1209 1f5c12ca-751b-0410-a591-d2e778427230
 2007-02-13 19:15:34 +00:00
hieuhoang1972
afe5f3bf2e undo update to regression data version 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1060 1f5c12ca-751b-0410-a591-d2e778427230
 2006-12-13 12:41:15 +00:00
hieuhoang1972
abd4adfb0c undo accidental checkin for MosesRegressionTesting.pm 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@999 1f5c12ca-751b-0410-a591-d2e778427230
 2006-11-22 15:50:59 +00:00
hieuhoang1972
dd721e1b4f Mac xcode proj 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@997 1f5c12ca-751b-0410-a591-d2e778427230
 2006-11-22 15:19:28 +00:00
nicolabertoldi
90d9147d9e Add simple regression test for nbest 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@986 1f5c12ca-751b-0410-a591-d2e778427230
 2006-11-20 13:42:04 +00:00
nicolabertoldi
7f7e7dfbd9 Add regression test for confusion network and multiple factors 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@985 1f5c12ca-751b-0410-a591-d2e778427230
 2006-11-20 13:27:24 +00:00
hieuhoang1972
4e2f8cf550 consistent file extension 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@967 1f5c12ca-751b-0410-a591-d2e778427230
 2006-11-09 06:55:23 +00:00
hieuhoang1972
a373d97ecf consistent file extension 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@966 1f5c12ca-751b-0410-a591-d2e778427230
 2006-11-08 22:44:29 +00:00
bojar
27fe632194 being safe and dying when unknown --cmdline-option supplied 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@824 1f5c12ca-751b-0410-a591-d2e778427230
 2006-09-21 14:45:03 +00:00
redpony
c69cfaf33e Allow the factor delimiter, that is, the string that separates the factors in a 'word' to be specified to moses and to train-factored-phrase-model.perl. The default is still to use '|'. Multi-character delimiters are allowed (for example, '+++'). Added a regression test for multi-character delimiters. 
Remove JHU dependencies on make release.  It now looks for GIZA++ and sets the BINDIR inside train-factored-phrase-model.perl at installation time (note: because of this, this script MUST BE released before it can be run now).



git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@812 1f5c12ca-751b-0410-a591-d2e778427230
 2006-09-12 15:53:50 +00:00
phkoehn
1d7bf5f8c2 regression tests had to adjusted to changes in verbose handling 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@801 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-30 17:05:45 +00:00
hieuhoang1972
1e63b42c06 fixed to run from any directory 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@661 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-11 18:36:23 +00:00
hieuhoang1972
47ab16ba5d fixed to work with most recent moses (8/11 1429) and to use good coding style (yay perl module!) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@660 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-11 18:25:01 +00:00
hieuhoang1972
80e0f752fa fixed to work with most recent moses as of now (8/11 1429) and to use good coding style (yay perl module!) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@659 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-11 18:24:15 +00:00
redpony
c8d3471bbe fix tests 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@657 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-11 18:16:00 +00:00
eherbst
b340381eb4 put a couple common functions into a utils perl module; fixed a bit of broken format-checking 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@652 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-11 16:35:50 +00:00
hieuhoang1972
99de4efd3c eclipse proj 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@650 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-11 16:13:42 +00:00
eherbst
0afacafddf adding characters to the printing of output dir 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@645 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-11 15:00:59 +00:00
hieuhoang1972
d11540e254 redirect cout to cerr using TRACE_ERR 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@586 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-08 22:32:29 +00:00
redpony
7f8914c6d5 fix to allow new hypothesis debugging format 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@566 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-08 18:50:28 +00:00
redpony
e9fd44ef13 check in regression tests to CVS 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@557 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-08 15:54:30 +00:00