moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2024-12-30 23:42:30 +03:00
Code Issues Packages Projects Releases Wiki Activity
2,336 Commits 136 Branches 26 Tags 147 MiB
1873030d24
Commit Graph

109 Commits

Author SHA1 Message Date
hieuhoang1972
1873030d24 forgot to add these files for regression tests 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4154 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-22 07:40:14 +00:00
bgottesman
436a285f18 stop using 'subtest' because it doesn't work for everyone, e.g. Hieu reports it doesn't work on a Mac even with an up-to-date Test::Simple module 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4144 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-16 10:51:43 +00:00
hieuhoang1972
87216f55be rename & make executable 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4136 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-09 09:07:34 +00:00
bgottesman
0fe1c629da if we fail to make the output directory for a test, just abort the test, don't exit the whole script 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4135 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-08 18:26:57 +00:00
bgottesman
24f5bf6723 when detokenizing, remove whitespace between a pair of CJK (Chinese/Japanese/Korean) words 
This gets the Chinese and Japanese tests working, so remove the failure expectation.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4134 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-08 15:30:54 +00:00
bgottesman
14587cdafc fix a detokenization bug that was preventing the removal of the whitespace following a contracted French or Italian article/pronoun (e.g. "l' immigration") when the contraction was the second-last word in the segment 
remove the expectation of failure on the corresponding unit test


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4133 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-08 15:02:56 +00:00
bgottesman
9d9977bc6f add TODO tests for detokenization of Chinese and Japanese 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4131 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-08 13:21:05 +00:00
bgottesman
c030dae094 Allow a test case to have an undefined language, since the detokenizer doesn't require a language to be passed in and, indeed, errors if a language is passed in for which there are no special rules (which seems dubious to me ...). Add test case TEST_GERMAN_NONASCII with an undefined language. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4130 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-05 19:14:01 +00:00
bgottesman
024bbe0bcc - factor out class DetokenizerTestCase 
- create an array of all of the test cases before running any of them
- in the case of an expected failure, move the TODO block deeper, just around the validation of the results

I'm not 100% I like this change, I think it makes the code slightly more elegant but it also makes it longer.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4128 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-05 18:48:37 +00:00
bgottesman
d521287a3f move commas to after here-docs, to hopefully make test cases more readable; and remove unused import 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4125 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-05 16:37:49 +00:00
bgottesman
76c3ef4dba a few more detokenization tests, including a TODO one that exposes a bug 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4124 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-05 16:23:47 +00:00
bgottesman
eda0f4e370 An initial test suite for detokenizer.perl. 
I realize this doesn't quite fit the paradigm if the existing moses test suite.  On the other hand, it's self-contained, easy to run, easy to add tests to (just follow the pattern in the section titled 'Definitions of individual test cases'), and uses an established Perl testing framework.  I don't think it will be infeasible to incorporate it into the existing test suite.

Usage:

run-test-detokenizer.t --results-dir <RESULTS-DIRECTORY>

where <RESULTS-DIRECTORY> is an empty existing directory where the output can be written


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4121 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-05 14:32:39 +00:00
hieuhoang1972
30ca534b86 faster scorer 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4119 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-05 10:27:15 +00:00
hieuhoang1972
b4c79f721e regression test for scorer 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4118 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-05 10:18:07 +00:00
hieuhoang1972
b618aadf8d regression test for scorer 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4117 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-05 09:23:48 +00:00
hieuhoang1972
b8a0b09206 regression test for scorer 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4116 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-05 02:48:30 +00:00
hieuhoang1972
fc176801d6 regression test for score 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4112 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-04 09:15:43 +00:00
hieuhoang1972
e988361d62 regression test for score 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4111 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-04 08:11:40 +00:00
nicolabertoldi
579d8b0760 added few regression tests explicitly working with IRSTLM; modified few regression tests wrongly working with IRSTLM/SRILM; modified the required data archive (now version is 6); 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4105 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-03 09:19:48 +00:00
hieuhoang1972
49e56f35bb regression test for score 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4102 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-02 10:28:36 +00:00
hieuhoang1972
d45a29d9c7 data for score regression test 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4101 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-02 10:10:09 +00:00
hieuhoang1972
ed4367ceb0 data for score regression test 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4100 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-02 10:04:53 +00:00
hieuhoang1972
69fe991923 data for score regression test 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4099 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-02 09:53:25 +00:00
hieuhoang1972
1ae8c53a08 executable perl script 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4098 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-02 09:31:01 +00:00
hieuhoang1972
acb7e984de starting regression test for score program 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4097 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-02 09:27:59 +00:00
hieuhoang1972
e389e9fec7 default decoders if none specified 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4094 1f5c12ca-751b-0410-a591-d2e778427230
 2011-07-27 10:14:33 +00:00
hieuhoang1972
6f8f1adf3b remove unnecessary parameters from ini files 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4010 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-11 04:52:05 +00:00
hieuhoang1972
b8e517d167 remove unnecessary parameters from ini files 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4009 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-11 04:48:22 +00:00
hieuhoang1972
265b4451ad remove unnecessary parameters from ini files 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4008 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-11 04:41:55 +00:00
hieuhoang1972
840f3915ce add xml markup back to regression testing 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3862 1f5c12ca-751b-0410-a591-d2e778427230
 2011-02-01 18:30:01 +00:00
hieuhoang1972
db404d0fc0 pass regression. Not sure why is passed before, wasn't beam threshold implemented? 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3861 1f5c12ca-751b-0410-a591-d2e778427230
 2011-02-01 18:17:44 +00:00
hieuhoang1972
c6e0391b21 spans must be in consistent format start-end, not start,end 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3860 1f5c12ca-751b-0410-a591-d2e778427230
 2011-02-01 15:17:13 +00:00
bhaddow
0e5fbcdb4a Add show-weights for moses_chart 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3745 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-29 17:05:16 +00:00
bhaddow
50f0e6c07d Add a show-weights option. It prints out the moses features and exits. May 
load tables as a side-effect.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3744 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-29 16:44:28 +00:00
phkoehn
2a594c0e2a fixed xml regression test 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3646 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-23 21:27:34 +00:00
hieuhoang1972
a875ef8361 regression test for ondisk 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3619 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-12 15:07:43 +00:00
hieuhoang1972
2d06b12a2c regression test for ondisk 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3618 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-12 15:04:31 +00:00
hieuhoang1972
ac875ffc81 mem leak 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3617 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-12 14:47:58 +00:00
hieuhoang1972
bdfa2beab8 delete eclipse build 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3422 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-10 09:16:50 +00:00
bhaddow
904133fcb7 Merge in the multiple models branch. These changes allow the moses server 
to support multiple translation, language and generation models within the
same process. The main design change is the introduction of a TranslationSystem
object to manage the models, which have been moved out of StaticData.
The changes should have no effect on existing systems.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3394 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-10 13:12:00 +00:00
rafpayen
1896cc7fff better messages 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3383 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-03 15:34:34 +00:00
hieuhoang1972
7221bf2dd4 alignment info, for chart decoding, updated regression 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3368 1f5c12ca-751b-0410-a591-d2e778427230
 2010-07-28 09:53:21 +00:00
hieuhoang1972
e8ab51f594 regression test 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3314 1f5c12ca-751b-0410-a591-d2e778427230
 2010-06-09 13:48:44 +00:00
hieuhoang1972
65194ba505 drop unknown option for chart decoding, xcode, minor adjustments to regression test formatting 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3312 1f5c12ca-751b-0410-a591-d2e778427230
 2010-06-09 12:56:31 +00:00
hieuhoang1972
f028e1d9bd add space @ end of line after each trans 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3251 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-11 23:07:15 +00:00
bojar
0118733a45 fixing traditional distortion penalty with -continue-partial-translation 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3222 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-03 21:41:28 +00:00
bojar
b7a48d3962 adding a simple regression test for continue-partial-translation 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3220 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-03 21:40:43 +00:00
hieuhoang1972
e9cd6d0fde regression 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3209 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-26 18:41:53 +00:00
hieuhoang1972
b73293f3a8 regression 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3208 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-26 18:39:05 +00:00
hieuhoang1972
a8f75cb789 regression 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3207 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-26 17:48:15 +00:00