moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2024-09-19 23:27:46 +03:00
Code Issues Packages Projects Releases Wiki Activity
14,471 Commits 136 Branches 26 Tags 147 MiB
5732129bd4
Commit Graph

480 Commits

Author SHA1 Message Date
Tetsuo Kiso
355f871457 Add const to member functions. 
Replace define macros with enum.
 2012-05-08 00:58:44 +09:00
Tetsuo Kiso
8f8b4693c3 Clean up headers. 2012-05-08 00:03:56 +09:00
Tetsuo Kiso
31c4bf3371 Clean up headers. 2012-05-08 00:03:56 +09:00
Tetsuo Kiso
42e07d23a8 Clean up including headers; delete using namespace std in headers. 2012-05-07 23:41:18 +09:00
Tetsuo Kiso
caa249d0f4 Clean up including headers; delete using namespace std in headers. 2012-05-07 23:41:18 +09:00
Tetsuo Kiso
1c8e2a2eb7 Fix passing arguments by const reference; add error checking. 
Please check the return values of fopen(), calloc(), and malloc().
 2012-05-07 23:26:32 +09:00
Tetsuo Kiso
dce212582f Fix passing arguments by const reference; add error checking. 
Please check the return values of fopen(), calloc(), and malloc().
 2012-05-07 23:26:32 +09:00
Tetsuo Kiso
5be31809af Use const instead of #define macros if possible. 2012-05-07 23:08:37 +09:00
Tetsuo Kiso
0ea2bc6fac Use const instead of #define macros if possible. 2012-05-07 23:08:37 +09:00
Tetsuo Kiso
4fcdffc29d Move/add #pragma once properly. 2012-05-07 22:59:37 +09:00
Tetsuo Kiso
245be8f63d Move/add #pragma once properly. 2012-05-07 22:59:37 +09:00
Tetsuo Kiso
e399131cfa Fix "-Wreorder". 2012-05-07 21:45:30 +09:00
Tetsuo Kiso
85c91b87da Fix "-Wreorder". 2012-05-07 21:45:30 +09:00
Eva Hasler
98da24a3a5 add condition for devtest chains, fix paths to moses/moses_chart 2012-04-24 05:17:19 +01:00
phikoehn
60ff4ff9fb minor fixes 2012-04-17 23:41:45 +01:00
phikoehn
a5a37c0371 minor fixes 2012-04-17 23:41:45 +01:00
phikoehn
d741e25494 re-run extract when using ghkm 2012-04-14 22:18:16 +01:00
phikoehn
05f02157ab re-run extract when using ghkm 2012-04-14 22:18:16 +01:00
Eva Hasler
0986ca1b5d dev 2012-04-13 15:45:06 +01:00
Eva Hasler
beb8096d81 set up pipeline for devtest set (mira tuning) 2012-04-13 15:43:01 +01:00
Eva Hasler
b85cf551c9 updates to ems 2012-04-13 13:18:35 +01:00
phikoehn
1a6e8dc6fd bug fix 2012-04-12 22:07:26 +01:00
phikoehn
4d133cf7ff bug fix 2012-04-12 22:07:26 +01:00
Eva Hasler
fc77c3e623 add [use-persistent-cache] 0 to mira configs 2012-04-12 12:30:56 +01:00
Eva Hasler
b008dabfce cherry-picked commit 2d47a5637b from master 2012-04-12 12:29:12 +01:00
phikoehn
366d427ce6 minor fixes 2012-04-12 00:25:57 +01:00
phikoehn
2d47a5637b minor fixes 2012-04-12 00:25:57 +01:00
Eva Hasler
77c743e65f enable mira for experiment.perl (.meta still needs some adjustment), clean up mira training script 2012-04-06 21:01:59 +01:00
Eva Hasler
f5a4d61fd4 resolving conflicts 2012-03-27 14:24:49 +01:00
phikoehn
f755b7b195 Merge branch 'master' of git://github.com/moses-smt/mosesdecoder 2012-03-23 07:17:34 +00:00
phikoehn
80e756b201 Merge branch 'master' of git://github.com/moses-smt/mosesdecoder 2012-03-23 07:17:34 +00:00
phikoehn
4d0fc996ba bug fix to filter hierarchical 2012-03-23 07:17:08 +00:00
phikoehn
4048734a59 bug fix to filter hierarchical 2012-03-23 07:17:08 +00:00
Tetsuo Kiso
7e876f0d5b Add error check when opening a file. 2012-03-22 21:51:00 +09:00
Tetsuo Kiso
90543c6c22 Add error check when opening a file. 2012-03-22 21:51:00 +09:00
phikoehn
1794bccd90 only filter rule and reordering tables, if the tables changes, not just when weights or other settings changed 2012-03-22 00:13:02 +00:00
phikoehn
76d2a19741 only filter rule and reordering tables, if the tables changes, not just when weights or other settings changed 2012-03-22 00:13:02 +00:00
Eva Hasler
95a1f8e36f update experiment.meta 2012-03-21 13:02:47 +00:00
phikoehn
15c83daa21 mop up 2012-03-21 00:38:05 +00:00
phikoehn
1213b9114a mop up 2012-03-21 00:38:05 +00:00
Eva Hasler
0f2a23f193 update experiment.perl 2012-03-20 13:13:52 +00:00
phikoehn
791b5a7676 lotsa minor changes: mostly bug fixes, tokenizer now esacapes special Moses characters (|<>&) 2012-03-20 04:57:37 +00:00
phikoehn
048e0f8de4 lotsa minor changes: mostly bug fixes, tokenizer now esacapes special Moses characters (|<>&) 2012-03-20 04:57:37 +00:00
Hieu Hoang
216c322587 cygwin 2012-02-24 14:18:22 +00:00
Hieu Hoang
dbb7980a72 cygwin 2012-02-24 14:18:22 +00:00
phikoehn
bca952e3e0 fix bug with < and > 2012-02-22 00:49:29 +00:00
phikoehn
d3e17f0ebd fix bug with < and > 2012-02-22 00:49:29 +00:00
Philipp Koehn
517fda5ada bug fixes 2012-02-16 05:14:56 +00:00
Philipp Koehn
faea0c3ea3 bug fixes 2012-02-16 05:14:56 +00:00
Barry Haddow
5ed5fc111a Fix to qsub args, from Guchun Zhang 2012-02-04 13:33:47 +00:00
Barry Haddow
f55c9921e7 Fix to qsub args, from Guchun Zhang 2012-02-04 13:33:47 +00:00
Michal Hrusecky
4bd84bca1a Alway return something in non-void functions 
There were functions defined as non-void but didn't have return value
for all possible passes. This can result in undefined behavior. Fixed
this issue and returning values that somehow makes sense hopefully.
 2012-02-01 14:03:49 +01:00
Michal Hrusecky
8ab6c7a655 Alway return something in non-void functions 
There were functions defined as non-void but didn't have return value
for all possible passes. This can result in undefined behavior. Fixed
this issue and returning values that somehow makes sense hopefully.
 2012-02-01 14:03:49 +01:00
phikoehn
a822db95db Merge branch 'master' of git://github.com/moses-smt/mosesdecoder 2012-01-24 10:14:04 +00:00
phikoehn
df6c71a276 Merge branch 'master' of git://github.com/moses-smt/mosesdecoder 2012-01-24 10:14:04 +00:00
phikoehn
2a97c6bed2 added comments to config files 2012-01-24 10:13:19 +00:00
phikoehn
79efdef247 added comments to config files 2012-01-24 10:13:19 +00:00
Eleftherios Avramidis
57688ae5b2 Catch errors produced after re-running on an existing run (with --continue) 2012-01-16 16:42:19 +01:00
Eleftherios Avramidis
d5df460835 Catch errors produced after re-running on an existing run (with --continue) 2012-01-16 16:42:19 +01:00
Eleftherios Avramidis
1022a53760 Merge commit 'upstream/master' 2012-01-14 00:22:57 +01:00
Eleftherios Avramidis
caa4ce78e1 Merge commit 'upstream/master' 2012-01-14 00:22:57 +01:00
Eleftherios Avramidis
8dd5c7bf2b Catch error, when for some reason train-model.perl is missing from installation (not properly compiled Moses) 2012-01-14 00:22:01 +01:00
Eleftherios Avramidis
28899b1f05 Catch error, when for some reason train-model.perl is missing from installation (not properly compiled Moses) 2012-01-14 00:22:01 +01:00
Eleftherios Avramidis
722c3ff6c7 Added support for optional external METEOR script. Workflow changes in experiment.meta and and corresponding sample given in example/config.basic. Results processing for report-experiment-scores now supports METEOR output. Lifted web interface restriction on the types of metrics that can be listed 2012-01-12 18:37:02 +01:00
Eleftherios Avramidis
a762145c2a Added support for optional external METEOR script. Workflow changes in experiment.meta and and corresponding sample given in example/config.basic. Results processing for report-experiment-scores now supports METEOR output. Lifted web interface restriction on the types of metrics that can be listed 2012-01-12 18:37:02 +01:00
phikoehn
9a5f61893a fixed ems bug when no interpolated LMs are used 2012-01-12 03:34:51 +00:00
phikoehn
ba469d27a5 fixed ems bug when no interpolated LMs are used 2012-01-12 03:34:51 +00:00
Philipp Koehn
9ad88ea09e allow for mix of interpolated and non-interpolated language models 2011-12-23 05:16:29 +00:00
Philipp Koehn
6561b32cbd allow for mix of interpolated and non-interpolated language models 2011-12-23 05:16:29 +00:00
Philipp Koehn
8d9c93e1aa allow for more than 10 language models by explicit or automatic grouping 2011-12-22 00:37:46 +00:00
Philipp Koehn
aba0925bf8 allow for more than 10 language models by explicit or automatic grouping 2011-12-22 00:37:46 +00:00
Philipp Koehn
b9622d0da3 better error message when no corpus defined, better integration of IRSTLM training 2011-12-21 05:50:59 +00:00
Philipp Koehn
cdf735b01b better error message when no corpus defined, better integration of IRSTLM training 2011-12-21 05:50:59 +00:00
Philipp Koehn
edd9c44519 updates to EMS: mteval-v13a.pl, parallel preparation, better paths and defaults in examples 2011-12-21 04:26:27 +00:00
Philipp Koehn
b95c372e3a updates to EMS: mteval-v13a.pl, parallel preparation, better paths and defaults in examples 2011-12-21 04:26:27 +00:00
Philipp Koehn
9ab5ac306b first cimmit to github: enable parallel data preparation in ems 2011-12-21 01:04:11 +00:00
Philipp Koehn
d6035a0612 first cimmit to github: enable parallel data preparation in ems 2011-12-21 01:04:11 +00:00
Barry Haddow
96217a0985 Fix handling of scientific notation 2011-12-19 15:27:54 +00:00
Barry Haddow
6f2068bddc Fix handling of scientific notation 2011-12-19 15:27:54 +00:00
Barry Haddow
072848a4df Merge branch 'master' into bjam. Doesn't compile. 
Conflicts:
	.gitignore
	Makefile.am
	config.h.in
	configure.in
	m4/boost.m4
	moses/src/LM/Factory.cpp
	moses/src/LM/ParallelBackoff.h
	moses/src/LM/RandLM.h
	moses/src/Makefile.am
	moses/src/PDTAimp.h
	moses/src/Phrase.h
	moses/src/PhraseDictionaryTree.cpp
	moses/src/ScoreComponentCollection.h
	moses/src/ScoreIndexManager.cpp
	moses/src/StaticData.cpp
	moses/src/TargetPhrase.cpp
	moses/src/TranslationOptionCollection.cpp
	moses/src/TranslationSystem.cpp
	scripts/Makefile
	scripts/released-files
 2011-12-08 23:17:30 +00:00
Kenneth Heafield
d6005b749a Change path to build_binary 2011-12-02 12:51:19 +00:00
Kenneth Heafield
138db687ed Change path to build_binary 2011-12-02 12:51:19 +00:00
Kenneth Heafield
a0d79781e5 Merge CreateOnDisk into OnDiskPt, pop out of src 2011-12-01 18:26:05 +00:00
Kenneth Heafield
9fe831c60c Merge CreateOnDisk into OnDiskPt, pop out of src 2011-12-01 18:26:05 +00:00
Kenneth Heafield
04dbfc3933 Trigger biconcor compilation 2011-11-30 15:23:13 +00:00
Kenneth Heafield
563dfd69d3 Trigger biconcor compilation 2011-11-30 15:23:13 +00:00
Kenneth Heafield
4393d00cbd biconcur jam and remove a Makefile 2011-11-25 10:39:28 +00:00
Kenneth Heafield
fafb835c5d biconcur jam and remove a Makefile 2011-11-25 10:39:28 +00:00
Jun Lang
6ac3d430a3 There is potential bug for parsing numbers in scientific notation. 2011-11-23 10:39:28 +08:00
Jun Lang
5ed75d236f There is potential bug for parsing numbers in scientific notation. 2011-11-23 10:39:28 +08:00
Phil Williams
ee0a6dbd5c extract-ghkm: add the features required for use as a drop-in replacement 
for extract-rules: composed rules, unaligned source word attachment,
non-lexical unary rule elimination, glue rule generation, unknown word
label generation, and EMS integration.
 2011-11-21 16:21:04 +00:00
Barry Haddow
afa4a310be Proper handling of weight file 2011-11-03 16:34:29 +00:00
Barry Haddow
fc51c94dd2 Implementation of corpus identifier feature 2011-10-14 12:44:37 +01:00
Barry Haddow
ff15f28d9c record corpus lengths 2011-10-12 13:06:17 +01:00
Barry Haddow
5311e94acf Remember to use sid version 2011-09-27 14:25:53 +01:00
Barry Haddow
9dce240ad0 correct output of sids file 2011-09-21 21:13:00 +01:00
Barry Haddow
66cbc5c3fc Changes to training pipeline to support sparse phrase features. 2011-09-21 15:57:33 +01:00
pjwilliams
a1ca2722df Add --MinNonInitialRuleCount option to filter-model-given-input.pl. This 
prunes non-initial rules (i.e. rules with non-terminals) from the rule table
based on their frequency counts.  In Zollmann, Venugopal, Och, and Ponte (2008),
pruning hierarchical rules that occur only once was found to significantly
decrease rule table size without harming translation quality.

Also, add TUNING:filter-settings and EVALUATION[:<set>]:filter-settings
variables so that this can be enabled in the EMS.

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4033 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-24 16:36:27 +00:00
phkoehn
6acd6a8684 improvements to ems analysis 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4026 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-21 21:52:13 +00:00
phkoehn
4285a8e236 various experiment.perl improvements: split filter and decode/tune; extensions to analysis.perl, especially precision by coverage graphs 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4018 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-16 23:43:29 +00:00
bhaddow
c7603add22 Option to specify language model type 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4013 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-13 12:13:55 +00:00
hieuhoang1972
5a0210855d minor warning 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3985 1f5c12ca-751b-0410-a591-d2e778427230
 2011-05-20 02:49:38 +00:00
hieuhoang1972
a3d97584a9 run beautify.perl. Consistent formatting for .h & .cpp files 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3902 1f5c12ca-751b-0410-a591-d2e778427230
 2011-02-24 13:57:11 +00:00
phkoehn
4c11bcd617 extensions to phrase table scoring options 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3893 1f5c12ca-751b-0410-a591-d2e778427230
 2011-02-23 10:27:54 +00:00
suzyh
994ccb12c2 Fix to EMS call of multi-bleu evaluation script 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3798 1f5c12ca-751b-0410-a591-d2e778427230
 2011-01-13 04:53:09 +00:00
phkoehn
7334d49191 minor experiment.perl fixes 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3668 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-27 12:42:34 +00:00
phkoehn
ace33d16dd bug fix 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3636 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-22 07:30:52 +00:00
phkoehn
85a5a13e4c improvements to web analysis, fixes to syntax wrappers 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3633 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-21 09:49:27 +00:00
hieuhoang1972
08739d8f49 add from josh's script 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3621 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-13 10:14:13 +00:00
hieuhoang1972
eedef63277 keep perl scripts with Unix line endings 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3612 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-11 11:32:27 +00:00
suzyh
d071296cde Fix to reuse-weights.perl to copy weights containing an exponent 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3590 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-29 04:04:12 +00:00
phkoehn
f34b37bad3 added hierarchical alignment view to web analysis tool 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3514 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-17 13:28:04 +00:00
suzyh
fa4eca6ccb Added loop to check_if_crashed in EMS experiment.perl to wait in case the .STDERR file is slow in appearing after the step has completed. Reinstated --old-sge and --filterfile command-line arguments to mert-moses.pl. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3420 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-09 11:40:40 +00:00
phkoehn
4a85fd95ce srilm as setting in interpolation 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3416 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-03 12:57:16 +00:00
bhaddow
12269b062c Only add queue-flags if non-empty. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3414 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-02 09:01:39 +00:00
phkoehn
4e0bc582f6 minor improvements: binarizing rule tables in filter script, multiple reference translation in ems analysis 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3284 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-28 22:19:58 +00:00
pjwilliams
05eb33d5ac In train-model.perl, write an 'unknown-lhs' line out to the config file if 
an unknown word label file has been generated.  Also, disable this option
by default since it can greatly increase number of hypotheses generated
and hasn't been shown to help translation yet.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3277 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-27 15:11:16 +00:00
phkoehn
b271862d7c various updates, mostly related to experiment.perl 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3262 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-18 17:39:16 +00:00
phkoehn
c15fc6f104 minor bug fixes 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3247 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-10 22:41:54 +00:00
phkoehn
524b1b12d2 added info for input phrase coverage 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3245 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-10 03:05:35 +00:00
phkoehn
883d12d482 added info for input word coverage to analysis + fixes 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3244 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-10 00:19:40 +00:00
phkoehn
447dccfc59 more analysis in experiment.perl 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3234 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-07 11:28:55 +00:00
phkoehn
45ecfa72d2 minor changes to experiment.perl 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3225 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-05 16:06:34 +00:00
phkoehn
2ed6804f12 official release of experiment.perl 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3224 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-04 23:04:10 +00:00
bojar
f36f347014 safer everything: tempdir, open, execution 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3144 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-20 11:10:31 +00:00
pjwilliams
4c6c4b71cf Remove redundant --ProperConditioning option from extract. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3118 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-12 10:41:32 +00:00
phkoehn
4d814e53d2 del 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2728 1f5c12ca-751b-0410-a591-d2e778427230
 2010-01-28 17:38:26 +00:00
phkoehn
f1f395e05d added web interface, organized files in sub directories 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2724 1f5c12ca-751b-0410-a591-d2e778427230
 2010-01-28 17:24:39 +00:00
phkoehn
244e334b3d bug fix 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2690 1f5c12ca-751b-0410-a591-d2e778427230
 2010-01-27 14:55:17 +00:00
phkoehn
def35604af initial release of experiment.perl 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2669 1f5c12ca-751b-0410-a591-d2e778427230
 2010-01-25 17:38:53 +00:00