moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2024-12-27 22:14:57 +03:00
Code Issues Packages Projects Releases Wiki Activity
9,454 Commits 136 Branches 26 Tags 147 MiB
33105a7ba7
Commit Graph

823 Commits

Author SHA1 Message Date
Hieu Hoang
487822ed14 don't write to stdout 2012-11-22 15:08:00 +00:00
Hieu Hoang
7d6d91a2e8 move zmert to contrib folder 2012-11-20 16:42:49 +00:00
Barry Haddow
2a88fd0730 Support for compact phrase table in EMS 
It should be sufficient to add a line like
ttable-binarizer = "/home/bhaddow/moses/dist/bin/processPhraseTableMin"
to your EMS config, and everything else will be taken care of. You can
add other arguments to the processPhraseTableMin, for example for
threading, by putting them in the quotes.

Note that this is not fully tested, since there are currently some
issues with the compact phrase table introduced by the sparse feature
merge.
 2012-11-16 15:07:07 +00:00
Barry Haddow
a90e1861c0 Alignments on by default for phrase-based 2012-11-15 12:35:43 +00:00
Mark Fishel
01eb79cf6f fixed berkeley parser wrapper's broken output in case of in-text parentheses 2012-11-07 11:18:36 +01:00
Barry Haddow
61ae24aa5d Merge remote-tracking branch 'origin/master' into miramerge 
Conflicts:
	moses/src/PhraseDictionary.cpp
	moses/src/TargetPhrase.cpp
	moses/src/TargetPhrase.h
 2012-10-14 14:18:03 +01:00
Ales Tamchyna
608f6ba607 handle binarized phrase table in clone_moses_model.pl 2012-10-09 16:05:12 +02:00
Barry Haddow
848aafb644 Merge remote branch 'github/master' into miramerge 
Conflicts:
	moses/src/AlignmentInfo.cpp
	moses/src/AlignmentInfo.h
	moses/src/ChartHypothesis.cpp
	moses/src/ChartTrellisNode.cpp
	moses/src/LM/Implementation.cpp
	moses/src/LM/Ken.cpp
	moses/src/TargetPhrase.cpp
	moses/src/TargetPhrase.h
 2012-10-08 17:54:59 +01:00
Phil Williams
b2b9751227 parse-de-bitpar.perl: fix special char handling 
Unescape special characters in input to BitPar and then re-escape
in output.
 2012-10-06 16:27:33 +01:00
Lane Schwartz
a323c8daf7 Send stderr to /dev/null when looking for pawd. 
This cleans up the logs a bit for those of us who don't have pawd.
Otherwise, messages like the following show up in the logs:

/usr/bin/which: no pawd in ...

bash: pawd: command not found
 2012-09-28 14:37:53 -04:00
Barry Haddow
0a950ee9f4 Merge remote branch 'github/master' into miramerge 
Compiles, but not tested. Had to disable relent filter. Strangely, it seems to contain the
whole of moses-cmd.

Conflicts:
	Jamroot
	OnDiskPt/TargetPhrase.cpp
	moses-cmd/src/Main.cpp
	moses/src/AlignmentInfo.cpp
	moses/src/AlignmentInfo.h
	moses/src/ChartTranslationOptionCollection.cpp
	moses/src/ChartTranslationOptionCollection.h
	moses/src/GenerationDictionary.cpp
	moses/src/Jamfile
	moses/src/Parameter.cpp
	moses/src/PhraseDictionary.cpp
	moses/src/StaticData.cpp
	moses/src/StaticData.h
	moses/src/TargetPhrase.h
	moses/src/TranslationSystem.cpp
	moses/src/TranslationSystem.h
	moses/src/Word.cpp
	phrase-extract/score.cpp
	regression-testing/Jamfile
	scripts/ems/experiment.meta
	scripts/ems/experiment.perl
	scripts/training/train-model.perl
 2012-09-26 22:49:33 +01:00
Philipp Koehn
4749e1b990 allow mert use weights in config file for first decoder run 2012-09-24 11:11:40 +01:00
Barry Haddow
58b6697dd3 Fix compile bug from merge 2012-09-13 20:00:20 +01:00
Eva Hasler
e6c73ec611 remove hardwired path 2012-09-11 17:29:15 +01:00
Rico Sennrich
4e2fc82854 new training option -write-lexical-counts 
(creates additional files lex.counts.e2f and lex.counts.f2e)
 2012-09-06 11:48:54 +02:00
Jonathan Clark
f5137c1a48 Accept compact phrase table and reordering models 2012-09-04 10:11:56 -04:00
phikoehn
5d9859ba0e merge issues 2012-09-03 07:27:41 +01:00
phikoehn
19ef785146 bug fixes 2012-09-03 07:24:31 +01:00
Hieu Hoang
c639cdbb38 binary hiero reordering feature. Integrated into train-model.perl and experiment.perl. In the 2nd to last position in phrase table, just in front of 2.718 2012-08-28 17:01:08 +01:00
Nadi Tomeh
e90744c21c Merge branch 'master' of ssh://github.com/moses-smt/mosesdecoder 2012-08-25 23:55:15 +02:00
Nadi Tomeh
02ddf45671 Remove a test from the function get_vocabulary which prevented vocabulary files from being generated if the option -no-lexical-weighting was specified. 2012-08-25 22:46:44 +02:00
Hieu Hoang
69fc00faf9 singleton feature in phrase table. Like similar feature in Adam's suffix array, as implemented in cdec 2012-08-24 00:54:05 +01:00
Jonathan Clark
3790c67750 Add a bit of documentation on how flags are being formed for the lexical-reordering-score program. 2012-08-21 15:27:18 -04:00
phikoehn
4a1a995878 a lot of changes 2012-08-18 23:48:26 +01:00
phikoehn
366ab93f8a a lot of changes 2012-08-18 23:47:05 +01:00
Lane Schwartz
1883090a3d Moved code for calculating lexical translation probabilities 
into a new perl module called LexicalTranslationModel.pm.

This commit moves the subroutine get_lexical (and its helper subroutines)
from train-model.perl into LexicalTranslationModel.pm. This new perl module
is now imported at the top of train-model.perl.

This change should not affect users of train-model.perl at all.

Doing this allows for the implementation of a stand-alone script
which can be used to create lexical translation model files directly,
given a word-aligned parallel corpus. This is often useful to do, and
should now be easier to do. The new script is get-lexical.perl.

Usage:
scripts/training/get-lexical.perl source target alignments output_prefix

Results:
output_prefix.f2e
output_prefix.e2f
 2012-08-15 10:45:36 -04:00
annirvine
9c5557f752 mert-moses.pl regex fixed to handle scientific notation 2012-07-31 14:56:43 -04:00
Hieu Hoang
d8daac9bb4 Merge branch 'master' of github.com:moses-smt/mosesdecoder 2012-07-19 22:22:53 +01:00
Hieu Hoang
d57216575a undo changes to make scripts handle directories with spaces. Karel gave up, and it messing up cruise control 2012-07-19 22:21:41 +01:00
Hieu Hoang
8206950336 under changes to make scripts handle directories with spaces. 2012-07-18 22:06:45 +01:00
Hieu Hoang
b609473645 under changes to make scripts handle directories with spaces. Karel gave up, and it messing up cruise control 2012-07-18 11:02:24 +01:00
Barry Haddow
2b4e61d826 Merge branch 'trunk' into miramerge 
Compiles, not tested.

Conflicts:
	Jamroot
	OnDiskPt/PhraseNode.h
	OnDiskPt/TargetPhrase.cpp
	OnDiskPt/TargetPhrase.h
	OnDiskPt/TargetPhraseCollection.cpp
	mert/BleuScorer.cpp
	mert/Data.cpp
	mert/FeatureData.cpp
	moses-chart-cmd/src/Main.cpp
	moses/src/AlignmentInfo.h
	moses/src/ChartManager.cpp
	moses/src/LM/Ken.cpp
	moses/src/LM/Ken.h
	moses/src/LMList.h
	moses/src/LexicalReordering.h
	moses/src/PhraseDictionaryTree.h
	moses/src/ScoreIndexManager.h
	moses/src/StaticData.h
	moses/src/TargetPhrase.h
	moses/src/Word.cpp
	scripts/ems/experiment.meta
	scripts/ems/experiment.perl
	scripts/training/train-model.perl
 2012-07-17 13:36:50 +01:00
Karel Bílek
ad257b27d9 spaces in filepath does not wreak havoc anymore 2012-07-17 04:36:27 +02:00
Ales Tamchyna
0594a7706d fixed off-by-one bug 2012-07-06 14:18:39 -04:00
Ales Tamchyna
19d8afe7df adding reduce-topt-count.pl, filters translation options that will be discarded by the decoder 2012-07-06 13:48:05 -04:00
Hieu Hoang
8d57651d0f move mert-moses-multi.pl 2012-07-06 12:58:06 +01:00
Barry Haddow
afd654eb74 Merge branch 'master' of github.com:moses-smt/mosesdecoder 2012-07-05 21:02:15 +01:00
Barry Haddow
278ec10603 Add warning 2012-07-05 21:01:59 +01:00
Eva Hasler
f592c222fe revert mode changes 2012-07-04 12:25:21 +01:00
Eva Hasler
027a20730e merge Jamfiles 2012-07-04 11:49:07 +01:00
phikoehn
c8b2e89573 integrated sparse lexical features into experiment.perl, needs some more testing. 2012-07-03 06:20:09 +01:00
phikoehn
ff79f9f054 fix conflict 
Merge branch 'master' of git://github.com/moses-smt/mosesdecoder

Conflicts:
	scripts/ems/experiment.perl
 2012-07-03 00:05:13 +01:00
phikoehn
ce65a47f0d count bin feature 2012-07-03 00:00:21 +01:00
Ales Tamchyna
4c6d2e7b88 updated path to mert in mert-moses.pl 2012-06-28 08:56:06 -04:00
phikoehn
462f5f992d bug fix 2012-06-28 13:03:41 +01:00
Hieu Hoang
debe090426 Change Bin to RealBin. Thanks to Tom Hoar 2012-06-26 11:57:23 -04:00
Hieu Hoang
5a7db584b8 Merge https://github.com/moses-smt/mosesdecoder 2012-06-16 00:46:03 +01:00
Colin Cherry
07a5c67ebc Merge branch 'master' into miramerge 
Conflicts:
	Jamroot
	misc/queryPhraseTable.cpp
	scripts/training/train-model.perl
 2012-06-14 17:08:16 -04:00
Colin Cherry
742f2d047f Fixed a bug in --return-best-dev 2012-06-12 23:02:04 -04:00
Colin Cherry
4d284b25ad Added --return-best-dev to mert-moses.pl 
Copies the run*.moses.ini with the highest development BLEU to
moses.ini, instead of copying the weights from the last run of
optimization.
Recommended when using kbmira (and probably pro).
 2012-06-12 09:16:11 -04:00