moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2024-12-26 05:14:36 +03:00
Code Issues Packages Projects Releases Wiki Activity
79cd40d2c4
mosesdecoder/scripts/training/bilingual-lm
History
Rico Sennrich 2271f295e6 nplm_train: more options
2015-03-20 14:12:41 +00:00
..
averageNullEmbedding.py typo 2015-03-04 10:06:03 +00:00
extract_test.py off-by-one error in previous commit 2015-03-04 17:25:19 +00:00
extract_training.py extract_test.py should also create numberized corpus 2015-03-04 17:10:06 +00:00
extract.py extract_test.py should also create numberized corpus 2015-03-04 17:10:06 +00:00
README move bilingual-lm training scripts 2014-11-26 10:32:37 +00:00
reduce_ngrams.py move bilingual-lm training scripts 2014-11-26 10:32:37 +00:00
tag.sh move bilingual-lm training scripts 2014-11-26 10:32:37 +00:00
test_nplm.py move bilingual-lm training scripts 2014-11-26 10:32:37 +00:00
train_nplm.py nplm_train: more options 2015-03-20 14:12:41 +00:00

README 

Example usage:
#create training and test corpus
/home/abmayne/code/deepathon/nnjm/extract_training.py --working-dir  /home/abmayne/experiments/2014-iwslt/nplm/en-cs10k --corpus corpus/europarl.clean.10k --target-language cs --source-language en --align corpus/europarl.clean.10k.align
/home/abmayne/code/deepathon/nnjm/extract_test.py --working-dir  /home/abmayne/experiments/2014-iwslt/nplm/en-cs10k --corpus corpus/europarl.test.10k --target-language cs --source-language en --align corpus/europarl.test.10k.align

#Train and test different language models with basic nplm training
/home/abmayne/code/deepathon/nnjm/train_nplm.py --working-dir /home/abmayne/experiments/2014-iwslt/nplm/en-cs10k --corpus europarl.clean.10k --minibatch-size 128 --epochs 40 --output-model europarl.10k.bbn  --nplm-home  /home/abmayne/code/deepathon/nplm_one_layer --hidden 0 --threads 1 --output-model europarl.10k.1layer
/home/abmayne/code/deepathon/nnjm/test_nplm.py --working-dir /home/abmayne/experiments/2014-iwslt/nplm/en-cs10k --corpus europarl.test.10k --train-corpus europarl.10k.1layer --nplm-home  /home/abmayne/code/deepathon/nplm_one_layer --threads 1