moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2024-12-27 22:14:57 +03:00
Code Issues Packages Projects Releases Wiki Activity
dfd3a53e9e
mosesdecoder/scripts/training/bilingual-lm
History
Rico Sennrich 4ca730a67c improve bilingualLM alignment heuristics consistency
2014-11-26 10:32:41 +00:00
..
averageNullEmbedding_baseline.py move bilingual-lm training scripts 2014-11-26 10:32:37 +00:00
extract_test.py move bilingual-lm training scripts 2014-11-26 10:32:37 +00:00
extract_training.py move bilingual-lm training scripts 2014-11-26 10:32:37 +00:00
extract.py improve bilingualLM alignment heuristics consistency 2014-11-26 10:32:41 +00:00
README move bilingual-lm training scripts 2014-11-26 10:32:37 +00:00
reduce_ngrams.py move bilingual-lm training scripts 2014-11-26 10:32:37 +00:00
tag.sh move bilingual-lm training scripts 2014-11-26 10:32:37 +00:00
test_nplm.py move bilingual-lm training scripts 2014-11-26 10:32:37 +00:00
train_nplm.py move bilingual-lm training scripts 2014-11-26 10:32:37 +00:00

README 

Example usage:
#create training and test corpus
/home/abmayne/code/deepathon/nnjm/extract_training.py --working-dir  /home/abmayne/experiments/2014-iwslt/nplm/en-cs10k --corpus corpus/europarl.clean.10k --target-language cs --source-language en --align corpus/europarl.clean.10k.align
/home/abmayne/code/deepathon/nnjm/extract_test.py --working-dir  /home/abmayne/experiments/2014-iwslt/nplm/en-cs10k --corpus corpus/europarl.test.10k --target-language cs --source-language en --align corpus/europarl.test.10k.align

#Train and test different language models with basic nplm training
/home/abmayne/code/deepathon/nnjm/train_nplm.py --working-dir /home/abmayne/experiments/2014-iwslt/nplm/en-cs10k --corpus europarl.clean.10k --minibatch-size 128 --epochs 40 --output-model europarl.10k.bbn  --nplm-home  /home/abmayne/code/deepathon/nplm_one_layer --hidden 0 --threads 1 --output-model europarl.10k.1layer
/home/abmayne/code/deepathon/nnjm/test_nplm.py --working-dir /home/abmayne/experiments/2014-iwslt/nplm/en-cs10k --corpus europarl.test.10k --train-corpus europarl.10k.1layer --nplm-home  /home/abmayne/code/deepathon/nplm_one_layer --threads 1