Hieu Hoang
018998247a
create script to convert phrase-table with alignment in Moses' dead-end format to standard format
2013-07-25 12:52:05 +01:00
Barry Haddow
f79746b3c2
Merge branch 'master' of github.com:moses-smt/mosesdecoder
2013-07-24 20:49:59 +01:00
Hieu Hoang
6fc21a32fc
Merge branch 'master' of github.com:moses-smt/mosesdecoder
2013-07-24 19:01:57 +01:00
Hieu Hoang
c104dee3b2
merge glue grammars, rather than writing them all to the same file. Required by Phil Williams & others when doing syntax extraction
2013-07-24 19:01:46 +01:00
Achim Ruopp
1813f9784b
Additional factoring to allow more NE recognizers; bug fixes
2013-07-24 12:44:53 -04:00
Barry Haddow
46ee1ca42d
More lattice fixes squashed by merge
2013-07-24 16:09:32 +01:00
Barry Haddow
0ce50a4c70
Merge branch 'master' of github.com:moses-smt/mosesdecoder
2013-07-24 15:58:08 +01:00
Phil Williams
1238041f98
Add option to do Penn Treebank style tokenization
...
tokenizer.perl and detokenizer.perl now have an option called -penn
which does Penn Treebank-like tokenization (English only). This is
useful if your pipeline involves processing the corpus with tools
trained on PTB-tokenized text.
Unlike PTB, the tokenizer splits on slashes (e.g. "Monday/Tuesday"
becomes "Monday", "@/@", "Tuesday"). If using parse-de-berkeley.perl,
the option -split-slash re-joins tokens that are separated by slashes
for parsing then splits them afterwards.
2013-07-24 13:41:21 +01:00
Barry Haddow
d5e40a5b08
Merge branch 'master' of github.com:moses-smt/mosesdecoder
2013-07-24 11:38:23 +01:00
Nadir Durrani
30544ae17e
Sample Config File
2013-07-23 12:29:23 +01:00
Nadir Durrani
61e56ecdcd
Sample Config File
2013-07-23 12:18:57 +01:00
Barry Haddow
50de0e06d1
Generate correct ini file for lattices
2013-07-23 11:46:37 +01:00
Barry Haddow
8ed8bcafc2
Merge branch 'master' of github.com:moses-smt/mosesdecoder
2013-07-23 11:21:47 +01:00
Barry Haddow
887d5dad62
Restore EMS lattice fixes, squashed by merge.
2013-07-23 10:38:11 +01:00
Phil Williams
91cc7c329e
parse-de-berkeley.perl: escape @ characters in input
2013-07-23 10:22:56 +01:00
Barry Haddow
ecc6c7177c
Reinstate lattice fixes squashed by merge
2013-07-22 17:25:01 +01:00
unknown
54eb50523b
Converted into modulino; added support for French numbers
2013-07-19 14:41:01 -04:00
Hieu Hoang
17366b9d7e
add mada wrapper from Phi's collection
2013-07-13 22:42:28 +01:00
Nadir Durrani
418abf42fa
Merge branch 'nadir_osm'
2013-07-09 11:44:14 +01:00
Hieu Hoang
2203bb3284
beautify
2013-07-08 19:08:31 +01:00
Nadir Durrani
deae3ac7b9
OSM entries in train-model.perl, experiment.*
2013-07-07 13:05:09 +01:00
Nadir Durrani
d2bc6a2584
In EMS
2013-07-04 19:58:19 +01:00
Hieu Hoang
0e46cd377c
Merge branch 'master' into nadir_osm
2013-07-03 20:24:20 +01:00
Nadir Durrani
fbdb07a94c
EMS
2013-07-03 10:54:38 +01:00
Nadir Durrani
82d6105f05
OSM Training Script
2013-07-02 13:59:47 +01:00
Hieu Hoang
4e4cf1e313
script to replace numbers with placeholder. /Achim Ruopp
2013-07-01 23:00:59 +01:00
Wilker Aziz
f3cd72537c
Merge branch 'master' of https://github.com/moses-smt/mosesdecoder
2013-06-24 15:39:18 +01:00
Wilker Aziz
2c19238c24
Patching up the suffix array wrappers
2013-06-24 15:38:10 +01:00
Wilker Aziz
b49e6a162f
Wrapper to lmplz
2013-06-24 12:20:20 +01:00
Hieu Hoang
a85f819a53
superceded
2013-06-24 11:33:11 +01:00
phikoehn
f5b8c47a2e
Merge branch 'master' of ssh://github.com/moses-smt/mosesdecoder
2013-06-23 17:19:37 +01:00
phikoehn
164b06cd7e
debugging
2013-06-23 17:19:22 +01:00
Hieu Hoang
dc33fa3d3d
redo parsing of feature function parameters
2013-06-20 12:50:41 +01:00
Hieu Hoang
029110c245
change table-limit specification to new format
2013-06-14 10:09:06 +01:00
Hieu Hoang
abe6bb7c22
refactor parsing of feature functiona args
2013-06-10 18:11:55 +01:00
phikoehn
54f2ea07bd
handle sparse features in translation table
2013-06-09 20:00:19 +01:00
phikoehn
ce372477c9
conversion script from Moses V1.0 moses.ini files to current format - may need some further tweaking
2013-06-09 14:28:56 +01:00
phikoehn
2e8fbe77a2
corrected example files
2013-06-08 14:45:55 +01:00
phikoehn
730da7edec
sparse feature specification bug fix
2013-06-08 13:39:15 +01:00
Hieu Hoang
a974bbafac
Merge pull request #37 from neubig/ems-interpolate-scientific
...
Fixed crash in interpolation for small lambdas
2013-06-07 16:24:32 -07:00
Rico Sennrich
8581fb9518
fix (minor) unicode warning, and update permissions
2013-06-03 13:48:31 +02:00
Graham Neubig
33d5aac6af
Fixed crash in interpolation for small lambdas
...
The EMS crashed when interpolating language models when the ideal lambdas included numbers so small that they required scientific notation (eg: 1.332e-07). This patch adds "e" and "-" to the acceptable numbers to fix this problem
2013-06-01 12:37:24 +09:00
phikoehn
68501f5a36
bug fix with weight substitution
2013-05-31 12:27:35 +01:00
Hieu Hoang
f83622b0b7
figure out which feature function to apply at which decode step. Book-keeping
2013-05-30 17:16:10 +01:00
Hieu Hoang
8f7c12ef40
beautify
2013-05-29 18:19:06 +01:00
Hieu Hoang
6249432407
beautify
2013-05-29 18:16:15 +01:00
Hieu Hoang
beaf295741
beautify
2013-05-27 17:47:54 +01:00
Hieu Hoang
6996973a56
beautify
2013-05-27 17:42:27 +01:00
phikoehn
8944ea541a
fast align parameter
2013-05-25 23:20:27 +01:00
phikoehn
542cd72c63
moved config creation back into train-model.perl
2013-05-19 03:28:02 +01:00
