Commit Graph

11534 Commits

Author SHA1 Message Date
Rico Sennrich
01bc3c111e swap position of alignment and scores in phrase table halves (before consolidate step).
ensures that multiple hierarchical rules with same source/target phrase, but different alignment, are sorted correctly
2014-03-02 16:55:42 +00:00
Kenneth Heafield
952bb76b77 We weren't linking against tcmalloc... 2014-03-01 07:42:32 +00:00
Jun-ya NORIMATSU
14d9d50b76 Merge remote-tracking branch 'origin/master' 2014-02-28 13:39:16 +09:00
Jun-ya NORIMATSU
19bc1c317d improve DALMwrapper again. 2014-02-28 13:38:35 +09:00
Hieu Hoang
b48f410bee don't check object pointer before deleting /Markus Elfring 2014-02-26 10:43:37 +00:00
Hieu Hoang
c48d8b5e31 update last weight override 2014-02-26 10:32:48 +00:00
Ulrich Germann
3f678fa689 Updated pretokenizer for French. 2014-02-25 03:10:58 +00:00
Ulrich Germann
2b181ee691 Fixed Mmsapt constructor. 2014-02-25 03:10:16 +00:00
Ulrich Germann
4c003edb0d Fixed #include-s. 2014-02-25 03:09:19 +00:00
Hieu Hoang
b9c6c7a2a6 need to zero out all weights, especially sparse weights 2014-02-23 21:35:34 +00:00
Hieu Hoang
7509e8bbaa reset sparse weights 2014-02-23 20:46:16 +00:00
Hieu Hoang
1716514eda create ResetWeights(). Only for dense weights 2014-02-23 20:07:28 +00:00
Ulrich Germann
e70766a6de Preprocessor for tokenization that fixes known errors in the raw input. 2014-02-22 00:28:33 +00:00
Ulrich Germann
a8d66cd68d Removed Mmsapt constuctor with both descriptor and config line. 2014-02-22 00:27:07 +00:00
Ulrich Germann
817e3695e0 Fixed some include paths. 2014-02-22 00:25:58 +00:00
Ulrich Germann
1252700c44 Removed constructor with both description and config line. 2014-02-22 00:25:02 +00:00
Ulrich Germann
d3d14ccf3d - added option -no-escape to skip escaping of special characters
- fixed bug in aggressive tokenization
  (est-a-dire was split as est @-@ a-dire
              instead of   est @-@ a @-@ dire)
2014-02-21 14:14:03 +00:00
Ulrich Germann
ef1d5b762e Merge branch 'dynamic-phrase-tables' of file:///fs/sif0/germann/git/mosesdecoder into dynamic-phrase-tables 2014-02-21 12:59:52 +00:00
Ulrich Germann
4b95c3a906 Merge branch 'dynamic-phrase-tables' of ssh://thor//home/germann/git/mosesdecoder into dynamic-phrase-tables
due to resetting the location of the remote repository.
2014-02-21 01:09:38 +00:00
Ulrich Germann
ef2ef881a4 Merge branch 'dynamic-phrase-tables' of file:///home/germann/git/mosesdecoder into dynamic-phrase-tables 2014-02-21 01:04:02 +00:00
Ulrich Germann
b410026804 Added test program try-align. 2014-02-20 23:56:45 +00:00
Ulrich Germann
ac238ef2d7 Changed construction from a given token sequence to allow partial matches. 2014-02-20 23:56:11 +00:00
Ulrich Germann
8afe62145b Minor fix to make the compiler stop complain about unused typedef. 2014-02-20 23:54:15 +00:00
Ulrich Germann
e1d07e7475 Added pid2str conversion method to convert from phrase ids to the string. 2014-02-20 23:53:15 +00:00
Ulrich Germann
9536cf49e9 Phrase look-up now also gathers phrase orientation info (work in progress). 2014-02-20 23:51:17 +00:00
Ulrich Germann
6c66b9c631 Added Jamfile to produce try-align 2014-02-20 23:50:07 +00:00
Ulrich Germann
683635ce25 Minor fix to make the compiler stop complaining about unused variables. 2014-02-20 23:48:56 +00:00
Ulrich Germann
061b861639 Small test program for phrase-based alignment via mmsapt. 2014-02-20 23:29:37 +00:00
Ulrich Germann
c259e10b23 Various changes. 2014-02-20 23:28:01 +00:00
Ulrich Germann
9bcc315644 Added phrase-based word alignment to mmsapt (work in progress!). 2014-02-20 23:25:36 +00:00
Jun-ya NORIMATSU
007799eb07 modify DALMwrapper. 2014-02-19 17:49:42 +09:00
Hieu Hoang
641145a5c6 Merge pull request #59 from akholy/master
Update train-model.perl
2014-02-18 16:52:26 +00:00
Jun-ya NORIMATSU
ba63f1eb97 improve DALMwrapper. 2014-02-18 00:00:28 +09:00
Jun-ya NORIMATSU
4488d97629 tune DALMWrapper. 2014-02-14 20:22:53 +09:00
Rico Sennrich
2ac7694f7e copy weights of features with 'tuneable=false' to final config (now also works with -return-best-dev) 2014-02-13 14:19:40 +00:00
phikoehn
049be8b71c Merge branch 'master' of ssh://github.com/moses-smt/mosesdecoder 2014-02-12 21:01:09 +00:00
phikoehn
d6b62db5b1 fix bug if interpolated lm on different factors 2014-02-12 21:00:55 +00:00
Hieu Hoang
50cadc754f use boost::unordered_map for CacheColl. Marginally faster 2014-02-11 03:43:58 +00:00
Ulrich Germann
a6ce081e15 Minor changes. 2014-02-08 18:25:46 +00:00
Ulrich Germann
6d2818fa57 Fixed include path for msapt.h 2014-02-08 18:22:33 +00:00
Ulrich Germann
af007c4cf0 Minor fixes related to mm. 2014-02-08 18:18:45 +00:00
Ulrich Germann
11aa1422a5 Added a few mm-related things. 2014-02-08 18:07:16 +00:00
Ulrich Germann
594272ce05 Changed function count_tokens so that it can be run without passing a filter explicitly. 2014-02-08 18:06:11 +00:00
Ulrich Germann
9899364c46 Added implicit add-1 smoothing. 2014-02-08 18:03:18 +00:00
Ulrich Germann
40fbe226e4 Added private members numSent and numWords. 2014-02-08 18:02:03 +00:00
Ulrich Germann
66822b279b Added append function to grow imTtracks dynamically in a thread-safe fashion. 2014-02-08 18:00:27 +00:00
Ulrich Germann
9f317f4849 Minor fix. 2014-02-08 17:58:05 +00:00
Ulrich Germann
5f8ae20d01 Added dynamicly updatable corpus; updated or added query functions. 2014-02-08 17:56:48 +00:00
Ulrich Germann
a74d7e6095 Bug fix? 2014-02-08 17:54:48 +00:00
Ulrich Germann
784654c831 Initial check-in. 2014-02-08 17:50:26 +00:00