moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2024-12-26 13:23:25 +03:00
Code Issues Packages Projects Releases Wiki Activity
11,314 Commits 136 Branches 26 Tags 147 MiB
e8f00eb420
Commit Graph

3200 Commits

Author SHA1 Message Date
phikoehn
e8f00eb420 Merge branch 'master' of ssh://github.com/moses-smt/mosesdecoder 2014-01-05 22:38:51 +00:00
Hieu Hoang
1431cc5688 delete check.hh 2014-01-05 18:19:51 +00:00
phikoehn
6c34ff207f additional time reporting and stardardization 2014-01-05 02:26:47 +00:00
phikoehn
6846089089 Merge branch 'master' of ssh://github.com/moses-smt/mosesdecoder 2014-01-03 18:45:45 +00:00
phikoehn
073a601eb0 make good use of the new precise timer... 2014-01-03 18:45:31 +00:00
Rico Sennrich
58ed19ce6a remove hacks for phrase penalty from MultiModel phrase type. 2014-01-02 12:29:38 +01:00
Kenneth Heafield
732a1b0744 KenLM 85c82bd, revamp Moses timer to have more precision 2014-01-01 13:19:06 -08:00
Hieu Hoang
18aaf4750a destroy feature functions at end. Otherwise it shows up on valgrind as memory leak 2013-12-31 20:15:43 +00:00
Hieu Hoang
02b78d72d9 update sample query to read ini file with relative file paths 2013-12-31 16:52:47 +00:00
Lane Schwartz
867eabd699 Re-enabled unit test for backward LM. 
The test is compiling and running with no errors.
 2013-12-20 15:49:55 -05:00
Lane Schwartz
f40e1ac8f8 Added support for hierarchical maximum entropy LM through SRILM. 
Currently, the code compiles, but has not been run or tested.

    Steps to compile with maxent LM are below:

    - Download SRILM 1.6.0

    - Download the maxent patch from
      http://www.phon.ioc.ee/~tanela/srilm-me/srilm-1.6.0-me.patch

    - Download the required libLBFGS library from
      http://www.chokkan.org/software/liblbfgs

    - Edit SRILM's sbin/machine-type script
      to return MACHINE_TYPE=i686-m64 for case x86_64

    - Compile libLBFGS and install it in the /tools/SRILM/SRILM-1.7.0 directory.
      Make sure that the liblbfgs.* library files are in /tools/SRILM/SRILM-1.7.0/lib//

    - In the /tools/SRILM/SRILM-1.7.0 dir, apply the maxent patch to SRILM:
      patch -p1 < srilm-1.6.0-me.patch

    - Point the appropriate SRILM makefile
      to see libLBFGS's include and lib dirs, following the instructions at:
      http://www.phon.ioc.ee/dokuwiki/doku.php?id=people:tanel:srilm-me.en

    - Compile SRILM

    - Compile Moses using --with-srilm and --with-maxent-srilm:
      ./bjam --with-srilm=/tools/SRILM/SRILM-1.7.0 --with-maxent-srilm=true

    NOTE: The above steps were for my original integration.
          Current versions of SRILM (such as 1.7.1) include the maxent patch.
          As such, it should be sufficient to compile a current version of SRILM
          with support form maxent enabled,
          and then compile Moses using --with-srilm and --with-maxent-srilm:
          ./bjam --with-srilm=/tools/SRILM/SRILM-1.7.0 --with-maxent-srilm=true
 2013-12-20 15:04:09 -05:00
Lane Schwartz
83f89ac2b2 Updated Jamfile for compiling and linking against NPLM 2013-12-20 13:49:56 -05:00
Hieu Hoang
dea1fe995f default factor to 0 for language models that don't specify what factor type 2013-12-18 20:54:53 +00:00
Hieu Hoang
1555bf3b9e Merge branch 'master' of github.com:moses-smt/mosesdecoder 2013-12-17 15:55:44 +00:00
Hieu Hoang
e8b2d2a269 bug - should get score for translation rule from the translation option, not the target phrase. 2013-12-17 14:54:12 +00:00
Hieu Hoang
f7cff6d692 Merge pull request #55 from jnory/master 
DALMWrapper using more efficient state class in DALM.
 2013-12-17 03:28:40 -08:00
Jun-ya NORIMATSU
da4e2719b2 modify DALMState to use the DALM compare function. 2013-12-17 14:36:19 +09:00
Jun-ya NORIMATSU
dbf90ce8e8 more efficient state for DALM. 2013-12-16 23:17:56 +09:00
Hieu Hoang
597cb4a20f added negate=true. Only create translation which DON'T match reference 2013-12-15 01:42:22 +00:00
Hieu Hoang
ea8837b5b9 delete debugging message 2013-12-14 16:15:27 +00:00
phikoehn
7763b4c41f sparse feature support for alternate weight setting 2013-12-14 02:56:01 +00:00
Hieu Hoang
ff3c8c195a compare translation option dimension using pointers to translation option, rather than comparing the phrase. Comparinf the phrase is incorrect - need to compare LHS and possibly alignments too. Unnecessary 2013-12-13 17:23:39 +00:00
Hieu Hoang
3bd5583976 more debug functions 2013-12-13 15:47:26 +00:00
Hieu Hoang
95a41c44fa more debug functions 2013-12-12 01:20:12 +00:00
Hieu Hoang
06b0b6ae87 c++ acting like a dumbass untyped scripting language and silently cast variables. Hashing of some obscure object didn't work on old boost. Broke target syntax when linking against those old boost libraries 2013-12-12 01:13:23 +00:00
Hieu Hoang
7c237bf389 delete -#if defined(BOOST... Trying to find target syntax bug when using old boost 2013-12-11 21:45:30 +00:00
Hieu Hoang
bc9ce334d0 more debug functions 2013-12-11 20:03:30 +00:00
Hieu Hoang
fb356606e9 debug function for ChartTranslationOptionList and ChartTranslationOptions 2013-12-11 19:19:25 +00:00
Hieu Hoang
ebc6e59b17 consistent global variable g_mosesDebug 2013-12-11 19:04:42 +00:00
Hieu Hoang
295c07e884 bug in constrained decoding 2013-12-10 18:47:13 +00:00
Hieu Hoang
e64368e7e4 Forgot to call SetFeaturesToApply() for SkeletonPT and Transliteration phrase table 2013-12-10 12:42:05 +00:00
Hieu Hoang
7bae70a2bc minor. More description of -feature-overwrite 2013-12-09 11:53:46 +00:00
Hieu Hoang
ba209202ef move all feature functions out of StaticData 2013-12-07 00:21:06 +00:00
Hieu Hoang
a5d0296699 make transliteration phrase table thread safe 2013-12-06 16:25:17 +00:00
Hieu Hoang
09d50f87b1 move word penalty feature out of StaticData 2013-12-06 15:57:49 +00:00
Hieu Hoang
f77fd7fb45 set parameters for fuzzy match. Not tested 2013-12-06 14:23:22 +00:00
Hieu Hoang
78eba93a6b minor change in argument key for consistency 2013-12-06 13:09:34 +00:00
Hieu Hoang
0e156afd5b Merge branch 'master' of github.com:moses-smt/mosesdecoder 2013-12-06 13:07:40 +00:00
Hieu Hoang
fc8f8ba657 minor change in argument key for consistency 2013-12-06 13:07:33 +00:00
Rico Sennrich
96ca814b86 special handling of partial rules that can only have terminal symbol next; 
we only need to search once for matching terminals.

speed: 336s -> 309s
VmPeak: 2914132kB -> 2783064kB
RSSMax: 2766976kB -> 2699148kB
 2013-12-06 12:54:58 +00:00
Hieu Hoang
3ab1a9996b comment 2013-12-06 11:32:53 +00:00
Hieu Hoang
cc9ecdff0a rename TransliterationPhraseDictionary to PhraseDictionaryTransliteration 2013-12-05 13:13:01 +00:00
Hieu Hoang
b33cf30bc1 transliteration pt obery backoff argument. Probably not threadable yet 2013-12-05 13:06:35 +00:00
Hieu Hoang
d3301fa1a3 move backoff parameter into decode graph class 2013-12-05 12:19:55 +00:00
Hieu Hoang
0dac035a12 arguments for transliteration phrase-table 2013-12-04 19:16:37 +00:00
Hieu Hoang
51f5bbea5d transliteration phrase-table now working. Just need to not hardcode some variables and make it obey -decoding-graph-backoff 2013-12-04 18:47:33 +00:00
Ales Tamchyna
64b629803d fixed a bug in CoveredRerefenceFeature 2013-12-04 10:21:01 +01:00
Ales Tamchyna
81ad92cc3f minor 2013-12-03 17:29:50 +01:00
Ales Tamchyna
fea32c7182 debugged CoveredReferenceFeature 2013-12-03 17:25:42 +01:00
Ales Tamchyna
8c0b5ffa6f early evaluation of CoveredReferenceFeature 2013-12-03 17:03:41 +01:00