moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2024-12-28 14:32:38 +03:00
Code Issues Packages Projects Releases Wiki Activity
11,313 Commits 136 Branches 26 Tags 147 MiB
a102bbb33c
Commit Graph

146 Commits

Author SHA1 Message Date
Lane Schwartz
f57ac23be9 Added comment in Jamroot on how to compile against NPLM 2013-12-20 15:06:49 -05:00
Lane Schwartz
f40e1ac8f8 Added support for hierarchical maximum entropy LM through SRILM. 
Currently, the code compiles, but has not been run or tested.

    Steps to compile with maxent LM are below:

    - Download SRILM 1.6.0

    - Download the maxent patch from
      http://www.phon.ioc.ee/~tanela/srilm-me/srilm-1.6.0-me.patch

    - Download the required libLBFGS library from
      http://www.chokkan.org/software/liblbfgs

    - Edit SRILM's sbin/machine-type script
      to return MACHINE_TYPE=i686-m64 for case x86_64

    - Compile libLBFGS and install it in the /tools/SRILM/SRILM-1.7.0 directory.
      Make sure that the liblbfgs.* library files are in /tools/SRILM/SRILM-1.7.0/lib//

    - In the /tools/SRILM/SRILM-1.7.0 dir, apply the maxent patch to SRILM:
      patch -p1 < srilm-1.6.0-me.patch

    - Point the appropriate SRILM makefile
      to see libLBFGS's include and lib dirs, following the instructions at:
      http://www.phon.ioc.ee/dokuwiki/doku.php?id=people:tanel:srilm-me.en

    - Compile SRILM

    - Compile Moses using --with-srilm and --with-maxent-srilm:
      ./bjam --with-srilm=/tools/SRILM/SRILM-1.7.0 --with-maxent-srilm=true

    NOTE: The above steps were for my original integration.
          Current versions of SRILM (such as 1.7.1) include the maxent patch.
          As such, it should be sufficient to compile a current version of SRILM
          with support form maxent enabled,
          and then compile Moses using --with-srilm and --with-maxent-srilm:
          ./bjam --with-srilm=/tools/SRILM/SRILM-1.7.0 --with-maxent-srilm=true
 2013-12-20 15:04:09 -05:00
Hieu Hoang
0eff7344d7 move Uli's new pt into it's own folder. Comment out bjam compile. Doesn't work on MacOSX yet 2013-09-26 10:28:52 +01:00
=
d8686f79db Reformatting. 2013-09-25 00:11:11 +01:00
Kenneth Heafield
5e506ed914 Make dl have the same linkage as glibc 2013-09-12 09:27:01 +01:00
Hieu Hoang
b534e8c4b7 f*cking link error 2013-09-12 09:17:18 +01:00
Kenneth Heafield
6b5c1a09e4 unwind library, cleanup 2013-08-22 10:16:12 +01:00
Kenneth Heafield
37c22cd4da Also use unwind for for tcmalloc 2013-08-21 14:29:00 +01:00
Kenneth Heafield
b1d7340b78 Add --full-tcmalloc to link against the non-minimal tcmalloc library 2013-08-21 10:16:33 +01:00
Hieu Hoang
5fa6e506b1 Jamroot error 2013-08-15 15:39:40 +01:00
phikoehn
b368085609 xml constraint 2013-08-15 11:46:45 +01:00
Barry Haddow
c127c58e9b fix to single thread build 2013-07-25 15:56:20 +01:00
Hieu Hoang
418652fe1d iconv for SRILM 1.7.0 on MacOSX 2013-07-01 10:57:06 +01:00
Kenneth Heafield
215e93377c Move iconv to everything but for OS X only 2013-07-01 09:01:56 +01:00
Kenneth Heafield
0d9020fefe Add message for missing tcmalloc and improve install directions 2013-06-18 13:31:37 -04:00
Lane Schwartz
aacc936204 Adding unit tests to Backward LM 2013-04-29 09:18:46 -04:00
Kenneth Heafield
030e2d40b2 Use tcmalloc_minimal and tell people how to build it 2012-11-14 20:29:55 +00:00
Kenneth Heafield
be7e78c6bf Use tcmalloc with multithread if detected. 
Change option and documentation to --without-tcmalloc.
 2012-11-14 20:01:07 +00:00
Hieu Hoang
f96b33de83 only include moses root when compiling 2012-11-14 13:43:04 +00:00
Hieu Hoang
0bc0b85358 resolved merge conflicts 2012-11-12 21:12:01 +00:00
Hieu Hoang
125fdbcff6 move moses-cmd/src/* to moses-cmd/ 2012-11-12 20:42:29 +00:00
Hieu Hoang
5e3ef23cef move moses/src/* to moses/ 2012-11-12 19:56:18 +00:00
Barry Haddow
62fa6d6f28 Feature function interface for use in scoring 2012-11-02 23:30:51 +00:00
Hieu Hoang
87b6520d82 add moses server back in. Changed ScoreComponentCollection to expose dense feature array 2012-10-22 14:41:23 +01:00
Hieu Hoang
6647ec4a41 don't compile moses server for the moment. Doesn't compile on some gcc/boost combinations 2012-10-22 11:40:26 +01:00
Kenneth Heafield
5f67527143 Bugfix vocabulary corruption on building 2012-10-19 12:00:10 +01:00
Barry Haddow
365e680115 Merge remote-tracking branch 'origin/master' into miramerge 
NB Untested

Conflicts:
	Jamroot
	moses-chart-cmd/src/Main.cpp
	moses/src/ChartManager.cpp
	moses/src/RuleTable/LoaderStandard.cpp
	moses/src/RuleTable/PhraseDictionaryALSuffixArray.cpp
	moses/src/Word.cpp
 2012-10-15 21:35:56 +01:00
Kenneth Heafield
0eb98df0fe Submodules are evil. Undo it. 2012-10-15 13:58:33 +01:00
Barry Haddow
a638736722 Reinstate Moses unit tests 2012-10-11 21:33:11 +01:00
Barry Haddow
0a950ee9f4 Merge remote branch 'github/master' into miramerge 
Compiles, but not tested. Had to disable relent filter. Strangely, it seems to contain the
whole of moses-cmd.

Conflicts:
	Jamroot
	OnDiskPt/TargetPhrase.cpp
	moses-cmd/src/Main.cpp
	moses/src/AlignmentInfo.cpp
	moses/src/AlignmentInfo.h
	moses/src/ChartTranslationOptionCollection.cpp
	moses/src/ChartTranslationOptionCollection.h
	moses/src/GenerationDictionary.cpp
	moses/src/Jamfile
	moses/src/Parameter.cpp
	moses/src/PhraseDictionary.cpp
	moses/src/StaticData.cpp
	moses/src/StaticData.h
	moses/src/TargetPhrase.h
	moses/src/TranslationSystem.cpp
	moses/src/TranslationSystem.h
	moses/src/Word.cpp
	phrase-extract/score.cpp
	regression-testing/Jamfile
	scripts/ems/experiment.meta
	scripts/ems/experiment.perl
	scripts/training/train-model.perl
 2012-09-26 22:49:33 +01:00
Kenneth Heafield
4faab6c68f Extract kenlm into a submodule, referencing lazy 2012-09-24 18:30:51 +01:00
wlin12
6202c7cc60 adding the code for relative entropy pruning to contrib/relent-filter 2012-09-07 17:33:44 +01:00
Kenneth Heafield
b317522563 Move max-order to lm directory and direct dependencies. 2012-08-18 12:07:53 -04:00
Lane Schwartz
2556dfc0e5 If --install-scripts is provided with no argument, install to $prefix/scripts 
If --prefix is specified, but --install-scripts is not, do not install scripts
 2012-08-08 16:50:33 -04:00
Lane Schwartz
a2afc98665 Add program to query the KenLM maximum n-gram order 2012-08-08 16:41:29 -04:00
Lane Schwartz
da5429318b KenLM maximum n-gram order can now be set via a compile-time flag 2012-08-08 16:22:13 -04:00
Kenneth Heafield
b52ed110b0 Build changes. --static will force static linkage. 
--without-libsegfault will skip libSegFault.
 2012-08-06 10:41:49 -04:00
Marcin Junczys-Dowmunt
767969af6d Jamroot 2012-08-04 22:49:42 +02:00
Marcin Junczys-Dowmunt
a245e9ea55 Correction of jamfiles etc. 2012-08-04 15:39:30 +02:00
Marcin Junczys-Dowmunt
88a860d776 Warnings, build issues 2012-08-03 19:29:21 +02:00
Marcin Junczys-Dowmunt
a35791893f Updated Jamsfiles, build instructions etc. 2012-08-03 16:38:45 +02:00
Kenneth Heafield
141861842d Change header file glob from an opt-out to an opt-in for Ondrej Bojar 2012-07-30 15:50:30 -04:00
Barry Haddow
2b4e61d826 Merge branch 'trunk' into miramerge 
Compiles, not tested.

Conflicts:
	Jamroot
	OnDiskPt/PhraseNode.h
	OnDiskPt/TargetPhrase.cpp
	OnDiskPt/TargetPhrase.h
	OnDiskPt/TargetPhraseCollection.cpp
	mert/BleuScorer.cpp
	mert/Data.cpp
	mert/FeatureData.cpp
	moses-chart-cmd/src/Main.cpp
	moses/src/AlignmentInfo.h
	moses/src/ChartManager.cpp
	moses/src/LM/Ken.cpp
	moses/src/LM/Ken.h
	moses/src/LMList.h
	moses/src/LexicalReordering.h
	moses/src/PhraseDictionaryTree.h
	moses/src/ScoreIndexManager.h
	moses/src/StaticData.h
	moses/src/TargetPhrase.h
	moses/src/Word.cpp
	scripts/ems/experiment.meta
	scripts/ems/experiment.perl
	scripts/training/train-model.perl
 2012-07-17 13:36:50 +01:00
Kenneth Heafield
272aeaa67e Merge some build system changes that were made for cdec 2012-06-21 09:52:25 -04:00
Colin Cherry
07a5c67ebc Merge branch 'master' into miramerge 
Conflicts:
	Jamroot
	misc/queryPhraseTable.cpp
	scripts/training/train-model.perl
 2012-06-14 17:08:16 -04:00
Barry Haddow
06f7615779 Merge branch 'miramerge' of github.com:moses-smt/mosesdecoder into miramerge 2012-06-07 00:26:47 +01:00
Barry Haddow
e53aee1275 Fix mpi build 2012-06-07 00:26:22 +01:00
Hieu Hoang
86a1886cb4 get rid of mention of --with-giza arg 2012-06-05 18:15:43 +01:00
Kenneth Heafield
833435561d Warning about dist directory. 2012-05-31 19:13:01 -04:00
Hieu Hoang
417d5ee57b move biconcor to / 2012-05-31 23:32:24 +01:00