moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2024-12-27 22:14:57 +03:00
Code Issues Packages Projects Releases Wiki Activity
11,654 Commits 136 Branches 26 Tags 147 MiB
55064961f7
Commit Graph

622 Commits

Author SHA1 Message Date
Hieu Hoang
88569e2e0a rand lm compiles 2013-11-04 20:09:15 +00:00
Hieu Hoang
cd00ca77f4 eclipse 2013-11-01 16:22:10 +00:00
Hieu Hoang
4b004b26d3 output list of available feature functions 2013-11-01 15:17:24 +00:00
Hieu Hoang
d51702f387 output list of available feature functions 2013-11-01 15:16:47 +00:00
Hieu Hoang
b926346bfe compiles with eclipse under OSX Mavericks 2013-10-30 21:23:16 +00:00
Hieu Hoang
bd82b7355b remove description argument from feature function constructor. Redundant 2013-10-29 18:44:33 +00:00
Hieu Hoang
37896ed854 add NeuralLM 2013-10-28 23:21:02 +00:00
Hieu Hoang
0af1df9f8b add NeuralLM 2013-10-28 22:11:37 +00:00
Barry Haddow
31aa9f2a63 Do not output phrase penalty by default 2013-10-24 21:12:47 +01:00
Phil Williams
69f5e4bec3 Merge branch 'master' into GHKMStruct 2013-10-09 16:21:49 +01:00
Hieu Hoang
28e4dcb304 compile error in moses server 2013-10-03 20:39:40 +01:00
Hieu Hoang
6e32bd3e19 correct creation of input paths for lattices 2013-10-02 18:42:56 +01:00
Ales Tamchyna
ea23629598 Moses server: optionally include alignment info in n-best lists 2013-10-02 13:56:16 +02:00
Hieu Hoang
233062a468 add DIMwid /Robin Kurtz 2013-10-02 11:33:10 +01:00
Hieu Hoang
6bce688517 eclipse 2013-09-30 17:29:05 +01:00
Phil Williams
2a28d1a73e Merge branch 'master' into GHKMStruct 
Conflicts:
	moses-chart-cmd/IOWrapper.cpp
	moses-chart-cmd/IOWrapper.h
	moses/FF/Factory.cpp
	moses/Parameter.cpp
	moses/StaticData.h
	phrase-extract/extract-ghkm/ScfgRuleWriter.cpp
	phrase-extract/score-main.cpp
 2013-09-29 15:27:09 +01:00
Ulrich Germann
050b089ea0 Merge branch 'master' of github.com:moses-smt/mosesdecoder 
Conflicts:
	moses/FF/ControlRecombination.cpp
	moses/TranslationModel/SkeletonPT.cpp
 2013-09-25 22:33:27 +01:00
Hieu Hoang
847f7c1ea2 skeleton PT 2013-09-25 20:59:13 +01:00
Ulrich Germann
82274ffda7 Merge branch 'dynamic-phrase-tables' 
Conflicts:
	moses/FF/ControlRecombination.cpp
	moses/Phrase.cpp
 2013-09-25 13:52:05 +01:00
Hieu Hoang
96df1f2659 lattices/confusion work with Hieu's binary pt, using new method 2013-09-25 13:03:09 +01:00
=
05ea3145a3 Added module mmbitext.m4m and mmsapt as phrase table type. 2013-09-25 00:40:40 +01:00
Hieu Hoang
b518b17cfa Merge branch 'master' of github.com:moses-smt/mosesdecoder 2013-09-18 21:17:06 +02:00
Hieu Hoang
4e4c6bd8b8 eclipse 2013-09-18 18:15:07 +02:00
Rico Sennrich
1113f83f98 documentation/permissions 2013-09-18 15:13:56 +02:00
Hieu Hoang
5625d30a26 add target phrase to arguments for Evaluate() during application to sentence 2013-09-18 12:35:31 +02:00
Hieu Hoang
6c739bcbf5 constrained decoding FF works for both pb and hiero 2013-09-17 15:26:25 +02:00
Hieu Hoang
0d94d7067e update convert-moses-ini-to-v2.perl with memory pt for SCFFG decoding. Start on constrained ff 2013-09-16 07:46:45 +02:00
Ulrich Germann
6ea0bb1f61 Merge branch 'master' of github.com:moses-smt/mosesdecoder 2013-09-14 11:21:15 +01:00
Ulrich Germann
7fad9c39de Added known issues to the m4m README. 2013-09-14 11:21:00 +01:00
maria nadejde
48fa62838c Merge branch 'master' into GHKMStruct 2013-09-13 17:30:50 +02:00
maria nadejde
d810a2c390 comment for Equal implementation 2013-09-13 17:30:16 +02:00
Hieu Hoang
c36d8d048f ExternalFeature /cdec & phil blumson 2013-09-12 09:48:17 +02:00
Ulrich Germann
52bb3bdf5a Merge branch 'master' of github.com:moses-smt/mosesdecoder 2013-09-10 12:07:12 +01:00
Ulrich Germann
6592265548 Initial check-in. 2013-09-10 12:06:29 +01:00
Lane Schwartz
7e1cfc981f Merge branch 'master' of www:/repos/git/Decoders/mosesdecoder 2013-09-06 14:01:32 -04:00
Lane Schwartz
226031ecf6 Checking in examples of CMake files. 
The example in contrib works, as does the one in biconcor.

The examples in lm and moses don't work at the moment,
but checking them in anyway as a starting point
for any potential work on them next week during the
MT Marathon.
 2013-09-06 13:39:46 -04:00
Chris Dyer
eb7afdaf4f fix for check plf 2013-09-06 10:40:10 -04:00
Chris Dyer
cd80fb6db8 fix to use current interface 2013-09-06 00:28:41 -04:00
Lane Schwartz
addc8be211 Merge branch 'master' of www:/repos/git/Decoders/mosesdecoder 2013-09-05 14:06:15 -04:00
Lane Schwartz
192bd8313c Adding more executables to XCode project 2013-09-05 14:06:09 -04:00
Lane Schwartz
31203ec308 XCode can now compile biconcor 2013-09-03 11:10:37 -04:00
Lane Schwartz
3be2f54a51 XCode can now compile moses_chart 2013-09-03 11:05:16 -04:00
Lane Schwartz
f1e4d6eb04 XCode can now compile moses and lmbrgrid 2013-09-03 10:59:53 -04:00
Lane Schwartz
fd81ca799d XCode can now compile queryOnDiskPt 
Also fixed some inconsistencies between
debug and release for some targets.
 2013-09-03 10:37:20 -04:00
Lane Schwartz
21801e85e7 XCode project can now compile CreateOnDiskPt 
If libmoses.a was compiled with LM_IRST,
then the following flags must be passed to the linker
when compiling CreateOnDiskPt:
-L $(XCODE_MOSES_LIBRARY_IRSTLM)
-lirstlm

If libmoses.a was compiled with LM_SRI,
then the following flags must be passed to the linker
when compiling CreateOnDiskPt:
-L $(XCODE_MOSES_LIBRARY_SRILM)
-ldstruct
-liconv
-lz
-lmisc
-loolm

Additionally, if libmoses.a was compiled with LM_SRI,
and SRILM was compiled with lbfgs,
then the following flags must be passed to the linker
when compiling CreateOnDiskPt:
-L $(XCODE_MOSES_LIBRARY_LBFGS)
-Xlinker -search_paths_first
-llbfgs

An note of explanation here may be warranted
with regard to the above lbfgs-related flags:
Mac OS X ships with a dynamic library that is loaded
with the flag -lz. SRILM compiles its own version of this
library. If SRILM is compiled statically, this becomes a problem.
We need XCode to link against the SRILM version of zlib,
but by default, the linker will search through all known library paths
for a dynamic version first. Only if no dynamic version is found will
it then search the library paths for a static version.
The fix for this problem is to use the -Xlinker flag to pass an additional
flag to the linker: -search_paths_first. This flag forces to linker to
iterate only once through each directory in its library path; within each
such directory it will first look for a dynamic version and
then, if necessary, for a static version. This solves the problem.
 2013-09-03 10:23:26 -04:00
Hieu Hoang
01b7f8a1e8 add skeleton feature functions 2013-09-01 16:58:23 +01:00
Lane Schwartz
19e94b4869 In XCode, removed headers from libmoses target. 2013-08-30 15:33:05 -04:00
Lane Schwartz
3bb4ac803a Adding targets to XCode project. 
Have working targets for kenlm binaries
and for libmoses, with the caveat that
only source files from the lm, moses, and util dirs
are currently included in libmoses.

The next step is adding the other relevant dirs
to libmoses.
 2013-08-30 15:26:59 -04:00
Hieu Hoang
fe36cbf737 move LexicalReordering to FF directory 2013-08-30 16:28:28 +01:00
Hieu Hoang
04783309ba move DecodeFeature to FF directory 2013-08-30 16:03:06 +01:00
Lane Schwartz
70eb3f13b9 More XCode configuration cleanup. 2013-08-29 16:26:49 -04:00
Lane Schwartz
27be036572 Clean up XCode configuration. 2013-08-29 16:02:19 -04:00
Lane Schwartz
f5a55e105b Cleaning up XCode project. 
Project is now configured such that it should compile
cleanly with or without optional libraries.

To compile the project, the user must (at minimum) have boost installed.

If boost is installed in a standard location, such as /usr/local,
then XCode should be able to find boost without any help from the user.

If boost is in a non-standard location, the user must configure XCode
such that the Source Tree variable XCODE_MOSES_INCLUDE_BOOST is set to
the directory where boost is installed.

For example, if you download and unzip boost in /path/to/boost_1_53_0,
then after compiling boost, headers would be located in
/path/to/boost_1_53_0/include and libraries would be located in
/path/too/boost_1_53_0/lib. In this case, set XCODE_MOSES_INCLUDE_BOOST=
/path/to/boost_1_53_0/include.

If you want to compile using IRSTLM, RandLM, SRILM, or the HHMM SynLM,
you will need to set the paths to those projects' respective include dirs
in the respective corresponding Source Tree variables:

For IRSTLM:
XCODE_MOSES_INCLUDE_IRSTLM

For RandLM:
XCODE_MOSES_INCLUDE_RANDLM

For SRILM:
XCODE_MOSES_INCLUDE_SRILM

For the HHMM syntactic LM:
XCODE_MOSES_INCLUDE_SYNLM_RVTL
XCODE_MOSES_INCLUED_SYNLM_WSJPARSE

At the moment, the project is configured with the
LM_SRI, LM_IRST, and LM_RAND preprocessor macros enabled,
which means that to compile the project, the respective source tree
variables must be defined. Alternatively, you could remove those macros
from the project definitions.
 2013-08-29 15:10:54 -04:00
Lane Schwartz
c0fe9bb4ea Updating XCode project 2013-08-29 12:56:25 -04:00
Hieu Hoang
de9decc30d mark GetTargetPhraseCollection with LEGACY 2013-08-23 21:02:03 +01:00
Hieu Hoang
5625f37eb1 delete ChartBasedFeatureContext 2013-08-23 15:00:47 +01:00
Hieu Hoang
1d653c87aa delete PhraseBasedFeatureContext 2013-08-23 14:25:25 +01:00
Hieu Hoang
841ce108a5 add phrase table caching 2013-08-15 20:20:44 +01:00
Hieu Hoang
154ca336d6 separate out ChartTranslationOption into its own files 2013-08-12 19:01:06 +01:00
Hieu Hoang
0bef0f63bf eclipse file 2013-08-01 13:53:06 +01:00
Hieu Hoang
bb1f49e10c Merge /home/hieu/workspace/github/mosesdecoder into hieu_opt_input 2013-07-31 14:29:18 +01:00
Hieu Hoang
ba187294d5 begin cleanup of SCFG parsing. Get rid of Sentence variable in ChartRuleLookupManager, replace with ChartParser variable 2013-07-31 12:25:34 +01:00
Rico Sennrich
a15bc05a33 rename multimodel weights in moses server (harmonization with the new config format) 2013-07-30 17:02:34 +02:00
Rico Sennrich
ccdcecc86f multimodel and mosesserver: instead of optimizing first model, select model by name. 2013-07-30 13:54:50 +02:00
Hieu Hoang
d1700435e0 eclipse file for extract 2013-07-26 14:00:19 +01:00
Hieu Hoang
4fde5f7ea2 eclipse file for extract-rules 2013-07-26 12:27:55 +01:00
Hieu Hoang
6ac4d4ddad Merge /home/hieu/workspace/github/mosesdecoder into hieu_opt_input 2013-07-24 19:04:54 +01:00
Ian Johnson
68779c66b9 Merge branch 'master' of github.com:moses-smt/mosesdecoder 2013-07-24 11:52:21 +01:00
Ian Johnson
08f64dea28 Arrow pipeline submodules now use https protocol. 2013-07-24 11:52:14 +01:00
Hieu Hoang
26c373ca3c Merge /home/hieu/workspace/github/mosesdecoder into hieu_opt_input 2013-07-23 13:13:17 +01:00
Hieu Hoang
9b9e8cc759 eclipse file with randlm 2013-07-23 12:41:02 +01:00
Hieu Hoang
a098227abe add ControlRecombination feature function 2013-07-20 23:10:50 +01:00
Hieu Hoang
5b7a9af588 refactor RandLM. Compiles with eclipse but not with bjam 2013-07-20 00:19:04 +01:00
Hieu Hoang
d4e641de80 eclipse 2013-07-19 23:19:17 +01:00
Hieu Hoang
24a9a7949e eclipse 2013-07-19 09:37:33 +01:00
Hieu Hoang
ba5ece8abe Merge github.com:moses-smt/mosesdecoder into hieu_opt_input 2013-07-19 09:34:32 +01:00
Hieu Hoang
5cbfcacccc Merge github.com:moses-smt/mosesdecoder into hieu_opt_input 2013-07-16 16:56:22 +01:00
Ian Johnson
1883db3f42 Updated PCL 2013-07-16 10:06:19 +01:00
Ian Johnson
4da9a8842b Updated PCL 2013-07-15 16:16:51 +01:00
Hieu Hoang
2a2c2856ed delete translation option caching 2013-07-12 22:46:24 +01:00
Hieu Hoang
ecef376aed rename InputLatticeNode.h to InputPath.h 2013-07-06 22:42:52 +01:00
Hieu Hoang
ac8d10af38 move lookup of all subphrases into PhraseDictionary class 2013-07-03 16:57:23 +01:00
Hieu Hoang
39f864d181 Merge github.com:moses-smt/mosesdecoder into hieu_opt_input2 2013-07-03 16:25:39 +01:00
Hieu Hoang
4995ea4f89 eclipse 2013-07-03 16:24:49 +01:00
Hieu Hoang
159911a92b Merge github.com:moses-smt/mosesdecoder into hieu_opt_input2 2013-07-03 13:35:10 +01:00
Hieu Hoang
bf9742f5f8 eclipse project for extract-ghkm 2013-07-03 12:35:45 +01:00
Ian Johnson
e20fbc0754 Merge branch 'master' of github.com:moses-smt/mosesdecoder 2013-07-03 10:55:30 +01:00
Ian Johnson
e0c9a05a9a Updated PCL, now with a manual. 2013-07-03 10:55:22 +01:00
Hieu Hoang
46e1b8df22 eclipse 2013-07-02 13:03:40 +01:00
Hieu Hoang
8123772b43 separate class InputLatticeNode into seprate file 2013-07-01 23:27:13 +01:00
Hieu Hoang
8d98d026d4 Merge github.com:moses-smt/mosesdecoder into hieu_opt_input2 2013-07-01 22:30:41 +01:00
Hieu Hoang
bff761b605 a little bit more towards using prefix subphrase to optimize translation model lookup 2013-07-01 22:30:29 +01:00
Hieu Hoang
e3a9694c60 eclipse 2013-07-01 19:56:48 +01:00
Hieu Hoang
fa4b92fc0a Merge branch 'master' into nadir_osm 2013-06-28 14:19:30 +01:00
Hieu Hoang
e3390393a3 Merge branch 'master' of github.com:moses-smt/mosesdecoder into nadir_osm 2013-06-27 21:12:29 +01:00
Hieu Hoang
3833d67ec0 merge with master 2013-06-26 17:12:22 +01:00
Hieu Hoang
d43c174dcd merge 2013-06-25 10:50:26 +01:00
Hieu Hoang
e15a4fc882 add phrase penalty feature function 2013-06-25 06:54:23 +01:00
Hieu Hoang
69acbfa1d3 compiles, but only on eclipse 2013-06-24 14:45:20 +01:00
Ian Johnson
3a524a4689 Updated PCL 2013-06-19 11:04:53 +01:00
Ian Johnson
d6fee6dd2f Merge branch 'master' of github.com:moses-smt/mosesdecoder 2013-06-19 10:27:09 +01:00
Ian Johnson
225a14f78b Added the Open Machine Translation Core submodule to contrib. 2013-06-19 10:27:00 +01:00
Rico Sennrich
9e9cbbc13b refactoring multimodel code out of StaticData; locking. 2013-06-12 14:47:40 +02:00
Ian Johnson
5cb0adaf95 Updated PCL 2013-06-11 15:40:57 +01:00
Ian Johnson
7d7493dd7d Merge branch 'master' of github.com:moses-smt/mosesdecoder 2013-06-11 12:10:09 +01:00
Ian Johnson
726a63b4e6 Bug fixes and updated PCL 2013-06-11 12:09:57 +01:00
Hieu Hoang
96d116dba0 refactor parsing of feature functiona args 2013-06-10 16:29:07 +01:00
Hieu Hoang
b4ca44e8af refactor TargetWordInsertionFeature::Load() 2013-06-09 20:09:21 +01:00
Hieu Hoang
82224a7a46 feature functions can be both sparse AND dense 2013-06-09 18:48:09 +01:00
Hieu Hoang
d281d025cf implement feature override as a run-time override of specific argument of a feature function. Only implemented table-limit for far 2013-06-07 17:42:23 +01:00
Hieu Hoang
dc6479fad9 eclipse 2013-06-06 15:09:05 +01:00
Ian Johnson
513af400ee Merge branch 'master' of github.com:moses-smt/mosesdecoder 2013-06-03 13:50:08 +01:00
Ian Johnson
15f197604d Updated PCL: fixes bugs in type derivation where fanout operations are used. 2013-06-03 13:49:58 +01:00
Hieu Hoang
266a81c9cf add class InputFeature. To replace 7 year old hack of using phrase table weights for input scores for confusion networks and lattice inputs 2013-05-28 17:25:59 +01:00
Ian Johnson
219d51754c Merge branch 'master' of github.com:moses-smt/mosesdecoder 2013-05-28 13:54:35 +01:00
Ian Johnson
58d5088f33 Updated PCL since it mow supports environment variables in configuration files. 2013-05-28 13:54:27 +01:00
Hieu Hoang
9977f6c393 compile error in moses server. Delete mention of ORLM 2013-05-27 19:16:18 +01:00
Hieu Hoang
87c7875391 delete LMList 2013-05-27 16:57:09 +01:00
Hieu Hoang
e8dadb58ce split classes in DummyScoreProducers into separate files 2013-05-25 00:57:37 +01:00
Hieu Hoang
49e69f19e2 split classes in DummyScoreProducers into separate files 2013-05-25 00:25:20 +01:00
Hieu Hoang
75a1849bcf split classes in FeatureFunction files 2013-05-24 19:11:15 +01:00
Hieu Hoang
a80d838b0f move feature functions into moses/FF 2013-05-24 18:02:49 +01:00
Hieu Hoang
6c322ba05e move and rename PhraseDictionarySCFG -> PhraseDictionaryMemory 2013-05-24 16:33:30 +01:00
Hieu Hoang
4a280c8ca7 move and rename PhraseDictionarySCFG -> PhraseDictionaryMemory 2013-05-24 16:23:31 +01:00
Ian Johnson
c839cf48c5 Udpated PCL 2013-05-22 15:39:36 +01:00
Ian Johnson
d9e3223e7c Re-wrote the README 2013-05-22 13:29:48 +01:00
Ian Johnson
10820e5bfd Added Makefiles 2013-05-22 12:11:41 +01:00
Ian Johnson
166eb7d0e0 Added PCL based arrow pipelining 2013-05-22 10:25:25 +01:00
Hieu Hoang
41da563645 port SRILM to new weights format 2013-05-19 12:44:45 +01:00
Hieu Hoang
f04ec4c56d 1. remove all code for MetaFeature from mira. 
2. in ShowWeights(), all print out dense feature weights. Don't print 'sparse' for sparse feature functions. All features functions can contains dense and sparse
 2013-05-16 19:05:08 +01:00
Rico Sennrich
5fa1752018 multimodel(counts): allow different features to have separate interpolation weights. 2013-05-16 14:11:03 +02:00
Rico Sennrich
876de13eba restore support for multimodel(counts) in moses server: 
set sentence-level weights and optimize weights
 2013-05-16 13:40:17 +02:00
Hieu Hoang
5f6f27dea5 compile error. Delete reference to TranslationSystem 2013-05-13 20:48:05 +01:00
Hieu Hoang
dcafe74892 port PhraseDictionaryMultiModelCount to new format 2013-05-13 17:20:14 +01:00
Christian Buck
5b69987373 mosesserver now supports nbest lists 2013-05-12 21:02:27 -07:00
Hieu Hoang
981351758a delete class TranslationSystem 2013-05-11 14:13:26 +01:00
Hieu Hoang
b75d194833 gut LMList. Get ready for deletion 2013-05-11 13:23:50 +01:00
Hieu Hoang
ed7ab8146f port PhraseDictionaryMultiModel to new format 2013-05-10 12:30:01 +01:00
Hieu Hoang
e8497d5a7e IsTuneable() can now be set in the ini file 2013-05-09 12:05:19 +01:00
Hieu Hoang
2b4e1fbea6 delete FindNumFeatures() and FindName() 2013-05-09 11:48:12 +01:00
Hieu Hoang
ac9496e74f merge 2013-05-08 15:39:00 +01:00
Hieu Hoang
644cac03f6 eclipse 2013-05-07 11:40:03 +01:00
Hieu Hoang
8a371f6547 eclipse 2013-05-07 09:25:42 +01:00
Hieu Hoang
b9373c7edf eclipse 2013-05-03 10:49:10 +01:00
Hieu Hoang
372d0505d2 eclipse 2013-05-02 21:22:05 +01:00
Hieu Hoang
f0bae8cbc8 eclipse 2013-05-02 19:56:04 +01:00
Hieu Hoang
25b1ef9e08 eclipse 2013-05-02 19:53:42 +01:00
Hieu Hoang
acaabb66d9 change StatelessFeatureType for class WordTranslationFeature. Test phrase.word-translation passes 2013-05-01 14:56:09 +01:00
Hieu Hoang
ce95c117f6 merge 2013-04-29 18:46:48 +01:00
Kenneth Heafield
972001e345 Compiler error 2013-04-27 01:02:52 +02:00