moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2024-09-20 15:48:05 +03:00
Code Issues Packages Projects Releases Wiki Activity
13,397 Commits 136 Branches 26 Tags 147 MiB
6d61db28fa
Commit Graph

557 Commits

Author SHA1 Message Date
Hieu Hoang
401c4940cf debugged FF NieceTerminal 2014-05-21 13:43:32 +01:00
Hieu Hoang
b60ac0321f rename DuplicateWord -> NieceTerminal 2014-05-20 20:02:49 +01:00
Hieu Hoang
3f93534624 Merge branch 'master' of github.com:moses-smt/mosesdecoder 2014-05-20 19:52:31 +01:00
Hieu Hoang
e31ce4d3be add FF DuplicateWord 2014-05-20 19:51:46 +01:00
Nicola Bertoldi
7040a7ee41 fixed typo 2014-05-20 11:58:44 +02:00
Nicola Bertoldi
fb5a03cc6b verbosity fixed 2014-05-20 11:38:54 +02:00
Matthias Huck
1740478238 Framework for additional phrase properties in decoding. 
Derive your property class from PhraseProperty. Do any expensive string
processing of the property value in there, not in the feature
implementation, and provide methods to access the information in
appropriate data formats. The property value string will thus have to
be processed only once (on loading) rather than each time the respective
phrase is applied and your feature needs to access the property value.
 2014-05-19 21:54:08 +01:00
Nicola Bertoldi
2f3cd5e2fe beautify 2014-05-19 15:34:27 +02:00
Nicola Bertoldi
b4857676be changes to allows the usage of multiple dynamic cache-based pseudo language models contemporarily 2014-05-19 15:26:48 +02:00
Hieu Hoang
65a1e176e5 bugs in FF MaxSpanFreeNonTermSource 2014-05-16 16:16:20 +01:00
Hieu Hoang
1e96ffcf3d rename FF RuleAmbiguity to RuleScope 2014-05-16 12:47:43 +01:00
Hieu Hoang
0989087a5b comment 2014-05-16 10:39:26 +01:00
Hieu Hoang
6c9bc9f988 delete chart translation option if -inf score. Add FF MaxSpanFreeNonTermSource to use it 2014-05-12 19:44:31 +01:00
Hieu Hoang
5f7c424f7f minor typo 2014-05-12 16:16:35 +01:00
Hieu Hoang
1d401817a5 backoff fix for memory and on-disk phrase-tables 2014-05-12 15:16:11 +01:00
Hieu Hoang
15566f7fc5 extend FF framework by add StackVec* argument to 2nd Evaluate(). NULL if phrase-based 2014-05-08 17:51:45 +01:00
Hieu Hoang
3ef5d76860 add MaxSpanFreeNonTermSource. Need to change FF framework to take in derivation 2014-05-08 16:34:04 +01:00
Hieu Hoang
814fb20aff 0 scores for FF SetSourcePhrase 2014-05-07 19:29:09 +01:00
Nicola Bertoldi
fe1ed42f81 Merge branch 'master' into dynamic-models 2014-04-30 08:32:46 +02:00
Nicola Bertoldi
20381cbf89 merged master into dynamic-models and solved conflicts 2014-04-28 19:18:38 +02:00
Ulrich Germann
7c145d045b Merge branch 'master' into dynamic-phrase-tables 
Conflicts:
	contrib/server/Jamfile
	contrib/server/mosesserver.cpp
 2014-04-28 10:00:07 +01:00
Ulrich Germann
f85bf72a8e Merge branch 'master' into dynamic-phrase-tables 
Conflicts:
	moses/TranslationModel/UG/mmsapt.cpp
 2014-04-11 19:24:37 +01:00
Hieu Hoang
e347020049 add FF ReferenceComparison 2014-04-11 11:03:50 +01:00
Hieu Hoang
0ec2fe0169 add FF RuleAmbiguity 2014-04-11 10:42:38 +01:00
Hieu Hoang
1e116a21ae add FF RuleAmbiguity 2014-04-11 10:29:57 +01:00
Hieu Hoang
9644a30858 add FF RuleAmbiguity 2014-04-11 10:22:03 +01:00
Hieu Hoang
1686686e65 start on FF ReferenceComparison 2014-04-10 21:47:06 +01:00
Hieu Hoang
91d6bfe0d5 start on FF ReferenceComparison 2014-04-10 21:34:34 +01:00
Hieu Hoang
0e98f1bc06 bug in ControlRecombinationState 2014-04-09 09:59:52 +01:00
Hieu Hoang
4c634023ce add FF CountNonTerms 2014-04-08 17:19:26 +01:00
Ulrich Germann
21f141ce0d Merge branch 'master' into dynamic-phrase-tables 
Conflicts:
	moses/InputPath.cpp
	moses/Word.cpp
 2014-04-08 14:58:29 +01:00
Hieu Hoang
2ea2224ba1 output source side of rule, if available 2014-04-07 12:00:13 +01:00
Hieu Hoang
09dc0e9913 add FF/SetSourcePhrase 2014-04-06 16:57:54 +01:00
germann
ec6fd21603 Make sure m_numInputScores is initialized properly. 2014-03-24 10:53:29 +00:00
Rico Sennrich
45630a5851 various optimizations to make CYK+ parser several times faster and eat less memory. 
speed-up of decoding depends on how much time is spent in parser:
10-50% speed-up for string-to-tree systems observed (more on long sentences and with high max-chart-span).

if you only use hiero or string-to-tree models (but none with source syntax), use compile-option --unlabelled-source for (small) efficiency gains.
 2014-03-21 11:12:24 +00:00
Ulrich Germann
76eb3d56b9 Proper initialization of class members of InputFeature. 2014-03-11 13:56:40 +00:00
Ulrich Germann
a7c85780ee Merge branch 'master' into dynamic-phrase-tables 
Conflicts:
	phrase-extract/score-main.cpp
 2014-03-10 14:25:45 +00:00
Ulrich Germann
fdc504d47a Changes on main branch files while I was working on dynamic phrase tables. 2014-03-10 14:08:00 +00:00
Ulrich Germann
6e4035fb12 Changes on main branch files while I was working on dynamic phrase tables. 2014-03-10 14:06:10 +00:00
Rico Sennrich
c27ecbe5c6 rename SyntaxConstraintFeature to TreeStructureFeature 
(makes it clearer what it does; build/print the internal tree structure in string-to-tree decoding)

no longer rely on name of FF for printing internal trees
 2014-03-03 13:56:19 +00:00
Hieu Hoang
b48f410bee don't check object pointer before deleting /Markus Elfring 2014-02-26 10:43:37 +00:00
Ulrich Germann
ef2ef881a4 Merge branch 'dynamic-phrase-tables' of file:///home/germann/git/mosesdecoder into dynamic-phrase-tables 2014-02-21 01:04:02 +00:00
Nicola Bertoldi
8d9bf2405d adding debugging to verify speed; not robust 2014-02-16 19:42:56 +01:00
Ulrich Germann
6d2818fa57 Fixed include path for msapt.h 2014-02-08 18:22:33 +00:00
Hieu Hoang
31857c47a2 Merge branch 'master' into nikolay_probingPT 2014-02-05 17:14:51 +00:00
Hieu Hoang
9745924937 output list of available feature functions, separated by space, rather than lines 2014-01-25 10:39:47 +00:00
Hieu Hoang
1aad3dfdce moved 2014-01-23 13:07:33 +00:00
Hieu Hoang
5800ff848c bjam 2014-01-22 20:57:06 +00:00
Hieu Hoang
63b9eac201 integration of Nikolay's probing pt into Moses 2014-01-22 13:06:10 +00:00
Hieu Hoang
8fe07d1c9c oops. Minor compile error 2014-01-20 17:01:53 +00:00
Hieu Hoang
839d1c6978 Merge branch 'master' of github.com:moses-smt/mosesdecoder 2014-01-20 16:26:45 +00:00
Hieu Hoang
b448e77639 register feature function HyperParameterAsWeight 2014-01-20 16:12:46 +00:00
Rico Sennrich
c8108ca71d some sparse feature functions couldn't be set to be non-tuneable in moses.ini 2014-01-20 15:56:36 +00:00
Hieu Hoang
20e7d078a6 add feature function HyperParameterAsWeight 2014-01-17 17:59:36 +00:00
Nicola Bertoldi
fd01180568 merge master into dynamic-models 2014-01-17 16:11:47 +01:00
Nicola Bertoldi
4b072f2097 merge master into this branch 2014-01-17 14:04:15 +01:00
Rico Sennrich
9e177cb472 SyntaxConstraintFeature (without any actual constraints; useful to build/output syntax tree from GHKM tree fragments) 2014-01-16 18:45:26 +00:00
Rico Sennrich
ed25bb2b99 soft matching of target-side nonterminals 2014-01-16 18:34:33 +00:00
jiejiang
5f1217d793 merged upstream with origin for mingw 2014-01-15 18:16:56 +00:00
Nicola Bertoldi
e452a13062 beautify 2014-01-15 16:49:57 +01:00
Nicola Bertoldi
bd83999264 beautify 2014-01-15 16:42:02 +01:00
Nicola Bertoldi
47bece6eac code cleanup; fixings to others' code/test 2014-01-15 16:16:37 +01:00
Nicola Bertoldi
c13bb8f8e8 partial code cleanup (not working) 2014-01-14 18:27:11 +01:00
Nicola Bertoldi
326d3bd508 code cleanup 2014-01-14 18:01:51 +01:00
Nicola Bertoldi
433986a7ac code cleanup 2014-01-14 17:40:32 +01:00
Nicola Bertoldi
e3f2ea1bd8 making dynamic models compliant with the master 2014-01-14 17:22:49 +01:00
Nicola Bertoldi
50970b2b59 merge master into this branch 2014-01-14 08:50:18 +01:00
Nicola Bertoldi
c2a053bd80 added a dynamic cache-based PhraseDitctionary; applied changes to the cache-based LanguageModel ; a lot of verbose message; some code cleanup 2014-01-13 15:41:52 +01:00
Hieu Hoang
b5558c36b8 get rid of all aborts 2014-01-13 14:37:05 +00:00
Hieu Hoang
07902ca550 minor memleak 2014-01-12 20:42:46 +00:00
phikoehn
073a601eb0 make good use of the new precise timer... 2014-01-03 18:45:31 +00:00
Hieu Hoang
18aaf4750a destroy feature functions at end. Otherwise it shows up on valgrind as memory leak 2013-12-31 20:15:43 +00:00
Lane Schwartz
f40e1ac8f8 Added support for hierarchical maximum entropy LM through SRILM. 
Currently, the code compiles, but has not been run or tested.

    Steps to compile with maxent LM are below:

    - Download SRILM 1.6.0

    - Download the maxent patch from
      http://www.phon.ioc.ee/~tanela/srilm-me/srilm-1.6.0-me.patch

    - Download the required libLBFGS library from
      http://www.chokkan.org/software/liblbfgs

    - Edit SRILM's sbin/machine-type script
      to return MACHINE_TYPE=i686-m64 for case x86_64

    - Compile libLBFGS and install it in the /tools/SRILM/SRILM-1.7.0 directory.
      Make sure that the liblbfgs.* library files are in /tools/SRILM/SRILM-1.7.0/lib//

    - In the /tools/SRILM/SRILM-1.7.0 dir, apply the maxent patch to SRILM:
      patch -p1 < srilm-1.6.0-me.patch

    - Point the appropriate SRILM makefile
      to see libLBFGS's include and lib dirs, following the instructions at:
      http://www.phon.ioc.ee/dokuwiki/doku.php?id=people:tanel:srilm-me.en

    - Compile SRILM

    - Compile Moses using --with-srilm and --with-maxent-srilm:
      ./bjam --with-srilm=/tools/SRILM/SRILM-1.7.0 --with-maxent-srilm=true

    NOTE: The above steps were for my original integration.
          Current versions of SRILM (such as 1.7.1) include the maxent patch.
          As such, it should be sufficient to compile a current version of SRILM
          with support form maxent enabled,
          and then compile Moses using --with-srilm and --with-maxent-srilm:
          ./bjam --with-srilm=/tools/SRILM/SRILM-1.7.0 --with-maxent-srilm=true
 2013-12-20 15:04:09 -05:00
jiejiang
744376b3fb moses windows build, with some TODO list 2013-12-18 20:15:39 +00:00
Hieu Hoang
597cb4a20f added negate=true. Only create translation which DON'T match reference 2013-12-15 01:42:22 +00:00
Hieu Hoang
295c07e884 bug in constrained decoding 2013-12-10 18:47:13 +00:00
Hieu Hoang
ba209202ef move all feature functions out of StaticData 2013-12-07 00:21:06 +00:00
Hieu Hoang
09d50f87b1 move word penalty feature out of StaticData 2013-12-06 15:57:49 +00:00
Hieu Hoang
78eba93a6b minor change in argument key for consistency 2013-12-06 13:09:34 +00:00
Hieu Hoang
fc8f8ba657 minor change in argument key for consistency 2013-12-06 13:07:33 +00:00
Hieu Hoang
cc9ecdff0a rename TransliterationPhraseDictionary to PhraseDictionaryTransliteration 2013-12-05 13:13:01 +00:00
Hieu Hoang
b33cf30bc1 transliteration pt obery backoff argument. Probably not threadable yet 2013-12-05 13:06:35 +00:00
Ales Tamchyna
64b629803d fixed a bug in CoveredRerefenceFeature 2013-12-04 10:21:01 +01:00
Ales Tamchyna
81ad92cc3f minor 2013-12-03 17:29:50 +01:00
Ales Tamchyna
fea32c7182 debugged CoveredReferenceFeature 2013-12-03 17:25:42 +01:00
Ales Tamchyna
8c0b5ffa6f early evaluation of CoveredReferenceFeature 2013-12-03 17:03:41 +01:00
Ales Tamchyna
cd63f30f15 modified FF interface: Evaluate() with input has access to future-cost estimation 2013-12-03 16:50:41 +01:00
Ales Tamchyna
fec97d2073 minor, added a check in CoveredReferenceFeature 2013-12-03 14:30:19 +01:00
Hieu Hoang
04cbbc48cf add phrase-table wrapper for Nadir's transliteration code 2013-12-02 15:29:13 +00:00
Ales Tamchyna
86a0dc337a adding CoveredReferenceFeature 2013-11-29 09:27:43 +01:00
Hieu Hoang
f13d0c8244 UTIL_THROW_IF -> UTIL_THROW_IF2 2013-11-22 20:27:46 +00:00
Hieu Hoang
c4dfc60f95 Merge branch 'master' of github.com:moses-smt/mosesdecoder 2013-11-22 15:26:23 +00:00
Hieu Hoang
5c6d88a463 add new phrase table for Scope-3 parsing. Doesn't quite work... 2013-11-22 15:26:14 +00:00
Hieu Hoang
193ee8aca6 minor compile error on Mac OSX 2013-11-21 19:19:34 +00:00
Hieu Hoang
0a8390981e delete includes for util/check.hh 2013-11-21 18:58:28 +00:00
Hieu Hoang
c3bd825d23 replace CHECK with UTIL_THROW_IF in Moses 2013-11-20 12:51:27 +00:00
Hieu Hoang
c2ffc022ea replace CHECK with UTIL_THROW_IF in Moses 2013-11-19 18:52:15 +00:00
Hieu Hoang
22864c2b4b replace CHECK with UTIL_THROW_IF in Moses 2013-11-19 17:23:19 +00:00
Hieu Hoang
2f00ae936b move collection of phrase-table from Staticdata to PhraseDictionary 2013-11-15 17:26:26 +00:00
Hieu Hoang
79fb8a1df4 provide mira program with BLEU Feature. Delete assert statement. 2013-11-15 16:43:39 +00:00
Hieu Hoang
2d8f9bc515 bug fixes for TargetNGramFeature 2013-11-11 15:32:58 +00:00
Hieu Hoang
ada80e6720 convert class TargetNgramFeature to new feature framework 2013-11-11 01:53:53 +00:00
Hieu Hoang
b8f67b51f3 convert class TargetNgramFeature to new feature framework 2013-11-11 01:50:40 +00:00
Hieu Hoang
08d888382d add DALM. Just a skeleton so far 2013-11-05 14:37:56 +00:00
Hieu Hoang
4b004b26d3 output list of available feature functions 2013-11-01 15:17:24 +00:00
Hieu Hoang
d51702f387 output list of available feature functions 2013-11-01 15:16:47 +00:00
Hieu Hoang
b926346bfe compiles with eclipse under OSX Mavericks 2013-10-30 21:23:16 +00:00
Hieu Hoang
a5f391e3f3 remove description argument from feature function constructor. Redundant 2013-10-29 18:59:53 +00:00
Hieu Hoang
bd82b7355b remove description argument from feature function constructor. Redundant 2013-10-29 18:44:33 +00:00
Hieu Hoang
d21e99e444 remove description argument from feature function constructor. Redundant 2013-10-29 17:46:03 +00:00
Hieu Hoang
37896ed854 add NeuralLM 2013-10-28 23:21:02 +00:00
Hieu Hoang
0af1df9f8b add NeuralLM 2013-10-28 22:11:37 +00:00
Phil Williams
69f5e4bec3 Merge branch 'master' into GHKMStruct 2013-10-09 16:21:49 +01:00
Hieu Hoang
6ccc17b804 speed up compiling by removing headers from StaticData.h 2013-10-03 11:05:53 +01:00
Phil Williams
841bf4b0a4 Fix merge error. 2013-09-29 17:07:48 +01:00
Phil Williams
4565d63995 Update InternalStructStatelessFF after merge. 2013-09-29 16:01:03 +01:00
Phil Williams
2a28d1a73e Merge branch 'master' into GHKMStruct 
Conflicts:
	moses-chart-cmd/IOWrapper.cpp
	moses-chart-cmd/IOWrapper.h
	moses/FF/Factory.cpp
	moses/Parameter.cpp
	moses/StaticData.h
	phrase-extract/extract-ghkm/ScfgRuleWriter.cpp
	phrase-extract/score-main.cpp
 2013-09-29 15:27:09 +01:00
Hieu Hoang
31ce9b510e beautify 2013-09-27 09:35:24 +01:00
Nadir Durrani
13e1adf000 Astetic Changes 2013-09-26 17:55:20 +01:00
Hieu Hoang
650f5150f8 Merge branch 'master' of github.com:moses-smt/mosesdecoder 2013-09-26 12:15:17 +01:00
Hieu Hoang
4285a6c7df move Uli's new pt into it's own folder. Comment out bjam compile. Doesn't work on MacOSX yet 2013-09-26 10:10:12 +01:00
Hieu Hoang
01c0708854 input feature score error when using anything other than bin pt 2013-09-26 00:05:52 +01:00
Ulrich Germann
050b089ea0 Merge branch 'master' of github.com:moses-smt/mosesdecoder 
Conflicts:
	moses/FF/ControlRecombination.cpp
	moses/TranslationModel/SkeletonPT.cpp
 2013-09-25 22:33:27 +01:00
Hieu Hoang
429a195c68 add type to ControlRecombination. type=2 disable recombination 2013-09-25 21:47:20 +01:00
Hieu Hoang
847f7c1ea2 skeleton PT 2013-09-25 20:59:13 +01:00
Hieu Hoang
56c1224c28 put some meat onto skeleton FFs 2013-09-25 20:19:05 +01:00
Hieu Hoang
deafd5922d skeleton PT 2013-09-25 16:57:01 +01:00
Ulrich Germann
82274ffda7 Merge branch 'dynamic-phrase-tables' 
Conflicts:
	moses/FF/ControlRecombination.cpp
	moses/Phrase.cpp
 2013-09-25 13:52:05 +01:00
Hieu Hoang
96df1f2659 lattices/confusion work with Hieu's binary pt, using new method 2013-09-25 13:03:09 +01:00
=
bed0cb5a68 Fixed signed/unsigned integer comparison issue. 2013-09-25 00:18:18 +01:00
=
933e4e0a8e Call to EmptyHypothesisState(const InputType &input) now throws an exception, since implementation is missing. 2013-09-25 00:17:30 +01:00
=
e6fdcda02d Reformatting. Max 80 chars per line, pretty please. 2013-09-25 00:15:43 +01:00
=
8545b4d15e Added Mmsapt as a feature function. 2013-09-25 00:14:57 +01:00
Hieu Hoang
cff36b6638 skeleton LM 2013-09-24 22:26:51 +01:00
Barry Haddow
266e36c440 stub out unit test 2013-09-18 21:58:38 +01:00
Hieu Hoang
ff196639de comment 2013-09-18 17:27:18 +02:00
Hieu Hoang
89fe7d3db2 feature functions to only allow recombination for the same output 2013-09-18 16:27:10 +02:00
Hieu Hoang
24f1760c05 add max-unknowns arg to ConstrainedDecoding FF 2013-09-18 15:47:49 +02:00
Hieu Hoang
5625d30a26 add target phrase to arguments for Evaluate() during application to sentence 2013-09-18 12:35:31 +02:00
Hieu Hoang
a4f5879fef take out -constraint from Parameter. Now handled exclusively in FF 2013-09-18 06:23:22 +02:00
Hieu Hoang
68974c9f20 check there are hypos in stack vector before create translation options. constrained decoding works for syntax model 2013-09-18 04:56:57 +02:00
Hieu Hoang
6c739bcbf5 constrained decoding FF works for both pb and hiero 2013-09-17 15:26:25 +02:00
Hieu Hoang
5ebb81a17a constrained decoding FF works for both pb and hiero 2013-09-17 15:06:17 +02:00
Hieu Hoang
764684bb6f Merge branch 'master' of github.com:moses-smt/mosesdecoder 2013-09-17 09:56:38 +02:00
Hieu Hoang
21319acb88 implement ConstrainedDecoding FF 2013-09-17 03:13:32 +02:00
Hieu Hoang
8b2bed458c implement ConstrainedDecoding FF 2013-09-17 02:40:25 +02:00
Hieu Hoang
ef2f6167e7 implement ConstrainedDecoding FF 2013-09-16 20:45:43 +02:00
Hieu Hoang
3b13068ce4 oops. forgot to check in new FF 2013-09-16 11:00:11 +01:00
Hieu Hoang
0d94d7067e update convert-moses-ini-to-v2.perl with memory pt for SCFFG decoding. Start on constrained ff 2013-09-16 07:46:45 +02:00
Barry Haddow
d737a352b3 Renamed 2013-09-14 17:22:43 +01:00
maria nadejde
5f37a545b1 fixed sparse feature output 2013-09-14 10:44:35 +02:00
Nicola Bertoldi
7dc6ad4255 beautify 2013-09-14 10:13:24 +02:00
Nicola Bertoldi
f93a1db381 added more commands, more scoring functions; cleanup 2013-09-14 10:09:59 +02:00
Nicola Bertoldi
74ca8c4285 cleanup 2013-09-14 10:08:06 +02:00
maria nadejde
bf5c32df6c stuff that probably doesn't work 2013-09-13 19:43:04 +02:00
Barry Haddow
f816a138ef feature name 2013-09-13 18:21:52 +01:00
Barry Haddow
0496363db3 Convert to stateless 2013-09-13 17:38:14 +01:00
maria nadejde
92d0932c27 Stateless Feature InternalStructStatelessFF 2013-09-13 18:23:24 +02:00
maria nadejde
7077a04c3b Stateless Feature InternalStructStatelessFF 2013-09-13 18:21:23 +02:00
Barry Haddow
82369968b5 vocabulary, type configuration 2013-09-13 16:52:42 +01:00
Barry Haddow
9d874b0ac1 Prints out feature values 2013-09-13 14:44:30 +01:00
Nicola Bertoldi
f841a20369 Added code for DynamicCacheBasedLanguageModel 2013-09-13 11:59:05 +02:00
Barry Haddow
8651f8da89 Extract blocks 2013-09-13 08:48:44 +01:00
Barry Haddow
a6e6660497 Merge branch 'master' into sparse-reordering 
Conflicts:
	moses/FF/Factory.cpp
 2013-09-12 18:57:10 +01:00
Barry Haddow
3dff330694 Configuration 2013-09-12 18:55:10 +01:00
Barry Haddow
7047b4e197 Sparse reordering for non-terminal pairs 2013-09-12 13:39:59 +01:00
Hieu Hoang
b534e8c4b7 f*cking link error 2013-09-12 09:17:18 +01:00
Hieu Hoang
c36d8d048f ExternalFeature /cdec & phil blumson 2013-09-12 09:48:17 +02:00
Barry Haddow
bd7f9344b8 Pairs of non-terminals 2013-09-11 22:10:23 +01:00
Barry Haddow
f7c53fef55 Set dense feature count to 0 2013-09-10 11:20:14 +01:00
Barry Haddow
565b4ea1ae Skeleton sparse reordering feature 2013-09-10 08:58:45 +01:00
Hieu Hoang
1adf7d0521 lattice decoding with sparse features 2013-09-08 15:57:31 +02:00
Hieu Hoang
65e2806b1d move lattice scoring into InputFeature class 2013-09-08 13:47:32 +02:00
Hieu Hoang
01b7f8a1e8 add skeleton feature functions 2013-09-01 16:58:23 +01:00
Hieu Hoang
fe36cbf737 move LexicalReordering to FF directory 2013-08-30 16:28:28 +01:00
Hieu Hoang
04783309ba move DecodeFeature to FF directory 2013-08-30 16:03:06 +01:00
Hieu Hoang
03fd06b38b convert Evaluate() methods into abstract, rather than empty functions. Every FF must implement them all 2013-08-30 15:49:00 +01:00
Hieu Hoang
1848946cb3 beautify 2013-08-28 11:06:27 +01:00
Nadir Durrani
1444837ff9 Factored-based OSM 2013-08-25 13:23:42 +01:00
Hieu Hoang
5625f37eb1 delete ChartBasedFeatureContext 2013-08-23 15:00:47 +01:00
Hieu Hoang
3ad3cd9004 delete ChartBasedFeatureContext 2013-08-23 14:39:42 +01:00
Hieu Hoang
1d653c87aa delete PhraseBasedFeatureContext 2013-08-23 14:25:25 +01:00
Hieu Hoang
0613d98beb delete PhraseBasedFeatureContext 2013-08-23 13:38:31 +01:00
Nicola Bertoldi
dc9353fd9e rpeparation for Dynamic cache-based LM 2013-08-17 22:10:15 +02:00
Nicola Bertoldi
ebfc21def8 remove useless files (generated by beautify) 2013-08-17 22:09:16 +02:00
Hieu Hoang
4d2fea9b38 prevent segfault in OSM model when used in factored model with decomposed phrase-tables 2013-08-16 16:37:40 +01:00
Hieu Hoang
9f92642e59 beautify 2013-08-15 21:14:04 +01:00
Hieu Hoang
abfe02f154 refactor input score handling to use Evaluate(inputtype, inputpath). Not a placeholder factor anymore 2013-08-14 16:53:06 +01:00
Hieu Hoang
2b030339e0 Merge /home/hieu/workspace/github/mosesdecoder into hieu_opt_input 2013-08-14 12:38:26 +01:00
Hieu Hoang
8419a3af35 beautify 2013-08-14 12:37:22 +01:00
Hieu Hoang
896b751066 Merge /Users/hieuhoang/unison/workspace/github/mosesdecoder into hieu_opt_input 2013-08-13 21:33:31 +01:00
Hieu Hoang
073b60db35 implement Evaluate(inputpath, input) for chart translation options. Plumbing for lattice input 2013-08-13 20:47:03 +01:00
Hieu Hoang
efc765ddff implement Evaluate(inputpath, input) for chart translation options. Plumbing for lattice input 2013-08-13 20:36:32 +01:00
Hieu Hoang
dcf170a047 refactor bleu score feature function. Still segfaults when running mira 2013-08-13 16:41:35 +01:00
Hieu Hoang
3775e37ee5 refactor bleu score feature function. Still segfaults when running mira 2013-08-13 16:39:37 +01:00
Nicola Bertoldi
5868653bd6 beautify 2013-08-11 23:41:23 +02:00
Nicola Bertoldi
7411227305 clean up related to the PhrrasePenalty producer 
transform the PhrasePenalty basic feature functions into a FF like
WordPenalty
 2013-08-11 23:32:54 +02:00
Hieu Hoang
8756ad8ffd beautify 2013-08-09 18:17:18 +01:00
Hieu Hoang
15132a04aa comment 2013-08-08 17:58:38 +01:00
Hieu Hoang
9341f28d75 Evaluate(InputType) --> Evaluate(InputType, Phrase source) 2013-08-08 17:35:42 +01:00
Hieu Hoang
9062194f46 comments 2013-08-08 12:26:25 +01:00
Hieu Hoang
585ec0f7fe DONE: delete source phrase from TargetPhrase class 2013-08-07 15:18:45 +01:00
Hieu Hoang
0ab6a441b2 get ready to delete source phrase from TargetPhrase class 2013-08-07 15:02:36 +01:00
Hieu Hoang
a1f5167f85 get ready to delete source phrase from TargetPhrase class 2013-08-07 11:37:40 +01:00
Hieu Hoang
a83c166678 delete Hypothesis::GetSourcePhrase(). Take it directly from TranslationOption::GetSourcePhrase() 2013-08-05 13:46:54 +01:00
Hieu Hoang
6872689bc2 TranslationOption::GetSourcePhrase() return a reference, not a pointer. It must always have the reference 2013-08-05 12:38:17 +01:00
Hieu Hoang
dbc545089b Merge /home/hieu/workspace/github/mosesdecoder into hieu_opt_input 2013-08-05 12:16:09 +01:00
Hieu Hoang
302eec8283 beautify 2013-08-05 12:11:59 +01:00
Hieu Hoang
bb1f49e10c Merge /home/hieu/workspace/github/mosesdecoder into hieu_opt_input 2013-07-31 14:29:18 +01:00
Rico Sennrich
29cde2a204 allow overriding table filtering in config (required for multimodelcounts) 2013-07-30 16:46:23 +02:00
Hieu Hoang
2aef439d8a Merge /home/hieu/workspace/github/mosesdecoder into hieu_opt_input 2013-07-25 15:26:38 +01:00
Hieu Hoang
e2c2bc59f1 beautify 2013-07-25 15:23:05 +01:00
Hieu Hoang
26c373ca3c Merge /home/hieu/workspace/github/mosesdecoder into hieu_opt_input 2013-07-23 13:13:17 +01:00
Hieu Hoang
1e906bea73 add ControlRecombination feature function 2013-07-23 01:38:08 +01:00
Hieu Hoang
42c1c908a5 add ControlRecombination feature function 2013-07-23 01:32:25 +01:00
Hieu Hoang
2590601708 add ControlRecombination feature function 2013-07-20 23:41:49 +01:00
Hieu Hoang
a098227abe add ControlRecombination feature function 2013-07-20 23:10:50 +01:00
Hieu Hoang
5b7a9af588 refactor RandLM. Compiles with eclipse but not with bjam 2013-07-20 00:19:04 +01:00
Hieu Hoang
11666a8359 RandLM is currently broken 2013-07-19 22:39:20 +01:00
Hieu Hoang
d7ab163b0e Merge /home/hieu/workspace/github/mosesdecoder into hieu_opt_input 2013-07-19 13:56:45 +01:00
Hieu Hoang
c77ec1b904 beautfiy 2013-07-19 13:56:02 +01:00
Hieu Hoang
320f7d575d Merge github.com:moses-smt/mosesdecoder into hieu_opt_input 2013-07-19 13:42:21 +01:00
Hieu Hoang
a95127b972 add default weights for feature functions that aren't tuneable, eg. OOV feature 2013-07-19 13:24:05 +01:00
Hieu Hoang
8a28178339 add default weights for feature functions that aren't tuneable, eg. OOV feature 2013-07-19 11:35:50 +01:00
Hieu Hoang
7896bbce5d Merge github.com:moses-smt/mosesdecoder into hieu_opt_input 2013-07-18 23:11:13 +01:00
Kenneth Heafield
b5e6b9c959 Factory 2013-07-18 22:54:52 +01:00
Hieu Hoang
2da41f7a56 Merge github.com:moses-smt/mosesdecoder into hieu_opt_input 2013-07-18 17:01:51 +01:00
Hieu Hoang
f81d7a446f starting placeholder 2013-07-18 17:00:07 +01:00
Hieu Hoang
89f74c130d Merge github.com:moses-smt/mosesdecoder into hieu_opt_input 2013-07-18 14:22:12 +01:00
Hieu Hoang
415c5ab693 tokenize only first occurence of = char for key=value. Allow paths with = chars 2013-07-18 14:01:08 +01:00
Hieu Hoang
49ede97304 merge 2013-07-09 12:41:08 +01:00
Hieu Hoang
0efcf64c28 Merge github.com:moses-smt/mosesdecoder into hieu_opt_input2 2013-07-04 23:37:21 +01:00
Hieu Hoang
6e56a9aed5 beautify 2013-07-04 20:29:05 +01:00
Hieu Hoang
f35750bc08 beautify 2013-07-04 20:19:51 +01:00
Nadir Durrani
e402714d2b Getting Rid of feature-path 2013-07-04 12:04:29 +01:00
Hieu Hoang
c38e1a7682 Merge github.com:moses-smt/mosesdecoder into hieu_opt_input2 2013-07-03 10:04:36 +01:00
Nadir Durrani
c05317dd82 Future-Cost 2013-07-02 13:19:55 +01:00
Hieu Hoang
8d98d026d4 Merge github.com:moses-smt/mosesdecoder into hieu_opt_input2 2013-07-01 22:30:41 +01:00
Nadir Durrani
582d31cb16 No SRILM 2013-07-01 13:10:58 +01:00
Nadir Durrani
ba72c70c6e Merge branch 'nadir_osm' of github.com:moses-smt/mosesdecoder into nadir_osm 2013-07-01 11:07:21 +01:00
Nadir Durrani
6a915253e1 Bug Fix 2013-07-01 11:06:55 +01:00
Hieu Hoang
4e773acd5f find all target phrases before search 2013-06-29 22:00:23 +01:00
Hieu Hoang
fa4b92fc0a Merge branch 'master' into nadir_osm 2013-06-28 14:19:30 +01:00
Nadir Durrani
8c19c2ba8a KenLM 2013-06-28 14:16:41 +01:00
Hieu Hoang
efc2c6145e beautify 2013-06-26 17:19:09 +01:00
Nadir Durrani
4335587698 Merge branch 'nadir_osm' of github.com:moses-smt/mosesdecoder into nadir_osm 2013-06-25 17:09:15 +01:00
Nadir Durrani
7aecb78d76 merge 2013-06-25 17:09:06 +01:00
Hieu Hoang
eefd94edb1 don't pollute namespace 2013-06-25 17:08:15 +01:00
Nadir Durrani
f66bbdd1f1 merge 2013-06-25 17:00:28 +01:00
Nadir Durrani
4f7a55e247 KenLM 2013-06-25 16:54:41 +01:00
Hieu Hoang
546d0add94 merge 2013-06-25 16:53:41 +01:00
Hieu Hoang
4b881dd74f don't pollute namespace 2013-06-25 16:51:56 +01:00
Nadir Durrani
537210b010 KenLM 2013-06-25 16:27:41 +01:00
Hieu Hoang
493d713075 Merge branch 'nadir_osm' of github.com:moses-smt/mosesdecoder into nadir_osm 2013-06-25 10:52:00 +01:00
Hieu Hoang
d43c174dcd merge 2013-06-25 10:50:26 +01:00
Hieu Hoang
e15a4fc882 add phrase penalty feature function 2013-06-25 06:54:23 +01:00
nadir
a243431d7a OSM-Feature 2013-06-24 19:46:33 +01:00
nadir
524653ae2b z 2013-06-24 16:13:27 +01:00
Hieu Hoang
cf00e966e4 call ReadParameters() 2013-06-24 15:34:25 +01:00
Hieu Hoang
2e1ba1d102 minor compile error 2013-06-24 15:29:26 +01:00
Hieu Hoang
9fe9b69c35 Merge branch 'nadir_osm' of github.com:moses-smt/mosesdecoder into nadir_osm 2013-06-24 15:27:46 +01:00
Hieu Hoang
19ce008c66 add plumbing for osm feature 2013-06-24 15:25:23 +01:00
Kenneth Heafield
14a970ec92 Get things to compile. Less header pollution. 2013-06-24 15:09:10 +01:00
Hieu Hoang
69acbfa1d3 compiles, but only on eclipse 2013-06-24 14:45:20 +01:00
nadir
cf55ab6678 OSM-Feature 2013-06-24 12:29:33 +01:00
Hieu Hoang
d68277d369 redo parsing of feature function parameters 2013-06-20 13:25:02 +01:00
Hieu Hoang
8103de2973 redo parsing of feature function parameters 2013-06-20 13:09:47 +01:00
Hieu Hoang
24d2edd790 redo parsing of feature function parameters 2013-06-20 13:06:03 +01:00
Hieu Hoang
dc33fa3d3d redo parsing of feature function parameters 2013-06-20 12:50:41 +01:00
Hieu Hoang
e3f9a5be9b redo parsing of feature function parameters 2013-06-20 10:35:25 +01:00
Hieu Hoang
c2f4c48758 clean up SetParameter() 2013-06-19 00:24:21 +01:00
Kenneth Heafield
1b87f29332 ChartDecoding isn't the only syntax algorithm 2013-06-18 10:21:47 -04:00
Hieu Hoang
047fff4dc5 implement IsUseable() correctly for SourceWordDeletionFeature 2013-06-14 19:49:51 +01:00
Hieu Hoang
f7371cf53d calculate m_featuresToApply in Load() for PhraseDictionary. Since every phrase dictionary now has to implement it, make Load() for PhraseDictionary abstract 2013-06-14 18:34:47 +01:00
Hieu Hoang
2615a93224 default number of dense features = 1 2013-06-14 11:02:22 +01:00
Hieu Hoang
f01ee737b0 refactor parsing of feature functiona args 2013-06-11 02:54:16 +01:00
Hieu Hoang
983f76b5cb refactor parsing of feature functiona args 2013-06-11 02:41:06 +01:00
Hieu Hoang
61e2b7a24f refactor parsing of feature functiona args 2013-06-11 01:46:04 +01:00
Hieu Hoang
b4ad093334 refactor parsing of feature functiona args 2013-06-11 00:05:12 +01:00
Hieu Hoang
b717114818 refactor parsing of feature functiona args 2013-06-10 23:16:28 +01:00
Hieu Hoang
efa42b451c refactor parsing of feature functiona args 2013-06-10 21:24:31 +01:00
Hieu Hoang
abe6bb7c22 refactor parsing of feature functiona args 2013-06-10 18:11:55 +01:00
Hieu Hoang
96d116dba0 refactor parsing of feature functiona args 2013-06-10 16:29:07 +01:00
Hieu Hoang
492bfaa27a refactor WordTranslationFeature::Load() 2013-06-09 20:23:10 +01:00
Hieu Hoang
b4ca44e8af refactor TargetWordInsertionFeature::Load() 2013-06-09 20:09:21 +01:00
Hieu Hoang
82224a7a46 feature functions can be both sparse AND dense 2013-06-09 18:48:09 +01:00
Hieu Hoang
a60f46a13e comment 2013-06-09 12:47:27 +01:00
Hieu Hoang
e291c5f4a4 implement feature override as a run-time override of specific argument of a feature function. Only implemented table-limit for far 2013-06-07 17:32:01 +01:00
Hieu Hoang
b9f54b195a implement GenerationDictionary.Load() 2013-06-05 13:42:56 +01:00
Hieu Hoang
fc7e4f4550 minor cleanup of InputFeature 2013-06-04 22:09:21 +01:00
Hieu Hoang
eaf53769ad consistent Load() for all feature functions. Can be used not to load if doing --show-weights 2013-05-31 20:21:02 +01:00
Hieu Hoang
b1993f6171 beautify 2013-05-30 12:51:40 +01:00
Hieu Hoang
83c4477841 figure out which feature function to apply at which decode step. Book-keeping 2013-05-30 12:41:08 +01:00
Hieu Hoang
b61a57d654 figure out which feature function to apply at which decode step. Book-keeping 2013-05-30 11:25:57 +01:00
Hieu Hoang
6249432407 beautify 2013-05-29 18:16:15 +01:00
Hieu Hoang
266a81c9cf add class InputFeature. To replace 7 year old hack of using phrase table weights for input scores for confusion networks and lattice inputs 2013-05-28 17:25:59 +01:00
Hieu Hoang
036ea4ee29 delete sourcePhrase variable in TargetPhrase class 2013-05-28 11:38:18 +01:00
Hieu Hoang
d647dd8aa3 delete sourcePhrase variable in TargetPhrase class 2013-05-28 10:51:28 +01:00
Hieu Hoang
1e7a51483f delete slow version of FindStringPiece() \Ken 2013-05-27 00:33:01 +01:00
Hieu Hoang
e8dadb58ce split classes in DummyScoreProducers into separate files 2013-05-25 00:57:37 +01:00