moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2024-09-20 07:42:21 +03:00
Code Issues Packages Projects Releases Wiki Activity
13,484 Commits 136 Branches 26 Tags 147 MiB
e932e96db7
Commit Graph

580 Commits

Author SHA1 Message Date
Ales Tamchyna
2efa4ef42f fixing more errors 2015-01-06 18:12:22 +01:00
Ales Tamchyna
2c0995d7a8 toward compilation 2015-01-06 18:02:53 +01:00
Ales Tamchyna
7c441bc035 Merge branch 'vw_integration' of github.com:moses-smt/mosesdecoder into vw_integration 2015-01-06 17:54:43 +01:00
Ales Tamchyna
27e3038157 remove other legacy files 2015-01-06 17:54:35 +01:00
Marcin Junczys-Dowmunt
967e02cade Merge branch 'vw_integration' of github.com:moses-smt/mosesdecoder into vw_integration 2015-01-06 17:52:21 +01:00
Marcin Junczys-Dowmunt
a2b9f87002 Added exception if now features are available 2015-01-06 17:52:07 +01:00
Ales Tamchyna
887392b8c2 conflicts fixed 2015-01-06 17:51:14 +01:00
Ales Tamchyna
cb2efe1919 drafted VW integration in FF 2015-01-06 17:49:23 +01:00
Marcin Junczys-Dowmunt
69e93f0b07 Cleaned up VWFeature 2015-01-06 17:41:06 +01:00
Ales Tamchyna
394e00cc82 fixed conflicts 2015-01-06 17:12:02 +01:00
Ales Tamchyna
0cdd8daca6 toward VW integration 2015-01-06 17:05:24 +01:00
Marcin Junczys-Dowmunt
c3667f1c7f Moved VW files to folder FF/VW, added VWFeature base class 2015-01-06 17:00:26 +01:00
Marcin Junczys-Dowmunt
320f824bee Added VWFeatureFeature 2015-01-06 16:04:54 +01:00
Marcin Junczys-Dowmunt
17faca575f Added VWFeatureFeature 2015-01-06 15:55:36 +01:00
Marcin Junczys-Dowmunt
c8b0476cee Added function to update future score and full score 2015-01-06 14:54:40 +01:00
Marcin Junczys-Dowmunt
7fac15e624 Merge branch 'vw_integration' of github.com:moses-smt/mosesdecoder into vw_integration 2015-01-06 14:07:25 +01:00
Marcin Junczys-Dowmunt
5caf98352b Added example TranslationOptionListFeature.h 2015-01-06 14:07:12 +01:00
Ales Tamchyna
d4879855e0 Merge branch 'vw_integration' of github.com:moses-smt/mosesdecoder into vw_integration 2015-01-06 14:02:21 +01:00
Ales Tamchyna
6471108e1d new interface for classifiers and implementations for VW training (file) and prediction (library) 2015-01-06 14:02:12 +01:00
Marcin Junczys-Dowmunt
12fe186164 Merge branch 'vw_integration' of github.com:moses-smt/mosesdecoder into vw_integration 2015-01-06 13:56:19 +01:00
Marcin Junczys-Dowmunt
18691c19a3 second attempt at new feature hook, needs review of constness etc. 2015-01-06 13:56:13 +01:00
Ales Tamchyna
416c819f79 minor 2015-01-06 10:55:11 +01:00
Marcin Junczys-Dowmunt
4e334f2b4e Merge branch 'vw_integration' of github.com:moses-smt/mosesdecoder into vw_integration 2015-01-06 10:46:40 +01:00
Marcin Junczys-Dowmunt
f932058af4 Added new feature function hook: EvaluateWithSourceContextCollecton(...) 2015-01-06 10:46:31 +01:00
Ales Tamchyna
02967d7e63 nicer names 2015-01-06 10:39:38 +01:00
Ales Tamchyna
ab0227ea48 VW feature 2015-01-05 18:23:55 +01:00
Hieu Hoang
1e0a2835bf add oovpt 2015-01-04 19:10:48 +05:30
Nicola Bertoldi
4e77665d30 better handling of cache-based models with inconsistent parameters 2014-12-15 17:42:41 +01:00
Hieu Hoang
7c159b3d58 rename ChartDecoding -> CYKPlus. Comment out CubeGrowing, not implemented 2014-12-14 13:19:38 +00:00
Hieu Hoang
10d41d025e move collection of DistortionScoreProducer to it own class 2014-12-13 23:19:09 +00:00
Nicola Bertoldi
e4eb201c52 merged master into dynamic-models and solved conflicts 2014-12-13 12:52:47 +01:00
Hieu Hoang
31375afa62 Merge pull request #86 from marxin/distortion-scorer-no-dyncast 
DistortionFunctionScorer dyn_cast removed in FF.
 2014-12-12 12:38:26 +00:00
Nicola Bertoldi
cea2d9d8bb beautify 2014-12-09 12:39:37 +01:00
Matthias Huck
a0b6b6a341 SoftSourceSyntacticConstraintsFeature: improved efficiency 2014-12-08 18:20:58 +00:00
Matthias Huck
57298f86fd PhraseOrientationFeature: removing state distinction again. 
This was not supposed to be committed at this point.
 2014-12-05 22:41:56 +00:00
Matthias Huck
6e43080dba PhraseOrientationFeature: a tiny code simplification 2014-12-05 22:33:40 +00:00
Matthias Huck
824bd174f1 FeatureFunction: default verbosity parameter value is "unset" ( == std::numeric_limits<std::size_t>::max() ). 
Global verbosity level applies in this case (cf. declaration of macro IFFEATUREVERBOSE in moses/Util.h)
 2014-12-04 19:21:36 +00:00
Matthias Huck
31eb7b9693 tuneable parameter is set in base class 2014-12-04 19:18:20 +00:00
Matthias Huck
24a8a6a511 PhraseOrientationFeature 2014-12-03 20:04:26 +00:00
mliska
0a264bf5e3 DistortionFunctionScorer dyn_cast removed in FF. 2014-11-29 17:02:10 +01:00
Paul Baltescu
4eaf58f100 Integrate OxLM tree models. 2014-11-27 01:34:22 +00:00
Paul Baltescu
fefd2b0ada Fix merge errors. 2014-11-13 16:52:24 +00:00
XapaJIaMnu
7858f74e9e Rename BilingualLM_NPLM so that it is not confused with a sparse feature 2014-11-13 16:13:04 +00:00
Paul Baltescu
61826cee8a Rename OxLM features. 2014-11-13 16:13:04 +00:00
Paul Baltescu
7588c4b8e3 Skeleton for source conditioned OxLM feature. 2014-11-13 16:08:10 +00:00
XapaJIaMnu
0271b451b7 Some file renaming and fixes 2014-11-13 15:51:47 +00:00
XapaJIaMnu
bceff44fcb Abstract BilingualLM to Binlm_NPLM. Doesn't compile 2014-11-13 15:51:47 +00:00
Hieu Hoang
41b207e623 fix Jamfile 2014-11-13 15:51:46 +00:00
XapaJIaMnu
2a9d83b375 Compiles, works, outputs some numbers 2014-11-13 15:50:06 +00:00
XapaJIaMnu
7b00ca1115 BilingualLM FF, compiles, untested. 2014-11-13 15:50:06 +00:00
XapaJIaMnu
37abe3f14a nearly done, compiles, untested 2014-11-13 15:50:06 +00:00
XapaJIaMnu
0af522e512 Getting source and target words, compiling 2014-11-13 15:50:06 +00:00
XapaJIaMnu
d1d38b4aa8 Started implementing evaluate when applied 2014-11-13 15:50:06 +00:00
XapaJIaMnu
b2b1d352d4 Don't do anything if we don't have enough ngrams 2014-11-13 15:50:06 +00:00
XapaJIaMnu
85db12549c EvaluateWithSource compiles 2014-11-13 15:50:06 +00:00
XapaJIaMnu
16eecbd35f Add scores for target side 2014-11-13 15:50:06 +00:00
XapaJIaMnu
598f4c194d Some neural integration 2014-11-13 15:50:06 +00:00
XapaJIaMnu
8f0ce75c33 my bad 2014-11-13 15:50:06 +00:00
XapaJIaMnu
683783d519 Compiles 2014-11-13 15:50:06 +00:00
XapaJIaMnu
4bea830188 doesn't work 2014-11-13 15:50:05 +00:00
Hieu Hoang
a7abd02fc6 add Bilingual LM class 2014-11-13 15:50:05 +00:00
Hieu Hoang
6dda0d19cf add Bilingual LM class 2014-11-13 15:50:05 +00:00
Phil Williams
5240c430ce Merge s2t branch 
This adds a new string-to-tree decoder, which can be enabled with the -s2t
option.  It's intended to be faster and simpler than the generic chart
decoder, and is designed to support lattice input (still WIP).  For a en-de
system trained on WMT14 data, it's approximately 40% faster in practice.

For background information, see the decoding section of the EMNLP tutorial
on syntax-based MT:

  http://www.emnlp2014.org/tutorials/5_notes.pdf

Some features are not implemented yet, including support for internal tree
structure and soft source-syntactic constraints.
 2014-11-04 13:13:56 +00:00
Rico Sennrich
ef2c9dc92d hiero model compatibility 2014-10-15 14:20:40 +01:00
Hieu Hoang
855d972852 refactor some stateless feature functions to avoid using EvaluateWhenApplied(). Not kosher 2014-10-13 10:42:25 +01:00
Phil Williams
05ecc914c2 Fix a few more compiler warnings (from Clang mostly). 2014-10-10 15:47:53 +01:00
Paul Baltescu
8f74ecd8f3 Fix OxLM. 2014-10-08 22:08:42 +01:00
Philipp Koehn
33fbbc5faa allow specification of default scores for lexicalized reordering model 2014-09-21 07:43:44 +01:00
Matthias Huck
7e359a8892 Merge branch 'master' of github.com:moses-smt/mosesdecoder 2014-09-13 00:20:52 +01:00
Matthias Huck
5ac6c42508 PhraseOrientationFeature: bugfixes 2014-09-13 00:20:17 +01:00
Rico Sennrich
0a127cb0f1 refactoring 2014-09-12 19:17:23 +02:00
Matthias Huck
152ddb7513 PhraseOrientationFeature 2014-09-12 18:05:27 +01:00
Matthias Huck
0ddea954f2 PhraseOrientationFeature: TODOs 
(1) For efficiancy reasons, the initialization of the phrase orientation scoring object should be done as a precomputation step
(2) Handle degenerate cases with boundary non-terminals
 2014-09-12 17:50:02 +01:00
Matthias Huck
90e6ec8d11 PhraseOrientationFeature: remove debug output 2014-09-12 17:17:40 +01:00
Matthias Huck
1523f3315d PhraseOrientationFeature for chart-based decoding: a first simple version, 
with lots of log output
 2014-09-12 13:51:04 +01:00
Matthias Huck
58842b8ffb PhraseOrientationFeature dummy. Does not do anything yet. 2014-09-09 18:17:05 +01:00
mnadejde
8ba6853dcd Delete InternalStructStatelessFF.h 2014-09-01 13:58:11 +01:00
mnadejde
76cd3df393 Delete InternalStructStatelessFF.cpp 2014-09-01 13:57:56 +01:00
Nicola Bertoldi
77e9e91b08 minor fixes 2014-08-18 19:13:51 +02:00
Hieu Hoang
734d3fc33f minor move 2014-08-09 17:33:45 +01:00
Matthias Huck
62e345a18b SoftSourceSyntacticConstraintsFeature 2014-08-08 17:33:11 +01:00
Matthias Huck
856e20b1a8 SoftSourceSyntacticConstraintsFeature 2014-08-08 16:41:07 +01:00
Matthias Huck
0d756c1d8d SoftSourceSyntacticConstraintsFeature 2014-08-08 16:30:02 +01:00
Hieu Hoang
f447a23067 TargetPhrase to have pointer to the phrase table that creates it 2014-08-05 10:26:42 +01:00
Hieu Hoang
fbbc756117 add virtual void ChangeSource(InputType *&input) const 
to FeatureFunction
 2014-08-01 22:41:24 +01:00
Marcin Junczys-Dowmunt
f616954a03 Put back destructor in OpSequenceModel.* 2014-07-31 23:00:09 +02:00
Marcin Junczys-Dowmunt
73a822975d Merge branch 'master' of github.com:moses-smt/mosesdecoder 2014-07-31 22:44:20 +02:00
Marcin Junczys-Dowmunt
8930d50076 Added support to OSM for all KenLM model types 2014-07-31 22:44:12 +02:00
Ulrich Germann
f9d167345a Changed feature and parameter names for Mmsapt / PhraseDictionaryBitextSampling as requested by PK. 2014-07-29 13:57:00 +01:00
Barry Haddow
52a1056f9e Cache FNames to make sparse lex reorder faster 2014-07-24 21:23:08 +01:00
Barry Haddow
2f31485c29 Avoid dynamic_cast in critcal path code. 2014-07-24 16:01:54 +01:00
Nicola Bertoldi
1063012892 added a flag do disable the decaying in the cache 2014-07-22 11:25:03 +02:00
Nicola Bertoldi
02bf6d5d5e fixings about file loading and precomputation of ascores 2014-07-22 09:45:41 +02:00
Hieu Hoang
69c27a3020 name change 2014-07-16 13:21:51 +01:00
Hieu Hoang
ce014e0b35 cleanup 2014-07-15 17:47:36 +01:00
Hieu Hoang
b753f26d67 Merge branch 'master' into hieu.oxlm 2014-07-14 21:50:01 +01:00
Hieu Hoang
20271b8e96 sort FF by name 2014-07-14 21:49:29 +01:00
Hieu Hoang
f340ede6b5 new LBLLM based on LanguageModel class 2014-07-14 19:15:20 +01:00
Hieu Hoang
8f5dc2b9e7 state info 2014-07-11 12:53:03 +01:00
Hieu Hoang
3c9e397132 compiles with eclipse 2014-07-10 16:14:22 +01:00
Hieu Hoang
5b2bafde76 compiles with eclipse 2014-07-10 12:16:38 +01:00
Hieu Hoang
98d464727b add LBLLM from Blunsom's group 2014-07-10 11:40:32 +01:00
Hieu Hoang
e197b110fc rename Evaluate() to EvaluateWhenApplied() 2014-07-09 23:54:16 +01:00
Hieu Hoang
12a14221e2 rename Evaluate() to EvaluateWhenApplied() 2014-07-09 23:41:08 +01:00
Hieu Hoang
587b5fd716 rename Evaluate() to EvaluateWithSourceContext() 2014-07-09 23:06:54 +01:00
Hieu Hoang
c9bd98fb0f rename Evaluate() to EvaluateInIsolation() 2014-07-09 22:35:59 +01:00
Barry Haddow
c4ca243b7a Improved debug for sparse reordering 2014-06-30 12:13:33 +01:00
Barry Haddow
e2312e879c Merge remote-tracking branch 'origin/master' into sparse-reordering 2014-06-30 10:21:18 +01:00
Hieu Hoang
dba4caa7c0 register PhraseDictionaryFuzzyMatch 2014-06-26 09:34:16 -04:00
Barry Haddow
961d72269a clusters for sparse reordering feature 2014-06-24 21:50:20 +01:00
Barry Haddow
69222ee32b Implement 'between' featuresr. 2014-06-24 13:37:54 +01:00
Barry Haddow
cc426190e2 Minor fix 2014-06-23 17:46:45 +01:00
Barry Haddow
6424da1387 Merge branch 'sparse-reordering' of github.com:moses-smt/mosesdecoder into sparse-reordering 2014-06-23 17:06:24 +01:00
Barry Haddow
1afa0bc8f6 Fix for hreo 2014-06-23 17:06:00 +01:00
Barry Haddow
77a4a5e133 Merge remote-tracking branch 'origin/master' into sparse-reordering 
Conflicts:
	moses/FF/Factory.cpp
	moses/FF/LexicalReordering/LexicalReordering.cpp
 2014-06-23 15:22:40 +01:00
Matthias Huck
15ac0821b1 removed unnecessary includes 2014-06-20 21:25:14 +01:00
Matthias Huck
f3907cd6a2 SourceGHKMTreeInputMatchFeature 2014-06-20 21:00:16 +01:00
Hieu Hoang
137126ca54 read parameter example for Skeleton FF 2014-06-18 09:08:19 +01:00
Hieu Hoang
f2d3f57d07 Merge branch 'hieu' of ../mosesdecoder.hieu 2014-06-16 10:57:02 +01:00
Hieu Hoang
e2dc889150 add input path to chart cell label 2014-06-16 00:55:36 +01:00
Hieu Hoang
6e3b6d013b add SyntaxRHS FF 2014-06-15 21:41:52 +01:00
Hieu Hoang
b5d3592a06 floor score if weight is -ve 2014-06-14 04:30:15 +01:00
Hieu Hoang
615c55b49c don't have -inf score if weight is -ve 2014-06-14 04:22:03 +01:00
Hieu Hoang
cef8fd3b72 merge 2014-06-13 17:42:13 +01:00
Hieu Hoang
5c57702664 merge 2014-06-13 17:08:22 +01:00
XapaJIaMnu
a797582e89 Remove more unnecessary dynamic casts 2014-06-13 17:02:08 +01:00
Matthias Huck
c40bf494ac incorporating Hieu's "redo property API" stuff again (from dea105f663) 2014-06-13 16:49:57 +01:00
Matthias Huck
a5467d89c4 Minor modification of the phrase properties framework. 
Properties can save memory by not storing the value string.
 2014-06-13 16:37:13 +01:00
XapaJIaMnu
a6f8a7c4c9 Remove unnecessary dynamic_cast 2014-06-13 16:10:08 +01:00
Hieu Hoang
e5184bc496 redo property API 2014-06-13 15:45:34 +01:00
Hieu Hoang
7142eb5c1f Merge ../mosesdecoder into hieu 2014-06-13 15:34:47 +01:00
Hieu Hoang
dea105f663 redo property API 2014-06-13 15:34:24 +01:00
Hieu Hoang
eb78782c5d merge with master 2014-06-13 10:35:35 +01:00
Barry Haddow
2785989e21 precalculation of feature names 2014-06-12 21:37:18 +01:00
Hieu Hoang
ad4825ad44 integrated probing pt into decoder 2014-06-11 14:25:18 +01:00
Hieu Hoang
8c55f98c7a bug in SpanLength FF 2014-06-10 10:29:49 +01:00
Barry Haddow
91ccf8ef72 bidirectional case 2014-06-10 10:23:48 +01:00
Barry Haddow
a5e5a6590b basic implementation of non-hierarchical sparse features 2014-06-09 22:17:05 +01:00
Hieu Hoang
1fb71da5b5 delete debugging messages 2014-06-09 16:17:20 +01:00
Nicola Bertoldi
c58649c4ca fix errors arisen from merging and solving conflicts 2014-06-08 11:59:48 +02:00
Nicola Bertoldi
c268d6f21b made complaint with last changes in master 2014-06-08 11:56:52 +02:00
Nicola Bertoldi
1fe4eb0528 beautify 2014-06-08 09:44:59 +02:00
Nicola Bertoldi
4d75c889f1 merged master into dynamic-models 2014-06-08 09:39:37 +02:00
Barry Haddow
3dec0abf0a Call sparse reordering 2014-06-06 21:08:09 +01:00
Barry Haddow
4aa4fe0a04 Pass scc, not scores 2014-06-06 20:25:45 +01:00
Hieu Hoang
091ce3f016 Merge ../mosesdecoder into hieu 2014-06-06 17:25:26 +01:00
Barry Haddow
18d1bceea0 Move sparse reordering object to LR config 2014-06-06 14:20:23 +01:00
Hieu Hoang
e25e7b69be span length 2014-06-06 10:40:40 +01:00
phikoehn
ceadacd3af Merge branch 'master' of ssh://github.com/moses-smt/mosesdecoder 2014-06-05 21:33:35 +01:00
Philipp Koehn
15288213be allow < > in factors 2014-06-05 21:31:09 +01:00