moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2024-09-11 11:25:40 +03:00
Code Issues Packages Projects Releases Wiki Activity
17,477 Commits 136 Branches 26 Tags 147 MiB
master
Commit Graph

206 Commits

Author SHA1 Message Date
Phil Williams
5240c430ce Merge s2t branch 
This adds a new string-to-tree decoder, which can be enabled with the -s2t
option.  It's intended to be faster and simpler than the generic chart
decoder, and is designed to support lattice input (still WIP).  For a en-de
system trained on WMT14 data, it's approximately 40% faster in practice.

For background information, see the decoding section of the EMNLP tutorial
on syntax-based MT:

  http://www.emnlp2014.org/tutorials/5_notes.pdf

Some features are not implemented yet, including support for internal tree
structure and soft source-syntactic constraints.
 2014-11-04 13:13:56 +00:00
Rico Sennrich
61c00ed636 n-best tree output 2014-09-12 19:17:23 +02:00
Barry Haddow
11d5fc3fef Remove, don't just comment out 2014-07-24 16:25:53 +01:00
Barry Haddow
d43f0e40fe Unused code 2014-07-24 16:20:49 +01:00
Nicola Bertoldi
1fe4eb0528 beautify 2014-06-08 09:44:59 +02:00
Nicola Bertoldi
4d75c889f1 merged master into dynamic-models 2014-06-08 09:39:37 +02:00
Nicola Bertoldi
c47d956e4c added support for segment id 2014-06-08 09:16:03 +02:00
Nicola Bertoldi
306dc26edd the management of passthrough xml-tags is introduced 2014-06-06 00:53:36 +02:00
Hieu Hoang
409c8d5349 Merge ../mosesdecoder into hieu 2014-05-20 00:16:58 +01:00
Matthias Huck
1740478238 Framework for additional phrase properties in decoding. 
Derive your property class from PhraseProperty. Do any expensive string
processing of the property value in there, not in the feature
implementation, and provide methods to access the information in
appropriate data formats. The property value string will thus have to
be processed only once (on loading) rather than each time the respective
phrase is applied and your feature needs to access the property value.
 2014-05-19 21:54:08 +01:00
Nicola Bertoldi
2f3cd5e2fe beautify 2014-05-19 15:34:27 +02:00
Nicola Bertoldi
20381cbf89 merged master into dynamic-models and solved conflicts 2014-04-28 19:18:38 +02:00
Hieu Hoang
6b1e147cff add -default-non-term-for-empty-range-only 2014-04-07 14:00:40 +01:00
Hieu Hoang
e5a6d63cf2 Merge branch 'master' of github.com:moses-smt/mosesdecoder into hieu 2014-03-21 21:41:04 +00:00
Rico Sennrich
45630a5851 various optimizations to make CYK+ parser several times faster and eat less memory. 
speed-up of decoding depends on how much time is spent in parser:
10-50% speed-up for string-to-tree systems observed (more on long sentences and with high max-chart-span).

if you only use hiero or string-to-tree models (but none with source syntax), use compile-option --unlabelled-source for (small) efficiency gains.
 2014-03-21 11:12:24 +00:00
Hieu Hoang
5e93c3f804 conflicts 2014-03-07 12:18:18 +00:00
Rico Sennrich
c27ecbe5c6 rename SyntaxConstraintFeature to TreeStructureFeature 
(makes it clearer what it does; build/print the internal tree structure in string-to-tree decoding)

no longer rely on name of FF for printing internal trees
 2014-03-03 13:56:19 +00:00
Hieu Hoang
1e5cd16fbf xml parsing for source syntax 2014-02-26 18:09:27 +00:00
Hieu Hoang
1716514eda create ResetWeights(). Only for dense weights 2014-02-23 20:07:28 +00:00
Hieu Hoang
cb2805151b delete -adjacent-only. Rubbish 2014-01-27 12:25:02 +00:00
Hieu Hoang
9745924937 output list of available feature functions, separated by space, rather than lines 2014-01-25 10:39:47 +00:00
Rico Sennrich
5d2b4d8efc sparse feature weights can now be set in moses.ini; overrides weights in weight-file. 2014-01-20 15:54:17 +00:00
Hieu Hoang
20e7d078a6 add feature function HyperParameterAsWeight 2014-01-17 17:59:36 +00:00
Nicola Bertoldi
4b072f2097 merge master into this branch 2014-01-17 14:04:15 +01:00
Rico Sennrich
ed25bb2b99 soft matching of target-side nonterminals 2014-01-16 18:34:33 +00:00
Nicola Bertoldi
bd83999264 beautify 2014-01-15 16:42:02 +01:00
Nicola Bertoldi
47bece6eac code cleanup; fixings to others' code/test 2014-01-15 16:16:37 +01:00
Nicola Bertoldi
c13bb8f8e8 partial code cleanup (not working) 2014-01-14 18:27:11 +01:00
Nicola Bertoldi
50970b2b59 merge master into this branch 2014-01-14 08:50:18 +01:00
Nicola Bertoldi
c2a053bd80 added a dynamic cache-based PhraseDitctionary; applied changes to the cache-based LanguageModel ; a lot of verbose message; some code cleanup 2014-01-13 15:41:52 +01:00
Hieu Hoang
ba209202ef move all feature functions out of StaticData 2013-12-07 00:21:06 +00:00
Hieu Hoang
09d50f87b1 move word penalty feature out of StaticData 2013-12-06 15:57:49 +00:00
Hieu Hoang
d3301fa1a3 move backoff parameter into decode graph class 2013-12-05 12:19:55 +00:00
Hieu Hoang
028edf8550 add -feature-name-overwrite. Specifically to replace PhraseDictionaryMemory with PhraseDictionaryScope3 for scope-3 decoding 2013-11-22 19:13:09 +00:00
Hieu Hoang
a57c9d6fe8 delete -parsing-algorithm. Scope-3 parsing depends on the phrase-table implementation used, not this argument 2013-11-22 17:35:22 +00:00
Hieu Hoang
749bd7fc8f add decoder arg -no-cache for mira tuning 2013-11-21 11:52:20 +00:00
Hieu Hoang
3f8972d67b move collection of generation-table from Staticdata to GenerationDictionary 2013-11-15 17:43:41 +00:00
Hieu Hoang
2f00ae936b move collection of phrase-table from Staticdata to PhraseDictionary 2013-11-15 17:26:26 +00:00
Hieu Hoang
d51702f387 output list of available feature functions 2013-11-01 15:16:47 +00:00
Phil Williams
69f5e4bec3 Merge branch 'master' into GHKMStruct 2013-10-09 16:21:49 +01:00
Hieu Hoang
6759af3821 beautify 2013-10-03 11:33:48 +01:00
Hieu Hoang
52215fdf79 speed up compiling by removing headers from StaticData.h 2013-10-03 11:16:06 +01:00
Hieu Hoang
6ccc17b804 speed up compiling by removing headers from StaticData.h 2013-10-03 11:05:53 +01:00
Hieu Hoang
f2d218f8ee move checking of legacy phrase-table to StaticData 2013-10-02 20:02:05 +01:00
Phil Williams
2a28d1a73e Merge branch 'master' into GHKMStruct 
Conflicts:
	moses-chart-cmd/IOWrapper.cpp
	moses-chart-cmd/IOWrapper.h
	moses/FF/Factory.cpp
	moses/Parameter.cpp
	moses/StaticData.h
	phrase-extract/extract-ghkm/ScfgRuleWriter.cpp
	phrase-extract/score-main.cpp
 2013-09-29 15:27:09 +01:00
Hieu Hoang
31ce9b510e beautify 2013-09-27 09:35:24 +01:00
braunefe
f23c1d7769 Added All Translation Options for DIMw 2013-09-23 16:30:28 +02:00
Hieu Hoang
574d716f23 placeholder for chart decoding 2013-09-22 15:09:49 +02:00
Hieu Hoang
a4f5879fef take out -constraint from Parameter. Now handled exclusively in FF 2013-09-18 06:23:22 +02:00
Nicola Bertoldi
7dc6ad4255 beautify 2013-09-14 10:13:24 +02:00
Matthias Huck
c39bed60c0 Tree fragments in GHKM glue rules; 
output of LHS tag in tree fragments for UNKs;
GHKMParse info is now denoted as Tree info
 2013-09-13 17:10:21 +02:00
Phil Williams
f3320c1ca0 Add -Tghkm parameter 
Writes internal tree structure (if available) for 1-best derivation.
Code contributed by Nina Seemann.
 2013-09-12 11:31:12 +02:00
Hieu Hoang
769f2fe00c start on forced decoding for SCFG 2013-08-30 17:45:56 +01:00
Hieu Hoang
21c91dede2 delete GetWordPenaltyProducer(). Need non-const version only for mira 2013-08-22 09:24:46 +01:00
Nicola Bertoldi
dc9353fd9e rpeparation for Dynamic cache-based LM 2013-08-17 22:10:15 +02:00
Jeremy Gwinnup
091bed03af Added option to output unknown words with prefix 'UNK' 2013-08-15 11:30:08 -04:00
Nicola Bertoldi
048f269e8e preparation for dynamic cache-based phrase dictionary 2013-08-14 16:11:43 +02:00
Nicola Bertoldi
5868653bd6 beautify 2013-08-11 23:41:23 +02:00
Nicola Bertoldi
7411227305 clean up related to the PhrrasePenalty producer 
transform the PhrasePenalty basic feature functions into a FF like
WordPenalty
 2013-08-11 23:32:54 +02:00
Hieu Hoang
6f7d239f13 Merge github.com:moses-smt/mosesdecoder into hieu_opt_input 2013-08-07 15:19:22 +01:00
phikoehn
67c3063574 Merge branch 'master' of ssh://github.com/moses-smt/mosesdecoder 2013-08-07 05:32:59 +01:00
phikoehn
ab4e3c63a6 enriched trace 2013-08-07 05:31:45 +01:00
Hieu Hoang
d3aa3ee65c source and target factors for placeholders 2013-07-19 18:23:38 +01:00
Hieu Hoang
2da41f7a56 Merge github.com:moses-smt/mosesdecoder into hieu_opt_input 2013-07-18 17:01:51 +01:00
Hieu Hoang
cb46bd3c00 beautify 2013-07-18 17:00:44 +01:00
Hieu Hoang
f81d7a446f starting placeholder 2013-07-18 17:00:07 +01:00
Hieu Hoang
475ebd2bbb get rid of translation option cache 2013-07-10 19:44:38 +01:00
Hieu Hoang
310b26f989 beautify 2013-07-08 20:52:14 +01:00
Hieu Hoang
3eba5782c2 beautify 2013-07-08 20:25:47 +01:00
Hieu Hoang
dc33fa3d3d redo parsing of feature function parameters 2013-06-20 12:50:41 +01:00
Rico Sennrich
9e9cbbc13b refactoring multimodel code out of StaticData; locking. 2013-06-12 14:47:40 +02:00
Hieu Hoang
abe6bb7c22 refactor parsing of feature functiona args 2013-06-10 18:11:55 +01:00
phikoehn
0538ac84d6 nuke silly debug message 2013-06-09 14:54:13 +01:00
Hieu Hoang
e291c5f4a4 implement feature override as a run-time override of specific argument of a feature function. Only implemented table-limit for far 2013-06-07 17:32:01 +01:00
Hieu Hoang
cf910955d0 cleanup LoadFeatureFunctions() 2013-06-06 15:51:31 +01:00
phikoehn
d0699b177d alternate weight settings can now ignore decoding graphs 2013-06-06 14:48:52 +01:00
phikoehn
081710ed4c alternate weight settings: added option to turn off features completely 2013-06-05 19:44:43 +01:00
phikoehn
d261106571 alternate weight setting now with correct caching 2013-06-05 14:06:04 +01:00
Hieu Hoang
0c48e687d1 delete information on input scores from StaticData 2013-06-05 01:41:52 +01:00
Hieu Hoang
fc7e4f4550 minor cleanup of InputFeature 2013-06-04 22:09:21 +01:00
Hieu Hoang
21ff6d4831 beautify 2013-06-03 12:33:18 +01:00
Hieu Hoang
eaf53769ad consistent Load() for all feature functions. Can be used not to load if doing --show-weights 2013-05-31 20:21:02 +01:00
phikoehn
d1650a5aa7 basic support for alternate weight settings 2013-05-31 12:28:57 +01:00
Hieu Hoang
6249432407 beautify 2013-05-29 18:16:15 +01:00
Hieu Hoang
06a5c86695 delete LMList 2013-05-27 16:50:21 +01:00
Hieu Hoang
195186b335 delete references to WordPenaltyProducer 2013-05-21 13:39:31 +01:00
Hieu Hoang
64b4a8805a remove all mention of MetaFeature 2013-05-16 19:41:35 +01:00
Hieu Hoang
f04ec4c56d 1. remove all code for MetaFeature from mira. 
2. in ShowWeights(), all print out dense feature weights. Don't print 'sparse' for sparse feature functions. All features functions can contains dense and sparse
 2013-05-16 19:05:08 +01:00
Hieu Hoang
981351758a delete class TranslationSystem 2013-05-11 14:13:26 +01:00
Hieu Hoang
7e398d3bca port PhraseDictionaryMultiModel to new format 2013-05-10 15:33:46 +01:00
Hieu Hoang
ed7ab8146f port PhraseDictionaryMultiModel to new format 2013-05-10 12:30:01 +01:00
Hieu Hoang
ce80626c9a remove SparsePhraseDictionaryFeature 2013-05-01 13:17:09 +01:00
Hieu Hoang
83494418e2 delete PhraseDictionaryMemory. Fix word penalty in compact phrase table 2013-04-26 15:55:32 +01:00
Hieu Hoang
2f78fe5fe5 Merge github.com:moses-smt/mosesdecoder into weight-new 2013-03-13 17:54:03 +00:00
Hieu Hoang
21c51194fa add -print-alignment-info 2013-03-13 12:12:33 +00:00
Hieu Hoang
bcecfe55db merge with Lane's search graph output 2013-02-24 00:31:29 +00:00
Hieu Hoang
30e0d1e0fa Merge github.com:moses-smt/mosesdecoder into weight-new 2013-02-22 22:39:24 +00:00
Lane Schwartz
764ce06726 More work on outputting search graph as hypergraph 2013-02-22 16:28:48 -05:00
Lane Schwartz
04f107fbb0 Add flag to output search graph in Kenneth's hypergraph format. 2013-02-22 16:28:48 -05:00
Lane Schwartz
774ed64f2e Work to allow output search graph in HTK standard lattice format 2013-02-22 16:28:47 -05:00