moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2024-12-26 05:14:36 +03:00
Code Issues Packages Projects Releases Wiki Activity
45630a5851
mosesdecoder/moses
History
Rico Sennrich 45630a5851 various optimizations to make CYK+ parser several times faster and eat less memory. 
speed-up of decoding depends on how much time is spent in parser:
10-50% speed-up for string-to-tree systems observed (more on long sentences and with high max-chart-span).

if you only use hiero or string-to-tree models (but none with source syntax), use compile-option --unlabelled-source for (small) efficiency gains.
2014-03-21 11:12:24 +00:00
..
FF various optimizations to make CYK+ parser several times faster and eat less memory. 2014-03-21 11:12:24 +00:00
LM Changes on main branch files while I was working on dynamic phrase tables. 2014-03-10 14:08:00 +00:00
TranslationModel various optimizations to make CYK+ parser several times faster and eat less memory. 2014-03-21 11:12:24 +00:00
AlignmentInfo.cpp replace CHECK with UTIL_THROW_IF in moses-chart-cmd 2013-11-18 19:03:30 +00:00
AlignmentInfo.h placeholder for chart decoding 2013-09-22 15:36:50 +02:00
AlignmentInfoCollection.cpp beautify 2013-05-29 18:16:15 +01:00
AlignmentInfoCollection.h beautify 2013-05-29 18:16:15 +01:00
AlignmentInfoTest.cpp beautify 2013-05-29 18:16:15 +01:00
BitmapContainer.cpp make good use of the new precise timer... 2014-01-03 18:45:31 +00:00
BitmapContainer.h speed up compiling by removing headers from StaticData.h 2013-10-03 11:05:53 +01:00
ChartCell.cpp delete -#if defined(BOOST... Trying to find target syntax bug when using old boost 2013-12-11 21:45:30 +00:00
ChartCell.h beautify 2013-05-29 18:16:15 +01:00
ChartCellCollection.cpp beautify 2013-05-29 18:16:15 +01:00
ChartCellCollection.h beautify 2013-05-29 18:16:15 +01:00
ChartCellLabel.h refactor parsing of feature functiona args 2013-06-10 18:11:55 +01:00
ChartCellLabelSet.h various optimizations to make CYK+ parser several times faster and eat less memory. 2014-03-21 11:12:24 +00:00
ChartHypothesis.cpp bug - should get score for translation rule from the translation option, not the target phrase. 2013-12-17 14:54:12 +00:00
ChartHypothesis.h constrained decoding FF works for both pb and hiero 2013-09-17 15:26:25 +02:00
ChartHypothesisCollection.cpp get rid of all aborts from decoder 2014-01-13 18:32:22 +00:00
ChartHypothesisCollection.h Compiler warning 2014-03-17 07:18:11 -07:00
ChartManager.cpp various optimizations to make CYK+ parser several times faster and eat less memory. 2014-03-21 11:12:24 +00:00
ChartManager.h moses_chart: implement -output-unknowns option. 2014-03-17 13:09:54 +00:00
ChartParser.cpp various optimizations to make CYK+ parser several times faster and eat less memory. 2014-03-21 11:12:24 +00:00
ChartParser.h moses_chart: implement -output-unknowns option. 2014-03-17 13:09:54 +00:00
ChartParserCallback.h various optimizations to make CYK+ parser several times faster and eat less memory. 2014-03-21 11:12:24 +00:00
ChartRuleLookupManager.cpp skelete pt 2013-09-28 09:42:17 +01:00
ChartRuleLookupManager.h various optimizations to make CYK+ parser several times faster and eat less memory. 2014-03-21 11:12:24 +00:00
ChartTranslationOption.cpp beautify 2013-09-27 09:35:24 +01:00
ChartTranslationOption.h beautify 2013-09-27 09:35:24 +01:00
ChartTranslationOptionList.cpp debug function for ChartTranslationOptionList and ChartTranslationOptions 2013-12-11 19:19:25 +00:00
ChartTranslationOptionList.h various optimizations to make CYK+ parser several times faster and eat less memory. 2014-03-21 11:12:24 +00:00
ChartTranslationOptions.cpp debug function for ChartTranslationOptionList and ChartTranslationOptions 2013-12-11 19:19:25 +00:00
ChartTranslationOptions.h more debug functions 2013-12-11 20:03:30 +00:00
ChartTrellisDetour.cpp beautify 2013-05-29 18:16:15 +01:00
ChartTrellisDetour.h beautify 2013-05-29 18:16:15 +01:00
ChartTrellisDetourQueue.cpp beautify 2013-05-29 18:16:15 +01:00
ChartTrellisDetourQueue.h beautify 2013-05-29 18:16:15 +01:00
ChartTrellisNode.cpp UTIL_THROW_IF -> UTIL_THROW_IF2 2013-11-22 20:27:46 +00:00
ChartTrellisNode.h beautify 2013-05-29 18:16:15 +01:00
ChartTrellisPath.cpp UTIL_THROW_IF -> UTIL_THROW_IF2 2013-11-22 20:27:46 +00:00
ChartTrellisPath.h beautify 2013-05-29 18:16:15 +01:00
ChartTrellisPathList.h move moses/src/* to moses/ 2012-11-12 19:56:18 +00:00
CMakeLists.txt Checking in examples of CMake files. 2013-09-06 13:39:46 -04:00
ConfusionNet.cpp Reformatting for readability. 2014-03-11 13:55:16 +00:00
ConfusionNet.h UTIL_THROW_IF -> UTIL_THROW_IF2 2013-11-22 20:27:46 +00:00
DecodeGraph.cpp transliteration pt obery backoff argument. Probably not threadable yet 2013-12-05 13:06:35 +00:00
DecodeGraph.h transliteration pt obery backoff argument. Probably not threadable yet 2013-12-05 13:06:35 +00:00
DecodeStep.cpp transliteration pt obery backoff argument. Probably not threadable yet 2013-12-05 13:06:35 +00:00
DecodeStep.h transliteration pt obery backoff argument. Probably not threadable yet 2013-12-05 13:06:35 +00:00
DecodeStepGeneration.cpp transliteration pt obery backoff argument. Probably not threadable yet 2013-12-05 13:06:35 +00:00
DecodeStepGeneration.h transliteration pt obery backoff argument. Probably not threadable yet 2013-12-05 13:06:35 +00:00
DecodeStepTranslation.cpp transliteration pt obery backoff argument. Probably not threadable yet 2013-12-05 13:06:35 +00:00
DecodeStepTranslation.h transliteration pt obery backoff argument. Probably not threadable yet 2013-12-05 13:06:35 +00:00
Factor.cpp move moses/src/* to moses/ 2012-11-12 19:56:18 +00:00
Factor.h beautify 2013-05-29 18:16:15 +01:00
FactorCollection.cpp various optimizations to make CYK+ parser several times faster and eat less memory. 2014-03-21 11:12:24 +00:00
FactorCollection.h various optimizations to make CYK+ parser several times faster and eat less memory. 2014-03-21 11:12:24 +00:00
FactorTypeSet.cpp beautify 2013-05-31 00:00:21 +01:00
FactorTypeSet.h figure out which feature function to apply at which decode step. Book-keeping 2013-05-30 18:34:10 +01:00
FeatureVector.cpp merged upstream with origin for mingw 2014-01-15 18:16:56 +00:00
FeatureVector.h UTIL_THROW_IF -> UTIL_THROW_IF2 2013-11-22 20:27:46 +00:00
FeatureVectorTest.cpp Re-enable unit tests. They all pass. 2013-09-14 17:15:41 +01:00
File.cpp move moses/src/* to moses/ 2012-11-12 19:56:18 +00:00
File.h merged upstream with origin for mingw 2014-01-15 18:16:56 +00:00
FilePtr.h move moses/src/* to moses/ 2012-11-12 19:56:18 +00:00
FloydWarshall.cpp UTIL_THROW_IF -> UTIL_THROW_IF2 2013-11-22 20:27:46 +00:00
FloydWarshall.h move moses/src/* to moses/ 2012-11-12 19:56:18 +00:00
GenerationDictionary.cpp UTIL_THROW_IF -> UTIL_THROW_IF2 2013-11-22 20:27:46 +00:00
GenerationDictionary.h move collection of generation-table from Staticdata to GenerationDictionary 2013-11-15 17:43:41 +00:00
gzfilebuf.h move moses/src/* to moses/ 2012-11-12 19:56:18 +00:00
hypergraph.proto move moses/src/* to moses/ 2012-11-12 19:56:18 +00:00
HypoList.h move moses/src/* to moses/ 2012-11-12 19:56:18 +00:00
Hypothesis.cpp make good use of the new precise timer... 2014-01-03 18:45:31 +00:00
Hypothesis.h UTIL_THROW_IF -> UTIL_THROW_IF2 2013-11-22 20:27:46 +00:00
HypothesisStack.cpp move moses/src/* to moses/ 2012-11-12 19:56:18 +00:00
HypothesisStack.h beautify 2013-05-29 18:16:15 +01:00
HypothesisStackCubePruning.cpp UTIL_THROW_IF -> UTIL_THROW_IF2 2013-11-22 20:27:46 +00:00
HypothesisStackCubePruning.h move moses/src/* to moses/ 2012-11-12 19:56:18 +00:00
HypothesisStackNormal.cpp get rid of all aborts from decoder 2014-01-13 18:32:22 +00:00
HypothesisStackNormal.h move moses/src/* to moses/ 2012-11-12 19:56:18 +00:00
Incremental.cpp various optimizations to make CYK+ parser several times faster and eat less memory. 2014-03-21 11:12:24 +00:00
Incremental.h beautify 2013-05-29 18:16:15 +01:00
InputFileStream.cpp move moses/src/* to moses/ 2012-11-12 19:56:18 +00:00
InputFileStream.h move moses/src/* to moses/ 2012-11-12 19:56:18 +00:00
InputPath.cpp various optimizations to make CYK+ parser several times faster and eat less memory. 2014-03-21 11:12:24 +00:00
InputPath.h various optimizations to make CYK+ parser several times faster and eat less memory. 2014-03-21 11:12:24 +00:00
InputType.cpp move checking of legacy phrase-table to StaticData 2013-10-02 20:02:05 +01:00
InputType.h faster compilation by taking some #includes from Word.h and Phrase.h 2013-10-04 15:40:23 +01:00
Jamfile Reverted to version prior to last check-in. 2014-03-12 19:18:25 +00:00
LVoc.cpp move moses/src/* to moses/ 2012-11-12 19:56:18 +00:00
LVoc.h replace CHECK with UTIL_THROW_IF in Moses 2013-11-19 18:52:15 +00:00
Manager.cpp merge c11 revert with mingw changes 2014-01-16 01:08:01 +00:00
Manager.h more enriched trace 2013-10-13 06:59:05 +01:00
MockHypothesis.cpp constrained decoding FF works for both pb and hiero 2013-09-17 15:06:17 +02:00
MockHypothesis.h all hypotheses have translation option. All translation options have target phrase 2013-08-05 16:53:15 +01:00
MosesTest.cpp move moses/src/* to moses/ 2012-11-12 19:56:18 +00:00
NonTerminal.cpp beautify 2013-08-07 14:18:12 +01:00
NonTerminal.h Add NonTerminalSet variable to InputPath 2013-08-02 15:54:49 +01:00
ObjectPool.h move moses/src/* to moses/ 2012-11-12 19:56:18 +00:00
OutputCollector.h compiles with eclipse under OSX Mavericks 2013-10-30 21:23:16 +00:00
Parameter.cpp output list of available feature functions, separated by space, rather than lines 2014-01-25 10:39:47 +00:00
Parameter.h sparse feature weights can now be set in moses.ini; overrides weights in weight-file. 2014-01-20 15:54:17 +00:00
PartialTranslOptColl.cpp replace nth_element() with macro that execute sort() instead for gcc 4.8.1 & 4.8.2 2013-11-15 10:55:38 +00:00
PartialTranslOptColl.h push InpuutPath down into phrase dictionary for legacy methods 2013-08-06 17:09:53 +01:00
PCNTools.cpp UTIL_THROW_IF -> UTIL_THROW_IF2 2013-11-22 20:27:46 +00:00
PCNTools.h beautify 2013-09-27 09:35:24 +01:00
PDTAimp.h get rid of all aborts from decoder 2014-01-13 18:32:22 +00:00
Phrase.cpp delete includes for util/check.hh 2013-11-21 18:58:28 +00:00
Phrase.h UTIL_THROW_IF -> UTIL_THROW_IF2 2013-11-22 20:27:46 +00:00
PrefixTree.h delete includes for util/check.hh 2013-11-21 18:58:28 +00:00
PrefixTreeMap.cpp replace CHECK with UTIL_THROW_IF in Moses 2013-11-21 17:51:01 +00:00
PrefixTreeMap.h beautify 2013-05-29 18:16:15 +01:00
ReorderingConstraint.cpp replace CHECK with UTIL_THROW_IF in Moses 2013-11-19 17:23:19 +00:00
ReorderingConstraint.h faster compilation by taking some #includes from Word.h and Phrase.h 2013-10-04 15:40:23 +01:00
ReorderingStack.cpp move moses/src/* to moses/ 2012-11-12 19:56:18 +00:00
ReorderingStack.h delete ScoreProducer 2013-02-21 20:03:35 +00:00
rule.proto move moses/src/* to moses/ 2012-11-12 19:56:18 +00:00
RuleCube.cpp delete -#if defined(BOOST... Trying to find target syntax bug when using old boost 2013-12-11 21:45:30 +00:00
RuleCube.h delete debugging message 2013-12-14 16:15:27 +00:00
RuleCubeItem.cpp UTIL_THROW_IF -> UTIL_THROW_IF2 2013-11-22 20:27:46 +00:00
RuleCubeItem.h compare translation option dimension using pointers to translation option, rather than comparing the phrase. Comparinf the phrase is incorrect - need to compare LHS and possibly alignments too. Unnecessary 2013-12-13 17:23:39 +00:00
RuleCubeQueue.cpp move moses/src/* to moses/ 2012-11-12 19:56:18 +00:00
RuleCubeQueue.h beautify 2013-05-29 18:16:15 +01:00
ScoreComponentCollection.cpp beautify 2013-09-27 09:35:24 +01:00
ScoreComponentCollection.h get rid of all aborts from decoder 2014-01-13 18:32:22 +00:00
ScoreComponentCollectionTest.cpp modified FF interface: Evaluate() with input has access to future-cost estimation 2013-12-03 16:50:41 +01:00
Search.cpp get rid of all aborts from decoder 2014-01-13 18:32:22 +00:00
Search.h speed up compiling by removing headers from StaticData.h 2013-10-03 11:05:53 +01:00
SearchCubePruning.cpp output list of available feature functions, separated by space, rather than lines 2014-01-25 10:39:47 +00:00
SearchCubePruning.h make good use of the new precise timer... 2014-01-03 18:45:31 +00:00
SearchNormal.cpp output list of available feature functions, separated by space, rather than lines 2014-01-25 10:39:47 +00:00
SearchNormal.h make good use of the new precise timer... 2014-01-03 18:45:31 +00:00
SearchNormalBatch.cpp get rid of all aborts from decoder 2014-01-13 18:32:22 +00:00
SearchNormalBatch.h Merge branch 'master' of github.com:moses-smt/mosesdecoder 2013-09-25 22:33:27 +01:00
Sentence.cpp bug fix with interaction of alternate weights and xml markup 2013-11-30 02:00:34 +00:00
Sentence.h get rid of some warnings 2013-09-22 18:24:32 +02:00
SentenceStats.cpp speed up compiling by removing headers from StaticData.h 2013-10-03 11:05:53 +01:00
SentenceStats.h make good use of the new precise timer... 2014-01-03 18:45:31 +00:00
SquareMatrix.cpp move moses/src/* to moses/ 2012-11-12 19:56:18 +00:00
SquareMatrix.h move moses/src/* to moses/ 2012-11-12 19:56:18 +00:00
StackVec.h move moses/src/* to moses/ 2012-11-12 19:56:18 +00:00
StaticData.cpp various optimizations to make CYK+ parser several times faster and eat less memory. 2014-03-21 11:12:24 +00:00
StaticData.h various optimizations to make CYK+ parser several times faster and eat less memory. 2014-03-21 11:12:24 +00:00
SyntacticLanguageModel.cpp beautify 2013-09-27 09:35:24 +01:00
SyntacticLanguageModel.h beautify 2013-05-29 18:16:15 +01:00
SyntacticLanguageModelFiles.h replace CHECK with UTIL_THROW_IF in Moses 2013-11-19 18:52:15 +00:00
SyntacticLanguageModelState.h beautify 2013-05-29 18:16:15 +01:00
TargetPhrase.cpp debugged CoveredReferenceFeature 2013-12-03 17:25:42 +01:00
TargetPhrase.h compare translation option dimension using pointers to translation option, rather than comparing the phrase. Comparinf the phrase is incorrect - need to compare LHS and possibly alignments too. Unnecessary 2013-12-13 17:23:39 +00:00
TargetPhraseCollection.cpp replace nth_element() with macro that execute sort() instead for gcc 4.8.1 & 4.8.2 2013-11-15 10:55:38 +00:00
TargetPhraseCollection.h skelete pt 2013-09-28 09:42:17 +01:00
Terminal.h faster compilation by taking some #includes from Word.h and Phrase.h 2013-10-04 15:40:23 +01:00
ThreadPool.cpp move moses/src/* to moses/ 2012-11-12 19:56:18 +00:00
ThreadPool.h beautify 2013-05-29 18:16:15 +01:00
Timer.cpp make good use of the new precise timer... 2014-01-03 18:45:31 +00:00
Timer.h make good use of the new precise timer... 2014-01-03 18:45:31 +00:00
TranslationOption.cpp UTIL_THROW_IF -> UTIL_THROW_IF2 2013-11-22 20:27:46 +00:00
TranslationOption.h implement Evaluate(inputpath, input) for chart translation options. Plumbing for lattice input 2013-08-13 20:36:32 +01:00
TranslationOptionCollection.cpp make good use of the new precise timer... 2014-01-03 18:45:31 +00:00
TranslationOptionCollection.h correct creation of input paths for lattices 2013-10-03 18:58:45 +01:00
TranslationOptionCollectionConfusionNet.cpp move all feature functions out of StaticData 2013-12-07 00:21:06 +00:00
TranslationOptionCollectionConfusionNet.h move checking of legacy phrase-table to StaticData 2013-10-02 20:02:05 +01:00
TranslationOptionCollectionLattice.cpp move all feature functions out of StaticData 2013-12-07 00:21:06 +00:00
TranslationOptionCollectionLattice.h correct creation of input paths for lattices 2013-10-04 13:08:14 +01:00
TranslationOptionCollectionText.cpp replace CHECK with UTIL_THROW_IF in Moses 2013-11-19 17:23:19 +00:00
TranslationOptionCollectionText.h xml constraint 2013-08-15 11:46:45 +01:00
TranslationOptionList.cpp move moses/src/* to moses/ 2012-11-12 19:56:18 +00:00
TranslationOptionList.h UTIL_THROW_IF -> UTIL_THROW_IF2 2013-11-22 20:27:46 +00:00
TreeInput.cpp various optimizations to make CYK+ parser several times faster and eat less memory. 2014-03-21 11:12:24 +00:00
TreeInput.h create target phrases, not Translation Options, for XML. Can be used in both pb and scfg 2013-07-24 15:55:17 +01:00
TrellisPath.cpp UTIL_THROW_IF -> UTIL_THROW_IF2 2013-11-22 20:27:46 +00:00
TrellisPath.h beautify 2013-05-29 18:16:15 +01:00
TrellisPathCollection.cpp move moses/src/* to moses/ 2012-11-12 19:56:18 +00:00
TrellisPathCollection.h move moses/src/* to moses/ 2012-11-12 19:56:18 +00:00
TrellisPathList.h move moses/src/* to moses/ 2012-11-12 19:56:18 +00:00
TypeDef.h delete -parsing-algorithm. Scope-3 parsing depends on the phrase-table implementation used, not this argument 2013-11-22 17:35:22 +00:00
UniqueObject.h move moses/src/* to moses/ 2012-11-12 19:56:18 +00:00
UserMessage.cpp move moses/src/* to moses/ 2012-11-12 19:56:18 +00:00
UserMessage.h move moses/src/* to moses/ 2012-11-12 19:56:18 +00:00
Util.cpp beautify 2013-05-29 18:16:15 +01:00
Util.h UTIL_THROW_IF -> UTIL_THROW_IF2 2013-11-22 20:27:46 +00:00
Word.cpp various optimizations to make CYK+ parser several times faster and eat less memory. 2014-03-21 11:12:24 +00:00
Word.h faster compilation by taking some #includes from Word.h and Phrase.h 2013-10-04 15:40:23 +01:00
WordLattice.cpp move all feature functions out of StaticData 2013-12-07 00:21:06 +00:00
WordLattice.h beautify 2013-10-03 11:33:48 +01:00
WordsBitmap.cpp output list of available feature functions, separated by space, rather than lines 2014-01-25 10:58:00 +00:00
WordsBitmap.h output list of available feature functions, separated by space, rather than lines 2014-01-25 10:39:47 +00:00
WordsRange.cpp move moses/src/* to moses/ 2012-11-12 19:56:18 +00:00
WordsRange.h UTIL_THROW_IF -> UTIL_THROW_IF2 2013-11-22 20:27:46 +00:00
XmlOption.cpp various optimizations to make CYK+ parser several times faster and eat less memory. 2014-03-21 11:12:24 +00:00
XmlOption.h get rid of some warnings 2013-09-22 18:24:32 +02:00