moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2024-10-06 08:18:02 +03:00
Code Issues Packages Projects Releases Wiki Activity
6,522 Commits 136 Branches 26 Tags 147 MiB
5b7c5ebdb5
Commit Graph

275 Commits

Author SHA1 Message Date
bhaddow
5b7c5ebdb5 Merge branch 'master' into miramerge 
Conflicts:
	moses-chart-cmd/src/Main.cpp
	moses-cmd/src/IOWrapper.cpp
	moses-cmd/src/Main.cpp
	moses/src/DummyScoreProducers.cpp
	moses/src/DummyScoreProducers.h
	moses/src/GenerationDictionary.cpp
	moses/src/GenerationDictionary.h
	moses/src/GlobalLexicalModel.h
	moses/src/LMList.h
	moses/src/LanguageModel.cpp
	moses/src/LanguageModel.h
	moses/src/LanguageModelImplementation.h
	moses/src/LanguageModelKen.h
	moses/src/LanguageModelMultiFactor.cpp
	moses/src/LanguageModelMultiFactor.h
	moses/src/LanguageModelSingleFactor.cpp
	moses/src/LanguageModelSingleFactor.h
	moses/src/LexicalReordering.h
	moses/src/PhraseDictionary.cpp
	moses/src/PhraseDictionary.h
	moses/src/ScoreIndexManager.cpp
	moses/src/ScoreProducer.h
	moses/src/StaticData.cpp
	moses/src/StaticData.h
	moses/src/TranslationSystem.cpp
 2011-09-20 11:23:38 +01:00
phkoehn
7927439ce5 option to get unpruned search graph for chart decoder 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4228 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-16 11:58:53 +00:00
bhaddow
e95c004145 Implement LM OOV feature 
- still have to run regression
 - doesn't work for moses_chart (awaiting H & K LM state merge)


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4211 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-09 18:03:00 +00:00
hieuhoang1972
5449839d75 option to sort word alignment info, as suggested by arianna bisazza 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4164 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-26 02:37:52 +00:00
bhaddow
1f3c11b441 Merge branch 'master' into miramerge 
Passes all regression except for the score tests, as I still have to merge
in the missing files for these tests.

Conflicts:
	.gitignore
	Makefile.am
	configure.in
	moses-chart/moses-chart.xcodeproj/project.pbxproj
	moses/src/ChartHypothesis.cpp
	moses/src/ChartTranslationOptionCollection.cpp
	moses/src/ChartTrellisPath.cpp
	moses/src/DotChart.cpp
	moses/src/LMList.h
	moses/src/LanguageModel.cpp
	moses/src/LanguageModel.h
	moses/src/LanguageModelFactory.cpp
	moses/src/LanguageModelRemote.h
	moses/src/LexicalReordering.h
	moses/src/Makefile.am
	moses/src/Manager.cpp
	moses/src/Parameter.cpp
	moses/src/RuleCubeQueue.h
	moses/src/ScoreIndexManager.cpp
	moses/src/StaticData.cpp
	moses/src/StaticData.h
	moses/src/TargetPhrase.cpp
	moses/src/TargetPhrase.h
	moses/src/TranslationOptionCollection.cpp
 2011-08-22 14:52:02 +01:00
bhaddow
1282500392 Merge commit 'a0b6abdfd3599e7fbdc6aac76fcd2cb4483d63ce' into miramerge 
Conflicts:
	moses/src/ConfusionNet.cpp
	moses/src/DecodeFeature.cpp
	moses/src/DecodeFeature.h
	moses/src/DotChartOnDisk.cpp
	moses/src/DummyScoreProducers.cpp
	moses/src/DummyScoreProducers.h
	moses/src/DynSAInclude/vocab.h
	moses/src/FeatureFunction.h
	moses/src/GenerationDictionary.cpp
	moses/src/GenerationDictionary.h
	moses/src/GlobalLexicalModel.cpp
	moses/src/GlobalLexicalModel.h
	moses/src/LMList.cpp
	moses/src/LMList.h
	moses/src/LanguageModel.cpp
	moses/src/LanguageModel.h
	moses/src/LanguageModelFactory.cpp
	moses/src/LanguageModelFactory.h
	moses/src/LanguageModelImplementation.h
	moses/src/LanguageModelKen.h
	moses/src/LanguageModelMultiFactor.cpp
	moses/src/LanguageModelMultiFactor.h
	moses/src/LanguageModelParallelBackoff.h
	moses/src/LanguageModelRemote.h
	moses/src/LanguageModelSingleFactor.cpp
	moses/src/LanguageModelSingleFactor.h
	moses/src/LexicalReordering.cpp
	moses/src/LexicalReordering.h
	moses/src/LexicalReorderingState.cpp
	moses/src/LexicalReorderingState.h
	moses/src/Manager.cpp
	moses/src/PDTAimp.h
	moses/src/Parameter.cpp
	moses/src/Parameter.h
	moses/src/Phrase.h
	moses/src/PhraseDictionary.cpp
	moses/src/PhraseDictionary.h
	moses/src/PhraseDictionaryMemory.cpp
	moses/src/ScoreComponentCollection.cpp
	moses/src/ScoreComponentCollection.h
	moses/src/ScoreIndexManager.cpp
	moses/src/ScoreIndexManager.h
	moses/src/ScoreProducer.h
	moses/src/StaticData.cpp
	moses/src/StaticData.h
	moses/src/TargetPhrase.cpp
	moses/src/TargetPhrase.h
	moses/src/TranslationOption.cpp
	moses/src/TranslationOptionCollection.cpp
	moses/src/TranslationSystem.cpp
	moses/src/TranslationSystem.h
	moses/src/TrellisPath.h
 2011-08-19 17:09:36 +01:00
bhaddow
cf81f9188f Merged up to 3895 from trunk 
git-svn-id: http://svn.statmt.org/repository/mira@3934 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:38 +02:00
pkoehn
69fdd15792 added WordTranslationFeature 
git-svn-id: http://svn.statmt.org/repository/mira@3930 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:38 +02:00
pkoehn
c815741145 added SourceWordDeletionFeature 
git-svn-id: http://svn.statmt.org/repository/mira@3929 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:38 +02:00
pkoehn
56a9a26c06 added TargetWordInsertionFeature 
git-svn-id: http://svn.statmt.org/repository/mira@3928 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:38 +02:00
pkoehn
7c4255c6f7 fix 
git-svn-id: http://svn.statmt.org/repository/mira@3927 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:38 +02:00
pkoehn
42e10d010b added reporting of sparse features in n-best list 
git-svn-id: http://svn.statmt.org/repository/mira@3926 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:38 +02:00
pkoehn
b919b862b8 added phrase length feature 
git-svn-id: http://svn.statmt.org/repository/mira@3925 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:38 +02:00
bhaddow
08c3efdf88 phrase boundary feature 
git-svn-id: http://svn.statmt.org/repository/mira@3893 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:37 +02:00
bhaddow
e664b4a4b3 Merge 3791-3842 from trunk 
git-svn-id: http://svn.statmt.org/repository/mira@3873 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:36 +02:00
bhaddow
c8e7c5da05 initialisation of source phrase member variable in TargetPhrase 
git-svn-id: http://svn.statmt.org/repository/mira@3849 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:35 +02:00
bhaddow
367ac683a3 Implementation of phrase pair feature 
git-svn-id: http://svn.statmt.org/repository/mira@3842 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:34 +02:00
bhaddow
3348dcd03e Configurable factor id for target bigram 
git-svn-id: http://svn.statmt.org/repository/mira@3812 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:33 +02:00
theleopardess
d7752b44fc I tested check-in by adding a test line in moses/src/StaticData.cpp, producing a trivial moses revision 4122. Now I have removed that line. Everything ok but sorry for the confusion. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4129 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-05 18:57:09 +00:00
theleopardess
f8a99e5d6d yanggao-softdep-v0 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4122 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-05 15:11:43 +00:00
hieuhoang1972
06af5d40d4 Improved error message 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4081 1f5c12ca-751b-0410-a591-d2e778427230
 2011-07-21 02:41:23 +00:00
pjwilliams
2451371ca2 Changes to chart decoder cube pruning: create one cube per dotted rule 
instead of one per translation and do 'non-lazy' scoring, i.e. fully
score the corner and neighbor hypotheses inside the rule cube instead
of waiting until an item is popped.  The old behaviour -- faster but
with more search errors -- is available via the
cube-pruning-lazy-scoring option.

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4039 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-27 15:13:15 +00:00
leven101
f64885599d added ClearTransOptCache() to StaticData class and call it when updating suffix PTs 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3996 1f5c12ca-751b-0410-a591-d2e778427230
 2011-05-31 09:42:27 +00:00
dowobeha
e04e155986 Syntactic LM can now be loaded properly by StaticData. This change should have been merged earlier, but was omitted. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3991 1f5c12ca-751b-0410-a591-d2e778427230
 2011-05-24 15:47:35 +00:00
dowobeha
9581b6aae2 Updated syntactic classes (cherry-picking from commit bf7a7bb8e7) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3973 1f5c12ca-751b-0410-a591-d2e778427230
 2011-05-13 19:28:23 +00:00
dowobeha
9375aa8846 Reverting changes. Revision 3971 was a bad commit. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3972 1f5c12ca-751b-0410-a591-d2e778427230
 2011-05-13 18:27:30 +00:00
dowobeha
bb941c01f6 Merge branch 'master' into local-trunk 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3971 1f5c12ca-751b-0410-a591-d2e778427230
 2011-05-13 18:07:21 +00:00
pjwilliams
75709a6c87 Memory efficiency: make the reserveSize argument non-optional in 
Moses::Phrase's constructor.  The default used to be ARRAY_SIZE_INCR = 10,
which will be excessive in many cases.  Where the default was used, I've
set the exact size where that was obvious and explicitly used ARRAY_SIZE_INCR 
otherwise.  If you know the code involved, it's probably worth reviewing.



git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3908 1f5c12ca-751b-0410-a591-d2e778427230
 2011-02-28 11:41:08 +00:00
hieuhoang1972
a0b6abdfd3 run beautify.perl. Consistent formatting for .h & .cpp files 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3901 1f5c12ca-751b-0410-a591-d2e778427230
 2011-02-24 13:14:42 +00:00
phkoehn
4e72cd91be added decoding-graph-backoff, still experimenting with it 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3869 1f5c12ca-751b-0410-a591-d2e778427230
 2011-02-03 13:41:44 +00:00
bojar
0bc0ece594 Ales Tamchyna's printing of alignments (-print-alignment-info did nothing) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3867 1f5c12ca-751b-0410-a591-d2e778427230
 2011-02-03 09:08:42 +00:00
bhaddow
7b6503680a Shortcut when trans opts cache is size 0. Avoids potential uninitialised read. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3825 1f5c12ca-751b-0410-a591-d2e778427230
 2011-01-21 21:25:14 +00:00
bhaddow
a2730c445d Merge up to 3791 from trunk. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3792 1f5c12ca-751b-0410-a591-d2e778427230
 2011-01-05 13:49:44 +00:00
evahasler
1de7aa4f81 enable batch optimisation, batch history updating, fix bug in Decoder.cpp (new: staticData.ReLoadBleuScoreFeatureParameter()) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3734 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-24 17:06:54 +00:00
chardmeier
837a667a95 Cleaned up language modelling code by disentangling the decoder's LM feature 
function from the LM toolkit abstraction layer. There are two different groups
of classes now:
- LanguageModel, which inherits from StatefulFeatureFunction and contains
  the n-gram model feature function.
- LanguageModelImplementation, which is the base class of the individual
  LM implementations (SRI, IRST, RandLM, KenLM) and provides methods to
  query LM probabilities and states.
Each LanguageModel controls a LanguageModelImplementation. Implementations can
be shared by more than one LanguageModel.
This should make it easier to use the LM libraries as a backend for other
feature functions while retaining the flexibility to use different LM toolkits.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3719 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-17 14:06:21 +00:00
bhaddow
9b438d9849 correction to error message 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3635 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-21 13:23:53 +00:00
bhaddow
0f09365a99 Enable usage of target bigram feature in decoding. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3626 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-15 15:19:17 +00:00
bhaddow
7e72ceea22 Goodbye ScoreIndexManager. 
Compiles ok, but haven't dared to run regression yet.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3608 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-06 22:06:49 +00:00
bhaddow
82dba81917 Merged out from trunk to revision 3604 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3605 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-05 16:02:25 +00:00
hieuhoang1972
03419ac02a it runs\!\! 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3542 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-17 20:27:26 +00:00
ales-t
5ca82431e4 Return true if GetParam returns empty vector. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3524 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-17 14:43:38 +00:00
hieuhoang1972
19ab98040d return weights as score component collection 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3521 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-17 14:25:08 +00:00
ales-t
540d6dd6e7 Fixed some compilation errors. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3516 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-17 13:46:40 +00:00
ales-t
4e48b1d9fe Prepared the integration of target bigram features. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3515 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-17 13:36:03 +00:00
hieuhoang1972
0eddbe99e1 bleu weight changeable 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3513 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-17 13:21:51 +00:00
hieuhoang1972
6a8ee95ebb bleu weight changeable 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3508 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-17 12:37:30 +00:00
smaj
36b6aad53e Calling of InitFeatureNames moved. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3496 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-16 20:40:20 +00:00
bhaddow
907766d67a reduce compile time dependencies 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3488 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-16 15:45:56 +00:00
bhaddow
ba5ccb7bc5 debugging and fixes for BleuScoreFeature 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3479 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-16 12:49:57 +00:00
smaj
5204e29681 Fixed usage of []. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3470 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-15 16:30:52 +00:00