moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2024-09-19 07:07:24 +03:00
Code Issues Packages Projects Releases Wiki Activity
14,471 Commits 136 Branches 26 Tags 147 MiB
5732129bd4
Commit Graph

139 Commits

Author SHA1 Message Date
Hieu Hoang
fdcfe95574 start framework for evaluating target phrase score 2013-05-02 17:19:45 +01:00
Hieu Hoang
ce95c117f6 merge 2013-04-29 18:46:48 +01:00
Kenneth Heafield
f1d3663810 Back FactorCollection with a memory pool. Less memory for large vocabularies. 2013-04-25 19:42:30 +01:00
Hieu Hoang
71a2b49a47 Merge github.com:moses-smt/mosesdecoder into weight-new 2013-04-01 16:43:32 +01:00
phikoehn
0a978e9f01 bug fixes 2013-04-01 14:31:32 +01:00
Hieu Hoang
194adf8130 change format for phrase model 2013-02-22 19:17:57 +00:00
Hieu Hoang
59449f2925 make TranslationModel subdirectory and move files from moses/ into it 2012-11-27 15:08:31 +00:00
Wilker Aziz
9e76804bde Python wrap for OnDiskWraper, docs and example. 2012-11-14 18:14:46 +01:00
Wilker Aziz
6bbf6db180 Wrapping OnDiskPt, Added OnDiskPt/Util.h with the tokenization procedure, so that they can be called from the Python wrap. 2012-11-13 17:27:33 +01:00
Hieu Hoang
44f24bd673 resolved merge conflicts 2012-11-12 20:35:32 +00:00
Hieu Hoang
f8438f80cc move moses/src/* to moses/ 2012-11-12 20:30:39 +00:00
Hieu Hoang
5e3ef23cef move moses/src/* to moses/ 2012-11-12 19:56:18 +00:00
Kenneth Heafield
d4f433c7f4 Put core Moses into one library libmoses. 2012-11-12 18:07:53 +00:00
Kenneth Heafield
c34ecf12de TargetPhrase doesn't have a FactorDirection constructor. 
It was implicitly doing TargetPhrase(Phrase(size_t)) wastefully
 2012-10-22 17:40:23 +01:00
Kenneth Heafield
f1bb9be091 D'oh missed an argument in usage 2012-10-20 18:08:20 +01:00
Hieu Hoang
f4c965e752 Merge branch 'master' of github.com:moses-smt/mosesdecoder 2012-10-19 15:10:20 +01:00
Hieu Hoang
397e704b49 terminal and non-terminal word alignments stored in 2 separate objects 2012-10-19 15:10:10 +01:00
Kenneth Heafield
8e23e441c3 Fix virtual destructors 2012-10-18 11:35:42 -04:00
Kenneth Heafield
f4116ec6f5 Add usage message for OnDiskPt 2012-10-17 14:45:58 +01:00
Kenneth Heafield
dd662e9b58 Heap->Stack variable 2012-10-17 12:35:28 +01:00
Hieu Hoang
e208c997d9 same version number check when reading and writing 2012-10-16 16:29:33 +01:00
Hieu Hoang
a6f63af172 same version number check when reading and writing 2012-10-16 16:26:31 +01:00
Barry Haddow
84f616fe63 Increase version number to 5 2012-10-16 16:20:56 +01:00
Barry Haddow
0a950ee9f4 Merge remote branch 'github/master' into miramerge 
Compiles, but not tested. Had to disable relent filter. Strangely, it seems to contain the
whole of moses-cmd.

Conflicts:
	Jamroot
	OnDiskPt/TargetPhrase.cpp
	moses-cmd/src/Main.cpp
	moses/src/AlignmentInfo.cpp
	moses/src/AlignmentInfo.h
	moses/src/ChartTranslationOptionCollection.cpp
	moses/src/ChartTranslationOptionCollection.h
	moses/src/GenerationDictionary.cpp
	moses/src/Jamfile
	moses/src/Parameter.cpp
	moses/src/PhraseDictionary.cpp
	moses/src/StaticData.cpp
	moses/src/StaticData.h
	moses/src/TargetPhrase.h
	moses/src/TranslationSystem.cpp
	moses/src/TranslationSystem.h
	moses/src/Word.cpp
	phrase-extract/score.cpp
	regression-testing/Jamfile
	scripts/ems/experiment.meta
	scripts/ems/experiment.perl
	scripts/training/train-model.perl
 2012-09-26 22:49:33 +01:00
Kenneth Heafield
ab60d1ad6f Slightly less wasteful OnDiskPt word conversion 
Ideally, OnDiskPt would convert its entire vocabulary to Word in advance.
 2012-09-26 12:52:11 +01:00
Barry Haddow
576bc09fa5 fix memory leaks in binarisation 2012-09-19 21:48:10 +01:00
Barry Haddow
1e4bff1def Fix memory leak in binarisation 2012-09-18 15:31:29 +01:00
Barry Haddow
2b4e61d826 Merge branch 'trunk' into miramerge 
Compiles, not tested.

Conflicts:
	Jamroot
	OnDiskPt/PhraseNode.h
	OnDiskPt/TargetPhrase.cpp
	OnDiskPt/TargetPhrase.h
	OnDiskPt/TargetPhraseCollection.cpp
	mert/BleuScorer.cpp
	mert/Data.cpp
	mert/FeatureData.cpp
	moses-chart-cmd/src/Main.cpp
	moses/src/AlignmentInfo.h
	moses/src/ChartManager.cpp
	moses/src/LM/Ken.cpp
	moses/src/LM/Ken.h
	moses/src/LMList.h
	moses/src/LexicalReordering.h
	moses/src/PhraseDictionaryTree.h
	moses/src/ScoreIndexManager.h
	moses/src/StaticData.h
	moses/src/TargetPhrase.h
	moses/src/Word.cpp
	scripts/ems/experiment.meta
	scripts/ems/experiment.perl
	scripts/training/train-model.perl
 2012-07-17 13:36:50 +01:00
Eva Hasler
027a20730e merge Jamfiles 2012-07-04 11:49:07 +01:00
Hieu Hoang
647e320629 commented all classes in OnDiskPt and moses-chart-cmd. Leave ken's lm directory 2012-06-28 20:12:31 -04:00
Hieu Hoang
e71f76a3e5 delete unused function 2012-06-16 00:38:50 +01:00
Hieu Hoang
d2416bbe26 up version number 2012-06-15 22:35:57 +01:00
Hieu Hoang
6297c67fb2 Binary pt works with factors and chart decoding 2012-06-15 22:32:00 +01:00
Hieu Hoang
6096f21740 factors in chart decoding runs. But target factor model has incorrect LM score 2012-06-15 21:44:45 +01:00
Eva Hasler
e1c1a5343c merge 2012-06-07 11:16:52 +01:00
Eva Hasler
6a6a35c65e fix start weights in experiment.perl, add hypothesis queue for picking hope and fear translations, add variations to 1slack formulation 2012-06-01 01:49:42 +01:00
Hieu Hoang
4c88377cbe change CreateOnDisk to CreateOnDiskPt 2012-05-31 14:28:58 +01:00
Phil Williams
2fab137aae Fix compile error. 2012-05-25 17:24:08 +01:00
Hieu Hoang
8844be6f25 Move queryOnDiskPt from contrib to OnDiskPt folder. Easier to amend Jam file 2012-05-25 13:41:06 +01:00
Eva
4e9babf040 enable binarization of rule tables with terminal alignments and new rule count 2012-05-18 11:09:04 -07:00
Hieu Hoang
299451235c Add util for querying on-disk pt 2012-05-15 16:46:16 +01:00
Hieu Hoang
3f3401f9ba Add util for querying on-disk pt 2012-05-15 16:35:00 +01:00
Hieu Hoang
12278a9f98 Add util for querying on-disk pt 2012-05-15 16:17:06 +01:00
Hieu Hoang
d25a2baa62 Add util for querying on-disk pt 2012-05-15 16:16:16 +01:00
Hieu Hoang
d924a709ea Add util for querying on-disk pt 2012-05-15 16:03:40 +01:00
Hieu Hoang
1ef0d6b7f6 bug. Unknown words given id starting from 1 2012-05-15 15:43:01 +01:00
Hieu Hoang
25d269390b Add util for querying on-disk pt 2012-05-15 15:17:59 +01:00
Hieu Hoang
fea7bf9d1a Add util for querying on-disk pt 2012-05-15 15:06:25 +01:00
Eva Hasler
30deedde9f changed permission, everything changed.. 2012-05-10 18:54:24 +01:00
Christian Federmann
4ccab95808 Merged in fixes for compiler warnings regarding comparing signed and unsigned integer expressions. 2012-05-10 14:48:51 +02:00
Tetsuo Kiso
857398dfbd Fix warnings: -Wreorder and -Wmismatched-tags. 2012-05-05 18:51:12 +09:00
Eva
6f39ad0b3e test 2012-04-28 23:11:30 -07:00
Eva Hasler
752c36cb31 use rule count feature in binary table 2012-04-24 05:14:59 +01:00
Eva Hasler
7e0c498096 OnDiskPt: use third count instead of second 2012-04-11 21:02:13 +01:00
Hieu Hoang
98fc6d9917 xcode 2012-01-08 14:36:54 +07:00
Hieu Hoang
39db162cc0 move vs and xcode project files into separate contrib folder 2011-12-02 08:33:39 +07:00
Hieu Hoang
71c78ef0ce move vs and xcode project files into separate contrib folder 2011-12-02 08:33:39 +07:00
Kenneth Heafield
a0d79781e5 Merge CreateOnDisk into OnDiskPt, pop out of src 2011-12-01 18:26:05 +00:00
Kenneth Heafield
9fe831c60c Merge CreateOnDisk into OnDiskPt, pop out of src 2011-12-01 18:26:05 +00:00
Kenneth Heafield
c53453b266 No more autotools (with bhaddow in the room) 2011-12-01 17:17:28 +00:00
Kenneth Heafield
42975b0635 No more autotools (with bhaddow in the room) 2011-12-01 17:17:28 +00:00
Kenneth Heafield
79eecac809 Actually, moses depends on OnDiskPt, not the other way around. 2011-11-25 08:33:16 -05:00
Kenneth Heafield
056b42f171 Actually, moses depends on OnDiskPt, not the other way around. 2011-11-25 08:33:16 -05:00
Hieu Hoang
ce0065ff2a Merge branch 'master' of github.com:moses-smt/mosesdecoder 2011-11-18 22:13:46 +07:00
Kenneth Heafield
5c9b28c55a Introducing Jamfiles 
Currently supports kenlm, irstlm, and srilm options.

bjam [--with-srilm=/path/to/sri] [--with-irstlm=/path/to/irst] -j8
Places binaries in dist/ that pass regression tests.
 2011-11-18 14:37:01 +00:00
Hieu Hoang
caac64a33b xcode 2011-11-18 21:33:58 +07:00
Kenneth Heafield
bf78f7a1ac Replace assert with CHECK until people learn how to use assert properly 2011-11-18 12:07:41 +00:00
Hieu Hoang
87500bc93d visual studio 2011-11-11 22:24:25 +00:00
Hieu Hoang
cb87f251b2 xcode 2011-10-24 14:44:36 +07:00
hieuhoang1972
897fe0f88b visual studio 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4356 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-14 10:50:08 +00:00
hieuhoang1972
9762a65bd8 xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4264 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-25 10:44:47 +00:00
hieuhoang1972
9c0d725cde visual studio 2010 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4079 1f5c12ca-751b-0410-a591-d2e778427230
 2011-07-19 03:07:15 +00:00
pjwilliams
2451371ca2 Changes to chart decoder cube pruning: create one cube per dotted rule 
instead of one per translation and do 'non-lazy' scoring, i.e. fully
score the corner and neighbor hypotheses inside the rule cube instead
of waiting until an item is popped.  The old behaviour -- faster but
with more search errors -- is available via the
cube-pruning-lazy-scoring option.

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4039 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-27 15:13:15 +00:00
pjwilliams
ab3460591c Share AlignmentInfo objects instead of storing one per TargetPhrase. 
This can save a significant amount of memory used on rule table storage,
though may increase loading time slightly.

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4017 1f5c12ca-751b-0410-a591-d2e778427230
 2011-06-16 21:20:20 +00:00
hieuhoang1972
0fdde952bc compiles with clang++ 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3940 1f5c12ca-751b-0410-a591-d2e778427230
 2011-04-01 23:31:11 +00:00
phkoehn
93ed0ca1a9 get rid of "unused variable" warnings 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3903 1f5c12ca-751b-0410-a591-d2e778427230
 2011-02-24 16:17:38 +00:00
hieuhoang1972
a3d97584a9 run beautify.perl. Consistent formatting for .h & .cpp files 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3902 1f5c12ca-751b-0410-a591-d2e778427230
 2011-02-24 13:57:11 +00:00
hieuhoang1972
abacb9166a xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3857 1f5c12ca-751b-0410-a591-d2e778427230
 2011-01-28 14:57:55 +00:00
hieuhoang1972
677f0f47a8 copyright notice & svn properties 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3620 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-12 16:17:41 +00:00
hieuhoang1972
ac875ffc81 mem leak 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3617 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-12 14:47:58 +00:00
hieuhoang1972
ef09298824 move function calls with side effects out of asserts 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3403 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-12 12:54:55 +00:00
bhaddow
904133fcb7 Merge in the multiple models branch. These changes allow the moses server 
to support multiple translation, language and generation models within the
same process. The main design change is the introduction of a TranslationSystem
object to manage the models, which have been moved out of StaticData.
The changes should have no effect on existing systems.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3394 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-10 13:12:00 +00:00
hieuhoang1972
7db72471ce xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3313 1f5c12ca-751b-0410-a591-d2e778427230
 2010-06-09 13:00:54 +00:00
hieuhoang1972
0d6d78b5a9 xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3308 1f5c12ca-751b-0410-a591-d2e778427230
 2010-06-08 11:42:28 +00:00
hieuhoang1972
fdfd4907f5 xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3306 1f5c12ca-751b-0410-a591-d2e778427230
 2010-06-08 00:59:48 +00:00
hieuhoang1972
a6b7807b2e less debugging info from OnDisk pt 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3227 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-06 11:19:53 +00:00
bhaddow
5340c16dcf Fix threaded build. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3121 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-12 11:11:33 +00:00
hieuhoang1972
5bab778f02 merge 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3079 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-08 17:57:38 +00:00
hieuhoang1972
c117ef7c17 Copy in changes from the chart_merge branch (doing it manually because the 
server doesn't seem to support subversion's --reintegrate option).


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3078 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-08 17:16:10 +00:00