moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2024-10-27 20:12:02 +03:00
Code Issues Packages Projects Releases Wiki Activity
9,773 Commits 136 Branches 26 Tags 147 MiB
01af82310b
Commit Graph

309 Commits

Author SHA1 Message Date
Hieu Hoang
63109d272b minor tweaks. Working now 2012-07-26 14:47:49 +01:00
Hieu Hoang
8329b46019 move old code into its own folder 2012-07-26 14:43:31 +01:00
Hieu Hoang
7dd138216f more debugging of tm-mt scripts 2012-07-26 14:41:34 +01:00
Hieu Hoang
824f34605d more debugging of tm-mt scripts 2012-07-26 13:04:33 +01:00
Hieu Hoang
41e990a814 more debugging of tm-mt scripts 2012-07-25 17:53:49 +01:00
Hieu Hoang
13a83ae0c7 split large cpp file into smaller .h & .cpp files 2012-07-25 12:24:31 +01:00
Hieu Hoang
7a44eaa7fb separate large cpp file into smaller header & cpp files 2012-07-25 12:22:13 +01:00
Hieu Hoang
46ed6eaa22 bring perl code into c++ 2012-07-24 16:42:35 +01:00
Hieu Hoang
c8a0107895 xcode for fuzzy-match program 2012-07-24 14:01:13 +01:00
Hieu Hoang
7787ae2a11 read in pt file. Compiles but and runs. Need to check scores 2012-07-23 16:31:38 +01:00
Hieu Hoang
8599b4fd24 read in pt file. Compiles but and runs. Need to check scores 2012-07-23 15:10:15 +01:00
Hieu Hoang
8bffcac320 error write input sentence to file for tm-mt extraction 2012-07-21 23:17:27 +01:00
Hieu Hoang
541fc740a6 Merge https://github.com/moses-smt/mosesdecoder 2012-07-20 14:23:38 +01:00
U-Hieu-PC\Hieu
1cbf457685 added paths to boost includes and libs 2012-07-20 14:15:11 +01:00
Hieu Hoang
fa5a7a92df Visual Studion project files. Submitted by a Windows developer. 2012-07-20 12:16:29 +01:00
Hieu Hoang
d8daac9bb4 Merge branch 'master' of github.com:moses-smt/mosesdecoder 2012-07-19 22:22:53 +01:00
Hieu Hoang
3f9c995e18 Add PhraseDictionaryTMExtract and ChartRuleLookupManagerMemoryPerSentence to deal with per per sentence grammar. eg. Adam's suffix array and Phi Koehn's TM extraction 2012-07-19 17:56:46 +01:00
Hieu Hoang
d988ca6ed4 Merge https://github.com/moses-smt/mosesdecoder 2012-07-19 16:45:52 +01:00
Rico Sennrich
dd2cf70f3f tmcombine: fix combination of lexical reordering models 2012-07-19 10:31:46 +02:00
Hieu Hoang
9a51e283db Visual Studion project files. Submitted by a Windows developer. 2012-07-19 08:59:46 +01:00
Hieu Hoang
23cfa630fe Added phrase dictionary for phi's tm extraction. Copy of SCFG phrase dictionart so far 2012-07-18 17:59:21 +01:00
Hieu Hoang
7aa3526471 Merge https://github.com/moses-smt/mosesdecoder 2012-07-18 16:32:44 +01:00
Hieu Hoang
b32c484c35 Added phrase dictionary for phi's tm extraction. Copy of SCFG phrase dictionart so far 2012-07-18 14:52:48 +01:00
U-Hieu-PC\Hieu
ceef0342cc visual studio build 2012-07-17 23:39:55 +01:00
Hieu Hoang
07498356d2 tm-mt integration. Runs all the way through to creating rule table without crashing 2012-07-17 18:20:53 +01:00
Hieu Hoang
fbd651c9d4 tm-mt integration. Runs all the ways through to creating rule table without crashing 2012-07-17 16:52:12 +01:00
Barry Haddow
2b4e61d826 Merge branch 'trunk' into miramerge 
Compiles, not tested.

Conflicts:
	Jamroot
	OnDiskPt/PhraseNode.h
	OnDiskPt/TargetPhrase.cpp
	OnDiskPt/TargetPhrase.h
	OnDiskPt/TargetPhraseCollection.cpp
	mert/BleuScorer.cpp
	mert/Data.cpp
	mert/FeatureData.cpp
	moses-chart-cmd/src/Main.cpp
	moses/src/AlignmentInfo.h
	moses/src/ChartManager.cpp
	moses/src/LM/Ken.cpp
	moses/src/LM/Ken.h
	moses/src/LMList.h
	moses/src/LexicalReordering.h
	moses/src/PhraseDictionaryTree.h
	moses/src/ScoreIndexManager.h
	moses/src/StaticData.h
	moses/src/TargetPhrase.h
	moses/src/Word.cpp
	scripts/ems/experiment.meta
	scripts/ems/experiment.perl
	scripts/training/train-model.perl
 2012-07-17 13:36:50 +01:00
Hieu Hoang
759aaf4e9d Merge https://github.com/moses-smt/mosesdecoder 2012-07-17 11:30:39 +01:00
Hieu Hoang
e31b4242e4 win32 build submitted by Windows developer. Includes small changes to order of hypothesis popping during cube pruning 2012-07-17 11:01:07 +01:00
Hieu Hoang
83e9e22690 working scripts 2012-07-16 19:16:39 +01:00
Rico Sennrich
9dd42dd9b6 tmcombine: fix crash with option --recompute-lexweights 2012-07-16 13:26:39 +02:00
Hieu Hoang
0c68957c20 Merge branch 'master' of github.com:moses-smt/mosesdecoder 2012-07-12 16:46:30 +01:00
Hieu Hoang
15b95cd042 use consistent alignment info for lexical probabilities for both forward and inverse scoring 2012-07-12 16:45:43 +01:00
Rico Sennrich
c9f2f73de3 sigtest-filter can now also filter lexical reordering tables. 
didn't have a significant effect on performance in my tests, but it's nice to have smaller tables.
 2012-07-12 16:04:23 +02:00
Hieu Hoang
3591f385d1 get philipp's code 2012-07-11 12:30:54 +01:00
U-Hieu-PC\Hieu
cfc1a71670 take out threading from vs build 2012-07-11 11:34:14 +01:00
U-Hieu-PC\Hieu
c7059e5e0b visual studio compiles but doesn't link 2012-07-11 10:54:21 +01:00
Hieu Hoang
4a34d5241e Merge https://github.com/hieuhoang/mosesdecoder 2012-07-10 17:09:28 +01:00
Hieu Hoang
89fc1fee05 xcode 2012-07-10 16:17:11 +01:00
Hieu Hoang
5dd0afb07b debugging lr scrore 2012-07-10 16:15:36 +01:00
Hieu Hoang
f4ca039d3c xcode 2012-07-10 15:00:56 +01:00
Hieu Hoang
16965f3469 convert new[] to vector. In case of leaks due to exception handling 2012-07-10 10:23:00 +01:00
Hieu Hoang
f335047981 windows-compatible. Thanks to Mike Lagwig 2012-07-10 10:05:14 +01:00
Hieu Hoang
f40f6393f3 A web interface to Moses. Thanks to Liang Tian 2012-07-09 10:27:28 +01:00
Hieu Hoang
388f5a2046 remove gpl code 2012-07-07 14:41:00 +01:00
Hieu Hoang
1737df448a remove gpl code 2012-07-07 14:24:02 +01:00
Hieu Hoang
8d57651d0f move mert-moses-multi.pl 2012-07-06 12:58:06 +01:00
U-Hieu-PC\Hieu
addb1f0645 Merge branch 'master' of https://github.com/hieuhoang/mosesdecoder 2012-07-05 18:30:57 +01:00
U-Hieu-PC\Hieu
81b916a963 visual studio project files 2012-07-05 18:30:06 +01:00
Eva Hasler
f592c222fe revert mode changes 2012-07-04 12:25:21 +01:00
Eva Hasler
027a20730e merge Jamfiles 2012-07-04 11:49:07 +01:00
Hieu Hoang
1088bdcb86 more comments 2012-07-03 18:11:53 +01:00
Rico Sennrich
ea4d0a4785 fix encoding issue 
Error that occurred for phrase tables without phrase alignment:

  File "tmcombine.py", line 377, in write_phrase_table
    line = b"%s ||| %s ||| %s 2.718 %s||| %s ||| %s\n" %(src,target,features,origin_features,alignment,comments)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 26: ordinal not in range(128)  File "tmcombine.py", line 377, in write_phrase_table
 2012-07-01 20:48:12 +02:00
Hieu Hoang
647e320629 commented all classes in OnDiskPt and moses-chart-cmd. Leave ken's lm directory 2012-06-28 20:12:31 -04:00
Hieu Hoang
f2878bcdeb commented all classes in moses lib 2012-06-28 18:29:46 -04:00
Hieu Hoang
ed18b2191d comment classes A-M with doxygen-compatible comments. Lots of @todo 2012-06-26 19:45:02 -04:00
Hieu Hoang
a4f69bb187 xcode for threaded mert 2012-06-26 16:30:17 -04:00
Hieu Hoang
a91120c12a xcode for kbmira 2012-06-24 09:41:05 -04:00
Hieu Hoang
0cb63edcb9 merge Lexi Birch's LRScore from mert_mtm5 branch. Compiles and run. Hack, must double check with barry or lexi 2012-06-23 22:51:48 -04:00
Hieu Hoang
7d19fe13ae merge Lexi Birch's LRScore from mert_mtm5 branch 2012-06-22 18:19:16 +01:00
Hieu Hoang
db06d9cf65 xcode for mert exec 2012-06-21 22:36:36 +01:00
Hieu Hoang
9a99d31a5c Merge branch 'master' of github.com:moses-smt/mosesdecoder 2012-06-20 15:01:15 +01:00
Hieu Hoang
112b4c768d use FindBin() to get directory of moses exec. Get ini file from command line, not hardcoded 2012-06-20 14:20:29 +01:00
Rico Sennrich
e5bec4a48b minor fix for hierarchical sigtest filter 
(rules that end with two nonterminals were all thrown out)
 2012-06-16 08:38:43 +02:00
Hieu Hoang
e72670700a Merge branch 'master' of github.com:moses-smt/mosesdecoder 2012-06-15 22:49:22 +01:00
Rico Sennrich
3549464943 bugfix (hierarchical sigtest filter failed for two consecutive nonterminals) 2012-06-15 18:42:34 +02:00
Rico Sennrich
9a0426543b sigtest-filter for hierarchical rule tables 
-h to enable:
split rules into terminal sequences, look up each in the suffix array,
and treat the intersection of the resulting sentence lists
as occurrences of the rule.

for a speedup, use unordered_set instead of set for SentIdSet
(requires GCC >= 4.3)
 2012-06-15 10:23:52 +02:00
Colin Cherry
07a5c67ebc Merge branch 'master' into miramerge 
Conflicts:
	Jamroot
	misc/queryPhraseTable.cpp
	scripts/training/train-model.perl
 2012-06-14 17:08:16 -04:00
Hieu Hoang
137d351621 xcode 2012-06-13 15:56:53 +01:00
Eva Hasler
e1c1a5343c merge 2012-06-07 11:16:52 +01:00
Eva Hasler
6a6a35c65e fix start weights in experiment.perl, add hypothesis queue for picking hope and fear translations, add variations to 1slack formulation 2012-06-01 01:49:42 +01:00
Rico Sennrich
fcf05e63c1 make translation model combination more general: allow non-default number and position of phrase table features 2012-05-31 14:23:38 +02:00
Colin Cherry
3c44d04baf Merge branch 'master' into miramerge 
Conflicts:
	Jamroot
	mert/FeatureStats.cpp
	moses-cmd/src/IOWrapper.h
	scripts/training/mert-moses.pl
	scripts/training/train-model.perl.missing_bin_dir
 2012-05-30 12:39:53 -04:00
Hieu Hoang
45870348ff xcode build supports threads. move 'using namespace' out from .h file to stop namespace pollution 2012-05-30 12:47:20 +01:00
Hieu Hoang
2b20de8ea9 xcode build supports threads. Abort when using Adam's suffix arrays with threads 2012-05-30 09:49:43 +01:00
Hieu Hoang
ef26388aff eclipse project 2012-05-28 17:29:46 +01:00
Hieu Hoang
7d602aff2f Move queryOnDiskPt from contrib to OnDiskPt folder. Easier to amend Jam file 2012-05-25 13:41:54 +01:00
Hieu Hoang
8844be6f25 Move queryOnDiskPt from contrib to OnDiskPt folder. Easier to amend Jam file 2012-05-25 13:41:06 +01:00
Barry Haddow
c397d2068b Merge branch 'trunk' into miramerge. Still to fix build. 
Conflicts:
	Jamroot
	mert/Data.cpp
	mert/Data.h
	mert/FeatureArray.cpp
	mert/FeatureArray.h
	mert/FeatureData.cpp
	mert/FeatureData.h
	mert/FeatureStats.cpp
	mert/FeatureStats.h
	mert/mert.cpp
	moses-chart-cmd/src/IOWrapper.h
	moses-chart-cmd/src/Main.cpp
	moses-cmd/src/IOWrapper.cpp
	moses-cmd/src/IOWrapper.h
	moses-cmd/src/Main.cpp
	moses/src/GlobalLexicalModel.cpp
	moses/src/Jamfile
	moses/src/Parameter.cpp
	moses/src/PhraseDictionary.cpp
	moses/src/ScoreIndexManager.h
	moses/src/TargetPhrase.h
	regression-testing/tests/phrase.lexicalized-reordering-bin/truth/results.txt
	regression-testing/tests/phrase.lexicalized-reordering-cn/truth/results.txt
	regression-testing/tests/phrase.lexicalized-reordering/truth/results.txt
	regression-testing/tests/phrase.multiple-translation-system-lr/truth/results.txt
	regression-testing/tests/phrase.show-weights.lex-reorder/truth/results.txt
	regression-testing/tests/phrase.show-weights/truth/results.txt
	scripts/ems/experiment.meta
	scripts/ems/experiment.perl
	scripts/training/filter-model-given-input.pl
	scripts/training/mert-moses.pl
 2012-05-24 21:11:35 +01:00
Hieu Hoang
299451235c Add util for querying on-disk pt 2012-05-15 16:46:16 +01:00
Hieu Hoang
f408401ff7 Add util for querying on-disk pt 2012-05-15 16:46:16 +01:00
Hieu Hoang
3f3401f9ba Add util for querying on-disk pt 2012-05-15 16:35:00 +01:00
Hieu Hoang
9467cc82e6 Add util for querying on-disk pt 2012-05-15 16:35:00 +01:00
Hieu Hoang
d25a2baa62 Add util for querying on-disk pt 2012-05-15 16:16:16 +01:00
Hieu Hoang
7ef9b5c635 Add util for querying on-disk pt 2012-05-15 16:16:16 +01:00
Hieu Hoang
d924a709ea Add util for querying on-disk pt 2012-05-15 16:03:40 +01:00
Hieu Hoang
f44e16d1fc Add util for querying on-disk pt 2012-05-15 16:03:40 +01:00
Hieu Hoang
0db6904548 Add util for querying on-disk pt 2012-05-15 15:19:23 +01:00
Hieu Hoang
ac1492e17b Add util for querying on-disk pt 2012-05-15 15:19:23 +01:00
Hieu Hoang
25d269390b Add util for querying on-disk pt 2012-05-15 15:17:59 +01:00
Hieu Hoang
5e889f70f8 Add util for querying on-disk pt 2012-05-15 15:17:59 +01:00
Hieu Hoang
fea7bf9d1a Add util for querying on-disk pt 2012-05-15 15:06:25 +01:00
Hieu Hoang
045f4965ad Add util for querying on-disk pt 2012-05-15 15:06:25 +01:00
Hieu Hoang
b9e203f267 begin adding util for querying on-disk pt. For Daniel Schaut 2012-05-15 14:35:31 +01:00
Hieu Hoang
b8f6958df0 begin adding util for querying on-disk pt. For Daniel Schaut 2012-05-15 14:35:31 +01:00
Eva Hasler
30deedde9f changed permission, everything changed.. 2012-05-10 18:54:24 +01:00
Eva Hasler
93bb01b2e4 some fixes to mosesserver and bleu feature 2012-05-07 05:20:21 +01:00
Eva
f147031576 modify mosesserver to allow weight updates, change BleuFeature compare function, add debug code to Hypothesis.cpp and FeatureVector.cpp, fix OutputPhraseNode in Manager.cpp, add client script for translation with weight updates 2012-05-06 16:04:55 -07:00
Rico Sennrich
b99ebd81cd documentation 2012-04-17 14:03:41 +02:00
Rico Sennrich
d9c07485b4 documentation 2012-04-17 14:03:41 +02:00
Barry Haddow
02ddca96ed Add support for chart-based models to server 2012-04-13 17:27:48 +01:00
Barry Haddow
806a614b6d Add support for chart-based models to server 2012-04-13 17:27:48 +01:00
Hieu Hoang
918bcafb80 xcode 2012-03-15 20:31:27 +07:00
Hieu Hoang
0faca41e11 xcode 2012-03-15 20:31:27 +07:00
Rico Sennrich
7927979298 use specified tmpdir when sorting 2012-02-28 10:05:30 +01:00
Rico Sennrich
bf54a5f38b use specified tmpdir when sorting 2012-02-28 10:05:30 +01:00
Rico Sennrich
e9d960d73e tmpdir option and bugfix to reordering 2012-02-27 14:11:47 +01:00
Rico Sennrich
230b93148c tmpdir option and bugfix to reordering 2012-02-27 14:11:47 +01:00
Hieu Hoang
2d3c2c47c4 compile error when using ORLM and SRI 2012-02-09 21:38:31 +07:00
Hieu Hoang
e85662045d compile error when using ORLM and SRI 2012-02-09 21:38:31 +07:00
Kenneth Heafield
b9455e0744 Compile ORLM. The existing code should to be refactored. 2012-02-08 13:29:16 -05:00
Kenneth Heafield
3c7271220d Compile ORLM. The existing code should to be refactored. 2012-02-08 13:29:16 -05:00
Rico Sennrich
8ee3ed6d64 tmcombine (translation model combination) 2012-01-31 10:50:20 +01:00
Rico Sennrich
1b229137fa tmcombine (translation model combination) 2012-01-31 10:50:20 +01:00
Hieu Hoang
c5b9d80171 xcode 2012-01-26 21:54:33 +07:00
Hieu Hoang
f2e475d48f xcode 2012-01-26 21:54:33 +07:00
Hieu Hoang
79c6528a51 add --sort-batch-size arg to train-model.perl 2012-01-24 18:00:45 +07:00
Hieu Hoang
5ae2a24878 add --sort-batch-size arg to train-model.perl 2012-01-24 18:00:45 +07:00
Hieu Hoang
f5a624fc4a xcode 2012-01-23 23:53:26 +07:00
Hieu Hoang
a47bfea3cd xcode 2012-01-23 23:53:26 +07:00
Hieu Hoang
80b16f0f6a xcode 2012-01-08 23:17:55 +07:00
Hieu Hoang
8e8e42b4c3 xcode 2012-01-08 23:17:55 +07:00
Hieu Hoang
7ade14b16e xcode 2012-01-08 23:17:17 +07:00
Hieu Hoang
7620258c51 xcode 2012-01-08 23:17:17 +07:00
Hieu Hoang
98fc6d9917 xcode 2012-01-08 14:36:54 +07:00
Hieu Hoang
180fa13a27 xcode 2012-01-08 14:36:54 +07:00
Hieu Hoang
7840ca9eef xcode 2012-01-08 14:19:23 +07:00
Hieu Hoang
ebbe0a3aba xcode 2012-01-08 14:19:23 +07:00
Hieu Hoang
81569aeead xcode 2012-01-08 14:17:29 +07:00
Hieu Hoang
007326e6fd xcode 2012-01-08 14:17:29 +07:00
Hieu Hoang
74194d9d90 Sample java client for moses server. By Marwen Azouzi 2012-01-03 22:17:12 +07:00
Hieu Hoang
48e91f5d04 Sample java client for moses server. By Marwen Azouzi 2012-01-03 22:17:12 +07:00
Hieu Hoang
e05eaf1419 visual studio 2011-12-26 17:25:13 +07:00
Hieu Hoang
8b2fe1be96 visual studio 2011-12-26 17:25:13 +07:00
Barry Haddow
072848a4df Merge branch 'master' into bjam. Doesn't compile. 
Conflicts:
	.gitignore
	Makefile.am
	config.h.in
	configure.in
	m4/boost.m4
	moses/src/LM/Factory.cpp
	moses/src/LM/ParallelBackoff.h
	moses/src/LM/RandLM.h
	moses/src/Makefile.am
	moses/src/PDTAimp.h
	moses/src/Phrase.h
	moses/src/PhraseDictionaryTree.cpp
	moses/src/ScoreComponentCollection.h
	moses/src/ScoreIndexManager.cpp
	moses/src/StaticData.cpp
	moses/src/TargetPhrase.cpp
	moses/src/TranslationOptionCollection.cpp
	moses/src/TranslationSystem.cpp
	scripts/Makefile
	scripts/released-files
 2011-12-08 23:17:30 +00:00
Hieu Hoang
7506d73610 minor bjam error for server 2011-12-02 08:50:11 +07:00
Hieu Hoang
8e64e75842 minor bjam error for server 2011-12-02 08:50:11 +07:00
Hieu Hoang
39db162cc0 move vs and xcode project files into separate contrib folder 2011-12-02 08:33:39 +07:00
Hieu Hoang
71c78ef0ce move vs and xcode project files into separate contrib folder 2011-12-02 08:33:39 +07:00
Kenneth Heafield
c53453b266 No more autotools (with bhaddow in the room) 2011-12-01 17:17:28 +00:00
Kenneth Heafield
42975b0635 No more autotools (with bhaddow in the room) 2011-12-01 17:17:28 +00:00
Kenneth Heafield
00805d5d6f Attempt to fix xmlrpc-c detection 2011-11-30 12:51:24 +00:00
Kenneth Heafield
7eedf62407 Attempt to fix xmlrpc-c detection 2011-11-30 12:51:24 +00:00
Kenneth Heafield
7451269923 More error checking for moses server 2011-11-30 08:53:13 +00:00
Kenneth Heafield
d4099a7455 More error checking for moses server 2011-11-30 08:53:13 +00:00
Kenneth Heafield
02b330588c shell-prefix was meant to be empty-string, not null-set 2011-11-28 13:41:21 -05:00
Kenneth Heafield
3d5a41baaf shell-prefix was meant to be empty-string, not null-set 2011-11-28 13:41:21 -05:00
Kenneth Heafield
d18f76098d Mike Ladwig complained about xmlrpc-c. 2011-11-28 18:24:23 +00:00
Kenneth Heafield
10d4e651cd Mike Ladwig complained about xmlrpc-c. 2011-11-28 18:24:23 +00:00
Kenneth Heafield
95fc3d6e60 Some people have xmlrpc without abyss so test for that 2011-11-26 11:57:09 +00:00
Kenneth Heafield
aadd7dee68 Some people have xmlrpc without abyss so test for that 2011-11-26 11:57:09 +00:00
Barry Haddow
21f3f19aaa move checkplf 2011-11-23 15:48:48 +00:00
Barry Haddow
9c952b586f move checkplf 2011-11-23 15:48:48 +00:00
Barry Haddow
ea49a01a80 Move from scripts to contrib 2011-11-23 15:31:16 +00:00
Barry Haddow
66c29e01c4 Move from scripts to contrib 2011-11-23 15:31:16 +00:00
Barry Haddow
0b57ba12f5 Fix server build 2011-11-23 15:22:45 +00:00
Barry Haddow
791831b0ce Fix server build 2011-11-23 15:22:45 +00:00
Barry Haddow
7b472960b1 Move projects from top level to contrib 
server build probably broken
 2011-11-23 14:48:50 +00:00
Barry Haddow
5e2731619c Move projects from top level to contrib 
server build probably broken
 2011-11-23 14:48:50 +00:00