moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2024-08-17 23:40:50 +03:00
Code Issues Packages Projects Releases Wiki Activity
17,477 Commits 136 Branches 26 Tags 147 MiB
master
Commit Graph

602 Commits

Author SHA1 Message Date
Barry Haddow
c6e36f4dd7 Fix compile error 2013-04-12 16:29:52 +01:00
Hieu Hoang
44a0e52e30 fixed ShowWeights() for confusion networks. This is a reason why we should get rid of ShortNames and move to refactored moses pdq 2013-04-09 14:44:32 +01:00
Hieu Hoang
72aa0a80bd Merge github.com:moses-smt/mosesdecoder into weight-new 2013-03-25 13:51:47 +00:00
Joan Puigcerver
4a954f2be7 Fixes compilation error using libboost_1_50 
Problem spotted in https://github.com/moses-smt/mosesdecoder/issues/32 
fixed.

According to the Boost doc, nbestPath.parent_path().filename() 
returns a path object, to get the correspondent std::string 
representation, one must call one of the methods listed in:
http://www.boost.org/doc/libs/1_53_0/libs/filesystem/doc/reference.html#path-native-format-observers

native() is supposed to return the path in the specific OS
path format (using backslashes for Windows). Anyway, since we 
are considering only the filename here, the result is the same.
 2013-03-24 20:01:59 +01:00
Hieu Hoang
fd4e954322 merge 2013-03-24 09:57:36 +00:00
Hieu Hoang
db005f6503 compile error caused by different versions of boost 2013-03-23 15:58:07 +00:00
Lane Schwartz
3a4e63c558 Ensure directory exists for outputting hypergraphs 2013-03-22 12:14:28 -04:00
Lane Schwartz
b2bba0bae3 Work on compression and defaults for outputting hypergraphs. 2013-03-21 16:48:47 -04:00
Lane Schwartz
28c980d58e Allow hypergraph output to be in plain text, gzip, or bzip2. 
The output-search-graph-hypergraph flag now takes two params:

* The first param must be "none", "gzip", or "bzip2"
* The second param is the hypergraph directory,
  which must already exist
 2013-03-21 15:19:31 -04:00
Lane Schwartz
22c77f7331 Work on decreasing memory requirement for outputting hypergraph 2013-03-21 12:17:49 -04:00
Hieu Hoang
7888c7ba23 eclipse project files 2013-03-15 22:02:00 +00:00
Hieu Hoang
2d252d2dd0 memory leak 2013-03-15 16:11:15 +00:00
Hieu Hoang
3da09b921c memory leak 2013-03-15 12:30:39 +00:00
Hieu Hoang
cedb41be2a change : back to = 2013-03-14 19:06:01 +00:00
Hieu Hoang
ca43a88bd2 change feature weight delimiter from = to : 2013-03-13 18:26:24 +00:00
Hieu Hoang
2f78fe5fe5 Merge github.com:moses-smt/mosesdecoder into weight-new 2013-03-13 17:54:03 +00:00
Hieu Hoang
21c51194fa add -print-alignment-info 2013-03-13 12:12:33 +00:00
Hieu Hoang
35bffae402 Merge github.com:moses-smt/mosesdecoder into weight-new 2013-03-04 18:19:28 +00:00
Lane Schwartz
ec69acf3d4 Don't open all hypergraph output files at once. 2013-03-04 12:07:37 -05:00
Lane Schwartz
8573a66da0 Close hypergraph output files. 2013-03-04 09:15:46 -05:00
Hieu Hoang
bcecfe55db merge with Lane's search graph output 2013-02-24 00:31:29 +00:00
Hieu Hoang
30e0d1e0fa Merge github.com:moses-smt/mosesdecoder into weight-new 2013-02-22 22:39:24 +00:00
Lane Schwartz
4adeb7e338 Output feature weights to a separate file when producing hypergraph 2013-02-22 16:28:48 -05:00
Lane Schwartz
04f107fbb0 Add flag to output search graph in Kenneth's hypergraph format. 2013-02-22 16:28:48 -05:00
Lane Schwartz
774ed64f2e Work to allow output search graph in HTK standard lattice format 2013-02-22 16:28:47 -05:00
Hieu Hoang
01af82310b delete ScoreProducer 2013-02-21 20:03:35 +00:00
Hieu Hoang
ebd7b85aa6 Merge github.com:moses-smt/mosesdecoder into weight-new 2013-02-20 10:11:21 +00:00
Lane Schwartz
2e6b5e7a23 Merge branch 'master' of www:/repos/git/Decoders/mosesdecoder 2013-02-14 12:03:35 -05:00
Lane Schwartz
f9a46adc31 Specify -start-translation-id to tell Moses the line number of the first sentence 2013-02-13 15:52:40 -05:00
Hieu Hoang
f00d781c37 merge with master 2013-02-13 14:17:20 +00:00
Hieu Hoang
1a8cd20b51 -print-id for christian buck and phi 2013-02-11 18:01:33 +00:00
Hieu Hoang
6885e04b42 move feature function variables into FeatureFunction.h 2012-12-31 00:57:21 +00:00
Hieu Hoang
efbc8bb6c4 move language model from TranslationSystem back to StaticData 2012-12-20 16:38:15 +00:00
Hieu Hoang
c2d9d84ba0 move feature function variables from TranslationSystem back to StaticData 2012-12-19 14:37:00 +00:00
Hieu Hoang
667b8a495f decoder -show-weight to output consistent new format 2012-12-17 17:17:44 +00:00
hieu
05045d574c don't display unknown weight penalty when showing weight, don't usually tune. Also, change delimiter in mert extractor from : to = 2012-12-16 18:29:53 +00:00
hieu
57b33f40cf don't display unknown weight penalty when showing weight. Don't usually tune 2012-12-15 19:20:07 +00:00
hieu
5ab3c3f081 output weight formatted according to discussions with Ken & Barry 2012-12-12 12:22:13 +00:00
Hieu Hoang
d1fcb37b06 make output of feature scores the same in moses-cmd as in moses-chart-cmd 2012-12-07 13:34:44 +00:00
Hieu Hoang
3d6d53bf49 delete hardcoded if() statements to show scores in n-best list. Excluded UnknownWordPenalty and made sure PhraseModel & Generation is in particular order 2012-12-06 17:28:56 +00:00
Hieu Hoang
08ca44b34e delete hardcoded if() statements for show weights. Excluded UnknownWordPenalty and made sure PhraseModel & Generation is in particular order 2012-12-06 17:13:00 +00:00
Hieu Hoang
28b70a5697 delete hardcoded if() statements for show weights. Excluded UnknownWordPenalty and made sure PhraseModel & Generation is in particular order 2012-12-06 16:59:54 +00:00
Hieu Hoang
4f3805a0d7 change \!UnknownWordPenalty to UnknownWordPenalty 2012-12-05 11:58:21 +00:00
Hieu Hoang
cb270a76be all regression tests passed 2012-12-04 18:39:06 +00:00
Hieu Hoang
08b508d686 indentation 2012-12-04 17:54:39 +00:00
Hieu Hoang
9fe742ce52 get rid function GetScoreProducerWeightShortName(). Fails 1 regression test 2012-12-04 17:09:23 +00:00
Hieu Hoang
33105a7ba7 get rid of int argument from GetScoreProducerWeightShortName() 2012-12-04 13:08:00 +00:00
Hieu Hoang
623289465c change argument -include-alignment-in-n-best to -include-segmentation-in-n-best. Avoids confusion with word alignment 2012-11-14 16:57:33 +00:00
Hieu Hoang
f96b33de83 only include moses root when compiling 2012-11-14 13:43:04 +00:00
Hieu Hoang
0bc0b85358 resolved merge conflicts 2012-11-12 21:12:01 +00:00
Hieu Hoang
125fdbcff6 move moses-cmd/src/* to moses-cmd/ 2012-11-12 20:42:29 +00:00
Hieu Hoang
44f24bd673 resolved merge conflicts 2012-11-12 20:35:32 +00:00
Hieu Hoang
6b354f1e90 move moses/src/* to moses/ 2012-11-12 20:21:32 +00:00
Hieu Hoang
5e3ef23cef move moses/src/* to moses/ 2012-11-12 19:56:18 +00:00
Kenneth Heafield
1bae8e1701 Why was mira pulling in IOWrapper? 2012-11-12 18:27:38 +00:00
Kenneth Heafield
b9b2874abc There's no need for the _lib suffix on IOWrapper. Also, why does this library exist? 2012-11-12 18:13:29 +00:00
Barry Haddow
0d26c2f521 Sync showweights order with nbest 2012-11-09 12:11:49 +00:00
Hieu Hoang
88b2f05e03 process empty lines regardless 2012-11-09 01:26:21 -08:00
Hieu Hoang
397e704b49 terminal and non-terminal word alignments stored in 2 separate objects 2012-10-19 15:10:10 +01:00
Barry Haddow
848aafb644 Merge remote branch 'github/master' into miramerge 
Conflicts:
	moses/src/AlignmentInfo.cpp
	moses/src/AlignmentInfo.h
	moses/src/ChartHypothesis.cpp
	moses/src/ChartTrellisNode.cpp
	moses/src/LM/Implementation.cpp
	moses/src/LM/Ken.cpp
	moses/src/TargetPhrase.cpp
	moses/src/TargetPhrase.h
 2012-10-08 17:54:59 +01:00
Kenneth Heafield
483e6b666b Print usage information before exit 2012-10-05 17:49:52 +01:00
Barry Haddow
0a950ee9f4 Merge remote branch 'github/master' into miramerge 
Compiles, but not tested. Had to disable relent filter. Strangely, it seems to contain the
whole of moses-cmd.

Conflicts:
	Jamroot
	OnDiskPt/TargetPhrase.cpp
	moses-cmd/src/Main.cpp
	moses/src/AlignmentInfo.cpp
	moses/src/AlignmentInfo.h
	moses/src/ChartTranslationOptionCollection.cpp
	moses/src/ChartTranslationOptionCollection.h
	moses/src/GenerationDictionary.cpp
	moses/src/Jamfile
	moses/src/Parameter.cpp
	moses/src/PhraseDictionary.cpp
	moses/src/StaticData.cpp
	moses/src/StaticData.h
	moses/src/TargetPhrase.h
	moses/src/TranslationSystem.cpp
	moses/src/TranslationSystem.h
	moses/src/Word.cpp
	phrase-extract/score.cpp
	regression-testing/Jamfile
	scripts/ems/experiment.meta
	scripts/ems/experiment.perl
	scripts/training/train-model.perl
 2012-09-26 22:49:33 +01:00
Barry Haddow
1ce788e2b8 Fixes for regression tests 2012-09-26 14:40:06 +01:00
Kenneth Heafield
15b0ae1aef Single-threaded compilation fixes 2012-09-21 18:34:48 -04:00
Barry Haddow
bfae09725c Add option to dump OOVs to file 2012-09-21 08:55:37 +01:00
Barry Haddow
54433bb221 Take notice of calc in t-option flag 2012-09-04 17:50:30 +01:00
Lane Schwartz
357845c1a4 Removed duplicate usage printout when moses is called with no parameters. 2012-09-04 17:32:55 +01:00
Jonathan Clark
6f008096ea Use Moses' Timer class, which is based on wall-time instead of the C clock() function, which measures clock ticks used within the *process* rather than each thread. This makes timings more meaningful in multi-threaded mode. However, these timings will still be inaccurate on Windows (second-level only). 2012-08-10 15:32:00 -04:00
Hieu Hoang
97f46537b7 wrap phi's suffix array implementation and extraction method in a wrapper class. Compiles 2012-07-30 21:07:19 +01:00
Barry Haddow
2b4e61d826 Merge branch 'trunk' into miramerge 
Compiles, not tested.

Conflicts:
	Jamroot
	OnDiskPt/PhraseNode.h
	OnDiskPt/TargetPhrase.cpp
	OnDiskPt/TargetPhrase.h
	OnDiskPt/TargetPhraseCollection.cpp
	mert/BleuScorer.cpp
	mert/Data.cpp
	mert/FeatureData.cpp
	moses-chart-cmd/src/Main.cpp
	moses/src/AlignmentInfo.h
	moses/src/ChartManager.cpp
	moses/src/LM/Ken.cpp
	moses/src/LM/Ken.h
	moses/src/LMList.h
	moses/src/LexicalReordering.h
	moses/src/PhraseDictionaryTree.h
	moses/src/ScoreIndexManager.h
	moses/src/StaticData.h
	moses/src/TargetPhrase.h
	moses/src/Word.cpp
	scripts/ems/experiment.meta
	scripts/ems/experiment.perl
	scripts/training/train-model.perl
 2012-07-17 13:36:50 +01:00
Eva Hasler
b00e4b3115 move deps into IOWrapper_lib 2012-07-03 16:22:59 +01:00
Hieu Hoang
75e038f4cf create namespace for all classes 2012-07-02 17:05:11 +01:00
Hieu Hoang
121e258e84 namespace all classes in mert directory 2012-06-30 21:39:10 +01:00
Hieu Hoang
b1ce27d01f commented all classes in OnDiskPt and moses-cmd 2012-06-28 23:19:28 -04:00
Eva Hasler
e1c1a5343c merge 2012-06-07 11:16:52 +01:00
Eva Hasler
6a6a35c65e fix start weights in experiment.perl, add hypothesis queue for picking hope and fear translations, add variations to 1slack formulation 2012-06-01 01:49:42 +01:00
Hieu Hoang
465c5cbf97 move all executables into bin 2012-05-31 12:55:05 +01:00
Colin Cherry
3c44d04baf Merge branch 'master' into miramerge 
Conflicts:
	Jamroot
	mert/FeatureStats.cpp
	moses-cmd/src/IOWrapper.h
	scripts/training/mert-moses.pl
	scripts/training/train-model.perl.missing_bin_dir
 2012-05-30 12:39:53 -04:00
Hieu Hoang
45870348ff xcode build supports threads. move 'using namespace' out from .h file to stop namespace pollution 2012-05-30 12:47:20 +01:00
Hieu Hoang
ef26388aff eclipse project 2012-05-28 17:29:46 +01:00
Barry Haddow
c397d2068b Merge branch 'trunk' into miramerge. Still to fix build. 
Conflicts:
	Jamroot
	mert/Data.cpp
	mert/Data.h
	mert/FeatureArray.cpp
	mert/FeatureArray.h
	mert/FeatureData.cpp
	mert/FeatureData.h
	mert/FeatureStats.cpp
	mert/FeatureStats.h
	mert/mert.cpp
	moses-chart-cmd/src/IOWrapper.h
	moses-chart-cmd/src/Main.cpp
	moses-cmd/src/IOWrapper.cpp
	moses-cmd/src/IOWrapper.h
	moses-cmd/src/Main.cpp
	moses/src/GlobalLexicalModel.cpp
	moses/src/Jamfile
	moses/src/Parameter.cpp
	moses/src/PhraseDictionary.cpp
	moses/src/ScoreIndexManager.h
	moses/src/TargetPhrase.h
	regression-testing/tests/phrase.lexicalized-reordering-bin/truth/results.txt
	regression-testing/tests/phrase.lexicalized-reordering-cn/truth/results.txt
	regression-testing/tests/phrase.lexicalized-reordering/truth/results.txt
	regression-testing/tests/phrase.multiple-translation-system-lr/truth/results.txt
	regression-testing/tests/phrase.show-weights.lex-reorder/truth/results.txt
	regression-testing/tests/phrase.show-weights/truth/results.txt
	scripts/ems/experiment.meta
	scripts/ems/experiment.perl
	scripts/training/filter-model-given-input.pl
	scripts/training/mert-moses.pl
 2012-05-24 21:11:35 +01:00
Hieu Hoang
4888e114ed Check factor is not null before outputting 2012-05-17 16:25:53 +01:00
Hieu Hoang
22ca90bd8a Check factor is not null before outputting 2012-05-17 16:25:53 +01:00
Eva Hasler
30deedde9f changed permission, everything changed.. 2012-05-10 18:54:24 +01:00
Jonathan Clark
19328231ff Add phrase segmentation information to n-best list 2012-04-18 17:09:02 -04:00
Jonathan Clark
bb36565e78 Add phrase segmentation information to n-best list 2012-04-18 17:09:02 -04:00
Eva Hasler
56f1101dfc ignore empty input lines 2012-04-07 21:03:30 +01:00
Lane Schwartz
1c455a2877 Fixed memory leak for single-threaded mode. 2012-03-30 15:25:42 -04:00
Lane Schwartz
8a03fd63ba Fixed memory leak for single-threaded mode. 2012-03-30 15:25:42 -04:00
Eva Hasler
03d3cb6a7a adapt Jamfiles, don't output bleu score feature in nbest 2012-03-29 20:58:53 +01:00
Eva Hasler
f5a4d61fd4 resolving conflicts 2012-03-27 14:24:49 +01:00
Eva Hasler
c64b8bc37f revert changes to nbest output for sparse features 2012-03-20 13:13:30 +00:00
Eva Hasler
da538961d4 nbest output for sparse features: report non-zero weights instead of scores 2012-03-11 16:26:25 +00:00
Eva Hasler
cb4a0d36bf change output format for sparse features 2012-03-08 14:58:12 +00:00
Barry Haddow
5a17ef82b3 Merge remote branch 'github/miramerge' into bjam 
Conflicts:
	mira/Decoder.h
	mira/Main.cpp
	mira/Main.h
	moses-chart-cmd/src/IOWrapper.cpp
	moses-chart-cmd/src/Main.cpp
	moses-cmd/src/Main.cpp
	moses/src/BleuScoreFeature.cpp
	moses/src/BleuScoreFeature.h
	moses/src/TargetNgramFeature.h
 2012-02-01 10:12:16 +00:00
Michal Hrusecky
ade1aa4acc Use make_pair without type 
Replacing typed make_pair with non-typed version to fix compilation
issues.
 2012-01-31 17:00:01 +01:00
Michal Hrusecky
c4397c1491 Use make_pair without type 
Replacing typed make_pair with non-typed version to fix compilation
issues.
 2012-01-31 17:00:01 +01:00
Barry Haddow
1e10bb7ef7 Merge from miramerge. 
Reverted ChartHypothesis as it breaks chart-decoding.
 2012-01-20 15:35:55 +00:00
Eva Hasler
b704381a42 restore order of scores returned by decoder and in nbestlist 2012-01-18 12:26:51 +00:00
Kenneth Heafield
06648fd832 Surround mains in try-catch blocks. 2012-01-13 10:20:42 -05:00
Kenneth Heafield
868255af03 Surround mains in try-catch blocks. 2012-01-13 10:20:42 -05:00
Eva Hasler
50d3a62bda change order of -show-weights output to match nbest list order 2011-12-14 10:51:35 +00:00
Eva Hasler
85ce6547fd -show-weights: output sparseProducerWeight if != 1, otherwise 'sparse' 2011-12-13 19:13:13 +00:00
Barry Haddow
072848a4df Merge branch 'master' into bjam. Doesn't compile. 
Conflicts:
	.gitignore
	Makefile.am
	config.h.in
	configure.in
	m4/boost.m4
	moses/src/LM/Factory.cpp
	moses/src/LM/ParallelBackoff.h
	moses/src/LM/RandLM.h
	moses/src/Makefile.am
	moses/src/PDTAimp.h
	moses/src/Phrase.h
	moses/src/PhraseDictionaryTree.cpp
	moses/src/ScoreComponentCollection.h
	moses/src/ScoreIndexManager.cpp
	moses/src/StaticData.cpp
	moses/src/TargetPhrase.cpp
	moses/src/TranslationOptionCollection.cpp
	moses/src/TranslationSystem.cpp
	scripts/Makefile
	scripts/released-files
 2011-12-08 23:17:30 +00:00
Hieu Hoang
71c78ef0ce move vs and xcode project files into separate contrib folder 2011-12-02 08:33:39 +07:00
Hieu Hoang
39db162cc0 move vs and xcode project files into separate contrib folder 2011-12-02 08:33:39 +07:00
Kenneth Heafield
9fe831c60c Merge CreateOnDisk into OnDiskPt, pop out of src 2011-12-01 18:26:05 +00:00
Kenneth Heafield
a0d79781e5 Merge CreateOnDisk into OnDiskPt, pop out of src 2011-12-01 18:26:05 +00:00
Kenneth Heafield
42975b0635 No more autotools (with bhaddow in the room) 2011-12-01 17:17:28 +00:00
Kenneth Heafield
c53453b266 No more autotools (with bhaddow in the room) 2011-12-01 17:17:28 +00:00
Eva Hasler
9084db96c4 add parameter weight-dlm as additional dense weight 2011-11-24 19:27:12 +00:00
Barry Haddow
9c952b586f move checkplf 2011-11-23 15:48:48 +00:00
Barry Haddow
21f3f19aaa move checkplf 2011-11-23 15:48:48 +00:00
Kenneth Heafield
a65ed7e587 Jam: install executables in legacy paths and stop leaving a.out files everywhere 2011-11-21 13:20:52 -05:00
Barry Haddow
7fd0e6ca7f Remove GetTargetPhrase() (already have GetCurrTargetPhrase()) 2011-11-21 12:35:58 +00:00
Barry Haddow
5baa595a91 Further direction removal 2011-11-21 11:14:05 +00:00
Barry Haddow
3dc30fd51a Remove direction from Phrase 2011-11-21 10:49:26 +00:00
Hieu Hoang
ce0065ff2a Merge branch 'master' of github.com:moses-smt/mosesdecoder 2011-11-18 22:13:46 +07:00
Hieu Hoang
ce773fc65a xcode 2011-11-18 22:07:18 +07:00
Kenneth Heafield
5c9b28c55a Introducing Jamfiles 
Currently supports kenlm, irstlm, and srilm options.

bjam [--with-srilm=/path/to/sri] [--with-irstlm=/path/to/irst] -j8
Places binaries in dist/ that pass regression tests.
 2011-11-18 14:37:01 +00:00
Hieu Hoang
caac64a33b xcode 2011-11-18 21:33:58 +07:00
Kenneth Heafield
024b411e85 One more assert 2011-11-18 13:41:24 +00:00
Kenneth Heafield
bf78f7a1ac Replace assert with CHECK until people learn how to use assert properly 2011-11-18 12:07:41 +00:00
Kenneth Heafield
404c4b3a73 Why are these executable? 2011-11-17 18:59:29 -05:00
Barry Haddow
d57d0c0656 Merge branch 'master' into miramerge 
Conflicts:
	Makefile.am
	config.h.in
	configure.in
	kenlm
	moses-chart-cmd/src/IOWrapper.cpp
	moses/src/Parameter.cpp
	scripts/Makefile
 2011-11-17 17:46:17 +00:00
Kenneth Heafield
72a4c8a0d3 Move kenlm up one level, simplify compilation 2011-11-17 12:49:55 +00:00
Hieu Hoang
253f32a2e5 load hiero format for reading suffix arrays 2011-11-14 00:14:40 +07:00
Hieu Hoang
d97cf2d9a5 Merge branch 'master' of github.com:moses-smt/mosesdecoder 2011-11-11 22:25:54 +00:00
Hieu Hoang
87500bc93d visual studio 2011-11-11 22:24:25 +00:00
Hieu Hoang
7da1af411c xcode build 2011-11-11 20:55:55 +07:00
Barry Haddow
6890472ba1 Merge branch 'master' into miramerge 
Conflicts:
	.gitignore
	config.h.in
	moses/src/ChartTrellisPath.cpp
	moses/src/Makefile.am
 2011-11-11 08:25:13 +00:00
Hieu Hoang
aac12ed58f xcode 2011-10-30 12:51:08 +07:00
Hieu Hoang
aff5978be4 Revert "xcode" 
This reverts commit c6b333a832.
 2011-10-30 12:37:51 +07:00
Hieu Hoang
c6b333a832 xcode 2011-10-29 20:00:45 +07:00
Barry Haddow
90820ad0c5 Merge branch 'master' into miramerge. 
Also fix mert-moses.pl to use correct flag for specifying
weights of non-core features.

Conflicts:
	.gitignore
	configure.in
	ltmain.sh
	moses/src/LM/Factory.cpp
	moses/src/LMList.cpp
	moses/src/LMList.h
	moses/src/LanguageModel.cpp
	moses/src/LanguageModel.h
	moses/src/LanguageModelKen.h
	moses/src/Makefile.am
	moses/src/Manager.cpp
	moses/src/PhraseDictionaryMemory.cpp
	moses/src/PhraseDictionaryTree.cpp
	moses/src/StaticData.cpp
	moses/src/TargetPhrase.h
 2011-10-28 15:54:23 +01:00
Hieu Hoang
a93f4691f6 win32 2011-10-23 09:37:47 +07:00
hieuhoang1972
897fe0f88b visual studio 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4356 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-14 10:50:08 +00:00
Barry Haddow
c83166087e Merge branch 'master' into miramerge 
Conflicts:
	moses/src/LanguageModel.cpp
	moses/src/TargetPhrase.h
	moses/src/TrellisPath.h
	moses/src/Util.h
	scripts/training/train-model.perl
 2011-10-12 17:14:23 +01:00
bhaddow
84d73700af Implementation of Lattice sampling (Chatterjee and Cancedda, emnlp 2010) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4295 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-04 15:46:24 +00:00
nicolabertoldi
23d9a9b55e normalization of output spaces before and after field separator 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4293 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-03 16:11:39 +00:00
hieuhoang1972
9762a65bd8 xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4264 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-25 10:44:47 +00:00
heafield
19879a8b1e -threads all option 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4252 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-22 22:29:56 +00:00
bhaddow
5b7c5ebdb5 Merge branch 'master' into miramerge 
Conflicts:
	moses-chart-cmd/src/Main.cpp
	moses-cmd/src/IOWrapper.cpp
	moses-cmd/src/Main.cpp
	moses/src/DummyScoreProducers.cpp
	moses/src/DummyScoreProducers.h
	moses/src/GenerationDictionary.cpp
	moses/src/GenerationDictionary.h
	moses/src/GlobalLexicalModel.h
	moses/src/LMList.h
	moses/src/LanguageModel.cpp
	moses/src/LanguageModel.h
	moses/src/LanguageModelImplementation.h
	moses/src/LanguageModelKen.h
	moses/src/LanguageModelMultiFactor.cpp
	moses/src/LanguageModelMultiFactor.h
	moses/src/LanguageModelSingleFactor.cpp
	moses/src/LanguageModelSingleFactor.h
	moses/src/LexicalReordering.h
	moses/src/PhraseDictionary.cpp
	moses/src/PhraseDictionary.h
	moses/src/ScoreIndexManager.cpp
	moses/src/ScoreProducer.h
	moses/src/StaticData.cpp
	moses/src/StaticData.h
	moses/src/TranslationSystem.cpp
 2011-09-20 11:23:38 +01:00
arianna_b
633b80f515 recovered missing blank in n-best-list output 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4170 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-02 11:40:32 +00:00
nicolabertoldi
75edc2eddd change to print the corrett name of the features with InputScores 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4168 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-30 12:25:50 +00:00
hieuhoang1972
5449839d75 option to sort word alignment info, as suggested by arianna bisazza 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4164 1f5c12ca-751b-0410-a591-d2e778427230
 2011-08-26 02:37:52 +00:00
bhaddow
1f3c11b441 Merge branch 'master' into miramerge 
Passes all regression except for the score tests, as I still have to merge
in the missing files for these tests.

Conflicts:
	.gitignore
	Makefile.am
	configure.in
	moses-chart/moses-chart.xcodeproj/project.pbxproj
	moses/src/ChartHypothesis.cpp
	moses/src/ChartTranslationOptionCollection.cpp
	moses/src/ChartTrellisPath.cpp
	moses/src/DotChart.cpp
	moses/src/LMList.h
	moses/src/LanguageModel.cpp
	moses/src/LanguageModel.h
	moses/src/LanguageModelFactory.cpp
	moses/src/LanguageModelRemote.h
	moses/src/LexicalReordering.h
	moses/src/Makefile.am
	moses/src/Manager.cpp
	moses/src/Parameter.cpp
	moses/src/RuleCubeQueue.h
	moses/src/ScoreIndexManager.cpp
	moses/src/StaticData.cpp
	moses/src/StaticData.h
	moses/src/TargetPhrase.cpp
	moses/src/TargetPhrase.h
	moses/src/TranslationOptionCollection.cpp
 2011-08-22 14:52:02 +01:00
bhaddow
de26a207d6 Merge commit '67dd80fb7bf71f36f2b29b4ba7a39e3ad35e5f81' into miramerge 
Conflicts:
	moses-cmd/src/IOWrapper.cpp
	moses-cmd/src/IOWrapper.h
	moses-cmd/src/Main.cpp
	moses-cmd/src/TranslationAnalysis.cpp
 2011-08-19 14:55:32 +01:00
bhaddow
cf81f9188f Merged up to 3895 from trunk 
git-svn-id: http://svn.statmt.org/repository/mira@3934 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:38 +02:00
pkoehn
42e10d010b added reporting of sparse features in n-best list 
git-svn-id: http://svn.statmt.org/repository/mira@3926 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:38 +02:00
bhaddow
e664b4a4b3 Merge 3791-3842 from trunk 
git-svn-id: http://svn.statmt.org/repository/mira@3873 cc96ff50-19ce-11e0-b349-13d7f0bd23df
 2011-08-18 12:59:36 +02:00
hieuhoang1972
b0ec298ce2 vs.net build 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4088 1f5c12ca-751b-0410-a591-d2e778427230
 2011-07-23 23:52:34 +00:00
hieuhoang1972
9c0d725cde visual studio 2010 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4079 1f5c12ca-751b-0410-a591-d2e778427230
 2011-07-19 03:07:15 +00:00
naditomeh
83d02c11c6 Fix a segmentation fault produced when both 'constraint' and 
'translation-details' are used, and no hypothesis that matches the
reference translation is found.
The problem resulted from a missing check if a vector is empty before
incrementing its iterator.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3950 1f5c12ca-751b-0410-a591-d2e778427230
 2011-04-17 19:46:46 +00:00
phkoehn
4ee7e5f673 added some comments 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3911 1f5c12ca-751b-0410-a591-d2e778427230
 2011-03-02 19:02:07 +00:00
pjwilliams
75709a6c87 Memory efficiency: make the reserveSize argument non-optional in 
Moses::Phrase's constructor.  The default used to be ARRAY_SIZE_INCR = 10,
which will be excessive in many cases.  Where the default was used, I've
set the exact size where that was obvious and explicitly used ARRAY_SIZE_INCR 
otherwise.  If you know the code involved, it's probably worth reviewing.



git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3908 1f5c12ca-751b-0410-a591-d2e778427230
 2011-02-28 11:41:08 +00:00
hieuhoang1972
67dd80fb7b run beautify.perl. Consistent formatting for .h & .cpp files 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3897 1f5c12ca-751b-0410-a591-d2e778427230
 2011-02-24 12:39:29 +00:00
phkoehn
4c11bcd617 extensions to phrase table scoring options 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3893 1f5c12ca-751b-0410-a591-d2e778427230
 2011-02-23 10:27:54 +00:00
bhaddow
6b8415bffb Write alignment info through OutputCollector so it gets ordered 
correctly when run with multiple threads.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3882 1f5c12ca-751b-0410-a591-d2e778427230
 2011-02-16 16:50:55 +00:00
ales-t
83e2406f42 Word alignment output also works with MBR decoding. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3875 1f5c12ca-751b-0410-a591-d2e778427230
 2011-02-08 17:15:50 +00:00
bojar
0bc0ece594 Ales Tamchyna's printing of alignments (-print-alignment-info did nothing) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3867 1f5c12ca-751b-0410-a591-d2e778427230
 2011-02-03 09:08:42 +00:00
pjwilliams
8051c5ad35 Use TranslationTask objects to perform sentence decoding in moses-chart. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3846 1f5c12ca-751b-0410-a591-d2e778427230
 2011-01-25 17:15:30 +00:00
pjwilliams
99bbfe938b Use OutputCollector to write moses-chart output. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3843 1f5c12ca-751b-0410-a591-d2e778427230
 2011-01-25 15:17:17 +00:00
bhaddow
a2bde7a16e Make sure internal libraries and paths go before boost. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3800 1f5c12ca-751b-0410-a591-d2e778427230
 2011-01-15 22:41:24 +00:00
bhaddow
a9cd71628a Change of boost macros - please make sure you favourite configuration still works 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3799 1f5c12ca-751b-0410-a591-d2e778427230
 2011-01-13 23:38:48 +00:00
heafield
a596b48971 Fix --enable-shared compilation. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3796 1f5c12ca-751b-0410-a591-d2e778427230
 2011-01-11 19:32:59 +00:00
bhaddow
a2730c445d Merge up to 3791 from trunk. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3792 1f5c12ca-751b-0410-a591-d2e778427230
 2011-01-05 13:49:44 +00:00
hieuhoang1972
41c5b3a1c2 xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3769 1f5c12ca-751b-0410-a591-d2e778427230
 2010-12-10 13:04:06 +00:00
bhaddow
50f0e6c07d Add a show-weights option. It prints out the moses features and exits. May 
load tables as a side-effect.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3744 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-29 16:44:28 +00:00
xandfraser
c0c617a8c4 Added print word alignment in nbest 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3726 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-22 15:34:53 +00:00
hieuhoang1972
735d5b682f xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3673 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-27 18:54:50 +00:00
hieuhoang1972
34e7c43114 xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3664 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-27 03:14:11 +00:00
hieuhoang1972
5f6baa9021 xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3627 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-18 05:47:07 +00:00
bhaddow
7e72ceea22 Goodbye ScoreIndexManager. 
Compiles ok, but haven't dared to run regression yet.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mira-mtm5@3608 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-06 22:06:49 +00:00
hieuhoang1972
e504b797b2 xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3599 1f5c12ca-751b-0410-a591-d2e778427230
 2010-10-01 00:21:27 +00:00
hieuhoang1972
559a5e3ece ken lm integration 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3570 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-26 22:29:01 +00:00
hieuhoang1972
bdfa2beab8 delete eclipse build 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3422 1f5c12ca-751b-0410-a591-d2e778427230
 2010-09-10 09:16:50 +00:00
redpony
570183461c plf checker 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3407 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-16 01:03:39 +00:00
bhaddow
904133fcb7 Merge in the multiple models branch. These changes allow the moses server 
to support multiple translation, language and generation models within the
same process. The main design change is the introduction of a TranslationSystem
object to manage the models, which have been moved out of StaticData.
The changes should have no effect on existing systems.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3394 1f5c12ca-751b-0410-a591-d2e778427230
 2010-08-10 13:12:00 +00:00
hieuhoang1972
31930eb6fc alignment info in pt 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3358 1f5c12ca-751b-0410-a591-d2e778427230
 2010-07-17 22:29:06 +00:00
hieuhoang1972
65194ba505 drop unknown option for chart decoding, xcode, minor adjustments to regression test formatting 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3312 1f5c12ca-751b-0410-a591-d2e778427230
 2010-06-09 12:56:31 +00:00
hieuhoang1972
c9eb463fe6 vs.net 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3301 1f5c12ca-751b-0410-a591-d2e778427230
 2010-06-05 13:10:26 +00:00
bhaddow
649bf24f62 One moses main 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3274 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-27 08:37:25 +00:00
bhaddow
f58f8397f4 Correct bug in use of scale factor 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3268 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-25 08:55:13 +00:00
bhaddow
c9baabe2ea Merging of moses mains. mosesmt now does single and multi-threeaded 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3264 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-19 16:42:18 +00:00
dowobeha
9fec69ce31 Merged fix (r3001-3260) for problem where multi-threaded moses would fail during linking if boost was compiled for 64 bit. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3261 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-17 01:41:58 +00:00
bhaddow
bb2431331c Fix to consensus decoding to stop it discarding hypos with zero score. 
Move hypothesis id management into Manager to make it thread-safe.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3257 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-14 09:33:03 +00:00
hieuhoang1972
f36759478b don't spew out param list if there's a genuine error 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3250 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-11 21:44:35 +00:00
hieuhoang1972
ecee5c5089 m_detailedTranslationReportingStream=NULL if not outputting trans opt 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3249 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-11 20:56:31 +00:00
bhaddow
0b6346a310 Add consensus decoding to mosesmt. Really need to get the mainlines merged! 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3248 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-11 14:55:59 +00:00
phkoehn
f52a2cc55d code cleanup & enable trace ("-t") for MBR decoding ("-mbr") 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3246 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-10 21:18:47 +00:00
pjwilliams
2a5b262503 Write translation details to a named file instead of cerr. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3243 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-08 15:51:59 +00:00
bhaddow
ffea53717e Refactoring of output search graph code to enable it to be used 
from moses server.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3226 1f5c12ca-751b-0410-a591-d2e778427230
 2010-05-05 16:53:41 +00:00
nicolabertoldi
1c615ea751 re-introduced the initialization and cleaning up into the constructor and destructor of Managers object. And I moved the check for LM cache cleanup into LanguageModelIRST 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3202 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-26 13:38:56 +00:00
nicolabertoldi
cf468273bc Safer way to cleanup LM caches after a given number of processed sentences. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3197 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-24 00:06:49 +00:00
nicolabertoldi
476528c05f I added a new parameter (-clean-lm-cache <value>) in order to control how often the LMs clean up their caches (if available). 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3195 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-23 15:01:06 +00:00
hieuhoang1972
45ecd333f9 delete std namespace in .h files 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3116 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-12 10:04:35 +00:00
hieuhoang1972
c117ef7c17 Copy in changes from the chart_merge branch (doing it manually because the 
server doesn't seem to support subversion's --reintegrate option).


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3078 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-08 17:16:10 +00:00
bhaddow
521d50fe63 Implementation of consensus decoding - first cut. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3071 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-07 15:47:58 +00:00
hieuhoang1972
202bcf2911 eclipse build 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3015 1f5c12ca-751b-0410-a591-d2e778427230
 2010-03-31 20:03:54 +00:00