Commit Graph

12592 Commits

Author SHA1 Message Date
Hieu Hoang
768ac1c6a8 add SpanLength to score 2014-06-12 13:26:01 +01:00
Hieu Hoang
f3ec79d278 Merge branch 'master' of github.com:moses-smt/mosesdecoder 2014-06-12 13:15:51 +01:00
Hieu Hoang
eeedaaa09c incorrectly used boost shared_ptr 2014-06-12 13:15:02 +01:00
Hieu Hoang
31f662264e incorrectly used boost shared_ptr 2014-06-12 13:10:56 +01:00
Matthias Huck
9a7e568760 SourceLabelsPhraseProperty 2014-06-11 21:08:22 +01:00
Matthias Huck
7380a83f75 CountsPhraseProperty: add comment to header 2014-06-11 20:26:24 +01:00
Matthias Huck
e693a27e4e A simple phrase property class to access the three phrase count values.
The counts are usually not needed during decoding and are not loaded
from the phrase table. This is just a workaround that can make them
available to features which have a use for them.

If you need access to the counts, copy the two marginal counts and the
joint count into an additional information property with key "Counts",
e.g. using awk:

$ zcat phrase-table.gz | awk -F' \|\|\| '  '{printf("%s {{Counts %s}}\n",$0,$5);}' | gzip -c > phrase-table.withCountsPP.gz

CountsPhraseProperty reads them from the phrase table and provides
methods GetSourceMarginal(), GetTargetMarginal(), GetJointCount().
2014-06-11 20:02:31 +01:00
Matthias Huck
d0e92da734 GHKM extraction can add a source labels phrase property 2014-06-11 19:27:18 +01:00
Matthias Huck
02848112d8 experiment.meta: skip-parse and mock-parse 2014-06-11 19:06:04 +01:00
Hieu Hoang
ad4825ad44 integrated probing pt into decoder 2014-06-11 14:25:18 +01:00
Hieu Hoang
253f8e35fd integrated probing pt into decoder 2014-06-11 13:52:20 +01:00
phikoehn
89a9c410c9 Merge branch 'master' of ssh://github.com/moses-smt/mosesdecoder 2014-06-11 13:44:22 +01:00
phikoehn
45648d03b9 support for lmplz training of osm in ems 2014-06-11 13:44:02 +01:00
XapaJIaMnu
095877742c Add ProbingPT files to Moses 2014-06-11 11:47:00 +01:00
Hieu Hoang
aa88c0815a Merge branch 'master' of github.com:moses-smt/mosesdecoder 2014-06-11 11:22:41 +01:00
XapaJIaMnu
2f752fe833 Truncate revision output 2014-06-11 10:51:31 +01:00
XapaJIaMnu
793aef6715 Fix small oversight 2014-06-11 10:24:44 +01:00
Hieu Hoang
8d5328a957 ignore 0 span. Don't bomb out 2014-06-11 05:05:22 +01:00
Hieu Hoang
af659446bd Merge branch 'hieu' of github.com:hieuhoang/mosesdecoder into hieu 2014-06-11 04:35:33 +01:00
Hieu Hoang
92089f9726 ignore 0 span. Don't bomb out 2014-06-11 04:35:09 +01:00
Hieu Hoang
a96b0fcce3 Merge branch 'hieu' of github.com:hieuhoang/mosesdecoder into hieu 2014-06-10 18:05:00 +01:00
Hieu Hoang
341969e61d Merge ../mosesdecoder into hieu 2014-06-10 18:04:55 +01:00
Hieu Hoang
a695c9b50b Merge branch 'master' of github.com:moses-smt/mosesdecoder 2014-06-10 17:27:34 +01:00
Hieu Hoang
4b98495e78 don't load data when just showing weightd 2014-06-10 17:27:20 +01:00
XapaJIaMnu
d1554c4cdc Add moses speedtesting framework and readmes. 2014-06-10 16:28:46 +01:00
Ulrich Germann
106a66eb17 Merge branch 'master' of https://github.com/moses-smt/mosesdecoder 2014-06-10 14:28:59 +01:00
Hieu Hoang
a27835727e make sure int are cast to float 2014-06-10 11:18:14 +01:00
Hieu Hoang
31a5b78b91 bug in SpanLength FF 2014-06-10 10:44:25 +01:00
Hieu Hoang
8c55f98c7a bug in SpanLength FF 2014-06-10 10:29:49 +01:00
Barry Haddow
91ccf8ef72 bidirectional case 2014-06-10 10:23:48 +01:00
Hieu Hoang
0ab1142238 Merge github.com:moses-smt/mosesdecoder into hieu 2014-06-10 10:17:05 +01:00
Hieu Hoang
8edb344492 minor const£ 2014-06-10 10:16:17 +01:00
Barry Haddow
a5e5a6590b basic implementation of non-hierarchical sparse features 2014-06-09 22:17:05 +01:00
Hieu Hoang
651967e109 eclipse 2014-06-09 16:24:19 +01:00
Hieu Hoang
da84cce8c4 Merge github.com:moses-smt/mosesdecoder into hieu 2014-06-09 16:20:29 +01:00
Hieu Hoang
1fb71da5b5 delete debugging messages 2014-06-09 16:17:20 +01:00
Hieu Hoang
649b31c226 span length 2014-06-09 16:01:37 +01:00
Rico Sennrich
169c3fce38 convert CoNNL-X to Moses XML format 2014-06-09 15:24:41 +01:00
Hieu Hoang
0178e5237e don't output SpanLength property if empty 2014-06-09 09:05:31 +01:00
Hieu Hoang
7c5208e6d6 Merge ../mosesdecoder into hieu 2014-06-08 17:18:16 +01:00
Hieu Hoang
29d83d94b1 delete any mention of SAFE_GETLINE so it doesn't reappear 2014-06-08 17:18:07 +01:00
Hieu Hoang
3c6a31128d Merge ../mosesdecoder into hieu 2014-06-08 17:07:41 +01:00
Hieu Hoang
1b667e3e24 delete any mention of SAFE_GETLINE so it doesn't reappear 2014-06-08 17:07:12 +01:00
Hieu Hoang
d68257c34d use standard c++ getline instead of old Moses SAFE_GETLINE 2014-06-08 16:37:12 +01:00
Hieu Hoang
cb94a3181b use standard c++ getline instead of old Moses SAFE_GETLINE 2014-06-08 16:23:14 +01:00
Hieu Hoang
23ba0de224 use standard c++ getline instead of old Moses SAFE_GETLINE 2014-06-08 15:41:27 +01:00
Hieu Hoang
d979b24314 use standard c++ getline instead of old Moses SAFE_GETLINE 2014-06-08 14:06:33 +01:00
Hieu Hoang
45ed0a5b1f Merge ../mosesdecoder into hieu 2014-06-08 13:22:34 +01:00
Hieu Hoang
16c941dd4e Merge branch 'master' of github.com:moses-smt/mosesdecoder 2014-06-08 13:17:31 +01:00
Hieu Hoang
f58c7fc831 use standard c++ getline instead of old Moses SAFE_GETLINE 2014-06-08 13:17:23 +01:00