Commit Graph

25 Commits

Author SHA1 Message Date
Matthias Huck
d1706a54dd OrientationPhraseProperty 2014-08-07 21:43:09 +01:00
Matthias Huck
c27cbf55ea source labels: integration into EMS 2014-08-07 21:02:51 +01:00
Hieu Hoang
a402523ef5 calculate baseline score without optimisation 2014-07-11 16:26:48 +01:00
Hieu Hoang
c460330430 debug output for PhraseProperty 2014-06-19 19:23:33 +01:00
Hieu Hoang
f2d3f57d07 Merge branch 'hieu' of ../mosesdecoder.hieu 2014-06-16 10:57:02 +01:00
Matthias Huck
9b9c182726 PhraseProperty destructor 2014-06-13 18:01:41 +01:00
Hieu Hoang
cef8fd3b72 merge 2014-06-13 17:42:13 +01:00
Hieu Hoang
5c57702664 merge 2014-06-13 17:08:22 +01:00
Matthias Huck
a5467d89c4 Minor modification of the phrase properties framework.
Properties can save memory by not storing the value string.
2014-06-13 16:37:13 +01:00
Hieu Hoang
eb78782c5d merge with master 2014-06-13 10:35:35 +01:00
Matthias Huck
9a7e568760 SourceLabelsPhraseProperty 2014-06-11 21:08:22 +01:00
Matthias Huck
7380a83f75 CountsPhraseProperty: add comment to header 2014-06-11 20:26:24 +01:00
Matthias Huck
e693a27e4e A simple phrase property class to access the three phrase count values.
The counts are usually not needed during decoding and are not loaded
from the phrase table. This is just a workaround that can make them
available to features which have a use for them.

If you need access to the counts, copy the two marginal counts and the
joint count into an additional information property with key "Counts",
e.g. using awk:

$ zcat phrase-table.gz | awk -F' \|\|\| '  '{printf("%s {{Counts %s}}\n",$0,$5);}' | gzip -c > phrase-table.withCountsPP.gz

CountsPhraseProperty reads them from the phrase table and provides
methods GetSourceMarginal(), GetTargetMarginal(), GetJointCount().
2014-06-11 20:02:31 +01:00
Hieu Hoang
a27835727e make sure int are cast to float 2014-06-10 11:18:14 +01:00
Hieu Hoang
31a5b78b91 bug in SpanLength FF 2014-06-10 10:44:25 +01:00
Hieu Hoang
8c55f98c7a bug in SpanLength FF 2014-06-10 10:29:49 +01:00
Hieu Hoang
0ab1142238 Merge github.com:moses-smt/mosesdecoder into hieu 2014-06-10 10:17:05 +01:00
Hieu Hoang
8edb344492 minor const£ 2014-06-10 10:16:17 +01:00
Hieu Hoang
649b31c226 span length 2014-06-09 16:01:37 +01:00
Hieu Hoang
e25e7b69be span length 2014-06-06 10:40:40 +01:00
Hieu Hoang
d95bd5bdae span length property 2014-06-05 20:57:00 +01:00
Hieu Hoang
c5aac0676e don't output SpanLength property if no non-terms 2014-06-04 17:15:54 +01:00
Hieu Hoang
4a3ac7411d span length 2014-06-04 16:52:57 +01:00
Matthias Huck
d921d23f7d comment 2014-05-19 22:09:27 +01:00
Matthias Huck
1740478238 Framework for additional phrase properties in decoding.
Derive your property class from PhraseProperty. Do any expensive string
processing of the property value in there, not in the feature
implementation, and provide methods to access the information in
appropriate data formats. The property value string will thus have to
be processed only once (on loading) rather than each time the respective
phrase is applied and your feature needs to access the property value.
2014-05-19 21:54:08 +01:00