Commit Graph

488 Commits

Author SHA1 Message Date
Ulrich Germann
5ff14f9798 Merge branch 'master' of https://github.com/moses-smt/mosesdecoder 2014-06-14 13:29:52 +01:00
Ulrich Germann
1a3d7d3266 Reorganization of phrase scorers in Mmsapt. 2014-06-14 13:03:31 +01:00
Ulrich Germann
c3db1a3a67 Better labeling of feature values in lookup_mmsapt. 2014-06-14 13:02:55 +01:00
Ulrich Germann
135ce20509 Smoothing parameter in LexicalPhraseScorer2 is now a configurable paramter. 2014-06-14 13:02:08 +01:00
Ulrich Germann
450c8c5b80 Moved phrase scorers to a separate file. 2014-06-14 12:58:33 +01:00
Hieu Hoang
9e43830c6e bjam for ProbingPT 2014-06-13 10:20:30 +01:00
Hieu Hoang
2c6cdb5889 bjam for ProbingPT 2014-06-13 09:48:32 +01:00
Hieu Hoang
4ebcfc2142 bjam for ProbingPT 2014-06-12 18:38:04 +01:00
Ulrich Germann
ead3fc565e Initial check-in. 2014-06-12 18:30:41 +01:00
Hieu Hoang
ad4825ad44 integrated probing pt into decoder 2014-06-11 14:25:18 +01:00
Hieu Hoang
253f8e35fd integrated probing pt into decoder 2014-06-11 13:52:20 +01:00
XapaJIaMnu
095877742c Add ProbingPT files to Moses 2014-06-11 11:47:00 +01:00
Ulrich Germann
106a66eb17 Merge branch 'master' of https://github.com/moses-smt/mosesdecoder 2014-06-10 14:28:59 +01:00
Hieu Hoang
d68257c34d use standard c++ getline instead of old Moses SAFE_GETLINE 2014-06-08 16:37:12 +01:00
Matthias Huck
a19aaada06 formatting 2014-06-06 20:16:40 +01:00
Matthias Huck
2ec50239e7 rule-limit=0: unlimited 2014-06-06 20:05:59 +01:00
Hieu Hoang
d29bf716a4 minor change to debugging output. Memory pt used by both pb and chart-based models 2014-06-05 17:15:08 +01:00
Ulrich Germann
5116f0072b Minor edits to ug_bitext.h. Added min_diverse to ug_bitext::job to ensure minimum number of translation alternatives before sampling stops. 2014-06-05 01:50:55 +01:00
Ulrich Germann
b92d599727 Bug fix in mmlex-lookup. 2014-06-05 01:48:11 +01:00
Ulrich Germann
2f109621bf Added configurable options and SetTableLimit to Mmsapt. 2014-06-05 01:47:29 +01:00
Ulrich Germann
a40fcbae02 Added utility lookup_mmsapt 2014-06-05 01:45:34 +01:00
Ulrich Germann
ce853731ae Added mmsapt lookup utility. 2014-06-05 01:38:55 +01:00
Ulrich Germann
8dee1725fb Removed Phrase penalty as a built-in feature function. 2014-06-03 21:36:04 +01:00
Ulrich Germann
9c27dc08e9 Merge branch 'master' of https://github.com/moses-smt/mosesdecoder
Conflicts:
	moses/FF/ConstrainedDecoding.cpp
2014-06-03 18:55:53 +01:00
Ulrich Germann
9cda39f802 Major bug fix in setting ug_bitext::jstats.my_cnt2. 2014-06-03 15:48:27 +01:00
Ulrich Germann
8c9a26a3be Added phrase feature function for log counts. 2014-06-03 15:45:29 +01:00
Ulrich Germann
0fc80660f3 TSA_tree_iterator::approxOccurrenceCount() now returns exact count for small values. 2014-06-03 15:44:20 +01:00
Ulrich Germann
a248349da3 Added more phrase table features and configuration options to Mmsapt. 2014-06-03 15:41:59 +01:00
Ulrich Germann
ce435d22bf Mmsapt now adds the phrase penalty only if num_features specified
on the respective line in moses.ini is odd.
2014-06-01 18:23:00 +01:00
Ulrich Germann
249c40ec74 Eliminated factor delimiter from the parameters of
Phrase.CreateFromString() and Sentence.CreateFromString(),
as it was never used in those functions anyway ---
Word.CreateFromString() retrieves the factor delimiter
from StaticData directly.
2014-05-31 14:33:31 +01:00
Ulrich Germann
c452875343 Mmsapt/ug_bitext:
- Sampling now uses a static seed for consistency between runs.
- (In order to have consistency guarantees, specify workers=1 in the
- Mmsapt line in moses.ini; otherwise the randomness of thread scheduling may
  cause inconsistencies between runs.
- Denominator for fwd phrase probabilities can now be chosen via an
  external argument.
2014-05-31 14:29:44 +01:00
Ulrich Germann
380b2da2d0 Bug fix in and in connection with Mmsapt::combine_pstats(). 2014-05-19 21:34:58 +01:00
Hieu Hoang
bf7c033271 backoff for default GetTargetPhraseCollectionBatch(), used by binary pt and compact pt 2014-05-12 15:50:32 +01:00
Hieu Hoang
2830ed9968 use Nadir's backoff code 2014-05-12 15:40:18 +01:00
Hieu Hoang
1d401817a5 backoff fix for memory and on-disk phrase-tables 2014-05-12 15:16:11 +01:00
Hieu Hoang
b436570ab0 delete unused files 2014-05-07 12:21:19 +01:00
Hieu Hoang
2beed771d2 minor error. Segfault if phrase length is bigger than max-phrase-length 2014-05-01 16:47:17 +01:00
Ulrich Germann
7c145d045b Merge branch 'master' into dynamic-phrase-tables
Conflicts:
	contrib/server/Jamfile
	contrib/server/mosesserver.cpp
2014-04-28 10:00:07 +01:00
Ulrich Germann
e5b739946f Merge branch 'master' of https://github.com/moses-smt/mosesdecoder 2014-04-23 18:29:39 +01:00
Hieu Hoang
00505ba048 minor leak 2014-04-23 11:36:25 +01:00
Ulrich Germann
24545ba1de Changes due to new pure virtual function in PhraseTable. 2014-04-11 04:28:34 +01:00
Ulrich Germann
21f141ce0d Merge branch 'master' into dynamic-phrase-tables
Conflicts:
	moses/InputPath.cpp
	moses/Word.cpp
2014-04-08 14:58:29 +01:00
Rico Sennrich
1f435340f0 faster pruning in chart decoding 2014-03-26 11:23:59 +00:00
Ulrich Germann
f63606a92b 1. Renamed PhraseDictionary member function release() to Release().
2. Added option to check phrase prefixes for early stopping of
   InputPath generation when translating confusion networks.
2014-03-25 01:49:24 +00:00
Ulrich Germann
80b031769d (1) Bug fixes related to locking. (2) pstats cache disabled when translating confusion networks. 2014-03-25 01:48:49 +00:00
Ulrich Germann
84299006da Added another constructor to TSA_tree_iterator. 2014-03-25 01:43:00 +00:00
Ulrich Germann
f5d969ab37 Added missing library for custom-pt.cc 2014-03-25 01:42:06 +00:00
Ulrich Germann
728ceb4e4c Added missing #include. 2014-03-24 14:51:51 +00:00
Ulrich Germann
14e52c9b94 Errors occurring when accessing data files now throw exceptions. 2014-03-24 13:39:15 +00:00
Ulrich Germann
ed825cd234 Bug fix: dependency .../UG/generic//generic was missing for custom-pt 2014-03-24 13:33:46 +00:00