moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2024-12-24 20:32:59 +03:00
Code Issues Packages Projects Releases Wiki Activity
14,110 Commits 136 Branches 26 Tags 147 MiB
704432cf0f
Commit Graph

741 Commits

Author SHA1 Message Date
Ulrich Germann
704432cf0f Bug fixes. 2015-06-05 22:25:13 +01:00
Ulrich Germann
623eb7bb77 Instantiation of btfix via boost::intrusive_ptr in Mmsapt. 
This is in preparation for distinct bitext samplers which need to
ensure the lifetime of the bitext while sampling.
 2015-06-05 21:15:47 +01:00
Ulrich Germann
e8ee56876e Initial check-in. 2015-06-05 17:24:53 +01:00
Ulrich Germann
8f4b2afe26 #include a few more things. 2015-06-05 16:30:07 +01:00
Ulrich Germann
1b4b3a5103 Mmsapt: btfix now instatiated via intrusive pointer 
... to prevent deletion while Mmsapt is live.
 2015-06-05 16:27:49 +01:00
Ulrich Germann
47fa99b61b Added member function size() to LRU_Cache. 2015-06-05 16:26:47 +01:00
Ulrich Germann
243a6a8b3b Added #define for intrusive pointer. 2015-06-05 16:23:00 +01:00
Ulrich Germann
576c743aee Simplified #include. 2015-06-05 16:22:03 +01:00
Ulrich Germann
5cb1d95e09 Added member function for retrieving nbest list items without sorting. 2015-06-05 16:21:09 +01:00
Ulrich Germann
5a56a5b496 Added target for forced relinking only (no forced recompilation); temporarily disabled tcmalloc. 2015-06-05 16:20:08 +01:00
Ulrich Germann
83fa1b6a88 Initial check-in. 2015-06-03 12:59:32 +01:00
Ulrich Germann
0afe139810 Initial check-in. 2015-06-03 12:55:58 +01:00
Ulrich Germann
debdd21899 Optional initialization of SentenceBias. 2015-06-03 12:53:38 +01:00
Ulrich Germann
f024eede74 Added ca() as short replacement for approxOccurrenceCount() to tsa_tree_iterator. 2015-06-03 12:51:44 +01:00
Ulrich Germann
d62d2dc95f Bug fix. 2015-06-01 23:10:50 +01:00
Ulrich Germann
aa4eed93d5 Bug fix related to getting rid of using namespace std; . 2015-06-01 18:55:40 +01:00
Ulrich Germann
cc800742b1 Updated Makefile for local compiles. 2015-06-01 18:26:27 +01:00
Ulrich Germann
99896cfd2c Untangling bitext class from Moses dependencies, so that the class can be used 
independently of Moses again.
 2015-06-01 18:25:04 +01:00
Ulrich Germann
349163f3fd Bug fix and in-line code documentation. 2015-06-01 18:21:52 +01:00
Ulrich Germann
25f98a446e Bug fix in building imTtrack directly from input stream. 2015-06-01 18:19:34 +01:00
Ulrich Germann
c82ee9a4e9 Bug fix. 2015-05-24 16:44:41 +01:00
Ulrich Germann
da052b7f2b Removed dependency on libcurlpp, as it was difficult to link that staticly. 2015-05-24 16:05:14 +01:00
Ulrich Germann
dcb8e5d3e0 Preparation for allowing context-aware decoding. 2015-05-19 02:35:39 +01:00
Hieu Hoang
39139e7a64 beautify. 2015-05-15 18:09:38 +01:00
Marcin Junczys-Dowmunt
7652ab9118 quick fix for out-of-bound alignment points 2015-05-15 09:12:51 +02:00
Jeroen Vermeulen
0859e9a844 Remove trailing whitespace from C++ files. 2015-05-13 17:05:43 +07:00
Jeroen Vermeulen
1364a7d599 Fix typo in mmap call. 
The case where !m_fixed passed m_map_size to mmap(), but the "else"
clause passed map_size.  In replacing mmap() with the portable wrapper,
I accidentally changed that to be m_map_size as well.

Besides fixing that, I'm changing the name of the variable to be more
clearly distinguishable from m_map_size.
 2015-05-12 09:58:47 +07:00
Ulrich Germann
7da7ce52da Added context buffering in IOWrapper for context-sensitive decoding. 
Unfortunately, this seems to slow things down quite a bit.
 2015-05-11 00:34:24 +01:00
Ulrich Germann
db5ccff364 Tweaks to logging for biased sampling. 2015-05-11 00:33:21 +01:00
Ulrich Germann
1778238d73 Logging of latency of bias lookup via server. 2015-05-11 00:32:20 +01:00
Ulrich Germann
8a174beb44 Additional check for document map if document bias is requested. 2015-05-11 00:30:32 +01:00
Nicola Bertoldi
90a982e579 merge remote into local 2015-05-04 09:42:44 +02:00
Nicola Bertoldi
c4f04670c2 made ProbingPT constructor compliant with PhraseDictionary signature 2015-05-04 09:25:50 +02:00
Hieu Hoang
cc8c6b7b10 beautify 2015-05-02 11:45:24 +01:00
Jeroen Vermeulen
eca5824100 Remove trailing whitespace in C++ files. 2015-04-30 12:05:11 +07:00
Ulrich Germann
324b1a9b56 Merge branch 'master' of https://github.com/moses-smt/mosesdecoder 2015-04-29 20:20:54 +01:00
Ulrich Germann
e4f5c69109 One step closer to eliminating the requirement to provide num-features=... in the config file. 
Some FF (Mmsapt, LexicalReordering, Many single-value FF) provide this number during "registration";
when missing, a default weight vector of uniform 1.0 is automatically generated. This eliminates the
need for the user to figure out what the exact number of features is for each FF, which can get complicated,
e.g. in the case of Mmsapt/PhraseDictionaryBitextSampling.
 2015-04-29 20:16:52 +01:00
Ulrich Germann
c76f1c338d Uninitialized variable. 2015-04-29 20:16:43 +01:00
Jeroen Vermeulen
616b589da3 Fix a bunch of compiler warnings. 
Warnings are useful, but only if there are few!
 2015-04-29 21:18:51 +07:00
Ulrich Germann
315610c02a Merge branch 'master' of https://github.com/moses-smt/mosesdecoder 2015-04-27 16:39:40 +01:00
Ulrich Germann
37bb1de9ed Unused variable. 2015-04-27 16:30:59 +01:00
Ulrich Germann
fbf8b1f8b8 Code design debizarrification: Indexes of feature functions into the dense vector of all feature 
values are now stored on the feature function instead of in a global map that is a static
member of ScoreComponentCollection.
 2015-04-26 16:46:36 +01:00
Ulrich Germann
e63561ae7f Unused variable. 2015-04-26 15:41:32 +01:00
Hieu Hoang
41529227b2 boost unique lock 2015-04-26 18:11:11 +04:00
Ulrich Germann
bafe60c3a1 Make sure things work when curl-based biasing is disabled. 2015-04-26 03:14:40 +01:00
Ulrich Germann
0d72cdd72c Merge branch 'master' of https://github.com/moses-smt/mosesdecoder into mmt-dev 
Conflicts:
	moses/Syntax/F2S/Manager-inl.h
	moses/TranslationModel/UG/mmsapt.cpp
 2015-04-26 02:12:16 +01:00
Jeroen Vermeulen
8ac91c8d97 Fix unqualified call to rand_excl(). 
The call needed to be made explicitly to util::rand_excl().  Sorry.
 2015-04-24 00:22:25 +07:00
Jeroen Vermeulen
38d790cac0 Add cross-platform randomizer module. 
The code uses two mechanisms for generating random numbers: srand()/rand(),
which is not thread-safe, and srandom()/random(), which is POSIX-specific.

Here I add a util/random.cc module that centralizes these calls, and unifies
some common usage patterns.  If the implementation is not good enough, we can
now change it in a single place.

To keep things simple, this uses the portable srand()/rand() but protects them
with a lock to avoid concurrency problems.

The hard part was to keep the regression tests passing: they rely on fixed
sequences of random numbers, so a small code change could break them very
thoroughly.  Util::rand(), for wide types like size_t, calls std::rand() not
once but twice.  This behaviour was generalized into utils::wide_rand() and
friends.
 2015-04-23 23:46:04 +07:00
Jeroen Vermeulen
02d1d9a4af Don't work around missing popen() in MinGW. 
Windows does not have popen()/pclose(), so FileHandler.cpp #define's them to
_popen()/_pclose().  But MinGW has similar macros built into <cstdio>, leading
to warnings.  So skip the workaround on MinGW.
 2015-04-22 11:24:32 +07:00
Jeroen Vermeulen
32722ab5b1 Support tokenize(const std::string &) as well. 
Convenience wrapper: the actual function takes a const char[], but many of
the call sites want to pass a string and have to call its c_str() first.
 2015-04-22 10:35:18 +07:00