Commit Graph

657 Commits

Author SHA1 Message Date
MosesAdmin
5696a59ae4 daily automatic beautifier 2015-06-04 13:41:46 +01:00
Ulrich Germann
5807ca4714 Mert programs won't link with shared linking without -lboost_filesystem. 2015-05-21 02:56:03 +00:00
Jeroen Vermeulen
ef5a17b2f9 Fix some new compile problems.
* file_piece.cc used isnan() instead of std::isnan().
 * Fdstream.h used close() but Windows doesn't have unistd.h.

Fixed Fdstream.h by using util::scoped_fd.  Thanks Ken.
2015-05-20 11:40:11 +07:00
Kenneth Heafield
a70d37e46f KenLM 7408730be415db9b650560a8b2bd3e4e3af49ec9.
unistd.hh is dead.
2015-05-19 15:27:30 -04:00
Hieu Hoang
39139e7a64 beautify. 2015-05-15 18:09:38 +01:00
Barry Haddow
294dcbb7a1 format of output 2015-05-13 12:12:20 +01:00
Barry Haddow
89b364bbf0 move hg decoding test to reg tests 2015-05-13 11:21:50 +01:00
Barry Haddow
e34db401ee Refactoring of weight loading, making it easier to test hg rescoring. 2015-05-13 10:21:43 +01:00
Hieu Hoang
cc8c6b7b10 beautify 2015-05-02 11:45:24 +01:00
Matthias Huck
4ee8f2dec1 sentence-bleu less greedy regarding memory
Don't load all references, read them line by line.
Corpora with millions of sentences can now be evaluated without consuming gigabytes of RAM.
2015-04-30 22:26:30 +01:00
Matthias Huck
34d1d3a904 sentence-bleu-nbest 2015-04-30 19:44:29 +01:00
Jeroen Vermeulen
eca5824100 Remove trailing whitespace in C++ files. 2015-04-30 12:05:11 +07:00
Ulrich Germann
1c37f92d77 Missing return value. 2015-04-29 20:06:08 +01:00
Jeroen Vermeulen
38d790cac0 Add cross-platform randomizer module.
The code uses two mechanisms for generating random numbers: srand()/rand(),
which is not thread-safe, and srandom()/random(), which is POSIX-specific.

Here I add a util/random.cc module that centralizes these calls, and unifies
some common usage patterns.  If the implementation is not good enough, we can
now change it in a single place.

To keep things simple, this uses the portable srand()/rand() but protects them
with a lock to avoid concurrency problems.

The hard part was to keep the regression tests passing: they rely on fixed
sequences of random numbers, so a small code change could break them very
thoroughly.  Util::rand(), for wide types like size_t, calls std::rand() not
once but twice.  This behaviour was generalized into utils::wide_rand() and
friends.
2015-04-23 23:46:04 +07:00
Jeroen Vermeulen
75bfb75882 Thread-safe, platform-agnostic randomizer.
Some places in mert use srandom()/random(), but these are POSIX-specific.
The standard alternative, srand()/rand(), is not thread-safe.  This module
wraps srand()/rand() in mutexes (very short-lived, so should not cost much)
so that it relies on just Boost and the C standard library, not on a Unix-like
environment.

This may reduce the width of the random numbers on some platforms: it goes
from "long int" to just "int".  If that is a problem, we may have to use
Boost's randomizer utilities, or eventually, the C++ ones.
2015-04-22 20:43:29 +07:00
Jeroen Vermeulen
1083999d3e Adapt test to poor Windows timer resolution.
TimerTest fails on Windows unless the sleep time is set to at least a
millisecond (1,000 microseconds).  Keep it nice and low for other platforms
though, because the sleep time is wasted.
2015-04-22 12:45:41 +07:00
Barry Haddow
e45c41e665 Testing of Viterbi decoding on hypergraph. 2015-04-17 12:29:41 +01:00
Kenneth Heafield
d6a66d39bd Delete unused code 2015-04-10 09:36:57 -04:00
Jeroen Vermeulen
b8793fb788 Address two TODO notes in mert/evaluator.cpp.
The notes were about two objects which were created on the free store
using "new", then cleaned up using "delete".  May have been a Java
habit; the solution was as simple as creating them on the stack.
2015-04-10 13:25:51 +07:00
Jeroen Vermeulen
8a3ae2fd5c Portability and include fixes.
Add <cstdlib> include for srand()/rand(), and <unistd.h> for open() etc.
Include <unistd.h> on Windows if using MinGW.  Disable MeteorScorer on
Windows, since it doesn't have fork() and pipe().
2015-04-10 12:54:34 +07:00
Barry Haddow
217f389230 Attempt at fixing sparse features for hgmira 2015-04-03 15:46:59 +01:00
Jeroen Vermeulen
789a2e2bc3 Fix some compile warnings (gcc 4.9.2).
Mostly signed/unsigned comparisons and reordered member
initializations; also a few unused variables.

There are more, but if I chip away at them for a while, who knows, it
may catch on and warnings may eventually become socially stigmatizing.
:)
2015-03-29 18:10:51 +07:00
Jeroen Vermeulen
536c6e375f Modernize "C" includes in mert.
This is one of those little chores in managing a long-lived C++
project: standard C headers like stdio.h and math.h now have their own
place in the C++ standard as resp. cstdio, cmath, and so on.  In this
branch the #include names are updated for the mert/ subdirectory; more
branches to follow.

C++11 adds cstdint, but to support compilation with the previous
standard, that change is left for later.
2015-03-28 20:20:58 +07:00
Ulrich Germann
8ca11d941d 1. Lifetime of tasks in ThreadPool is now managed via shared pointers.
2. Code cleanup in IOWrapper and a bit elsewhere.
2015-03-21 16:12:52 +00:00
mjdenkowski
0714521367 Meteor compatibility with batch MIRA 2015-03-13 17:41:53 -04:00
Hieu Hoang
32de075022 beautify 2015-02-19 12:27:23 +00:00
Christophe SERVAN
90471e22a8 Change Namespace in TER library 2015-02-16 19:46:25 +01:00
Christophe SERVAN
d0ff70decc Change Namespace in TER library 2015-02-16 19:34:41 +01:00
Christophe SERVAN
87a4f19546 Memory leak correction in TER algorithm 2015-02-16 19:02:46 +01:00
Ulrich Germann
8a1c8af7a7 Bug fix to commenting out unused variable. 2015-02-09 23:11:17 +00:00
Ulrich Germann
435b1edea8 Bug fix to commenting out unused variable. 2015-02-09 23:09:40 +00:00
Ulrich Germann
be5799dca3 Merge branch 'master' of https://github.com/moses-smt/mosesdecoder
Conflicts:
	moses/TranslationOptionCollection.cpp
	moses/TranslationOptionCollectionLattice.cpp
	moses/TranslationOptionCollectionLattice.h
	moses/TranslationOptionList.h
2015-02-06 01:30:00 +00:00
Ulrich Germann
6cb520d1b2 Commented out unused variable to reduce compiler warnings. 2015-02-05 22:18:11 +00:00
Ulrich Germann
9438c8c1f0 Commented out unused variable to reduce compiler warnings. 2015-02-05 22:14:47 +00:00
Ulrich Germann
bafdc3fa70 Commented out unused variable to reduce compiler warnings. 2015-02-05 22:14:12 +00:00
Hieu Hoang
6d61db28fa use astyle 2.01. It's on Edinburgh server and doesn't screw up enum 2015-01-14 19:21:11 +00:00
Hieu Hoang
05ead45e71 beautify 2015-01-14 11:07:42 +00:00
Barry Haddow
e5a91812e9 fix crash in hypergraph mira for hiero 2014-12-17 17:41:29 +00:00
Nicola Bertoldi
e4eb201c52 merged master into dynamic-models and solved conflicts 2014-12-13 12:52:47 +01:00
Paul Guyot
52eced7572 Fix incorrect integer concatenation in error messages 2014-10-14 13:45:50 +00:00
Phil Williams
50e7e129c8 Fix compile error if !(defined(__GLIBCXX__) || defined(__GLIBCPP__)) 2014-10-09 16:23:48 +01:00
Michael Denkowski
44b22ece69 More streaming for kbmira 2014-09-30 14:50:10 -04:00
Rico Sennrich
84ad576750 explicitly set BLEU as default scorer (for return-best-dev)
(evaluator doesn't accept --scconfig without --sctype)
2014-09-24 14:47:58 +01:00
Rico Sennrich
d39cbca0b9 (optionally) use n-best file for evaluator/return-best-dev
this adds support for metrics that rely on alignment / trees
2014-09-22 10:49:20 +01:00
Rico Sennrich
3d00e5dc8c basic support for more metrics with kbmira
metrics need getReferenceLength (for background smoothing) to work with kbmira
2014-09-22 10:49:20 +01:00
Rico Sennrich
6810b225cc calculateScore with float (for smoothing support) 2014-09-22 10:49:20 +01:00
Rico Sennrich
f40bb2c53c HWCM for MERT 2014-09-22 10:49:20 +01:00
Barry Haddow
d44fd8ed57 Merge branch 'master' of github.com:moses-smt/mosesdecoder 2014-09-07 23:20:17 +01:00
Christophe SERVAN
be9b3cb1c6 Bug fix about the TER calculation 2014-08-29 14:46:56 +02:00
Barry Haddow
1f14ba45e6 Ignore # with in edge lists 2014-08-12 11:33:37 +01:00