Commit Graph

670 Commits

Author SHA1 Message Date
Prashant Mathur
c817980025 Update email 2018-05-18 16:20:47 +02:00
Prashant Mathur
e315438bea Make CHRFscorer compile 2018-05-18 16:18:47 +02:00
Prashant Mathur
fb478bf1db Include chrf as a metric 2018-05-18 16:18:19 +02:00
Prashant Mathur
8b59644945 Adding chrf scorers 2018-05-18 16:16:22 +02:00
MosesAdmin
b16e6a5219 daily automatic beautifier 2017-07-22 00:01:14 +01:00
Prashant Mathur
f07e60aece Merge remote-tracking branch 'upstream/master' 2017-07-20 14:04:32 +02:00
Hieu Hoang
7c62f9c8a4 compile error on Centos 6 2017-01-20 16:00:45 +00:00
Prashant Mathur
85dca3a4da Mira works with BLEU and WER both 2016-11-08 15:49:10 +01:00
MosesAdmin
ea306f62b7 daily automatic beautifier 2016-06-02 00:00:39 +01:00
Marcin Junczys-Dowmunt
73ffe51bd8 Added Grammatical Error Correction specific scorer (M^2) and features 2016-06-01 18:36:43 +02:00
Philipp Koehn
a052837a3a streamlining license to LGPL 2.1 2015-09-25 17:20:09 -04:00
MosesAdmin
c3424ce541 daily automatic beautifier 2015-07-21 00:00:42 +01:00
Rico Sennrich
bec950cf72 support factors in InternalTree 2015-07-20 10:39:01 +01:00
MosesAdmin
5696a59ae4 daily automatic beautifier 2015-06-04 13:41:46 +01:00
Ulrich Germann
5807ca4714 Mert programs won't link with shared linking without -lboost_filesystem. 2015-05-21 02:56:03 +00:00
Jeroen Vermeulen
ef5a17b2f9 Fix some new compile problems.
* file_piece.cc used isnan() instead of std::isnan().
 * Fdstream.h used close() but Windows doesn't have unistd.h.

Fixed Fdstream.h by using util::scoped_fd.  Thanks Ken.
2015-05-20 11:40:11 +07:00
Kenneth Heafield
a70d37e46f KenLM 7408730be415db9b650560a8b2bd3e4e3af49ec9.
unistd.hh is dead.
2015-05-19 15:27:30 -04:00
Hieu Hoang
39139e7a64 beautify. 2015-05-15 18:09:38 +01:00
Barry Haddow
294dcbb7a1 format of output 2015-05-13 12:12:20 +01:00
Barry Haddow
89b364bbf0 move hg decoding test to reg tests 2015-05-13 11:21:50 +01:00
Barry Haddow
e34db401ee Refactoring of weight loading, making it easier to test hg rescoring. 2015-05-13 10:21:43 +01:00
Hieu Hoang
cc8c6b7b10 beautify 2015-05-02 11:45:24 +01:00
Matthias Huck
4ee8f2dec1 sentence-bleu less greedy regarding memory
Don't load all references, read them line by line.
Corpora with millions of sentences can now be evaluated without consuming gigabytes of RAM.
2015-04-30 22:26:30 +01:00
Matthias Huck
34d1d3a904 sentence-bleu-nbest 2015-04-30 19:44:29 +01:00
Jeroen Vermeulen
eca5824100 Remove trailing whitespace in C++ files. 2015-04-30 12:05:11 +07:00
Ulrich Germann
1c37f92d77 Missing return value. 2015-04-29 20:06:08 +01:00
Jeroen Vermeulen
38d790cac0 Add cross-platform randomizer module.
The code uses two mechanisms for generating random numbers: srand()/rand(),
which is not thread-safe, and srandom()/random(), which is POSIX-specific.

Here I add a util/random.cc module that centralizes these calls, and unifies
some common usage patterns.  If the implementation is not good enough, we can
now change it in a single place.

To keep things simple, this uses the portable srand()/rand() but protects them
with a lock to avoid concurrency problems.

The hard part was to keep the regression tests passing: they rely on fixed
sequences of random numbers, so a small code change could break them very
thoroughly.  Util::rand(), for wide types like size_t, calls std::rand() not
once but twice.  This behaviour was generalized into utils::wide_rand() and
friends.
2015-04-23 23:46:04 +07:00
Jeroen Vermeulen
75bfb75882 Thread-safe, platform-agnostic randomizer.
Some places in mert use srandom()/random(), but these are POSIX-specific.
The standard alternative, srand()/rand(), is not thread-safe.  This module
wraps srand()/rand() in mutexes (very short-lived, so should not cost much)
so that it relies on just Boost and the C standard library, not on a Unix-like
environment.

This may reduce the width of the random numbers on some platforms: it goes
from "long int" to just "int".  If that is a problem, we may have to use
Boost's randomizer utilities, or eventually, the C++ ones.
2015-04-22 20:43:29 +07:00
Jeroen Vermeulen
1083999d3e Adapt test to poor Windows timer resolution.
TimerTest fails on Windows unless the sleep time is set to at least a
millisecond (1,000 microseconds).  Keep it nice and low for other platforms
though, because the sleep time is wasted.
2015-04-22 12:45:41 +07:00
Barry Haddow
e45c41e665 Testing of Viterbi decoding on hypergraph. 2015-04-17 12:29:41 +01:00
Kenneth Heafield
d6a66d39bd Delete unused code 2015-04-10 09:36:57 -04:00
Jeroen Vermeulen
b8793fb788 Address two TODO notes in mert/evaluator.cpp.
The notes were about two objects which were created on the free store
using "new", then cleaned up using "delete".  May have been a Java
habit; the solution was as simple as creating them on the stack.
2015-04-10 13:25:51 +07:00
Jeroen Vermeulen
8a3ae2fd5c Portability and include fixes.
Add <cstdlib> include for srand()/rand(), and <unistd.h> for open() etc.
Include <unistd.h> on Windows if using MinGW.  Disable MeteorScorer on
Windows, since it doesn't have fork() and pipe().
2015-04-10 12:54:34 +07:00
Barry Haddow
217f389230 Attempt at fixing sparse features for hgmira 2015-04-03 15:46:59 +01:00
Jeroen Vermeulen
789a2e2bc3 Fix some compile warnings (gcc 4.9.2).
Mostly signed/unsigned comparisons and reordered member
initializations; also a few unused variables.

There are more, but if I chip away at them for a while, who knows, it
may catch on and warnings may eventually become socially stigmatizing.
:)
2015-03-29 18:10:51 +07:00
Jeroen Vermeulen
536c6e375f Modernize "C" includes in mert.
This is one of those little chores in managing a long-lived C++
project: standard C headers like stdio.h and math.h now have their own
place in the C++ standard as resp. cstdio, cmath, and so on.  In this
branch the #include names are updated for the mert/ subdirectory; more
branches to follow.

C++11 adds cstdint, but to support compilation with the previous
standard, that change is left for later.
2015-03-28 20:20:58 +07:00
Ulrich Germann
8ca11d941d 1. Lifetime of tasks in ThreadPool is now managed via shared pointers.
2. Code cleanup in IOWrapper and a bit elsewhere.
2015-03-21 16:12:52 +00:00
mjdenkowski
0714521367 Meteor compatibility with batch MIRA 2015-03-13 17:41:53 -04:00
Hieu Hoang
32de075022 beautify 2015-02-19 12:27:23 +00:00
Christophe SERVAN
90471e22a8 Change Namespace in TER library 2015-02-16 19:46:25 +01:00
Christophe SERVAN
d0ff70decc Change Namespace in TER library 2015-02-16 19:34:41 +01:00
Christophe SERVAN
87a4f19546 Memory leak correction in TER algorithm 2015-02-16 19:02:46 +01:00
Ulrich Germann
8a1c8af7a7 Bug fix to commenting out unused variable. 2015-02-09 23:11:17 +00:00
Ulrich Germann
435b1edea8 Bug fix to commenting out unused variable. 2015-02-09 23:09:40 +00:00
Ulrich Germann
be5799dca3 Merge branch 'master' of https://github.com/moses-smt/mosesdecoder
Conflicts:
	moses/TranslationOptionCollection.cpp
	moses/TranslationOptionCollectionLattice.cpp
	moses/TranslationOptionCollectionLattice.h
	moses/TranslationOptionList.h
2015-02-06 01:30:00 +00:00
Ulrich Germann
6cb520d1b2 Commented out unused variable to reduce compiler warnings. 2015-02-05 22:18:11 +00:00
Ulrich Germann
9438c8c1f0 Commented out unused variable to reduce compiler warnings. 2015-02-05 22:14:47 +00:00
Ulrich Germann
bafdc3fa70 Commented out unused variable to reduce compiler warnings. 2015-02-05 22:14:12 +00:00
Hieu Hoang
6d61db28fa use astyle 2.01. It's on Edinburgh server and doesn't screw up enum 2015-01-14 19:21:11 +00:00
Hieu Hoang
05ead45e71 beautify 2015-01-14 11:07:42 +00:00