Commit Graph

22 Commits

Author SHA1 Message Date
Kenneth Heafield
a70d37e46f KenLM 7408730be415db9b650560a8b2bd3e4e3af49ec9.
unistd.hh is dead.
2015-05-19 15:27:30 -04:00
Jeroen Vermeulen
75bfb75882 Thread-safe, platform-agnostic randomizer.
Some places in mert use srandom()/random(), but these are POSIX-specific.
The standard alternative, srand()/rand(), is not thread-safe.  This module
wraps srand()/rand() in mutexes (very short-lived, so should not cost much)
so that it relies on just Boost and the C standard library, not on a Unix-like
environment.

This may reduce the width of the random numbers on some platforms: it goes
from "long int" to just "int".  If that is a problem, we may have to use
Boost's randomizer utilities, or eventually, the C++ ones.
2015-04-22 20:43:29 +07:00
Jeroen Vermeulen
d56f317f2e New helper classes: temp_dir & temp_file.
I'm adding these because boost::filesystem::unique_path introduces
encoding issues: on Windows the path is in wchar_t, breaking use of
those strings in various places!  Encoding the strings is just too
much work.

It's still possible that the current temp_file implementation won't
build on Windows (it uses POSIX mkstemp() and close()) but that can
be fixed underneath the API.
2015-04-17 22:57:55 +07:00
Kenneth Heafield
dd03f9fb69 KenLM 5a7efd8fe1db88ee0a9f7e9479b24ac3ca348221 with Hieu's patch to exception.hh 2014-06-02 10:29:40 -07:00
Kenneth Heafield
8ae4d153c8 Consolidated, consistent rt target fixes single-threaded build 2014-03-11 11:29:40 -07:00
Kenneth Heafield
e1d8f5c2ae Do not compile pcqueue_test for single-threaded builds 2014-03-11 11:11:24 -07:00
Kenneth Heafield
14e02978fc KenLM 5cc905bc2d214efa7de2db56a9a672b749a95591
Avoid unspecified behavior of mmap when a file is resized reported by Christian Hardmeier
Fixes for Mavericks and a workaround for Boost's broken semaphore
Clean clang compile (of kenlm)

Merged some of 744376b3fb but also undid some of it because it was just masking a fundaemntal problem with pread rather than working around windows limitations
2014-01-27 16:51:35 -08:00
Kenneth Heafield
36c17f3c93 librt only for single-threaded linux 2013-06-05 12:30:05 +01:00
Kenneth Heafield
611df02326 librt for single threaded 2013-06-05 09:41:44 +01:00
Kenneth Heafield
fc5868d0ff KenLM df5be22 lmplz for estimation 2013-01-18 15:59:51 +00:00
Kenneth Heafield
d4f433c7f4 Put core Moses into one library libmoses. 2012-11-12 18:07:53 +00:00
Kenneth Heafield
630b4d4ea9 Get more paranoid by checking for headers too 2012-10-30 20:02:59 +00:00
Kenneth Heafield
b6b2e35190 Change test_lib to obey --static 2012-10-30 19:42:11 +00:00
Kenneth Heafield
4b98b0937c Add bzip and xz input for FilePiece KenLM 06f78db2 2012-10-30 19:33:22 +00:00
Kenneth Heafield
96bc9d05dc Move pool to util 2012-10-16 17:35:27 +01:00
Kenneth Heafield
0eb98df0fe Submodules are evil. Undo it. 2012-10-15 13:58:33 +01:00
Kenneth Heafield
4faab6c68f Extract kenlm into a submodule, referencing lazy 2012-09-24 18:30:51 +01:00
Kenneth Heafield
c7e02eb677 KenLM e3b5c55910 including rest costs for probing 2012-06-28 10:58:59 -04:00
Kenneth Heafield
f564876e5e Avoid naming conflict with glibc's libutil 2011-11-28 09:19:45 +00:00
Kenneth Heafield
463f2e232e Link dynamically against libz if we have to 2011-11-21 15:09:38 +00:00
Kenneth Heafield
f15eee0abf Jam: Clean up boost library handling, assimilate mert 2011-11-18 11:27:57 -05:00
Kenneth Heafield
5c9b28c55a Introducing Jamfiles
Currently supports kenlm, irstlm, and srilm options.

bjam [--with-srilm=/path/to/sri] [--with-irstlm=/path/to/irst] -j8
Places binaries in dist/ that pass regression tests.
2011-11-18 14:37:01 +00:00