Commit Graph

103 Commits

Author SHA1 Message Date
Kenneth Heafield
168baf281c Compile phrase_table_vocab 2016-03-08 10:13:52 +00:00
Kenneth Heafield
2a74f3a521 KenLM e06ec4dc59f765482d7545b3cb797b8fc128ab9f 2016-01-12 23:39:27 +00:00
Hieu Hoang
b6231e8c73 make util::StringStream more like std::stringstream 2015-10-16 19:03:22 +00:00
Kenneth Heafield
ea8e19f286 KenLM a590a3a4dadf516a1cff28c8f1c06aa89766f519 including StringStream
TODO: kill istream
2015-09-29 16:58:02 +01:00
Ulrich Germann
58d9d88a9f Merge branch 'master' of http://github.com/moses-smt/mosesdecoder 2015-09-14 09:58:30 +01:00
Kenneth Heafield
5732129bd4 Update kenlm 2015-09-10 16:04:09 +01:00
Ulrich Germann
5f563035e6 Merge branch 'master' into ranked-sampling 2015-09-07 10:11:16 +01:00
Ulrich Germann
33ff9d91b7 Missing #include. 2015-09-03 23:58:45 +01:00
Rico Sennrich
8df6cb5bc6 synchronize nplm jamfile options between moses/LM/Jamfile and lm/Jamfile 2015-09-03 18:15:29 +01:00
Kenneth Heafield
09ecd071f9 KenLM 2a3e8fae3633c890cb3b342d461f9130c8e343fa excluding unfinished interpolation directory 2015-08-27 10:55:52 +01:00
Ulrich Germann
80ee9813c6 Added missing header for std::cerr. 2015-05-20 17:36:48 +01:00
Jeroen Vermeulen
1c250dac92 Fix _fpclass build failure on MinGW.
On MinGW, use the Unix-style fpclassify() code rather than the Window
version.  The latter doesn't seem to compile.
2015-05-20 15:12:43 +07:00
Kenneth Heafield
a70d37e46f KenLM 7408730be415db9b650560a8b2bd3e4e3af49ec9.
unistd.hh is dead.
2015-05-19 15:27:30 -04:00
Jeroen Vermeulen
eca5824100 Remove trailing whitespace in C++ files. 2015-04-30 12:05:11 +07:00
Kenneth Heafield
6626d59cfc Less error-like complaint when substituting fallback discounts / Matthias Huck 2015-03-31 21:51:38 -04:00
Jeroen Vermeulen
b3c0a29044 Modernize "C" includes in lm.
This is one of those little chores in managing a long-lived C++
project: standard C headers like stdio.h and math.h now have their own
place in the C++ standard as resp. cstdio, cmath, and so on.  In this
branch the #include names are updated for the lm/ subdirectory.

C++11 adds cstdint, but to support compilation with the previous
standard, that change is left for later.
2015-03-28 20:28:45 +07:00
Kenneth Heafield
206d0c9698 Make NPLM wrapper compile again. Fixes #102. 2015-03-26 11:57:41 -04:00
Kenneth Heafield
8b323abbca KenLM 240ea65a021574261a38d45eb68143f26ad177e5 2015-03-25 10:40:21 -04:00
Kenneth Heafield
769c19d10c KenLM a6d57501dcac95a31719a8628f6cbd288f6741e2 including Marcin's fixed pruning 2015-01-22 11:42:46 -05:00
Rico Sennrich
4a94a41310 compatibility with NPLM 0.3 (fork on https://github.com/rsennrich/nplm)
breaks compatibility with older versions of NPLM, and older models.
2014-11-17 15:12:18 +00:00
Kenneth Heafield
36da8d1e0c KenLM 370f97fa549f02e162a3a0f17bf3ad6cce2c3813 2014-10-08 08:42:53 -04:00
Kenneth Heafield
02ab8f5102 Bugfix / Stephan Peitz and more paranoid error checking 2014-08-27 23:23:39 -04:00
Marcin Junczys-Dowmunt
061aac9e3b First attempt at fixing missing-backoffs bug in lmplz 2014-07-27 20:35:15 +02:00
Kenneth Heafield
c83c5a3ee6 D'ph forgot to copy util 2014-07-19 06:54:01 +08:00
Hieu Hoang
07d325876c dump_counts doesn't compile 2014-07-18 21:30:49 +01:00
Kenneth Heafield
ff1d5ca0d6 KenLM 8386117fdc026b8532adac85c5f921388754d569 2014-07-18 22:24:28 +08:00
Rico Sennrich
eb5336ad9c thread-safety, precomputation and caching for NeuralLMWrapper
doesn't work with default nplm, but with the fork at https://github.com/rsennrich/nplm
2014-07-17 16:50:08 +01:00
Rico Sennrich
0d8d77e3da add nplm wrapper from Kenneth's kenlm repo 2014-07-17 16:10:30 +01:00
Kenneth Heafield
6b22097cbe KenLM e94d0b3ea9082d8aa443e15e33aee6fb9c2af990 including possible deadlock fix 2014-06-22 05:54:03 -07:00
Kenneth Heafield
6d9173ba72 KenLM f81d02792087a837ea17e6ce2b33f9b7aaecca68 should fix segfault with ArrayTrie 2014-06-04 16:03:39 -07:00
Kenneth Heafield
dd03f9fb69 KenLM 5a7efd8fe1db88ee0a9f7e9479b24ac3ca348221 with Hieu's patch to exception.hh 2014-06-02 10:29:40 -07:00
Kenneth Heafield
14e02978fc KenLM 5cc905bc2d214efa7de2db56a9a672b749a95591
Avoid unspecified behavior of mmap when a file is resized reported by Christian Hardmeier
Fixes for Mavericks and a workaround for Boost's broken semaphore
Clean clang compile (of kenlm)

Merged some of 744376b3fb but also undid some of it because it was just masking a fundaemntal problem with pread rather than working around windows limitations
2014-01-27 16:51:35 -08:00
jiejiang
5f1217d793 merged upstream with origin for mingw 2014-01-15 18:16:56 +00:00
Kenneth Heafield
732a1b0744 KenLM 85c82bd, revamp Moses timer to have more precision 2014-01-01 13:19:06 -08:00
jiejiang
744376b3fb moses windows build, with some TODO list 2013-12-18 20:15:39 +00:00
Kenneth Heafield
5cca6fafcb KenLM 6f7913cc7ca0f7672c6d899358365f047a742bbb
Mostly fixes from Tetsuo Kiso and Jonathan Graehl
2013-09-09 10:43:20 +01:00
Lane Schwartz
226031ecf6 Checking in examples of CMake files.
The example in contrib works, as does the one in biconcor.

The examples in lm and moses don't work at the moment,
but checking them in anyway as a starting point
for any potential work on them next week during the
MT Marathon.
2013-09-06 13:39:46 -04:00
Kenneth Heafield
1a8977d890 EmptyContextState -> NullContextState / Nadir 2013-07-01 16:22:25 +01:00
Kenneth Heafield
794867c555 KenLM 6b4a1c7940a36026de1d96693ccb6ec0f16de8dc 2013-06-24 16:05:47 +01:00
Kenneth Heafield
59bd7deb4b Tetsuo's two patches 2013-05-29 05:30:39 -04:00
Kenneth Heafield
50652382e9 KenLM 10ddf7d923355b35a7de9a5219673eca9e18be98 except Hieu's slow string_piece_hash 2013-05-19 10:12:06 -04:00
Xiang Li
1b63277e66 Update search_trie.hh
There is no array compression header for order 2 models
2013-05-16 10:27:46 +08:00
Kenneth Heafield
03b077364a KenLM 31a6644 resizable probing hash table, build fixes 2013-01-24 12:07:46 +00:00
Kenneth Heafield
22bf1c77e9 Ales Tamchyna patch for ancient Boost. 2013-01-23 13:28:15 +00:00
Kenneth Heafield
e2fc306231 KenLM 58da33: compiler warnings, correctly handle bare K, kill HAVE_BOOST 2013-01-22 20:23:35 +00:00
Kenneth Heafield
cfe7d00ea2 Compiler warnings, rename to ${binary}_main.cc 2013-01-20 13:43:19 +00:00
Kenneth Heafield
a52d97f99f Cross platform fixes 2013-01-20 12:25:58 +00:00
Hieu Hoang
8a3b93c72e compile error on mac 2013-01-20 11:52:18 +00:00
Kenneth Heafield
a35299497f Tolerate odd characters in corpora. 2013-01-19 13:35:00 +00:00
Kenneth Heafield
fc5868d0ff KenLM df5be22 lmplz for estimation 2013-01-18 15:59:51 +00:00