Commit Graph

64 Commits

Author SHA1 Message Date
Kenneth Heafield
59bd7deb4b Tetsuo's two patches 2013-05-29 05:30:39 -04:00
Kenneth Heafield
50652382e9 KenLM 10ddf7d923355b35a7de9a5219673eca9e18be98 except Hieu's slow string_piece_hash 2013-05-19 10:12:06 -04:00
Xiang Li
1b63277e66 Update search_trie.hh
There is no array compression header for order 2 models
2013-05-16 10:27:46 +08:00
Kenneth Heafield
03b077364a KenLM 31a6644 resizable probing hash table, build fixes 2013-01-24 12:07:46 +00:00
Kenneth Heafield
22bf1c77e9 Ales Tamchyna patch for ancient Boost. 2013-01-23 13:28:15 +00:00
Kenneth Heafield
e2fc306231 KenLM 58da33: compiler warnings, correctly handle bare K, kill HAVE_BOOST 2013-01-22 20:23:35 +00:00
Kenneth Heafield
cfe7d00ea2 Compiler warnings, rename to ${binary}_main.cc 2013-01-20 13:43:19 +00:00
Kenneth Heafield
a52d97f99f Cross platform fixes 2013-01-20 12:25:58 +00:00
Hieu Hoang
8a3b93c72e compile error on mac 2013-01-20 11:52:18 +00:00
Kenneth Heafield
a35299497f Tolerate odd characters in corpora. 2013-01-19 13:35:00 +00:00
Kenneth Heafield
fc5868d0ff KenLM df5be22 lmplz for estimation 2013-01-18 15:59:51 +00:00
Kenneth Heafield
5f7b91e702 Workaround for bjam inverting argument order 2013-01-18 13:40:56 +00:00
Kenneth Heafield
a7731ce432 KenLM 014bced6 2013-01-17 11:58:58 +00:00
Kenneth Heafield
1530ae4f5f Fix state comparison (impacted 32-bit) 2013-01-09 13:15:04 +00:00
Kenneth Heafield
f9ee7ae4b3 KenLM 0e5d259 including read_compressed fix 2013-01-04 21:02:47 +00:00
Ales Tamchyna
35cb7e68be correct parameter in comment for KenLM maximum order 2012-11-15 16:06:51 +01:00
Kenneth Heafield
d653d45abc KenLM 85e5fe4: mostly fixes for cygwin/windows 2012-11-15 12:00:15 +00:00
Kenneth Heafield
d4f433c7f4 Put core Moses into one library libmoses. 2012-11-12 18:07:53 +00:00
Kenneth Heafield
14392acc8f KenLM 36b746 including -w mmap <unk> offset fix 2012-11-04 15:36:42 -05:00
Kenneth Heafield
9a1ac30889 Increase paranoia about --kenlm-max-order and --max-factors 2012-10-31 14:48:29 +00:00
Kenneth Heafield
4b98b0937c Add bzip and xz input for FilePiece KenLM 06f78db2 2012-10-30 19:33:22 +00:00
Kenneth Heafield
5f67527143 Bugfix vocabulary corruption on building 2012-10-19 12:00:10 +01:00
Kenneth Heafield
791a7b68fc Make max-factors a compile option and add recompilation safety 2012-10-19 10:16:45 +01:00
Kenneth Heafield
de5276630b Restore kenlm max order 2012-10-19 10:00:01 +01:00
Kenneth Heafield
0eb98df0fe Submodules are evil. Undo it. 2012-10-15 13:58:33 +01:00
Kenneth Heafield
4faab6c68f Extract kenlm into a submodule, referencing lazy 2012-09-24 18:30:51 +01:00
Kenneth Heafield
b317522563 Move max-order to lm directory and direct dependencies. 2012-08-18 12:07:53 -04:00
Kenneth Heafield
128829ebee Don't segfault trie building when there is no n-gram of a given order. Jon Clark. 2012-08-16 16:01:43 -04:00
Kenneth Heafield
6efb6c0ffe Throw exception for binary files with too large order / Ondrej Bojar 2012-08-15 11:08:08 -04:00
Lane Schwartz
a2afc98665 Add program to query the KenLM maximum n-gram order 2012-08-08 16:41:29 -04:00
Lane Schwartz
da5429318b KenLM maximum n-gram order can now be set via a compile-time flag 2012-08-08 16:22:13 -04:00
Kenneth Heafield
650fcd6919 Cast everything to double before BOOST_CHECK_CLOSE for Yvette 2012-08-06 08:31:02 -04:00
Kenneth Heafield
d5c2868f97 Fix slow backoff lookup skip path 2012-07-16 10:47:47 -04:00
U-Hieu-PC\Hieu
c7059e5e0b visual studio compiles but doesn't link 2012-07-11 10:54:21 +01:00
Hieu Hoang
f335047981 windows-compatible. Thanks to Mike Lagwig 2012-07-10 10:05:14 +01:00
Kenneth Heafield
c7e02eb677 KenLM e3b5c55910 including rest costs for probing 2012-06-28 10:58:59 -04:00
Hieu Hoang
465c5cbf97 move all executables into bin 2012-05-31 12:55:05 +01:00
Hieu Hoang
ef26388aff eclipse project 2012-05-28 17:29:46 +01:00
Kenneth Heafield
f8d88920a1 KenLM b1daeaf for clang 2012-05-05 00:55:46 -04:00
Kenneth Heafield
175b7aaf49 KenLM c1dba12
- Reject NaNs
- Fix ChartState hashing (unused in Moses)
- Expose CreateOrThrow
- Minor portability improvement in getopt
2012-03-11 13:47:38 -04:00
Kenneth Heafield
e48de47c23 KenLM 98814b2 including faster malloc-backed building and portability improvements 2012-02-28 13:58:00 -05:00
Hieu Hoang
5284d0b819 Merge pull request #9 from miska/master
Few compilation fixes
2012-02-03 01:25:33 -08:00
Kenneth Heafield
09061e53e8 Force kenutil to be linked with dependencies 2012-02-02 17:49:41 -05:00
Michal Hrusecky
60354d2d5f Fix library dependencies 2012-01-31 17:00:11 +01:00
Kenneth Heafield
d49cc6c9f8 Make lm constructor explicit 2012-01-14 17:07:31 +00:00
Kenneth Heafield
23c79b8169 Remove uncalled and broken function 2012-01-14 01:23:54 -05:00
Hieu Hoang
98fc6d9917 xcode 2012-01-08 14:36:54 +07:00
Hieu Hoang
a6229dd52f move some functions to .hh file for easier integration with other products 2011-12-20 16:44:45 +07:00
Kenneth Heafield
c53453b266 No more autotools (with bhaddow in the room) 2011-12-01 17:17:28 +00:00
Kenneth Heafield
e20a098a29 Get more passive-aggressive about IRSTLM and <s>. 2011-12-01 13:32:15 +00:00