Taku Kudo
27103a9e02
Merge pull request #701 from monologg/docs/typo
...
Fix typo in doc
2022-02-16 10:16:10 +09:00
Taku Kudo
4c8a377627
Merge pull request #721 from matteobaccan/patch-1
...
Fixed typo error
2022-02-16 10:14:50 +09:00
Matteo Baccan
dbc83417bb
Fixed typo error
...
Fixed unknow -> unknown
2022-01-25 19:49:26 +01:00
xiefangqi
f144921644
fix address sanitizers on clang problem
2021-12-21 19:52:47 +08:00
zy566
d972932e33
Fix typo in cmakelist
2021-10-28 14:28:26 -07:00
monologg
38278a0098
Fix typo in doc
2021-10-03 07:09:36 +00:00
Alexis Praga
142662e585
Add FreeBSD support
...
<endian.h> should be <sys/endian.h> instead.
2021-09-10 22:06:15 +02:00
Taku Kudo
8420f21790
Merge pull request #669 from reiyw/patch-1
...
Fix link to nfkc.tsv in normalization.md
2021-07-03 00:44:09 +09:00
Ryo Takahashi
cefb97b59d
Fix link to nfkc.tsv in normalization.md
2021-07-02 16:06:52 +09:00
Taku Kudo
995460307e
Merge pull request #666 from felixdae/patch-1
...
Update options.md
2021-07-01 11:33:29 +09:00
Zhangqiu Yu
31505e065a
Update options.md
...
fix doc
2021-06-30 23:30:19 +08:00
Taku Kudo
d8711f55d9
Merge pull request #664 from danieldk/remove-build-path
...
Strip build directory prefix in the __FILE__ macro
2021-06-18 01:10:09 +09:00
Taku Kudo
cbfc6b3c2c
updated *.tsv file.
2021-06-18 01:06:29 +09:00
Daniël de Kok
cab2e3c486
Strip build directory prefix in the __FILE__ macro
...
Various macros use __FILE__ to embed the source file name in
errors/logs. However, the full path is encoder, making reproducible
builds harder.
This change uses the -fmacro-prefix-map of GCC >= 8.0 and clang >= 10.0
to remove the source directory prefix. Since only the prefix is removed,
the relevant source file names are still included.
Background:
https://reproducible-builds.org/docs/build-path/
2021-06-17 17:23:10 +02:00
Taku Kudo
62eafec17a
updated the comment.
2021-06-17 19:32:05 +09:00
Taku Kudo
a61f985cde
fixed build error.
2021-06-17 13:08:46 +09:00
Taku Kudo
f4f8309cfe
fixed build error.
2021-06-17 12:53:39 +09:00
Taku Kudo
705cdc1194
added endian.h
2021-06-17 11:11:02 +09:00
Taku Kudo
cc2d2c8d31
fixed build error
2021-06-17 02:31:48 +09:00
Taku Kudo
d1e33330a7
fixed build error
2021-06-17 02:07:59 +09:00
Taku Kudo
5c194ecc34
fixed link error
2021-06-17 01:56:17 +09:00
Taku Kudo
fab966ad21
sync from internal
2021-06-16 19:04:14 +09:00
Taku Kudo
3a5bc5815b
Revert "sync from internal"
...
This reverts commit 05db0894d8
.
2021-06-16 14:51:52 +09:00
Taku Kudo
05db0894d8
sync from internal
2021-06-16 01:12:46 +09:00
Taku Kudo
897fe9d50c
fixed build error on mac
2021-06-15 18:56:02 +09:00
Taku Kudo
faa76a03e7
fixed build error on mac
2021-06-15 14:27:56 +09:00
Taku Kudo
687885bd98
update the mac pywhell script to support py3.5
2021-06-15 13:21:20 +09:00
Taku Kudo
8a176d8ca6
use latest ubuntu image
2021-06-14 17:17:44 +09:00
Taku Kudo
6a9706c9d7
Merge pull request #658 from George-Ogden/patch-1
...
Fixed mistake in README
2021-05-26 14:45:33 +09:00
George Ogden
7316ef15d2
Fixed mistake in README
2021-05-25 12:41:39 +01:00
Taku Kudo
6256ef2438
Merge pull request #651 from wanchichen/master
...
Fix typo in readme
2021-05-08 10:31:08 +09:00
William Chen
c9ea0f3e08
fix typo
2021-04-22 19:46:42 -04:00
Taku Kudo
351600c297
Update README.md
2021-04-20 11:48:37 +09:00
Taku Kudo
2f4da31b56
Merge pull request #642 from AdolfVonKleist/restore_boundaries
...
Restore the sentence boundary marker insertion for the unigram model trainer
2021-04-20 11:39:29 +09:00
Taku Kudo
35cc60b39b
Merge pull request #632 from brettfazio/patch-1
...
Fix typo in readme
2021-04-20 11:38:19 +09:00
joe
9f544a66ff
Restore the sentence boundary marker insertion for the unigram
...
trainer. Dramatically speeds up training time.
2021-03-22 17:26:20 +00:00
Brett Fazio
72be462dcf
Fix typo in readme
2021-02-25 22:20:25 -05:00
Taku Kudo
bc53923a91
Merge pull request #630 from sarubi/zwj_fix
...
Prevent Zero Width Joiner replaced with whitespace
2021-02-26 01:25:37 +09:00
Taku Kudo
d429804123
Merge pull request #623 from erasaur/master
...
add spm proto headers to install
2021-02-24 01:34:46 +09:00
Sarubi
c970dedd8f
Removed codes where Zero Width Joiner replaced with whitespace.
2021-02-23 20:47:25 +05:30
mark
f78087f696
only install proto headers if not using builtin proto
2021-02-10 10:59:56 -08:00
mark
e03761f36d
add spm proto headers to install
2021-02-10 00:14:40 -08:00
Taku Kudo
ba7e11a17f
Merge pull request #617 from mhsmith/master
...
Add missing #include for BYTE_ORDER
2021-01-13 02:23:24 +09:00
Malcolm Smith
4bc9ae0df2
Add missing #include for BYTE_ORDER
2021-01-12 13:43:28 +00:00
Taku Kudo
d9a0892980
add ubuntu focal image to travis target
2021-01-10 22:23:07 +09:00
Taku Kudo
0e6dfbf86e
fixed python module to check the id range.
2021-01-10 14:49:13 +09:00
Taku Kudo
3589bfbebd
fixed build break.
2021-01-10 00:26:09 +09:00
Taku Kudo
8083d4f6e7
checks the range of id in Decode method
2021-01-09 16:58:41 +09:00
Taku Kudo
0e03b57a33
change the type of input_sentence_size from int32 to uint64
2021-01-09 01:44:49 +09:00
Taku Kudo
fe046e1c97
change the type of input_sentence_size from int32 to uint64
2021-01-08 16:40:03 +09:00