Taku Kudo
|
f2fcd859b3
|
Fixes cross build yaml
|
2023-05-14 10:29:06 +00:00 |
|
Taku Kudo
|
6c901b0fb5
|
Fixes build test errors in big-endian machines
|
2023-05-14 09:54:52 +00:00 |
|
Taku Kudo
|
17f9c6bd2c
|
Fixes build test errors in big-endian machines
|
2023-05-14 09:53:35 +00:00 |
|
Taku Kudo
|
827591a0c5
|
Fixes build test errors in big-endian machines
|
2023-05-14 09:08:39 +00:00 |
|
Taku Kudo
|
3863f7648e
|
increases the max number of threads
|
2023-04-30 17:37:15 +00:00 |
|
Taku Kudo
|
25b64fc630
|
Fix the test error on windows
|
2023-04-28 06:20:50 +00:00 |
|
Taku Kudo
|
bb0b610fae
|
Fix the ULM training bugs
|
2023-04-27 17:32:57 +00:00 |
|
Taku Kudo
|
ba44ab1ca0
|
Fix bugs in the handling of duplicated bigrams
|
2023-04-24 07:25:10 +00:00 |
|
Taku Kudo
|
69d34c7171
|
prepare for v0.1.99
|
2023-04-15 06:33:01 +00:00 |
|
Taku Kudo
|
d9a2b216b1
|
Fix bugs the seed score computation.
|
2023-04-15 05:59:52 +00:00 |
|
Taku Kudo
|
518c57c335
|
build wheel from sdist for testing
|
2023-04-12 07:41:58 +00:00 |
|
Taku Kudo
|
fabfe3095b
|
build wheel from sdist for testing
|
2023-04-12 07:35:45 +00:00 |
|
Taku Kudo
|
d6e597b391
|
build wheel from sdist for testing
|
2023-04-12 07:24:31 +00:00 |
|
Taku Kudo
|
f2884a17e9
|
test loacl sdist build on github actions
|
2023-04-12 04:42:27 +00:00 |
|
Taku Kudo
|
609a2b7d88
|
test loacl sdist build on github actions
|
2023-04-12 02:17:27 +00:00 |
|
Taku Kudo
|
8fd5c6b587
|
test loacl sdist build on github actions
|
2023-04-12 01:43:39 +00:00 |
|
Taku Kudo
|
e07ebf74d7
|
support pretokenization in BPE mode.
|
2023-04-11 06:48:08 +00:00 |
|
Taku Kudo
|
119e58d97a
|
Fixes include path when using external protobuf
|
2023-04-10 10:15:46 +00:00 |
|
Taku Kudo
|
2b0713791a
|
fixes IS_BIGENDIAN macro places
|
2023-04-10 02:28:20 +00:00 |
|
Taku Kudo
|
e58bb684d0
|
add pretokenization_delimiter options. Initialize seed pieces more accurately.
|
2023-04-10 02:11:37 +00:00 |
|
Taku Kudo
|
6c9fd791cf
|
Merge pull request #845 from chris-ha458/patch-1
Update sentencepiece_python_module_example.ipynb
|
2023-04-09 17:13:58 +09:00 |
|
Chris Ha
|
9b53e211e8
|
Update sentencepiece_python_module_example.ipynb
fix typo
|
2023-04-08 23:26:13 +09:00 |
|
Taku Kudo
|
c032c261c2
|
automatically detect -latomic linker option
|
2023-04-05 00:01:15 +00:00 |
|
Taku Kudo
|
5489c0a56a
|
add -latomic in static linking
|
2023-04-04 17:26:29 +00:00 |
|
Taku Kudo
|
c945229958
|
updated set-output commands
|
2023-04-04 15:51:06 +00:00 |
|
Taku Kudo
|
799c025aea
|
creates sdist with build_sdist.sh
|
2023-04-04 05:05:44 +00:00 |
|
Taku Kudo
|
59d84babc9
|
Ubuntu 18.04 to 20.04 migration
|
2023-04-04 03:53:27 +00:00 |
|
Taku Kudo
|
f54d8ba070
|
includes the sentencepiece source files in python source package
|
2023-04-04 03:15:11 +00:00 |
|
Taku Kudo
|
d0d1066dbf
|
use /MD to build wheel package on windows
|
2023-04-03 18:18:29 +00:00 |
|
Taku Kudo
|
573cc39aab
|
make the error message more descriptive. null termnate string in Utf8ToWide
|
2023-04-03 02:24:52 +00:00 |
|
Taku Kudo
|
359c04397c
|
handle the exception of std::random_device
|
2023-04-02 18:56:19 +00:00 |
|
Taku Kudo
|
d4c58fc779
|
handle the exception of std::random_device
|
2023-04-02 18:23:42 +00:00 |
|
Taku Kudo
|
ba466a6bae
|
prepare for 0.1.98
|
2023-04-02 18:06:40 +00:00 |
|
Taku Kudo
|
c0766c9870
|
added option to /MT flag
|
2023-04-02 16:56:20 +00:00 |
|
Taku Kudo
|
7e0137c1ba
|
added option to /MT flag
|
2023-04-02 16:17:36 +00:00 |
|
Taku Kudo
|
8772159684
|
Merge pull request #837 from ilya-lavrenov/msvc-remove-static-runtime
Removed replacing of /MD with /MT for MSVC
|
2023-03-29 15:52:06 +09:00 |
|
Ilya Lavrenov
|
1983663e62
|
Removed replacing of /MD with /MT for MSVC
|
2023-03-26 22:10:01 +04:00 |
|
Taku Kudo
|
9ffb33a14c
|
Merge pull request #820 from juliusfrost/patch-2
Fix setup-python version not detected
|
2023-02-21 10:50:26 +09:00 |
|
Julius Frost
|
f2dacdff52
|
setup-python@v4 parity
|
2023-02-17 10:56:51 -05:00 |
|
Julius Frost
|
4de04ccd6a
|
Fix setup-python version not detected
|
2023-02-17 10:48:29 -05:00 |
|
Taku Kudo
|
9c211b6c2d
|
Merge pull request #819 from juliusfrost/patch-1
Add Python 3.11 builds
|
2023-02-18 00:00:20 +09:00 |
|
Julius Frost
|
4c2a713fbe
|
Use latest setup-python==4.5
|
2023-02-15 15:28:24 -05:00 |
|
Julius Frost
|
de2fabe5df
|
Update wheel.yml
|
2023-02-15 15:28:01 -05:00 |
|
Taku Kudo
|
c5a49eb5de
|
Merge pull request #808 from jacek-michalak/patch-1
Update README.md
|
2023-01-24 13:57:30 +09:00 |
|
jacek-michalak
|
14b67a4e05
|
Update README.md
Two unnecessary characters
|
2023-01-17 21:31:20 +01:00 |
|
Taku Kudo
|
31656da0c9
|
Merge pull request #799 from kyoto7250/fix_tutorial
fix the path in add_new_vocab.ipynb
|
2022-12-12 23:55:52 +09:00 |
|
kyoto7250
|
2ba0a5aae3
|
fix the path in add_new_vocab.ipynb
Because the location of the path is different from when it was committed
|
2022-12-12 15:39:18 +09:00 |
|
Taku Kudo
|
77a65e0230
|
Merge pull request #795 from A2va/master
CMake need endif
|
2022-11-29 11:57:18 +09:00 |
|
A2va
|
225fb19811
|
CMake need endif
|
2022-11-28 21:19:09 +01:00 |
|
Taku Kudo
|
f9dac76274
|
Merge pull request #793 from A2va/master
Disable shared build on windows
|
2022-11-28 12:30:38 +09:00 |
|