guillaume-be/rust-bert
1
1
Fork 0
mirror of https://github.com/guillaume-be/rust-bert.git synced 2024-08-17 00:20:37 +03:00
Code Issues Packages Projects Releases Wiki Activity
1,109 Commits 5 Branches 0 Tags 5.8 MiB
main
Commit Graph

2 Commits

Author SHA1 Message Date
guillaume-be
1f4d344668
Change generate return type to Result (#437) 
* - Changed the return type of generate method to be `Result`, removed fallible unwraps

* Fix doctests
 2023-12-04 17:58:21 +00:00
guillaume-be
fd1e66b1c7
Support for HF Tokenizers (#408) 
* tokenizers output type conversion

* WIP hf Tokenizers support (2)

* finalize interface methods for hf tokenizers

* Addition of GPT2 example with hf tokenizers

* Made hf-tokenizers optional, added doc for HFTokenizer

* Addition of tests for hf tokenizers, addition to CI

* Updated changelog, extended documentation

* Fix Clippy warnings
 2023-08-13 11:09:02 +01:00