guillaume-be/rust-bert
1
1
Fork 0
mirror of https://github.com/guillaume-be/rust-bert.git synced 2024-08-16 16:10:25 +03:00
Code Issues Packages Projects Releases Wiki Activity
1,109 Commits 5 Branches 0 Tags 5.8 MiB
main
Commit Graph

266 Commits

Author SHA1 Message Date
Guillaume B
c40a218b37 Initial implementation of scores output 2021-06-15 19:09:20 +02:00
Guillaume B
5907b7d954 Updated documentation, cleaned examples, added integration tests 2021-06-06 13:01:33 +02:00
Guillaume B
d401fea891 Updated tests and docstrings 2021-06-03 10:17:52 +02:00
Guillaume B
a9518c94fa Addition of GPT-Neo 2.7B pretrained weights, added example, updated changelog 2021-05-06 16:57:12 +02:00
Guillaume B
71c196b0ce Updated documentation, fixed Clippy warnings 2021-04-08 14:50:02 +02:00
Guillaume B
4c4ef41a80 Fixed Marian to be compatible with BART refactoring 2021-03-27 18:36:42 +01:00
Guillaume B
5e6b84a7a0 Updated BART embeddings for compatibility with Marian model 2021-03-27 18:03:45 +01:00
Guillaume B
32537da610 Fixed attention value calculation 2021-03-27 16:51:10 +01:00
Guillaume B
b6f722984b updated attention reshape method 2021-03-27 16:05:06 +01:00
Guillaume B
c378b02bbe BART refactoring (initial draft) 2021-03-27 15:55:25 +01:00
Guillaume B
d5321a8940 Updated README and documentation 2021-03-20 17:03:21 +01:00
Guillaume B
65da7afbb6 Update punctuation POS tags with low score 2021-03-15 16:41:00 +01:00
Guillaume B
02819c0a71 initial version of POS tagging pipeline 2021-03-12 09:30:21 +01:00
Guillaume B
6a6bd74533 Fixed Clippy warnings 2021-02-21 08:56:36 +01:00
Guillaume B
545d52ec9d Longformer integration tests 2021-02-16 10:16:09 +01:00
Guillaume B
25392ef19b Reverted DistilBertForQuestionAnswering example 2021-02-14 10:53:00 +01:00
Guillaume B
e9102de7be Fixed LongformerForQuestionAnswering, fixed QA start/end masking 2021-02-14 10:52:11 +01:00
Guillaume B
83afd4ce54 Merge remote-tracking branch 'origin/master' into prophetnet_implementation 
# Conflicts:
#	src/pipelines/generation_utils.rs
#	src/pipelines/summarization.rs
 2021-01-25 18:25:36 +01:00
Guillaume B
43512134ae Refactored the text generation methods for models to the model definition file 2021-01-23 09:03:27 +01:00
Guillaume B
930a6b4f42 Addition of integration tests for ProphetNet 2021-01-20 17:16:48 +01:00
Guillaume B
9c0edeebf1 Fix and validation of ProphetNet summarization 2021-01-18 12:41:58 +01:00
Guillaume B
732a8e1272 ProphetNetModel output validated 2021-01-16 10:50:27 +01:00
Guillaume B
dbd7469807 Minor fixes for ProphetNetEncoder 2021-01-16 10:30:47 +01:00
Guillaume B
f8ed5079c1 initial commit for ProphetNet 2021-01-07 10:41:50 +01:00
Guillaume B
bf00f90d55 Working MobileBert masked LM 2020-12-19 15:46:33 +01:00
Guillaume B
78eb970679 Updated position ids computation 2020-12-19 15:29:13 +01:00
Guillaume B
0c1af736e2 Addition of MobileBertForMaskedLM 2020-12-19 14:20:32 +01:00
Guillaume B
fed8908a83 Addition of MobileBERT config and resources 2020-12-05 10:01:16 +01:00
Guillaume B
594ac71102 Addition of integration tests for Reformer model 2020-11-15 10:32:10 +01:00
Guillaume B
f3eb2e189f Fixed masking for LSH attention, added pipeline and example 2020-11-15 09:27:06 +01:00
Guillaume B
4678011d29 Working forward pass (long output, no cache) 2020-11-13 17:34:15 +01:00
Guillaume B
3995a3ee92 Working forward pass (short output, no cache) 2020-11-13 16:58:03 +01:00
Guillaume B
aea41062ef Reformer forward pass running (short sequence, no history) 2020-11-13 16:19:47 +01:00
Guillaume B
8ec1704ff1 Fixed weights loading error 2020-11-10 08:24:38 +01:00
Guillaume B
ed346d34ac Definition and loading of Reformer config 2020-11-01 09:40:53 +01:00
Guillaume B
6ebfb7a98c Created a specific TextGenerationConfig for the TextGenerationModel 2020-10-30 18:26:39 +01:00
Guillaume B
4b11d7b76c Updated documentation, kept GenerateConfig unchanged for backward compatibility 2020-10-28 17:37:51 +01:00
Guillaume B
3d4bb6535d Updated integration tests 2020-10-27 18:52:48 +01:00
Guillaume B
297c9d2c8d Addition of TextGenerationModel 2020-10-27 18:22:34 +01:00
Guillaume B
ec766b1714 Addition of integration test for T5-based summarization 2020-10-25 09:10:56 +01:00
Guillaume B
ef8c8f84db Extended summarization pipeline to align with Enum unpacking, added support for T5 summarization 2020-10-25 09:04:30 +01:00
Guillaume B
3c6f35ea06 Merge branch 'master' into updated_tokenizers 
# Conflicts:
#	src/pipelines/zero_shot_classification.rs
 2020-10-23 16:47:11 +02:00
Guillaume B
a22c89b010 Fixed greedy generation method 2020-10-13 18:52:09 +02:00
Guillaume B
fbff61507a Updated README 2020-10-13 07:14:00 +02:00
Guillaume B
97ee8ee928 Generalization of input types for pipelines 2020-10-11 16:18:44 +02:00
Guillaume B
9e1887e0f6 Compatibility with updated tokenizers crate 2020-10-10 12:46:43 +02:00
Guillaume B
6dde3241a2 initial docs for XLNet 2020-10-01 19:13:04 +02:00
Guillaume B
2ce687cd1e Addition of integration tests for XLNet 2020-09-24 18:49:05 +02:00
Guillaume B
fb5a35cf4f Addition of entity chunking 2020-09-22 21:07:28 +02:00
Guillaume B
0fe17cacb4 Fixed beam_search generation with XLNet 2020-09-20 12:02:17 +02:00
Guillaume B
95d3bf5a53 Updated generation for XLNet 2020-09-19 17:31:56 +02:00
Guillaume B
536b0b02db updated relative shift 2020-09-18 17:41:28 +02:00
Guillaume B
7db7b2b34c updated weights shape 2020-09-18 17:12:07 +02:00
Guillaume B
abe177c1c6 XLNetModel forward pass implemented 2020-09-18 16:41:29 +02:00
Guillaume B
6f907ff995 Merge branch 'master' of https://github.com/guillaume-be/rust-bert into xlnet_implementation 2020-09-16 17:19:22 +02:00
Guillaume B
b9e221f322 Merge branch 'master' into xlnet_implementation 2020-09-16 17:17:00 +02:00
Guillaume B
93277439e5 Updated documentation and RoBERTa model ouputs 2020-09-15 17:45:39 +02:00
guillaume-be
59e6cb54f0 updated ALBERT documentation 2020-09-14 18:33:58 +02:00
Guillaume B
47d9a1017d Updated T5 (clippy) 2020-09-13 12:25:22 +02:00
Guillaume B
6cef45787d Updated generation (clippy) 2020-09-13 11:08:49 +02:00
Guillaume B
5aa1e635ba Updated BERT and DistilBERT (clippy) 2020-09-12 17:12:49 +02:00
Guillaume B
7e9d6d7e39 Updated Bart & Marian (clippy warnings) 2020-09-12 16:17:58 +02:00
Guillaume B
daa6dba2d2 Updated Albert (clippy warnings) 2020-09-12 15:11:56 +02:00
Evan Pete Walsh
34e061c0de
add cached-path file utils (#74) 
* add cached-path

* updates

* fix typo, add deprecation notice

* fix up

* update cached-path dependency

* update cached-path

* remove usage of 'download_resource' in examples

* clean up
 2020-09-12 13:47:23 +02:00
Guillaume B
64320b55c8 Initial commit for XLNetRelativeAttention 2020-09-10 18:10:54 +02:00
Guillaume B
eabab31aa3 Definition of XLNet configuration 2020-09-08 16:47:45 +02:00
Guillaume B
6e9a0310e3 Updated examples and tests for zero shot classification 
Multilabel example for zero-shot classification
 2020-09-06 10:39:18 +02:00
Guillaume B
080efc6791 Formatting update 2020-09-06 10:28:32 +02:00
Guillaume B
5a833c7caa Moved csv to dev dependencies 2020-09-06 10:27:29 +02:00
Guillaume B
b52b0cb005 Addition of integration tests for zero-shot classification 2020-09-05 15:02:34 +02:00
Guillaume B
56747f7033 reversed error handling for pipelines to maintain backward compatibility 2020-09-01 18:00:38 +02:00
Guillaume B
71121e56d8 Addition of BART as an option for sequence classification pipelines, updated return type of pipelines to be a Result 2020-09-01 17:52:49 +02:00
Guillaume B
c6886096d2 Updated summarization output resulting from tokenization updates 2020-08-30 10:18:38 +02:00
Guillaume B
50ee1a1e4e Updated for additional tokenizer settings & configuration validity check 2020-08-30 09:35:38 +02:00
Guillaume B
3200026c0b Updated to make prefix space optional for models using Roberta tokenizer 2020-08-29 15:58:03 +02:00
guillaume-be
325f97b676 Improved error handling 2020-08-22 12:00:02 +02:00
Sacha Arbonel
9205df49b4
replace failure with anyhow (#69) 
* replace failure with anyhow

* formatting update with `fmt`

Co-authored-by: guillaume-be <guillaume.becquin@gmail.com>
 2020-08-12 13:39:04 +02:00
Guillaume B
5b00074ab5 Addition of integration test for XLMRoberta for NER (German) 2020-07-23 19:34:52 +02:00
Guillaume B
c9262a310a Addition of integration tests for BERT and RoBERTa QA 2020-07-12 09:49:22 +02:00
Guillaume B
ec54a213ad Addition of Albert to pipeline options 2020-07-12 09:35:06 +02:00
Guillaume B
e6938e1dcb Addition of BERT option for QA 2020-07-08 21:59:31 +02:00
Guillaume B
154fe2a4d0 Added integration tests for T5 and dependencies download 2020-07-07 19:29:45 +02:00
Guillaume B
1e865ef6eb Addition of T5_BASE dependency and download script 2020-07-07 18:15:15 +02:00
Guillaume B
a6b11d5449 Implementation of generation for T5 2020-07-07 18:07:31 +02:00
Guillaume B
7938abc76d Implementation of T5ForConditionalGeneration 2020-07-06 17:43:27 +02:00
Guillaume B
857b4bf7a5 Updated T5 attention, base model output validated w/o cache 2020-07-05 17:49:27 +02:00
Guillaume B
fde80e4470 Updated T5 model, weights loading 2020-07-05 12:27:26 +02:00
Guillaume B
8e7696ff27 T5 dependencies download and configuration setup 2020-06-30 19:11:40 +02:00
Guillaume B
23310dfc1e Added documentation, updated tests & doctests 2020-06-28 11:04:36 +02:00
Guillaume B
68def5a912 Addition of tests for conversation 2020-06-28 09:31:41 +02:00
Guillaume B
caa2ba9108 Updated conversation manager and example 2020-06-28 09:09:34 +02:00
Guillaume B
dbdc75a9bb Fixed generation bug, enable multiple batched conversations 2020-06-27 18:44:14 +02:00
Guillaume B
372c357463 Update of conversation manager and conversations 2020-06-27 18:27:43 +02:00
Guillaume B
c946791b58 Initial draft for conversation and conversation manager 2020-06-27 10:51:59 +02:00
Guillaume B
f9895752bd Addition of history tokens for multi-turn conversation 2020-06-27 09:32:28 +02:00
Guillaume B
c045b3212f Updated generation process to allow for direct ids input 2020-06-25 21:32:45 +02:00
Guillaume B
9081bc3318 Initial single turn conversation 2020-06-25 19:32:36 +02:00
Guillaume B
47e36c4e8c Code formatted using rustfmt 2020-06-23 16:54:46 +02:00
Guillaume B
b5b6f68410 addition of albert dependencies download 2020-06-22 20:59:36 +02:00
Guillaume B
d49a1e0935 Updated documentation, addition of weights download for ALBERT 2020-06-22 20:51:01 +02:00