rust-bert/examples
guillaume-be 540c9268e7
ONNX Support (#346)
* Fixed Clippy warnings

* Revert "Shallow clone optimization (#243)"

This reverts commit ba584653bc.

* updated dependencies

* tryouts

* GPT2 tryouts

* WIP GPT2

* input mapping

* Cache storage

* Initial GPT2 prototype

* Initial ONNX Config and decoder implementation

* ONNXDecoder first draft

* Use Decoders in example

* Automated tch-ort conversion, decoder implementation

* ONNXCausalDecoder implementation

* Refactored _get_var_store to be optional, added get_device to gen trait

* updated example

* Added decoder_start_token_id to ConfigOption

* Addition of ONNXModelConfig, make max_position_embeddigs optional

* Addition of forward pass function for ONNXModel

* Working ONNX causal decoder

* Simplify tensor conversion

* refactor translation to facilitate ONNX integration

* Implementation of ONNXEncoder

* Implementation of ONNXConditionalGenerator

* working ONNXCausalGenerator

* - Reworked model resources type for pipelines and generators

* Aligned ONNXConditionalGenerator with other generators to use GenerateConfig for creation

* Moved force_token_id_generation to common utils function, fixed tests, Translation implementation

* generalized forced_bos and forced_eos tokens generation

* Aligned the `encode_prompt_text` method across language models

* Fix prompt encoding for causal generation

* Fix prompt encoding for causal generation

* Support for ONNX models for SequenceClassification

* Support for ONNX models for TokenClassification

* Support for ONNX models for POS and NER pipelines

* Support for ONNX models for ZeroShotClassification pipeline

* Support for ONNX models for QuestionAnswering pipeline

* Support for ONNX models for MaskedLM pipeline

* Added token_type_ids , updated layer cache i/o parsing for ONNX pipelines

* Support for ONNX models for TextGenerationPipeline, updated examples for remote resources

* Remove ONNX zero-shot classification example (lack of correct pretrained model)

* Addition of tests for ONNX pipelines support

* Made onnx feature optional

* Fix device lookup with onnx feature enabled

* Updates from main branch

* Flexible tokenizer creation for M2M100 (NLLB support), make NLLB test optional du to their size

* Fixed Clippy warnings

* Addition of documentation for ONNX

* Added documentation for ONNX support

* upcoming tch 1.12 fixes

* Fix merge conflicts

* Fix merge conflicts (2)

* Add download libtorch feature to ONNX tests

* Add download-onnx feature

* attempt to enable onnx download

* add remote resources feature

* onnx download

* pin ort version

* Update ort version
2023-05-30 07:20:25 +01:00
..
async-sentiment.rs Tokenizer special token map update (#330) 2023-01-30 17:53:18 +00:00
buffer_resource.rs ONNX Support (#346) 2023-05-30 07:20:25 +01:00
codebert.rs ONNX Support (#346) 2023-05-30 07:20:25 +01:00
conversation.rs Tokenizer special token map update (#330) 2023-01-30 17:53:18 +00:00
generation_gpt2.rs Tokenizer special token map update (#330) 2023-01-30 17:53:18 +00:00
generation_gpt_neo.rs ONNX Support (#346) 2023-05-30 07:20:25 +01:00
generation_gptj.rs ONNX Support (#346) 2023-05-30 07:20:25 +01:00
generation_reformer.rs ONNX Support (#346) 2023-05-30 07:20:25 +01:00
generation_xlnet.rs ONNX Support (#346) 2023-05-30 07:20:25 +01:00
keyword_extraction.rs Keyword/Keyphrase extraction (#295) 2022-11-13 08:51:10 +00:00
masked_language.rs ONNX Support (#346) 2023-05-30 07:20:25 +01:00
named_entities_recognition.rs Tokenizer special token map update (#330) 2023-01-30 17:53:18 +00:00
natural_language_inference_deberta.rs Introduce in-memory resource abstraction (#375) 2023-05-26 18:23:28 +01:00
onnx-masked-lm.rs ONNX Support (#346) 2023-05-30 07:20:25 +01:00
onnx-question-answering.rs ONNX Support (#346) 2023-05-30 07:20:25 +01:00
onnx-sequence-classification.rs ONNX Support (#346) 2023-05-30 07:20:25 +01:00
onnx-text-generation.rs ONNX Support (#346) 2023-05-30 07:20:25 +01:00
onnx-token-classification.rs ONNX Support (#346) 2023-05-30 07:20:25 +01:00
onnx-translation.rs ONNX Support (#346) 2023-05-30 07:20:25 +01:00
part_of_speech_tagging.rs Tokenizer special token map update (#330) 2023-01-30 17:53:18 +00:00
question_answering_bert.rs ONNX Support (#346) 2023-05-30 07:20:25 +01:00
question_answering_longformer.rs ONNX Support (#346) 2023-05-30 07:20:25 +01:00
question_answering_squad.rs Updated documentation, cleaned examples, added integration tests 2021-06-06 13:01:33 +02:00
question_answering.rs Tokenizer special token map update (#330) 2023-01-30 17:53:18 +00:00
sentence_embeddings_local.rs Tokenizer special token map update (#330) 2023-01-30 17:53:18 +00:00
sentence_embeddings.rs Tokenizer special token map update (#330) 2023-01-30 17:53:18 +00:00
sentiment_analysis_fnet.rs ONNX Support (#346) 2023-05-30 07:20:25 +01:00
sentiment_analysis_sst2.rs Fixed Clippy warnings (#204) 2021-12-09 09:33:27 +01:00
sentiment_analysis.rs Tokenizer special token map update (#330) 2023-01-30 17:53:18 +00:00
sequence_classification_multilabel.rs Tokenizer special token map update (#330) 2023-01-30 17:53:18 +00:00
sequence_classification.rs Tokenizer special token map update (#330) 2023-01-30 17:53:18 +00:00
summarization_bart.rs ONNX Support (#346) 2023-05-30 07:20:25 +01:00
summarization_pegasus.rs ONNX Support (#346) 2023-05-30 07:20:25 +01:00
summarization_prophetnet.rs ONNX Support (#346) 2023-05-30 07:20:25 +01:00
summarization_t5.rs ONNX Support (#346) 2023-05-30 07:20:25 +01:00
token_classification.rs ONNX Support (#346) 2023-05-30 07:20:25 +01:00
translation_builder.rs Tokenizer special token map update (#330) 2023-01-30 17:53:18 +00:00
translation_m2m100.rs ONNX Support (#346) 2023-05-30 07:20:25 +01:00
translation_marian.rs ONNX Support (#346) 2023-05-30 07:20:25 +01:00
translation_mbart.rs ONNX Support (#346) 2023-05-30 07:20:25 +01:00
translation_t5.rs ONNX Support (#346) 2023-05-30 07:20:25 +01:00
zero_shot_classification.rs Tokenizer special token map update (#330) 2023-01-30 17:53:18 +00:00