mirror of
https://github.com/browsermt/bergamot-translator.git
synced 2024-09-11 05:35:33 +03:00
Patch WASM artifacts to run optimized (wormhole enabled) inference (#68)
* A script to patch the wasm artifacts to use wormhole via APIs that instantiate WASM module * Updated README * Load just production ready models * Shallow clone bergamot-models repo since it has such a large history * Improved wasm test_page - test page can load all 5 language pairs - Use intgemm.alpha* models * Refactor the code that patches wasm artifacts to enable wormhole Co-authored-by: Andre Natal <anatal@gmail.com> Co-authored-by: Motin <motin@motin.eu>
This commit is contained in:
parent
34228d37bf
commit
12e9232066
@ -33,6 +33,10 @@ jobs:
|
||||
working-directory: build-wasm
|
||||
run: emmake make -j2
|
||||
|
||||
- name: Instantiate simd wormhole
|
||||
working-directory: build-wasm
|
||||
run: bash ../wasm/patch-artifacts-enable-wormhole.sh
|
||||
|
||||
- name: Check artifacts
|
||||
working-directory: build-wasm
|
||||
run: |
|
||||
|
15
README.md
15
README.md
@ -46,8 +46,8 @@ Bergamot translator provides a unified API for ([Marian NMT](https://marian-nmt.
|
||||
If you want to package bergamot project specific models, please follow these instructions:
|
||||
```bash
|
||||
mkdir models
|
||||
git clone https://github.com/mozilla-applied-ml/bergamot-models
|
||||
cp -rf bergamot-models/* models
|
||||
git clone --depth 1 --branch main --single-branch https://github.com/mozilla-applied-ml/bergamot-models
|
||||
cp -rf bergamot-models/prod/* models
|
||||
gunzip models/*/*
|
||||
```
|
||||
|
||||
@ -75,11 +75,16 @@ Bergamot translator provides a unified API for ([Marian NMT](https://marian-nmt.
|
||||
emmake make -j
|
||||
```
|
||||
|
||||
The artefacts (.js and .wasm files) will be available in `wasm` folder of build directory ("build-wasm" in this case).
|
||||
The wasm artifacts (.js and .wasm files) will be available in `wasm` folder of build directory ("build-wasm" in this case).
|
||||
|
||||
3. Enable SIMD Wormhole via Wasm instantiation API in generated artifacts
|
||||
```bash
|
||||
bash ../wasm/patch-artifacts-enable-wormhole.sh
|
||||
```
|
||||
|
||||
#### Recompiling
|
||||
As long as you don't update any submodule, just follow steps in `4.ii` to recompile.\
|
||||
If you update a submodule, execute following command before executing steps in `4.ii` to recompile.
|
||||
As long as you don't update any submodule, just follow steps in `4.ii` and `4.iii` to recompile.\
|
||||
If you update a submodule, execute following command before executing steps in `4.ii` and `4.iii` to recompile.
|
||||
```bash
|
||||
git submodule update --init --recursive
|
||||
```
|
||||
|
7
wasm/patch-artifacts-enable-wormhole.sh
Normal file
7
wasm/patch-artifacts-enable-wormhole.sh
Normal file
@ -0,0 +1,7 @@
|
||||
#!/bin/bash
|
||||
|
||||
echo "Patching wasm artifacts to enable wormhole via APIs that compile and instantiate wasm module"
|
||||
sed -i.bak 's/var result = WebAssembly.instantiateStreaming(response, info);/var result = WebAssembly.instantiateStreaming(response, info, {simdWormhole:true});/g' wasm/bergamot-translator-worker.js
|
||||
sed -i.bak 's/return WebAssembly.instantiate(binary, info);/return WebAssembly.instantiate(binary, info, {simdWormhole:true});/g' wasm/bergamot-translator-worker.js
|
||||
sed -i.bak 's/var module = new WebAssembly.Module(bytes);/var module = new WebAssembly.Module(bytes, {simdWormhole:true});/g' wasm/bergamot-translator-worker.js
|
||||
echo "Done"
|
@ -32,6 +32,9 @@
|
||||
<label>Choose the model to use</label>
|
||||
<input type="radio" name="modellang" value="enes"/><label>English to Spanish</label>
|
||||
<input type="radio" name="modellang" value="esen" checked/><label>Spanish to English</label>
|
||||
<input type="radio" name="modellang" value="eten" checked/><label>Estonian to English</label>
|
||||
<input type="radio" name="modellang" value="enet" checked/><label>English to Estonian</label>
|
||||
<input type="radio" name="modellang" value="ende" checked/><label>English to German</label>
|
||||
<input type="button" id="load" value="Load Model"/>
|
||||
</div>
|
||||
|
||||
@ -70,7 +73,7 @@ En consecuencia, durante el año 2011 se introdujeron 180 proyectos de ley que r
|
||||
// Set the Model Configuration as YAML formatted string.
|
||||
// For available configuration options, please check: https://marian-nmt.github.io/docs/cmd/marian-decoder/
|
||||
const modelConfig = `models:
|
||||
- /${languagePair}/model.${languagePair}.npz
|
||||
- /${languagePair}/model.${languagePair}.intgemm.alphas.bin
|
||||
vocabs:
|
||||
- /${vocabLanguagePair}/vocab.${vocabLanguagePair}.spm
|
||||
- /${vocabLanguagePair}/vocab.${vocabLanguagePair}.spm
|
||||
|
@ -1,8 +1,10 @@
|
||||
#!/bin/bash
|
||||
|
||||
echo "Start: Copying artifacts in local folder------"
|
||||
cp ../../build-wasm/wasm/bergamot-translator-worker.data .
|
||||
cp ../../build-wasm/wasm/bergamot-translator-worker.js .
|
||||
cp ../../build-wasm/wasm/bergamot-translator-worker.wasm .
|
||||
cp ../../build-wasm/wasm/bergamot-translator-worker.worker.js .
|
||||
|
||||
npm install
|
||||
echo "Start httpserver"
|
||||
node bergamot-httpserver.js
|
Loading…
Reference in New Issue
Block a user