mirror of
https://github.com/browsermt/bergamot-translator.git
synced 2024-09-17 16:47:18 +03:00
Patch WASM artifacts to run optimized (wormhole enabled) inference (#68)
* A script to patch the wasm artifacts to use wormhole via APIs that instantiate WASM module * Updated README * Load just production ready models * Shallow clone bergamot-models repo since it has such a large history * Improved wasm test_page - test page can load all 5 language pairs - Use intgemm.alpha* models * Refactor the code that patches wasm artifacts to enable wormhole Co-authored-by: Andre Natal <anatal@gmail.com> Co-authored-by: Motin <motin@motin.eu>
This commit is contained in:
parent
34228d37bf
commit
12e9232066
@ -33,6 +33,10 @@ jobs:
|
|||||||
working-directory: build-wasm
|
working-directory: build-wasm
|
||||||
run: emmake make -j2
|
run: emmake make -j2
|
||||||
|
|
||||||
|
- name: Instantiate simd wormhole
|
||||||
|
working-directory: build-wasm
|
||||||
|
run: bash ../wasm/patch-artifacts-enable-wormhole.sh
|
||||||
|
|
||||||
- name: Check artifacts
|
- name: Check artifacts
|
||||||
working-directory: build-wasm
|
working-directory: build-wasm
|
||||||
run: |
|
run: |
|
||||||
|
15
README.md
15
README.md
@ -46,8 +46,8 @@ Bergamot translator provides a unified API for ([Marian NMT](https://marian-nmt.
|
|||||||
If you want to package bergamot project specific models, please follow these instructions:
|
If you want to package bergamot project specific models, please follow these instructions:
|
||||||
```bash
|
```bash
|
||||||
mkdir models
|
mkdir models
|
||||||
git clone https://github.com/mozilla-applied-ml/bergamot-models
|
git clone --depth 1 --branch main --single-branch https://github.com/mozilla-applied-ml/bergamot-models
|
||||||
cp -rf bergamot-models/* models
|
cp -rf bergamot-models/prod/* models
|
||||||
gunzip models/*/*
|
gunzip models/*/*
|
||||||
```
|
```
|
||||||
|
|
||||||
@ -75,11 +75,16 @@ Bergamot translator provides a unified API for ([Marian NMT](https://marian-nmt.
|
|||||||
emmake make -j
|
emmake make -j
|
||||||
```
|
```
|
||||||
|
|
||||||
The artefacts (.js and .wasm files) will be available in `wasm` folder of build directory ("build-wasm" in this case).
|
The wasm artifacts (.js and .wasm files) will be available in `wasm` folder of build directory ("build-wasm" in this case).
|
||||||
|
|
||||||
|
3. Enable SIMD Wormhole via Wasm instantiation API in generated artifacts
|
||||||
|
```bash
|
||||||
|
bash ../wasm/patch-artifacts-enable-wormhole.sh
|
||||||
|
```
|
||||||
|
|
||||||
#### Recompiling
|
#### Recompiling
|
||||||
As long as you don't update any submodule, just follow steps in `4.ii` to recompile.\
|
As long as you don't update any submodule, just follow steps in `4.ii` and `4.iii` to recompile.\
|
||||||
If you update a submodule, execute following command before executing steps in `4.ii` to recompile.
|
If you update a submodule, execute following command before executing steps in `4.ii` and `4.iii` to recompile.
|
||||||
```bash
|
```bash
|
||||||
git submodule update --init --recursive
|
git submodule update --init --recursive
|
||||||
```
|
```
|
||||||
|
7
wasm/patch-artifacts-enable-wormhole.sh
Normal file
7
wasm/patch-artifacts-enable-wormhole.sh
Normal file
@ -0,0 +1,7 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
|
||||||
|
echo "Patching wasm artifacts to enable wormhole via APIs that compile and instantiate wasm module"
|
||||||
|
sed -i.bak 's/var result = WebAssembly.instantiateStreaming(response, info);/var result = WebAssembly.instantiateStreaming(response, info, {simdWormhole:true});/g' wasm/bergamot-translator-worker.js
|
||||||
|
sed -i.bak 's/return WebAssembly.instantiate(binary, info);/return WebAssembly.instantiate(binary, info, {simdWormhole:true});/g' wasm/bergamot-translator-worker.js
|
||||||
|
sed -i.bak 's/var module = new WebAssembly.Module(bytes);/var module = new WebAssembly.Module(bytes, {simdWormhole:true});/g' wasm/bergamot-translator-worker.js
|
||||||
|
echo "Done"
|
@ -32,6 +32,9 @@
|
|||||||
<label>Choose the model to use</label>
|
<label>Choose the model to use</label>
|
||||||
<input type="radio" name="modellang" value="enes"/><label>English to Spanish</label>
|
<input type="radio" name="modellang" value="enes"/><label>English to Spanish</label>
|
||||||
<input type="radio" name="modellang" value="esen" checked/><label>Spanish to English</label>
|
<input type="radio" name="modellang" value="esen" checked/><label>Spanish to English</label>
|
||||||
|
<input type="radio" name="modellang" value="eten" checked/><label>Estonian to English</label>
|
||||||
|
<input type="radio" name="modellang" value="enet" checked/><label>English to Estonian</label>
|
||||||
|
<input type="radio" name="modellang" value="ende" checked/><label>English to German</label>
|
||||||
<input type="button" id="load" value="Load Model"/>
|
<input type="button" id="load" value="Load Model"/>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
@ -70,7 +73,7 @@ En consecuencia, durante el año 2011 se introdujeron 180 proyectos de ley que r
|
|||||||
// Set the Model Configuration as YAML formatted string.
|
// Set the Model Configuration as YAML formatted string.
|
||||||
// For available configuration options, please check: https://marian-nmt.github.io/docs/cmd/marian-decoder/
|
// For available configuration options, please check: https://marian-nmt.github.io/docs/cmd/marian-decoder/
|
||||||
const modelConfig = `models:
|
const modelConfig = `models:
|
||||||
- /${languagePair}/model.${languagePair}.npz
|
- /${languagePair}/model.${languagePair}.intgemm.alphas.bin
|
||||||
vocabs:
|
vocabs:
|
||||||
- /${vocabLanguagePair}/vocab.${vocabLanguagePair}.spm
|
- /${vocabLanguagePair}/vocab.${vocabLanguagePair}.spm
|
||||||
- /${vocabLanguagePair}/vocab.${vocabLanguagePair}.spm
|
- /${vocabLanguagePair}/vocab.${vocabLanguagePair}.spm
|
||||||
|
@ -1,8 +1,10 @@
|
|||||||
#!/bin/bash
|
#!/bin/bash
|
||||||
|
echo "Start: Copying artifacts in local folder------"
|
||||||
cp ../../build-wasm/wasm/bergamot-translator-worker.data .
|
cp ../../build-wasm/wasm/bergamot-translator-worker.data .
|
||||||
cp ../../build-wasm/wasm/bergamot-translator-worker.js .
|
cp ../../build-wasm/wasm/bergamot-translator-worker.js .
|
||||||
cp ../../build-wasm/wasm/bergamot-translator-worker.wasm .
|
cp ../../build-wasm/wasm/bergamot-translator-worker.wasm .
|
||||||
cp ../../build-wasm/wasm/bergamot-translator-worker.worker.js .
|
cp ../../build-wasm/wasm/bergamot-translator-worker.worker.js .
|
||||||
|
|
||||||
npm install
|
npm install
|
||||||
|
echo "Start httpserver"
|
||||||
node bergamot-httpserver.js
|
node bergamot-httpserver.js
|
Loading…
Reference in New Issue
Block a user