mirror of
https://github.com/browsermt/bergamot-translator.git
synced 2024-09-11 05:35:33 +03:00
e34420647d
* Rework WASM compilation options Necessary to work with newer versions of emscripten that are more picky about which option goes to the compiler, and which to the linker. Also took the opportunity to remove the need for the patching of the bergamot-translation-worker.js file, this can now easily be done through supported apis. Furthermore, I tried to downsize the generated javascript and wasm code a bit. Initial estimates show that bergamot-translator compiled with emscripten 3.0.0 runs at about 3x the speed of 2.0.9 (when using embedded intgemm). Speed-up when using mozIntGemm is less dramatic. * Updated marian-dev submodule * Revert changes specific to patching external gemm modules for wasm * Better Compilation and Link flags - Added "-O3" optimization flag for linking as well - "-g2" only for release and debug builds - "-g1" for release builds - Replaced deprecated "--bind" flag with "-lembind" - Removed redundant link flag * Upgraded emsdk to 3.1.8 * Enclosed EXPORTED_FUNCTIONS values in a list * Fixed the remaining 2.0.9 reference in circle ci build script * Updated README Co-authored-by: Jelmer van der Linde <jelmer@ikhoefgeen.nl>
88 lines
3.1 KiB
Markdown
88 lines
3.1 KiB
Markdown
# Bergamot Translator
|
|
|
|
[](https://circleci.com/gh/browsermt/bergamot-translator/)
|
|
|
|
Bergamot translator provides a unified API for ([Marian NMT](https://marian-nmt.github.io/) framework based) neural machine translation functionality in accordance with the [Bergamot](https://browser.mt/) project that focuses on improving client-side machine translation in a web browser.
|
|
|
|
## Build Instructions
|
|
|
|
### Build Natively
|
|
Create a folder where you want to build all the artifacts (`build-native` in this case) and compile
|
|
|
|
```bash
|
|
mkdir build-native
|
|
cd build-native
|
|
cmake ../
|
|
make -j2
|
|
```
|
|
|
|
### Build WASM
|
|
#### Prerequisite
|
|
|
|
Building on wasm requires Emscripten toolchain. It can be downloaded and installed using following instructions:
|
|
|
|
* Get the latest sdk: `git clone https://github.com/emscripten-core/emsdk.git`
|
|
* Enter the cloned directory: `cd emsdk`
|
|
* Install the sdk: `./emsdk install 3.1.8`
|
|
* Activate the sdk: `./emsdk activate 3.1.8`
|
|
* Activate path variables: `source ./emsdk_env.sh`
|
|
|
|
#### <a name="Compile"></a> Compile
|
|
|
|
To build a version that translates with higher speeds on Firefox Nightly browser, follow these instructions:
|
|
|
|
1. Create a folder where you want to build all the artifacts (`build-wasm` in this case) and compile
|
|
```bash
|
|
mkdir build-wasm
|
|
cd build-wasm
|
|
emcmake cmake -DCOMPILE_WASM=on ../
|
|
emmake make -j2
|
|
```
|
|
|
|
The wasm artifacts (.js and .wasm files) will be available in the build directory ("build-wasm" in this case).
|
|
|
|
2. Enable SIMD Wormhole via Wasm instantiation API in generated artifacts
|
|
```bash
|
|
bash ../wasm/patch-artifacts-enable-wormhole.sh
|
|
```
|
|
|
|
3. Patch generated artifacts to import GEMM library from a separate wasm module
|
|
```bash
|
|
bash ../wasm/patch-artifacts-import-gemm-module.sh
|
|
```
|
|
|
|
To build a version that runs on all browsers (including Firefox Nightly) but translates slowly, follow these instructions:
|
|
|
|
1. Create a folder where you want to build all the artifacts (`build-wasm` in this case) and compile
|
|
```bash
|
|
mkdir build-wasm
|
|
cd build-wasm
|
|
emcmake cmake -DCOMPILE_WASM=on -DWORMHOLE=off ../
|
|
emmake make -j2
|
|
```
|
|
|
|
2. Patch generated artifacts to import GEMM library from a separate wasm module
|
|
```bash
|
|
bash ../wasm/patch-artifacts-import-gemm-module.sh
|
|
```
|
|
|
|
#### Recompiling
|
|
As long as you don't update any submodule, just follow [Compile](#Compile) steps.\
|
|
If you update a submodule, execute following command in repository root folder before executing
|
|
[Compile](#Compile) steps.
|
|
```bash
|
|
git submodule update --init --recursive
|
|
```
|
|
|
|
|
|
## How to use
|
|
|
|
### Using Native version
|
|
|
|
The builds generate library that can be integrated to any project. All the public header files are specified in `src` folder.\
|
|
A short example of how to use the APIs is provided in `app/main.cpp` file.
|
|
|
|
### Using WASM version
|
|
|
|
Please follow the `README` inside the `wasm` folder of this repository that demonstrates how to use the translator in JavaScript.
|