mirror of
https://github.com/browsermt/bergamot-translator.git
synced 2024-09-11 05:35:33 +03:00
Imported CI scripts from mozilla/bergamot-translator-old (#1)
* CircleCI config, docs and badge * Increase CircleCI RAM from 4gb to 16gb Co-authored-by: Motin <motin@motin.eu>
This commit is contained in:
parent
d3ef1a9bc3
commit
c64deb50a8
19
.circleci/config.yml
Normal file
19
.circleci/config.yml
Normal file
@ -0,0 +1,19 @@
|
||||
version: 2.1
|
||||
jobs:
|
||||
build:
|
||||
docker:
|
||||
- image: 'emscripten/emsdk:2.0.9'
|
||||
resource_class: xlarge
|
||||
|
||||
working_directory: ~/checkout
|
||||
|
||||
steps:
|
||||
- checkout
|
||||
|
||||
- run:
|
||||
name: Build WASM
|
||||
command: bash build-wasm.sh
|
||||
|
||||
- store_artifacts:
|
||||
path: "build-wasm/wasm"
|
||||
destination: "build-wasm/wasm"
|
2
.gitignore
vendored
2
.gitignore
vendored
@ -17,6 +17,6 @@ _deps
|
||||
|
||||
|
||||
wasm/test_page/node_modules
|
||||
build-*
|
||||
build-wasm
|
||||
models
|
||||
wasm/test_page/bergamot-translator-worker.*
|
||||
|
@ -1,5 +1,7 @@
|
||||
# Bergamot Translator
|
||||
|
||||
[![CircleCI badge](https://img.shields.io/circleci/project/github/mozilla/bergamot-translator/main.svg?label=CircleCI)](https://circleci.com/gh/mozilla/bergamot-translator/)
|
||||
|
||||
Bergamot translator provides a unified API for ([Marian NMT](https://marian-nmt.github.io/) framework based) neural machine translation functionality in accordance with the [Bergamot](https://browser.mt/) project that focuses on improving client-side machine translation in a web browser.
|
||||
|
||||
## Build Instructions
|
||||
|
70
build-wasm.sh
Executable file
70
build-wasm.sh
Executable file
@ -0,0 +1,70 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
# Usage: ./build-wasm.sh
|
||||
|
||||
set -e
|
||||
set -x
|
||||
|
||||
# Run script from the context of the script-containing directory
|
||||
cd "$(dirname $0)"
|
||||
|
||||
# This file replicates the instructions found in ./README.md under "Build WASM"
|
||||
# with slight adjustments to be able to run the build script multiple times without having to clone all dependencies
|
||||
# as per "As long as you don't update any submodule, just follow steps in `4.ii` to recompile."
|
||||
|
||||
# 1. Download and Install Emscripten using following instructions (unless the EMSDK env var is already set)
|
||||
if [ "$EMSDK" == "" ]; then
|
||||
EMSDK_UPDATE_REQUIRED=0
|
||||
if [ ! -d "emsdk" ]; then
|
||||
git clone https://github.com/emscripten-core/emsdk.git
|
||||
EMSDK_UPDATE_REQUIRED=1
|
||||
else
|
||||
cd emsdk
|
||||
git fetch
|
||||
# Only pull if necessary
|
||||
if [ $(git rev-parse HEAD) != $(git rev-parse @{u}) ]; then
|
||||
git pull --ff-only
|
||||
EMSDK_UPDATE_REQUIRED=1
|
||||
fi
|
||||
cd -
|
||||
fi
|
||||
if [ "$EMSDK_UPDATE_REQUIRED" == "1" ]; then
|
||||
cd emsdk
|
||||
./emsdk install latest
|
||||
./emsdk activate latest
|
||||
cd -
|
||||
fi
|
||||
source ./emsdk/emsdk_env.sh
|
||||
fi
|
||||
|
||||
# 3. Download models (only required if you want to package files in wasm binary)
|
||||
if [ ! -d "bergamot-models" ]; then
|
||||
git clone https://github.com/mozilla-applied-ml/bergamot-models
|
||||
else
|
||||
cd bergamot-models
|
||||
git fetch
|
||||
# Only pull if necessary
|
||||
if [ $(git rev-parse HEAD) != $(git rev-parse @{u}) ]; then
|
||||
git pull --ff-only
|
||||
fi
|
||||
cd -
|
||||
fi
|
||||
mkdir -p models
|
||||
rm -rf models/*
|
||||
cp -rf bergamot-models/* models
|
||||
gunzip models/*/*
|
||||
|
||||
# 4. Compile
|
||||
# 1. Create a folder where you want to build all the artefacts (`build-wasm` in this case)
|
||||
if [ ! -d "build-wasm" ]; then
|
||||
mkdir build-wasm
|
||||
fi
|
||||
cd build-wasm
|
||||
|
||||
# 2. Compile the artefacts
|
||||
emcmake cmake -DCOMPILE_WASM=on -DPACKAGE_DIR="../models/" ../
|
||||
emmake make -j
|
||||
|
||||
# The artefacts (.js and .wasm files) will be available in `wasm` folder of build directory ("build-wasm" in this case).
|
||||
|
||||
exit 0
|
22
doc/CI.md
Normal file
22
doc/CI.md
Normal file
@ -0,0 +1,22 @@
|
||||
# Continuous Integration
|
||||
|
||||
[Circle CI](https://circleci.com/) is used for continuous integration. Configured via `./.circleci/config.yml`.
|
||||
|
||||
## Run Circle CI locally (requires Docker)
|
||||
|
||||
1. [Install the CircleCI local cli](https://circleci.com/docs/2.0/local-cli/#installation)
|
||||
2. Validate Circle CI configuration (useful exercise before pushing any changes to the configuration)
|
||||
|
||||
```shell
|
||||
circleci config validate -c .circleci/config.yml
|
||||
```
|
||||
|
||||
3. To better mimic the starting point for CI, commit your changes and clone your repository into a clean directory then run CircleCI inside that directory:
|
||||
|
||||
```shell
|
||||
git clone . /tmp/$(basename $PWD)
|
||||
cd /tmp/$(basename $PWD)
|
||||
circleci build
|
||||
```
|
||||
|
||||
Note: Steps related to caching and uploading/storing artifacts will report as failed locally. This is not necessarily a problem, they are designed to fail since the operations are not supported locally by the CircleCI build agent.
|
Loading…
Reference in New Issue
Block a user