Abhishek Aggarwal
f5dffeb5ca
Downgraded resource class to 'medium' for circle ci
...
- Also restricted parallel make compilation to 3
2021-04-14 14:19:13 +02:00
Abhishek Aggarwal
bf28edad82
Improved wasm test_page
...
- test page can load all 5 language pairs
- Use intgemm.aplha* models
- start_server.sh script automatically enable simdwormhole via
APIs that instantiate WASM module
2021-03-11 14:38:16 +01:00
Motin
d1ecd007a6
Shallow clone bergamot-models repo since it has such a large history
2021-03-11 14:38:16 +01:00
Andre Natal
a2d6650097
Patch to load just production ready models
2021-03-11 14:38:16 +01:00
Abhishek Aggarwal
6e7b7c71ec
Updates README for enabling simdwormhole in WASM APIs
2021-03-11 13:44:14 +01:00
Abhishek Aggarwal
4f124e7976
Enabled simdwormhole in github workflows
2021-03-11 13:44:14 +01:00
Abhishek Aggarwal
8c8913e2ef
Use intgemm models in wasm test_page
2021-03-11 13:44:14 +01:00
Andre Natal
e96d7047a7
Enable SIMD wormhole in circle ci build scripts
...
- Modify the APIs that compile & instantiate WASM module
2021-03-11 13:44:14 +01:00
abhi-agg
c64deb50a8
Imported CI scripts from mozilla/bergamot-translator-old ( #1 )
...
* CircleCI config, docs and badge
* Increase CircleCI RAM from 4gb to 16gb
Co-authored-by: Motin <motin@motin.eu>
2021-03-10 09:30:39 -08:00
Abhishek Aggarwal
d3ef1a9bc3
Updated marian submodule
...
- This fixes the binary model loading problem for wasm
2021-03-10 15:50:27 +01:00
Ulrich Germann
f17f02a544
Update submodule ssplit-cpp
2021-03-03 11:48:56 +01:00
abhi-agg
a9e0d800ae
Merge pull request #8 from browsermt/integration
...
Integration: marian new on-the-fly-decoder for bergamot
2021-02-26 18:56:29 +01:00
abhi-agg
e94f6da8b6
Merge pull request #44 from browsermt/jp/find-segfault
...
Improve threaded/non-threaded paths
2021-02-26 15:36:33 +01:00
Abhishek Aggarwal
0be73705d9
Fixed native builds while using wasm compatible sources
...
- main-mts and marian-decoder-new can't be used because
it uses multi-threaded variant of Service class
2021-02-26 14:55:30 +01:00
Abhishek Aggarwal
4d4acf6b8b
Cleanup CMakeFiles.txt
...
- Renamed USE_WASM_COMPATIBLE_MARIAN to USE_WASM_COMPATIBLE_SOURCES
- Removed COMPILE_THREAD_VARIANT cmake option and removed
corresponding compile definition
- Updated workflows and READMEs accordingly
2021-02-26 14:17:48 +01:00
Jerin Philip
e1b74bccab
Reverting moot COMPILE_WASM guards in app folder
2021-02-26 11:42:23 +00:00
Jerin Philip
7c14b737a4
Improving abort error message.
2021-02-26 11:41:10 +00:00
Jerin Philip
9ead41d879
Adds documentation, makes enqueue() private
2021-02-26 10:33:07 +00:00
Jerin Philip
66e3b4493e
LoadVocabularies inlined in service_base.h
...
To fix WASM Mac builds on CI. loadVocabularies function is now inlines
and available through service_base.h, from where it seems to propogate
to all places of use.
2021-02-26 00:43:36 +00:00
Jerin Philip
dad3a4088c
Marking enqueue as override()
2021-02-26 00:39:38 +00:00
Jerin Philip
2c57f4b498
Adding ci-sandbox branch for GitHub CI test-use
2021-02-26 00:11:58 +00:00
Jerin Philip
570865e799
Getting rid of the ifdef from BatchTranslator as well
2021-02-25 23:50:57 +00:00
Jerin Philip
cd01d7552a
ServiceBase -> [NonThreadedService, Service]
...
Through inheritance, a non-threaded and multithreaded Service are
created, both derived of the same ServiceBase class which holds the
common elements.
In preparation to solve SIGSEGV in #41 . First inspections gave aborts in
thread part, and repeated SIGSEGV's in lock-policy's of shared_pointers
even in non-threaded paths.
Solving this first, to avoid ifdef or tricky paths. The non-threaded
implementation is not included in WASM builds at all, by separating out
the single-threaded logic. DRY is achieved through inheritance and
operator overloading.
2021-02-25 23:11:09 +00:00
abhi-agg
dd3dc6f932
Merge pull request #43 from browsermt/ci-test
...
CI Fixes: vanilla bergamot-translator
2021-02-25 17:35:59 +01:00
Jerin Philip
51da121057
CI Fixes: vanilla bergamot-translator builds
...
* Updating vanilla workflows with -DUSE_WASM_COMPATIBLE_MARIAN=off
* Boost: Replacing with OS Boost for Ubuntu Builds
2021-02-25 16:29:49 +00:00
Abhishek Aggarwal
18b4c7a016
Improved README
...
- Removed 'wasm-integration' branch from wasm build instructions
- Improved native build instructions
2021-02-25 15:22:52 +01:00
Abhishek Aggarwal
2538fb6007
Added workflows for compilation with custom marian
...
- Custom marian means only those marian features that
are required for wasm
- Added workflow for native builds
- Added workflow for wasm builds
2021-02-25 13:12:16 +01:00
Abhishek Aggarwal
c2b1c6eab4
Use system installed PCRE2 for builds using full blown marian
...
- USE_INTERNAL_PCRE2 is ON for custom marian builds while OFF
for full marian builds
2021-02-24 20:02:48 +01:00
Abhishek Aggarwal
b845ed3693
Update marian submodule
...
- Fixes the compilation while building with full blown marian
2021-02-24 19:54:38 +01:00
abhi-agg
31cdf861fa
Merge pull request #40 from browsermt/wasm-integration
...
Wasm integration
2021-02-23 18:47:27 +01:00
Abhishek Aggarwal
eb5284fb20
Removed erroneous '?' in example text in wasm test page
2021-02-23 18:43:51 +01:00
Abhishek Aggarwal
4369a56f90
Enable building marian executables for vanilla marian builds
...
- COMPILE_LIBRARY_ONLY is set to ON only for wasm compatible marian
builds
2021-02-23 18:15:33 +01:00
abhi-agg
c28687fffb
Merge pull request #38 from browsermt/wasm-integration
...
wasm-integration -> integration
2021-02-23 16:34:22 +01:00
abhi-agg
c0efc21c6e
Merge pull request #30 from browsermt/jp/absorb-batch-translator
...
Sync thread adjustments with wasm-integration
2021-02-23 16:32:26 +01:00
Abhishek Aggarwal
415d16bd1d
Single cmake option to enable/disable wasm compatible marian compilation
...
- USE_WASM_COMPATIBLE_MARIAN=off will start using vanilla Marian
i.e. with full threading support, with exceptions, with MKL
- Changed the relevant documentation
2021-02-23 16:15:05 +01:00
Abhishek Aggarwal
458176c050
Enable building pcre2 from sources for ssplit submodule
...
- USE_INTERNAL_PCRE2 is set to ON
- Sentence splitting is working (tested it via wasm test page)
2021-02-22 18:51:48 +01:00
Abhishek Aggarwal
462a850d8a
Changed Sentences to Paragraphs in test page of WASM
...
- Sentence Splitter works now => No more sentence splitting in
test code
- Changed example to include some paragraphs
2021-02-22 18:48:59 +01:00
Abhishek Aggarwal
fa4a1ed67d
Adapted model config in test example of bergamot
...
- Replaced deprecated names with new names
mini-batch-words and max-length-break
- Set cpu-threads to 0
2021-02-22 18:29:26 +01:00
Abhishek Aggarwal
5dcbb721fa
Update ssplit submodule to master branch
...
- This submodule brings pcre2 lib compiled from sources
2021-02-22 18:03:53 +01:00
Jerin Philip
fd9e79a817
Merge remote-tracking branch 'origin/wasm-integration' into jp/absorb-batch-translator
2021-02-22 16:33:46 +00:00
Motin
51f702ea6c
Remove Docker-based builds since they are no more reproducible than metal builds. Fixes https://github.com/browsermt/bergamot-translator/issues/31
2021-02-18 15:02:44 +02:00
Abhishek Aggarwal
79571bada5
Improved wasm/README
...
- Clarified that the Demo and API usage section assumes
bergamot models were packaged into wasm binary
- Formatting changes
2021-02-18 12:48:45 +01:00
Abhishek Aggarwal
c2371dd771
Replaced "build-wasm-docker" with "build-wasm"
...
- Now things are consistent with the top level README
instructions that suggest to build in "build-wasm"
folder
2021-02-18 12:20:06 +01:00
Jerin Philip
fbff7389d1
Temporary: Switch to abhi-agg/ssplit-cpp@wasm
2021-02-18 11:20:01 +00:00
Jerin Philip
ca9aa64926
Switch to work with ssplit-cpp both pcre2 and pcrecpp
2021-02-18 11:07:31 +00:00
Abhishek Aggarwal
b75e72e65d
Added more explanation for FILES_TO_PACKAGE in README
2021-02-18 10:42:06 +01:00
Jerin Philip
d249dcbfaa
Build doc updated with wasm-branch compatible command
2021-02-17 21:15:35 +00:00
Jerin Philip
b9d081dd45
Temporary: Updating marian-dev to wasm branch
2021-02-17 19:51:57 +00:00
Abhishek Aggarwal
9feebe5cb2
Allow using relative paths for packaging files
...
- PACKAGE_DIR cmake option can now accept relative paths
2021-02-17 20:06:04 +01:00
Jerin Philip
d72343567c
BatchTranslator doesn't do thread_, residue from merge removed
2021-02-17 16:41:04 +00:00