* Refactored wasm bindings code
- Replaced TranslationModel, TranslationRequest and TranslationResult
with Service, ResponseOptions and Response
- Corresponding documentation changes
- Names of the bindings files changed
- Moved Vector<Response> definition in Response specific bindings
file
* Partial test applications
Previously service-cli was used to generate output and accomplish
regression testing for all of: (1) translated-text (2) alignment tokens
+ scores (3) quality scores (4) indirectly annotation and tokenizations.
The --mode native now only outputs a faithful to source translated text
of the input source on stdin.
Test apps are separated into testing only individual functionalities.
This can help in independently testing ssplit-cpp, quality-scores for
the quality estimation implementation etc.
Separating numbers and text have the advantage of being able to compare
one with tolerance using BLEU (text) and some allowed error-rates
(numbers).
* Removing #mac tag
* Moving test apps to src/tests
* Tests are always on for CI
Unit tests are turned off looking for WASM_COMPATIBLE_SOURCES.
* Fixing WASM_COMPATIBLE_SOURCE -> USE_WASM_COMPATIBLE_SOURCE
* Workaround for now; CMakeLists.txt horrors are starting to bite
* BRT: use bergamot-test instead of bergamot now
* This should fix issues: CMakeLists.txt has so many paths
* Casing to camelCase and removing legacyServiceCli
* removing leftover service-cli declaration, some doc updates
* #pragma once is starting to look easier
* All the more reasons to do #pragma once
* Updating marian-dev with intgemm::kCPU print, resolved from INTGEMM_CPUID
* BRT: Use --gemm-highest-arch instead of python script
* Adding intgemm resolve here, where always(?) have intgemm on?
* intgemm-resolve in default binary directory
* BRT: Update to use intgemm-resolve
* marian-dev: Reset to without --gemm-highest-precision
Co-authored-by: Kenneth Heafield <kpu@users.noreply.github.com>
* Use CMAKE_CURRENT_SOURCE_DIR instead of CMAKE_SOURCE_DIR for project bound version string
* marian-dev cmake fix
* Generate project.h in binary dir
* We don't want people asking about extra spaces
* Matrix is now more organized, Ubuntu 20.04-gcc9.3, Ubuntu-18.04-gcc7.5 is added.
* ccache is extended to MacOS, and brings down CI run times to <5m when
ccache works.
* The compiler hash scripts are gone, ccache already covers most ground
by default. The shell script is unnecessary. Cache works by preprocessor
mode output of running the compiler with -E, which includes the
necessary information. ccache-docs:How the cache works.
* BRT if failed prints the final 20 lines of the test*.log to inspect
what's going wrong without having to artifact download.
* Pull request on any branch triggers workflow.
* Push on main and ci-sandbox triggers workflow.
* Collapsing executables
* Adding new test executable
* Deleting old executable sources
* Updating brt to operate with modes
* cli-framework -> cli
* Updating workflows to check for bergamot instead of bergamot-translator-app
* Adding documentation
* Making fn pure virtual
* Shuffling apps into app namespace, alongside class documentation
* Include app folder in documentation
* BRT update service-cli -> native
* parser.h: service-cli -> native
* Updates to marian-integration.md
* Cleanup: Remove templates, interface proper
* change 4 to 2 cores for build instructions
* service-cli -> native
* Commenting the string constructor explanation
* Not doing halfway interface / inheritance
* Nick hates state, let's try this one
* Revert "Nick hates state, let's try this one"
This reverts commit e56db9f474.
* class -> struct before trying std::function stuff
* oop -> functional?
* Hints on what is happening
* app::ftable -> app::REGISTRY
* We have if-else and functions now.
And we won't have test apps.
* Doc linking to usage examples in brt
* Remove unordered_map
* Documentation updates
* Fix warning
Adds a GitHub workflow that builds documentation from sources through doxygen through sphinx on push to the main branch or on push of any semantic version tags. The built documentation is deployed at https://github.com/browsermt/docs@gh-pages, which is rendered at https://browser.mt/docs/<suffix>, where <suffix> is 'main' or a tag vM.m.p corresponding to a semantic version.
On pull request artifacts are uploaded for reviewers to inspect if need be.
* Adding a first version of clang-format
* Adding run-clang-format.py
* Adding coding styles to workflow
* Fix indentation on coding-styles workflow
* run-clang-format.'py'
* -style -> --style in python
* Updating ColumnLimit: 120
* Format update with clang-format
* Revert "Format update with clang-format"
This reverts commit 5340b19eae.
* Apply update after sync
* Removing a few empty lines
* Removing one more empty line
* Removing empty in workflow file
* Updating README with coding style instructions
* clang-format-* provided in this repository doc update
Co-authored-by: Nikolay Bogoychev <nheart@gmail.com>
* Use binary lexical shortlist in documentation
* MKL/AppleAccelerate note
Co-authored-by: Nikolay Bogoychev <nheart@gmail.com>
Co-authored-by: Jerin Philip <jphilip@ed.ac.uk>
This does make the batcher a critical section across job submission and
cleaving though. If that becomes a problem, we should go back to
incoming and outgoing queues with a batcher thread.
Also removes blocking mode from native compiles.
Note that translateMultiple no longer guarantees great batching. Guess
we could lease the mutex from ThreadsafeBatcher and create a session.
There is the risk that one sentence comes in at a time and each thread
grabs one sentence at a time instead of better batching. Not sure what
to do about that other than some sort of Nagle algorithm.
Due to non-deterministic batching, even with one thread, the regression
tests will go haywire.
* CI Changes to add tiny regression tests
* Adding an inspect cache step
* Removing ccache, pursue in another
* Incorporating Nick's changes through submodule merge
* Submodule now points to master
* Restoring ccache enabled workflow file
* Restoring ccache enabled CMakeLists
* cache -> ccache typo fix
* Moving CCACHE setup to GitHub runner file
* Find also uses CCACHE dir
* Updating CMakeLists not to override env
* Cache compiler binary's contents
* Changing a few names to trigger new build; Testing cache looks fun
* USE_CCACHE=on, -L for inspection
* Adding a ccache_cmd, but will only use in next commit
* Using ccache_cmd
* Removing "
* Adding compiler hash script
* Bunch of absolute paths
* GITHUB_WORKSPACE typo
* Nah, I'll keep -L and trigger another build
* Trying something with compiler hash on cache key backup as well
* builtin, bash it seems
* Empty commit #1
* Move ccache stats to after compile
* Reshuffling ccache vars
* No comments
* Updates to Github output set syntax
* Empty Commit 1
* Empty Commit 2
* Empty commit 3
* /bin/bash -> bash; ccache_cmd for consistency
* Adding ccache -s before and after build
* Adding comments to compiler-hash script
* Let's build cached and non-cached variants together for comparison
* Fixing quotes, /bin/bash -> bash
* Minor var/env adjustment
* Adding ccache -z before the job
* Reverting CMakeLists.txt without CCACHE
* Switching to CMAKE_LANG_COMPILER_LAUNCHER instead of CMakeLists.txt rule
* 5G -> 1G cache size
* 1G -> 2G; Hyperparameter tuning
- Removed PACKAGE_DIR cmake option
- Removed Workerfs, FORCE_FILESYSTEM=1 in wasm builds
-- File system support is not needed any more (since model,
shortlist and vocabs are being passed as bytes now)
