* Collapsing executables
* Adding new test executable
* Deleting old executable sources
* Updating brt to operate with modes
* cli-framework -> cli
* Updating workflows to check for bergamot instead of bergamot-translator-app
* Adding documentation
* Making fn pure virtual
* Shuffling apps into app namespace, alongside class documentation
* Include app folder in documentation
* BRT update service-cli -> native
* parser.h: service-cli -> native
* Updates to marian-integration.md
* Cleanup: Remove templates, interface proper
* change 4 to 2 cores for build instructions
* service-cli -> native
* Commenting the string constructor explanation
* Not doing halfway interface / inheritance
* Nick hates state, let's try this one
* Revert "Nick hates state, let's try this one"
This reverts commit e56db9f474.
* class -> struct before trying std::function stuff
* oop -> functional?
* Hints on what is happening
* app::ftable -> app::REGISTRY
* We have if-else and functions now.
And we won't have test apps.
* Doc linking to usage examples in brt
* Remove unordered_map
* Documentation updates
* Fix warning
Adds a GitHub workflow that builds documentation from sources through doxygen through sphinx on push to the main branch or on push of any semantic version tags. The built documentation is deployed at https://github.com/browsermt/docs@gh-pages, which is rendered at https://browser.mt/docs/<suffix>, where <suffix> is 'main' or a tag vM.m.p corresponding to a semantic version.
On pull request artifacts are uploaded for reviewers to inspect if need be.
* Adding a first version of clang-format
* Adding run-clang-format.py
* Adding coding styles to workflow
* Fix indentation on coding-styles workflow
* run-clang-format.'py'
* -style -> --style in python
* Updating ColumnLimit: 120
* Format update with clang-format
* Revert "Format update with clang-format"
This reverts commit 5340b19eae.
* Apply update after sync
* Removing a few empty lines
* Removing one more empty line
* Removing empty in workflow file
* Updating README with coding style instructions
* clang-format-* provided in this repository doc update
Co-authored-by: Nikolay Bogoychev <nheart@gmail.com>
* Use binary lexical shortlist in documentation
* MKL/AppleAccelerate note
Co-authored-by: Nikolay Bogoychev <nheart@gmail.com>
Co-authored-by: Jerin Philip <jphilip@ed.ac.uk>
This does make the batcher a critical section across job submission and
cleaving though. If that becomes a problem, we should go back to
incoming and outgoing queues with a batcher thread.
Also removes blocking mode from native compiles.
Note that translateMultiple no longer guarantees great batching. Guess
we could lease the mutex from ThreadsafeBatcher and create a session.
There is the risk that one sentence comes in at a time and each thread
grabs one sentence at a time instead of better batching. Not sure what
to do about that other than some sort of Nagle algorithm.
Due to non-deterministic batching, even with one thread, the regression
tests will go haywire.
* CI Changes to add tiny regression tests
* Adding an inspect cache step
* Removing ccache, pursue in another
* Incorporating Nick's changes through submodule merge
* Submodule now points to master
* Restoring ccache enabled workflow file
* Restoring ccache enabled CMakeLists
* cache -> ccache typo fix
* Moving CCACHE setup to GitHub runner file
* Find also uses CCACHE dir
* Updating CMakeLists not to override env
* Cache compiler binary's contents
* Changing a few names to trigger new build; Testing cache looks fun
* USE_CCACHE=on, -L for inspection
* Adding a ccache_cmd, but will only use in next commit
* Using ccache_cmd
* Removing "
* Adding compiler hash script
* Bunch of absolute paths
* GITHUB_WORKSPACE typo
* Nah, I'll keep -L and trigger another build
* Trying something with compiler hash on cache key backup as well
* builtin, bash it seems
* Empty commit #1
* Move ccache stats to after compile
* Reshuffling ccache vars
* No comments
* Updates to Github output set syntax
* Empty Commit 1
* Empty Commit 2
* Empty commit 3
* /bin/bash -> bash; ccache_cmd for consistency
* Adding ccache -s before and after build
* Adding comments to compiler-hash script
* Let's build cached and non-cached variants together for comparison
* Fixing quotes, /bin/bash -> bash
* Minor var/env adjustment
* Adding ccache -z before the job
* Reverting CMakeLists.txt without CCACHE
* Switching to CMAKE_LANG_COMPILER_LAUNCHER instead of CMakeLists.txt rule
* 5G -> 1G cache size
* 1G -> 2G; Hyperparameter tuning
- Removed PACKAGE_DIR cmake option
- Removed Workerfs, FORCE_FILESYSTEM=1 in wasm builds
-- File system support is not needed any more (since model,
shortlist and vocabs are being passed as bytes now)
* Change WASM_COMPATIBLE_SOURCE=OFF by default
The default was WASN_COMPATIBLE_SOURCE=ON COMPILE_WASM=OFF which is a
testing configuration, not a sensible default for native or wasm.
* Always USE_WASM_COMPATIBLE_SOURCE with COMPILE_WASM
* Set CMP0077 to fix variable handling
* first attempt to enable vocabs pass as byte arrays
* pass vocabs bytes as AlignedMemory
* add vocabIndices to avoid double loading
* small fix on parameter names and documentation
* fix windows build plus tiny update on documentation
* update marian-dev submodule
* move validate model bytearray in BatchTranslator
* small refactors on validateBinaryModel()
* switch vocab memories to std::vector<marian::Ptr<AlignedMemory>>
* update marian-dev submodule
* replace marian::Ptr to std::shared_ptr for vocab memories
* add note for vocab memories
* First draft of faithful translation
* Comments explaining pre and post
* Comments on response_builder
* Updating bergamot-translator-tests with new outputs
* Cosmetic changes in response target text construction
* Replacing &(x[0]) -> x.data() to avoid illegal indices
* Removing nullptr given both branches init pointer with legal values
* pre, post -> gap(i) addressing review comments
Functions which were pre and post before are subsumed by gap(i), and the
algorithm in ResponseBuilder adjusted to fix.
`x = nullptr` is back, should be harmless.
* Updating brt with paragraph outputs
* Bumping brt with updated outputs, buffer text at begin as well
* Bumping BRT with sync after bytearray collapse merge
* Pointing BRT to main after merge
Co-authored-by: Nikolay Bogoychev <nheart@gmail.com>
* Adding bytearray option
* collapse intermediate for bytearray apps
* Removing service-cli-bytearray
* Removing the bergamot bytearray app
* Bumping updates to brt collapsing apps
* Reasonable defaults and hard check when cmd enabled
* Update documentation for flags
* Bump brt with MKL check and skip
* Bumping BRT with MKL_FOUND instead of USE_MKL
* Bumping BRT with no mkl enforce
* Bumping BRT with ssse3 output
* Let's try disabling OpenBLAS
* Trying to disable apple accelerate
* Using WASM compatible BLAS can enable intgemm
* Adding a CMake -L to see what exactly is the diff
* Revert "Let's try disabling OpenBLAS"
This reverts commit 9a6b9bc53b.
* Revert "Using WASM compatible BLAS can enable intgemm"
This reverts commit 936a592e18.
* Restricting mac tests through tags and on GitHub CI
* Using only check-bytearray
* Bumping BRT with change of default behaviour
