browsermt/bergamot-translator
1
1
Fork 0
mirror of https://github.com/browsermt/bergamot-translator.git synced 2024-09-11 05:35:33 +03:00
Code Issues Packages Projects Releases Wiki Activity

Add ability to load .npz models (#342)

Browse Source 
Changes `ABORT` on non `.bin` model to an additional check for a `.npz` 
extension. If `.bin`, the fast load path is activated by returning `AlignedMemory`. 
Otherwise, the return of empty `AlignedMemory` causes fallback to
filesystem-based loads.

BRT: A test that checks if translation using `.npz` is approximately similar to 
that of default CLI translation is checked in to ensure stability going ahead.

Previously, we only supported `.bin` models' loading via a fast mmap 
path. While we had the underlying capability to load non `.bin` models, this 
was not exposed, encouraging fast loads. Loading `.npz` models are helpful 
for quick debugging and broader coverage of models available, which will 
enhance user experience at translateLocally and python bindings. 


Fixes #341.
See also: XapaJIaMnu/translateLocally#89
This commit is contained in:
Jerin Philip 2022-02-09 19:37:30 +00:00 committed by GitHub
parent 80bd4e7651
commit 34786520cd
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 14 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
bergamot-translator-tests

@ -1 +1 @@
Subproject commit aaf315c80b7339c2ae6fcca080ca30d3f0a5e2f5
Subproject commit 3c0f95a1775a74f5db441aa2f17ceb7437679022

17
src/translator/byte_array_util.cpp
View File

@ -3,6 +3,7 @@
#include <cstdlib>
#include <memory>
#include "common/io.h"
#include "data/shortlist.h"
namespace marian {
@ -93,10 +94,18 @@ AlignedMemory loadFileToMemory(const std::string& path, size_t alignment) {
AlignedMemory getModelMemoryFromConfig(marian::Ptr<marian::Options> options) {
auto models = options->get<std::vector<std::string>>("models");
ABORT_IF(models.size() != 1, "Loading multiple binary models is not supported for now as it is not necessary.");
marian::filesystem::Path modelPath(models[0]);
ABORT_IF(modelPath.extension() != marian::filesystem::Path(".bin"), "The file of binary model should end with .bin");
AlignedMemory alignedMemory = loadFileToMemory(models[0], 256);
return alignedMemory;
// If binary model we load into aligned memory. If .npz we leave it be to
// return empty aligned memory, thus allowing traditional file system loads.
if (marian::io::isBin(models[0])) {
AlignedMemory alignedMemory = loadFileToMemory(models[0], 256);
return alignedMemory;
} else if (marian::io::isNpz(models[0])) {
return AlignedMemory();
} else {
ABORT("Unknown extension for model: {}, should be one of `.bin` or `.npz`", models[0]);
}
return AlignedMemory();
}
AlignedMemory getShortlistMemoryFromConfig(marian::Ptr<marian::Options> options) {