BatchTranslators are now held in Service. Threads are separate, and
constructed via lambdas. Retaining BatchTranslator class and member
function (Probably a matter of taste I guess).
This should eliminate complaints in (#10), hopefully.
To avoid confusion, this commit renames
marian::bergamot::TranslationResult -> marian::bergamot::Response.
Usages of marian::bergamot::TranslationResults are updated across the
source to be consistent with the change and get source back working.
marian-TranslationResult has more guards in place. Switching to a
construction on demand model for sentenceMappings. These changes
propogate to bergamot translation results.
Integration broke with the change in marian's internals, which are
updated accordingly to get back functionality.
Changes revealed a few bugs, which are fixed:
- ConfigParser already discovered in wasm-integration
(a06530e92b).
- Lambda captures and undefined values in DeviceId
Batch Ids cannot be set by outside classes to values < 0.
Batch.Id_ =
-1 : Poison, for use in PCQueue
0 : Default constructed, invalid batch.
>0 : Legit batch.
Book-keeping for batch metrics (maxLength, numTokens, etc) and logging
are now moved to Batch. Batch is now a class instead of a struct with
accessors controlling how members can be modified to suit above.
Guided by an objective to move batching mechanism and queueing request
to generate batches into a diffenrent thread. This commit is in
preparation for this functionality.
First, PCItem from the looks of it is *Batch*. Renamed to reflect the
same. Fingers crossed, hopefully no naming conflicts with marian.
BatchTranslator translates a "Batch" now, instead of
vector<RequestSentence>. Additional data members are setup at Batch to
enable development.
Workflows previously in Service, but more adequate in Batcher are now
moved, preparing to move Batcher/enqueuing of a request into a new
thread making it non-blocking. This will allow service to queue requests
into the batcher thread and exit, without waiting until the full-request
is queued.
Batcher now has a path with and without pcqueue.
