Hieu Hoang
a332e550a5
debug
2021-06-16 12:56:36 -07:00
Marcin Junczys-Dowmunt
85eb6adce0
update sentencepiece pointer to version with case-awareness
2021-06-16 12:40:38 -07:00
Hieu Hoang
9b4a845cc7
clean up bias
2021-06-16 11:19:23 -07:00
Hieu Hoang
892554129e
lemma Et is optional
2021-06-16 10:19:24 -07:00
Hieu Hoang
395a4f94d0
init vector
2021-06-15 18:08:45 -07:00
Hieu Hoang
6981b21f4e
Merge branch 'hihoan/lsh7' of vs-ssh.visualstudio.com:v3/machinetranslation/Marian/marian-dev into hihoan/lsh7
2021-06-15 16:54:57 -07:00
Hieu Hoang
488a532bdf
get lemma size from vocab class
2021-06-15 16:54:17 -07:00
Hieu Hoang
7e6ec58507
delete variables altogether
2021-06-14 19:07:17 -07:00
Hieu Hoang
82fa059a03
'use' variables
2021-06-14 19:03:53 -07:00
Hieu Hoang
5362c2cc0e
don't define BLAS_FOUND
2021-06-14 18:47:08 -07:00
Hieu Hoang
5b7b1f7e5c
no need for args in getIndicesExpr(). Deleted debugging
2021-06-14 18:44:05 -07:00
Hieu Hoang
8c04f66474
reverse batch beam argument order
2021-06-15 00:10:08 +00:00
Hieu Hoang
dffbb47eea
rename broadcast -> createCachedTensors
2021-06-14 23:18:21 +00:00
Hieu Hoang
cc295938ce
incorrect dimension order
2021-06-14 22:29:27 +00:00
Hieu Hoang
8649034760
no need to broadcast
2021-06-11 23:26:10 +00:00
Hieu Hoang
700dc7fdd1
don't transpose lastIndices. Works for lsh & sl
2021-06-11 22:55:26 +00:00
Hieu Hoang
49998217d9
don't transpose lastIndices. Works for lsh
2021-06-11 22:47:15 +00:00
Hieu Hoang
f0251889f2
debug
2021-06-11 00:57:02 -07:00
Hieu Hoang
fef7202bc8
batch-beam -> beam-batch
2021-06-10 23:58:25 -07:00
Hieu Hoang
5a93c67185
origBatchIdx -> currentBatchIdx. Doesn't crash but bad results
2021-06-09 23:31:06 +00:00
Hieu Hoang
fe97259d3d
debug
2021-06-09 22:58:25 +00:00
Hieu Hoang
6f0f534a4a
debug
2021-06-09 22:36:34 +00:00
Hieu Hoang
1e3db86a94
batch based filtering. COmment out debug
2021-06-09 21:56:51 +00:00
Hieu Hoang
4b9082bc39
don't manually broadcast lemma
2021-06-09 20:57:02 +00:00
Hieu Hoang
79dbde7efc
don't manually broadcast weights
2021-06-09 20:45:40 +00:00
Hieu Hoang
0bc9b22b15
separate broadcast
2021-06-09 20:18:19 +00:00
Hieu Hoang
5d1946ebd3
filter & broadcast every word. SL works
2021-06-09 20:07:57 +00:00
Hieu Hoang
e07e0368c9
debug
2021-06-08 18:20:56 -07:00
Hieu Hoang
92c6c07786
reshape cachedShortWt_
2021-06-07 15:43:54 -07:00
Hieu Hoang
eb3f540d42
debug
2021-06-07 15:37:22 -07:00
Hieu Hoang
b5f97dc605
reshape cachedShortLemmaEt
2021-06-07 15:35:18 -07:00
Hieu Hoang
acdff77688
reduce tranform for no-shortlist
2021-06-07 15:29:20 -07:00
Hieu Hoang
0949a4c914
start using bdot
2021-06-07 15:05:56 -07:00
Hieu Hoang
bc4ad2408c
Merge branch 'mjd/bdot' into hihoan/lsh7
2021-06-07 14:16:41 -07:00
Marcin Junczys-Dowmunt
1c0b899444
Merge branch 'pmaster'
2021-06-07 11:51:22 -07:00
Marcin Junczys-Dowmunt
ce34df4d98
add legacy code on gpu
2021-06-07 11:25:40 -07:00
Marcin Junczys-Dowmunt
1d96d7b6eb
add legacy code on cpu
2021-06-07 11:24:31 -07:00
Marcin Junczys-Dowmunt
2e6f0293ab
add legacy bdot
2021-06-07 11:23:03 -07:00
Marcin Junczys-Dowmunt
77c0cac1f2
broadcasting bdot
2021-06-07 09:14:39 -07:00
Hieu Hoang
f19ebbae69
debug
2021-06-04 14:21:26 -07:00
Hieu Hoang
28e5e2260a
filter once for shortlist
2021-06-04 13:39:03 -07:00
Rohit Jain
2c1b16f43e
Merged PR 19252: Update sentencepiece module to include CMake changes
...
Update SPM module to include CMake changes.
2021-06-04 10:13:00 +00:00
Roman Grundkiewicz
6e87f16e48
Merged PR 18763: Fix adding new validation metrics with --valid-reset-stalled
...
This fixes a bug that's been discovered recently by checking if a validator exists before resetting its stalled validations.
Regression test for it is in: https://github.com/marian-nmt/marian-regression-tests/pull/80
2021-05-26 06:12:33 +00:00
Young Jin Kim
9fa166be88
Online quantization ( #847 )
...
* Enable on-line packing/quantization
* Add half precision min/max quantization for model weights
* Change default quantization of B matrix to min/max, revert a false commit for AggregateAll
* Fixed missing half quantization
* Fix quantization range for A
* Set all default values for the quantize range to 0.f
* Use 7 bits clip for the weight matrix quantization to avoid an overflow of VPMADDUBSW
2021-05-24 22:51:37 -07:00
Marcin Junczys-Dowmunt
3133a9b27b
resolve conflict
2021-05-24 11:19:20 -07:00
Marcin Junczys-Dowmunt
84a20f65a1
Merge branch 'master' into pmaster
2021-05-24 11:17:53 -07:00
Marcin Junczys-Dowmunt
8b818b7c07
Avoid Ampere misaligment issue
2021-05-17 13:25:13 -07:00
Roman Grundkiewicz
fe74576dc3
Update VERSION
2021-05-04 12:36:37 +01:00
Nikolay Bogoychev
379212b75c
Enable compute86 where supported ( #863 )
...
* Enable compute86 where supported
2021-05-04 12:36:10 +01:00
Hieu Hoang
7faebf77ca
use args
2021-04-30 23:59:05 +00:00