Roman Grundkiewicz
6f7766f837
Merged PR 25465: Choose top checkpoints from train.log for averaging
...
Added `--from-log logfile N metric asc|desc` option to `average.py`, which selects top N checkpoint paths from the provided train.log file according to the selected metric. Last 3 arguments to this option are optional. If the last argument is omitted, "asc" is assumed for perplexity and "desc" for other metrics.
2022-09-15 06:19:18 +00:00
Alex Muzio
a90950ea25
Merged PR 25154: Add model shapes flag to model_info.py script
...
Add model shapes flag to model_info.py script through `--matrix-shapes` flag
This will print something like:
```
...
encoder_l6_ffn_W1 (1024, 4096)
encoder_l6_ffn_W2 (4096, 1024)
encoder_l6_ffn_b1 (1, 4096)
encoder_l6_ffn_b2 (1, 1024)
encoder_l6_ffn_ffn_ln_bias (1, 1024)
encoder_l6_ffn_ffn_ln_scale (1, 1024)
encoder_l6_self_Wk (1024, 1024)
encoder_l6_self_Wo (1024, 1024)
encoder_l6_self_Wo_ln_bias (1, 1024)
encoder_l6_self_Wo_ln_scale (1, 1024)
encoder_l6_self_Wq (1024, 1024)
encoder_l6_self_Wv (1024, 1024)
encoder_l6_self_bk (1, 1024)
encoder_l6_self_bo (1, 1024)
encoder_l6_self_bq (1, 1024)
encoder_l6_self_bv (1, 1024)
special:model.yml (1264,)
```
2022-08-10 22:23:47 +00:00
Roman Grundkiewicz
e0e3287a3b
Merged PR 23840: Update CUDA installation script for Ubuntu
...
Updates CUDA deb/key fetching
https://developer.nvidia.com/blog/updating-the-cuda-linux-gpg-repository-key/
2022-05-12 16:23:58 +00:00
Marcin Junczys-Dowmunt
b0275e7754
merge with internal master
2022-02-11 06:03:16 -08:00
Marcin Junczys-Dowmunt
b3feecc82b
Merged PR 22483: Make C++17 the official standard for Marian
...
Make C++17 the official standard for Marian
2022-02-10 16:34:23 +00:00
Graeme Nail
4d44627f26
PyYaml safe_load instead of load ( #913 )
...
* pyyaml safe_load instead of load
* Update CHANGELOG
2022-02-10 11:20:27 +00:00
Roman Grundkiewicz
b97645846a
Update release workflow ( #915 )
...
* Add CUDA 11.x to Windows installation script
* Update release.yml workflow
2022-02-09 18:56:56 +00:00
Roman Grundkiewicz
8d0a3c0c27
Add --allow-unauthenticated when installing CUDA ( #878 )
2021-09-07 11:11:58 +01:00
Roman Grundkiewicz
3b468e4628
Remove ::set-env from GitHub checks for Windows ( #766 )
2020-11-19 20:12:28 +00:00
Roman Grundkiewicz
a21e48fb4f
Add GitHub workflows with CUDA and MKL ( #700 )
...
* Add GitHub workflows with CUDA
* Install MKL on Windows
* Upload CPU-only executables from Windows and Ubuntu as artifacts
* Clean up workflows
2020-08-21 18:50:07 +01:00
Frank Seide
00f10c2288
Merged PR 14593: bug fix in ONNX exporter
...
The number of layers was confused with the number of output states.
2020-08-06 03:35:53 +00:00
Frank Seide
38bd181937
Merged PR 14349: edited some comments in ONNX converter
...
edited some comments in ONNX converter
2020-07-24 18:12:28 +00:00
Frank Seide
435aa9505e
Merged PR 14334: full ONNX conversion script
...
This PR adds a full ONNX conversion script that exports a Marian model and wraps it in a greedy-search implemented in ONNX.
2020-07-24 17:23:05 +00:00
Martin Junczys-Dowmunt
c3fb60cbcd
Merged PR 13476: Add LASER reimplementation and code for embeddings sentences
...
This reimplements the LASER encoder from:
```
Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond
Mikel Artetxe, Holger Schwenk
https://arxiv.org/abs/1812.10464
```
and adds functionality to embed sentences with any Marian encoder, also different from LASER. Some early attempts to train a transformer model with Encoder-Decoder bottle-neck. This is quite early code, so some code-duplication is to be expected. Nevertheless, it's functional and I would like to have it in master as we will slowly put that into production in various places. I will make the code "nicer" as we go along.
2020-06-24 01:54:27 +00:00
Frank Seide
6d2bfa68c0
Merged PR 13081: minor regression fix in ONNX expoeter
...
This fixes a silly little regression that snuck in to the last commit.
2020-05-27 06:07:28 +00:00
Frank Seide
c8a62dd2c8
Merged PR 13053: minor updates to ONNX code
...
will not affect anything else
2020-05-26 17:51:44 +00:00
Frank Seide
f9029638ff
Merged PR 13039: FAISS builds with Windows
...
FAISS builds with Windows
2020-05-22 02:49:04 +00:00
Frank Seide
77a420740c
Merged PR 12958: ONNX support
...
This branch adds functionality to export ONNX models (with limitations).
2020-05-21 05:51:18 +00:00
Roman Grundkiewicz
f2347a827f
Update Simple-WebSocket-Server and move it to submodules ( #639 )
...
* Fix server build with current boost, move simple-websocket-server to submodule
* Change submodule to marian-nmt/Simple-WebSocket-Server
* Update submodule simple-websocket-server
Co-authored-by: Gleb Tv <glebtv@gmail.com>
2020-04-27 10:34:10 +01:00
Kenneth Heafield
3c0c1e133b
python3 shebang from #620 ( #621 )
...
* python3 shebang from #620
* Add changelog entry for python3 change
2020-04-16 11:15:42 +01:00
Roman Grundkiewicz
533604024b
Update script exporting embeddings to support tied embeddings ( #569 )
2020-01-29 13:19:21 -08:00
Marcin Junczys-Dowmunt
b55c257af6
address comments from PR
2019-09-04 12:48:44 -07:00
Marcin Junczys-Dowmunt
c6740287f3
merge CMakeLists.txt
2019-09-03 11:11:05 -07:00
Roman Grundkiewicz
993025327f
Add script for checkpoint averaging
2019-07-19 14:29:21 +01:00
Roman Grundkiewicz
ec2d66e852
Add execute permission
2019-05-27 14:52:11 +01:00
Frank Seide
356f804293
minor bug fixes, to get last few changes to pass tests
2019-05-15 11:43:20 -07:00
Marcin Junczys-Dowmunt
209c1d4a4e
add back execution rights for python/perl scripts
2019-04-30 12:52:13 -07:00
Frank Seide
eacfd2de11
weird mode change back
2019-04-29 19:01:29 -07:00
Frank Seide
cd329be400
weird mode change
2019-04-29 17:52:23 -07:00
Frank Seide
b570638533
weirdo change of access permissions
2019-04-26 14:59:40 -07:00
Marcin Junczys-Dowmunt
d121ba4726
address code review comments
2019-02-04 20:26:46 -08:00
Marcin Junczys-Dowmunt
abe9467471
hacky Nan handling
2019-01-27 23:03:46 -08:00
Marcin Junczys-Dowmunt
83fbd248d0
rename bert-specific options
2019-01-27 12:43:04 -08:00
Marcin Junczys-Dowmunt
cb8c249ec6
added checking of number of layers
2019-01-26 15:24:55 -08:00
Marcin Junczys-Dowmunt
664e87624a
first try for bert to marian conversion script
2019-01-26 14:33:27 -08:00
Roman Grundkiewicz
8ced995144
Add scripts generating shortlists
2019-01-24 18:44:26 +00:00
Roman Grundkiewicz
df526c1fe3
Use single connection
2018-10-19 12:04:10 +01:00
Roman Grundkiewicz
aed31cd5d5
Add script for injecting 'decoder_c_tt'
2018-10-09 13:36:17 +00:00
Dominik Stańczak
e95320a6d1
Remove leftover references to Python bindings
2018-06-06 20:04:47 +02:00
Roman Grundkiewicz
9101c00d31
Add option -f for printing whole arrays
2018-04-25 08:12:06 +00:00
Roman Grundkiewicz
ba0ea7491f
Add printing value for any key from model.npz
2018-01-13 11:12:27 +00:00
Roman Grundkiewicz
43fbaa6c10
Add scripts printing special:mode.yml from model.npz
2018-01-12 11:39:19 +00:00
Roman Grundkiewicz
89b23ce8ab
Fix decoding error with python2
2018-01-11 20:45:54 +00:00
Kenneth Heafield
76c8bb541e
chmod +x client
2017-11-13 00:05:12 +00:00
Roman Grundkiewicz
bbb96e2ad7
Rename script name
2017-11-07 12:47:15 +00:00
Roman Grundkiewicz
040caa0f6e
Remove unwanted newline
2017-09-04 09:44:54 +00:00
Roman Grundkiewicz
9c30760615
Add script injecting s2s options to nematus .npz models
2017-09-01 14:13:38 +00:00
Roman Grundkiewicz
a6b5c27a75
Update python example scripts
2017-08-19 12:24:59 +01:00
Roman Grundkiewicz
036693b531
Update python client
2017-08-19 12:15:11 +01:00
Roman Grundkiewicz
585c3f26f0
Fix python bindings
2017-08-19 12:11:49 +01:00