marian-nmt/marian
1
1
Fork 0
mirror of https://github.com/marian-nmt/marian.git synced 2024-09-17 09:47:34 +03:00
Code Issues Packages Projects Releases Wiki Activity
4,889 Commits 41 Branches 10 Tags 44 MiB
a90950ea25
Commit Graph

262 Commits

Author SHA1 Message Date
Marcin Junczys-Dowmunt
9562338ff7 include nccl in source tree 2018-11-22 16:20:05 -08:00
Marcin Junczys-Dowmunt
17551afe80 merge with internal 2018-11-15 06:31:41 -08:00
Frank Seide
30ac255f38 fixed build error due to chrono 2018-11-10 10:47:04 -08:00
Marcin Junczys-Dowmunt
37822d60a8 add -g -rdynamic to Release compilation options to make stack backtrace readable 2018-11-08 00:13:26 -08:00
Marcin Junczys-Dowmunt
9af35a8a9d add back -Wl,--no-as-needed, MKL needs this apparently 2018-11-06 23:14:08 -08:00
Marcin Junczys-Dowmunt
61839a2122 remove a few warnings from clang with cuda 10.0 2018-11-06 11:11:54 -08:00
Marcin Junczys-Dowmunt
f6eb9cc569 add -Werror to nvcc, fix deprecated headers 2018-11-06 10:24:18 -08:00
Marcin Junczys-Dowmunt
8b8a49c312 small fixes to compile with different compilers 2018-11-06 08:36:48 -08:00
Frank Seide
015f392e82 reenabled -Wunused-variable and dealt with fallout 2018-11-05 21:24:02 -08:00
Frank Seide
fca0510133 enabled most warnings on gcc, and fixed them 2018-11-05 20:21:50 -08:00
Frank Seide
4fcdd191ab Merged PR 5379: bug fix: git revision should be updated accurately 
This PR separates the old `version.h` (now renamed to `project_version.h`, which is really meant for the project version and therefore gets created only at time of `cmake`, and a new process that creates a new file `git_revision.h` whenever the HEAD changes.

By using a clever trick from StackOverflow that makes `git_revision.h` depend directly on git's `HEAD` file, this new header file only gets recreated when the HEAD changes (e.g. a commit or checkout command). As a result, if the revision has does not change, a repeated `make` invocation will not compile or link anything.

This version is now logged directly at program startup, together with hostname, process id, and the command line.

What is still missing is a "dirty" flag in the version number.

Related work items: #96098
 2018-10-29 21:24:09 +00:00
Frank Seide
88d141060a added -Werror; added warning for missing 'override' and fixed the fallout; now always compiles quicksand.cpp to ensure it builds 2018-10-25 14:51:39 -07:00
Frank Seide
d906b9c759 made gcc happy 2018-10-24 18:06:54 -07:00
Frank Seide
7dd1763fae git_revision.h is now created 2018-10-24 17:10:57 -07:00
Frank Seide
ab8c3e9009 renamed version.h* to project_version.h*; created new version.cpp that is the only place that includes version header(s) 2018-10-24 15:24:08 -07:00
Ulrich Germann
ede2932b42 The previous fix didn't work on old nehalem CPUs. 
This fix lets the user freely specify the BUILD_ARCH for which to compile.
It also adds the switch -msse4.1, as that is required for compilation with MKL.
 2018-10-23 20:05:34 +01:00
Ulrich Germann
88915642d0 Allow compilation with -march=x86-64 -mavx instead of -march=native. 
The previous attempt (with -DCMAKE_BUILD_TYPE=Nonative) had no effect.
Now -DCOMPILE_NATIVELY=off allows compilation of portable binaries that
will also run on CPUs older than the one Marian was compiled on.
 2018-10-22 22:11:47 +01:00
Marcin Junczys-Dowmunt
6e1addb787 merge with multi-node branch 2018-10-12 16:43:33 -07:00
Marcin Junczys-Dowmunt
ed4e579f55 copy sentencepiece executables to main directory 2018-10-07 16:26:32 -07:00
Marcin Junczys-Dowmunt
a244acca24 Add comments to CMakeLists.txt 2018-10-07 16:09:29 -07:00
Marcin Junczys-Dowmunt
6cfe9ba0b6 surround more parts of SentencePiece with #ifdef 2018-10-07 15:43:54 -07:00
Frank Seide
4d26c3e363 fixed the link error 2018-10-04 21:42:08 -07:00
Frank Seide
108a3e8980 addressed some merge problems. This still does not link 2018-10-04 17:48:28 -07:00
Marcin Junczys-Dowmunt
188697ba0c move submodule checkout higher 2018-10-02 17:02:52 -07:00
Frank Seide
6b2579dd94 towards logging the call stack on segfault 2018-09-30 18:31:10 -07:00
Marcin Junczys-Dowmunt
cedbb71033 Merge branch 'master' into rm-boost-2 2018-09-27 20:28:48 -07:00
Roman Grundkiewicz
e2294a90a3 Remove tabs and whitespaces 2018-09-27 14:52:39 +01:00
Marcin Junczys-Dowmunt
5bc551c1a8
Merge branch 'master' into rm-boost-2 2018-09-26 13:42:07 -07:00
Cédric Rousseau
d4f2cb49bb Merge branch 'master' into build_on_win 2018-09-17 14:25:59 +02:00
Marcin Junczys-Dowmunt
7b45358c43 remove more obsolete references to boost 2018-09-16 21:48:43 +02:00
Marcin Junczys-Dowmunt
78c6c7722c add examples submodule, update README 2018-09-16 14:37:30 +02:00
Marcin Junczys-Dowmunt
5a42fdd3e6 checkout examples automatically 2018-09-16 09:50:53 -07:00
Cédric Rousseau
77516c9239 Update scripts to build on Windows 2018-09-14 15:52:42 +02:00
Marcin Junczys-Dowmunt
88d968fcc2 get rid of boost::hash_combine 2018-09-14 02:04:19 -07:00
Cédric Rousseau
2e38a2a919 Merge branch 'master' into build_on_win 2018-09-13 17:56:36 +02:00
Cédric Rousseau
a445635cbd Ninja and MSBuild compilation 2018-09-13 17:27:46 +02:00
Cédric Rousseau
f792b899dc Make it build directly on VS with CMake 2018-09-12 18:28:36 +02:00
Cédric Rousseau
e89995ee32 Fix Warnings 2018-09-07 17:58:45 +02:00
Cédric Rousseau
4fd2360918 Fix CUDA+MSVC incompatibility with Boost.Preprocessor 2018-09-05 15:42:16 +02:00
Cédric Rousseau
0659b2c5d5 Build in Release, with improved debug info 2018-09-04 16:58:49 +02:00
Cédric Rousseau
1ab5f0ccb7 Fix VS compiler flags 2018-09-04 16:44:04 +02:00
Roman Grundkiewicz
1a052ae99d Remove program_options from required Boost components 2018-08-24 16:52:17 +01:00
Marcin Junczys-Dowmunt
ac0cdc8502 add build type Nonative 2018-08-17 17:39:40 -07:00
Roman Grundkiewicz
c73a2cf6fb Fix compiler warnings 2018-08-01 16:43:05 +01:00
Marcin Junczys-Dowmunt
468678f545 change option name 2018-07-01 00:15:30 -07:00
Marcin Junczys-Dowmunt
f68cf82f64 clean NCCL support 2018-06-24 23:07:41 -07:00
Marcin Junczys-Dowmunt
afabe4e7e9 try to test nccl 2018-06-24 12:47:33 -07:00
Marcin Junczys-Dowmunt
ca73a7c0bd add batch-size based on words for decoder 2018-05-13 23:16:34 +00:00
Marcin Junczys-Dowmunt
1cffa82f2d small changes for shared task 2018-05-13 20:03:33 +00:00
Marcin Junczys-Dowmunt
992494426c handle regex problems with gcc-4.8 2018-04-18 15:14:10 -07:00
Marcin Junczys-Dowmunt
5b868db6db make static tests optional 2018-04-18 13:23:16 -07:00
Marcin Junczys-Dowmunt
5a9bb9792b continue linking against shared zlib 2018-04-18 12:07:22 -07:00
Marcin Junczys-Dowmunt
3f1667726e get rid of obsolete python depdencies 2018-04-18 10:17:11 -07:00
Marcin Junczys-Dowmunt
d49a711474 Prefer static linking wherever possible 2018-04-18 08:26:46 -07:00
Marcin Junczys-Dowmunt
54a533d45e disable type checking for tensor::set 2018-04-14 20:59:57 -07:00
Marcin Junczys-Dowmunt
f0b18f240f add CMake build profiles 2018-04-13 11:30:14 -07:00
Marcin Junczys-Dowmunt
07e5105075 update test 2018-04-11 23:18:00 -07:00
Marcin Junczys-Dowmunt
78a9947374 working memoization 2018-04-11 16:50:45 -07:00
Marcin Junczys-Dowmunt
9928625001 add gemm from sharp models, dull hardware 2018-04-08 22:13:30 -07:00
CatarinaSilva
ade15ca0a2 Fix CMakeLists.txt 2018-04-05 15:09:28 +01:00
Marcin Junczys-Dowmunt
83e9e8927a add missing flag to CUDA_NVCC_FLAGS 2018-03-04 17:50:21 -08:00
Marcin Junczys-Dowmunt
8ec6eef9d5 more coda separation 2018-03-02 18:06:26 -08:00
Roman Grundkiewicz
059a45e661 Add CMake option: COMPILE_CPU 2018-03-01 12:36:27 +00:00
Marcin Junczys-Dowmunt
36e8077a7d fix issue #183 by using boost regex, version prior to g++-4.9 fail 2018-02-26 10:38:27 -08:00
Marcin Junczys-Dowmunt
b241011336 make search for MPI optional 2018-02-25 22:08:25 -08:00
Marcin Junczys-Dowmunt
cead1b4feb Restrict to OpenBLAS 2018-02-24 21:03:06 -08:00
Marcin Junczys-Dowmunt
dbba0f220d add cudnn back 2018-02-24 20:11:02 -08:00
Marcin Junczys-Dowmunt
845063b342 re-enable mpi 2018-02-24 17:56:33 -08:00
Marcin Junczys-Dowmunt
ccbbbb95d3 fix FindMKL 2018-02-24 16:11:53 -08:00
Marcin Junczys-Dowmunt
73c38cf82c clean up cpu version 2018-02-22 17:51:52 -08:00
Marcin Junczys-Dowmunt
8d3f70599f batched sgemm 2018-02-21 22:08:36 -08:00
Marcin Junczys-Dowmunt
f224db8257 add FinMKL 2018-02-21 21:39:29 -08:00
Marcin Junczys-Dowmunt
d9d66f416b prototype cpu version 2018-02-21 17:44:04 -08:00
Marcin Junczys-Dowmunt
4834267989 clean up allocators and devices 2018-02-15 15:23:45 -08:00
Roman Grundkiewicz
ae827f8bbe Merge branch 'master' of https://github.com/tneck/marian-dev into tneck-master 2018-01-31 11:23:05 +00:00
Marcin Junczys-Dowmunt
bfe6839880 sqlite3-based corpus source 2018-01-23 18:35:55 -08:00
r0452425
0a1cdbc9d5 Add logic and compile config for multi-node 2018-01-21 07:06:51 +00:00
Marcin Junczys-Dowmunt
97ddb30948 disable server if no openssl found 2018-01-13 11:24:03 -08:00
Roman Grundkiewicz
d9063529ea Add COMPILE_SERVER option to CMake 2018-01-09 14:18:57 +00:00
Tomasz Dwojak
1fbd30674a Merge branch 'master' of https://github.com/marian-nmt/marian-dev into charS2S 2017-11-27 13:26:01 +00:00
Hieu Hoang
fe7ca63a66 compilable with cuda 8 2017-11-24 17:13:19 +00:00
Hieu Hoang
a1b13e8cf0 gpu arch consistent with amun 2017-11-24 16:27:56 +00:00
Marcin Junczys-Dowmunt
39e15a6d3e remove tcmaloc_minimal 2017-11-24 17:06:25 +01:00
Tomasz Dwojak
537fccc3e8 Merge branch 'master' of https://github.com/marian-nmt/marian-dev into charS2S 2017-11-23 11:26:38 +00:00
Marcin Junczys-Dowmunt
e378a764ce update changelog 2017-11-21 09:45:46 +00:00
Tomasz Dwojak
8d33f14785 Set cudnn version at least 7 2017-11-15 23:30:41 +00:00
Marcin Junczys-Dowmunt
c8c7f2f62a remove lib 2017-11-02 23:25:54 +01:00
Marcin Junczys-Dowmunt
7a80d2a390 Merge branch 'master' into functional 2017-11-02 21:06:42 +01:00
Marcin Junczys-Dowmunt
ca64c429e4 new functional templates 2017-11-02 20:37:43 +01:00
Roman Grundkiewicz
a3b72c2ffe Add TCMalloc 2017-11-02 10:33:47 +00:00
Marcin Junczys-Dowmunt
0d20bb57c1 clean up permutation 2017-11-01 22:33:47 +01:00
Marcin Junczys-Dowmunt
c7d236762a more gpu-specific TMP 2017-11-01 20:09:42 +01:00
Marcin Junczys-Dowmunt
4f14c9175a unroll loop for variable GPU shape length 2017-11-01 10:28:14 +01:00
Marcin Junczys-Dowmunt
0daa611030 simplify shape 2017-11-01 01:07:02 +01:00
Roman Grundkiewicz
1f9fef5836 Show commit SHA with the project version 2017-08-30 13:35:47 +00:00
Roman Grundkiewicz
41f99c5136 Clean Boost components 2017-08-19 13:49:55 +01:00
Roman Grundkiewicz
2214a90181 Make OpenSSL library optional 2017-08-19 12:55:01 +01:00
Roman Grundkiewicz
95fe229d26 Add Simple-WebSocket-Server library 2017-08-19 08:01:31 +01:00
Roman Grundkiewicz
6bf7a14f6d Add python module returning version 2017-08-18 18:04:10 +01:00
Roman Grundkiewicz
7aaa3ba985 Switch to manual project versioning 2017-07-13 13:02:43 +01:00
Marcin Junczys-Dowmunt
0f4d113148 revert spdlog submodule for now 2017-07-11 16:27:45 +00:00
Ulrich Germann
0b0d2a4b18 Automatic submodule initialization should work now. 2017-07-10 14:12:08 +01:00
Ulrich Germann
e46aa14ae8 Made spdlog a submodule that will be updated automatically when cmake is run. 2017-07-09 00:40:56 +01:00
Roman Grundkiewicz
dfcb4ab1ba Add project versioning 2017-07-04 14:06:26 +01:00
Marcin Junczys-Dowmunt
b42141d613 merge with master 2017-06-29 21:43:25 +02:00
Roman Grundkiewicz
51618fec22 Add USE_CUDNN options to CMake; fix tests and examples compilation 2017-06-27 15:20:32 +01:00
Marcin Junczys-Dowmunt
7e79bd5d73 temporarily disable CUDNN, add missing ifdef 2017-06-26 19:15:07 +02:00
Tomasz Dwojak
ad468d55e8 Add CUDNN to CMakeLists 2017-06-21 11:10:51 +00:00
Marcin Junczys-Dowmunt
84c3121885 some more refactoring 2017-06-17 21:22:27 +02:00
Roman Grundkiewicz
3f51d65bd6 Add a framework for unit tests 2017-06-14 14:06:20 +02:00
Marcin Junczys-Dowmunt
77dfe53ed1 timing and compilation options 2017-06-02 12:43:36 +02:00
Marcin Junczys-Dowmunt
eae71b42e4 remove cudaHostRegister - +10% speed for multi-gpu 2017-06-01 00:03:55 +02:00
Roman Grundkiewicz
e7bcc2ea57 Add Iris example 2017-05-29 12:50:10 +02:00
Marcin Junczys-Dowmunt
7300b52727 more ensembles 2017-05-13 19:11:40 +02:00
Marcin Junczys-Dowmunt
e466abce2f scorers and ensembling 2017-05-10 19:43:00 +02:00
Marcin Junczys-Dowmunt
a2b10c9ecd fix scripts 2017-04-23 23:09:03 +02:00
Marcin Junczys-Dowmunt
98c84dd8df fix missing embeddings CDI transducer 2017-04-23 21:00:07 +02:00
Marcin Junczys-Dowmunt
e75d0dc748 lexprobs 2017-04-19 17:08:44 +02:00
Marcin Junczys-Dowmunt
63ed92d7ce translation-based validation 2017-04-13 14:15:14 +02:00
Ulrich Germann
8498134cf0 set(CMAKE_LIBRARY_PATH /usr/local/cuda-8.0/lib64) 2017-03-29 23:51:50 +01:00
Marcin Junczys-Dowmunt
ba91eda85d fix cmake 2017-03-23 20:30:21 +01:00
Ulrich Germann
d80502cd53 Allow Marian being built as library-only (no executables) within Amunmt. 
Building executables also may be possible, but will require fiddling with the cmake setup.
 2017-03-22 01:53:45 +00:00
Marcin Junczys-Dowmunt
af1177eab9 fixes #42 and #41 2017-03-20 16:46:16 +01:00
Marcin Junczys-Dowmunt
cdd4bd087c handle curand better in cmake 2017-03-02 16:34:05 +01:00
Marcin Junczys-Dowmunt
fcd99c49f7 scaling dropout across rnn time steps and layers 2017-02-26 20:24:17 +00:00
Marcin Junczys-Dowmunt
61db9defe3 beam search 2017-02-20 00:24:12 +00:00
Marcin Junczys-Dowmunt
a0daeff6d3 towards beam search 2017-02-18 23:04:29 +00:00
Marcin Junczys-Dowmunt
07bcb1d9d2 restore compile flags 2017-02-18 17:53:25 +00:00
Marcin Junczys-Dowmunt
e84c6d63fe greedy search 2017-02-18 17:52:45 +00:00
Tomasz Dwojak
d1e90650dc Small changes in dropout tests 2017-02-13 09:56:03 +00:00
Tomasz Dwojak
bc73140de3 Work on dropout 2017-02-06 14:06:05 +00:00
Marcin Junczys-Dowmunt
5f6889183c proper model loading for graph groups, some more refactoring, fixed tensor test 2017-02-06 00:22:08 +00:00
Marcin Junczys-Dowmunt
2ad81a1de0 enabled stream per thread 2017-02-04 00:26:34 +00:00
Marcin Junczys-Dowmunt
516531f9d0 preparations for multi-gpu 2017-02-01 01:07:33 +00:00
Marcin Junczys-Dowmunt
7646a3c01e works on different gpus 2017-01-28 23:47:59 +00:00
Marcin Junczys-Dowmunt
3ea3a46847 fixed order for cuda includes in CMake 2017-01-21 15:11:40 +00:00
Marcin Junczys-Dowmunt
3d7912e3fd added architecture for GPU 2017-01-21 14:07:58 +00:00
Marcin Junczys-Dowmunt
9bcbef8cc9 gradient clipping and fast reduce 2017-01-21 12:31:48 +00:00
Marcin Junczys-Dowmunt
8a5bfbf28f simplify code, remove clutter, prepare for grouped processing 2017-01-12 01:44:53 +01:00
Marcin Junczys-Dowmunt
6410d2a937 simple command line interface 2017-01-11 19:32:28 +01:00
Hieu Hoang
ec47eef77d compile error 2017-01-07 17:26:47 +00:00
Marcin Junczys-Dowmunt
566be8ebc1 towards working batching for parallel corpora 2017-01-05 20:42:49 +01:00
Marcin Junczys-Dowmunt
619fff5902 refactored a bit, added saving to nematus format, trai first models on three sentences 2017-01-04 23:14:44 +01:00
Marcin Junczys-Dowmunt
7fc232e3d5 a number of optimizations and bug fixes 2016-12-28 15:30:16 +00:00
Marcin Junczys-Dowmunt
07134aef36 improved reshaping for multi-dimensional tensors when doing 2D dot product 2016-12-26 21:53:25 +00:00
Marcin Junczys-Dowmunt
cbf57e13c5 more work on nematus reimplementation, temporary fix for mnist 2016-12-02 16:26:11 +01:00
Marcin Junczys-Dowmunt
59b434e479 towards finished decoder 2016-11-29 18:01:24 +01:00
Marcin Junczys-Dowmunt
47859e4ecd fast forward for GRU 2016-11-16 02:26:06 +01:00
Marcin Junczys-Dowmunt
7b3bd14893 working memory allocation 2016-11-13 00:11:52 +01:00
Marcin Junczys-Dowmunt
c74e749698 partially fixed memory problem 2016-11-12 02:12:35 +01:00