Hieu Hoang
7159f45ab7
make sure file open even if it doesn't exist
2019-10-23 13:28:08 -07:00
Ulrich Germann
9a89c25664
Roll-back of all changes not directly related to graceful shutdown.
2019-10-23 20:58:51 +01:00
Hieu Hoang
46b8ebae24
delete old temp file class
2019-10-23 11:25:53 -07:00
Hieu Hoang
8f591f67a2
use new temp file class everywhere
2019-10-23 11:21:06 -07:00
Ulrich Germann
d1ed2c2506
More helpful error message when CUDA libraries cannot be found.
2019-10-23 18:54:34 +01:00
Hieu Hoang
f4056af240
use new temp file class in corpus
2019-10-23 10:51:02 -07:00
Hieu Hoang
999b8fa0bb
start temp file class
2019-10-23 09:51:37 -07:00
Hieu Hoang
cdf9b5281f
debuggging message
2019-10-23 09:51:12 -07:00
Hieu Hoang
33f60022cf
delete OutputFileStream and other crappy classes
2019-10-22 17:56:09 -07:00
Hieu Hoang
c5c1a4c449
replace all OutputFileStream with OutputFileStreamNew
2019-10-22 17:50:32 -07:00
Hieu Hoang
1f53deaefd
flush both filters
2019-10-22 17:22:23 -07:00
Hieu Hoang
e4182e5446
flush stream to make sure everything comes out. Doens't work
2019-10-22 16:38:49 -07:00
Hieu Hoang
908dcad06f
try using OutputFileStream
2019-10-22 16:18:31 -07:00
Hieu Hoang
a10f5fbe6e
start OutputFileStream
2019-10-22 16:03:06 -07:00
Hieu Hoang
b8e2f06be3
delete InputFileStream
2019-10-22 15:44:13 -07:00
Hieu Hoang
d342e44818
replace all InputFileStream with InputFileStreamNew
2019-10-22 15:38:28 -07:00
Hieu Hoang
750d204e37
replace InputFileStream with InputFileStreamNew in everything except Corpus classes
2019-10-22 15:11:57 -07:00
Hieu Hoang
84ce4cd64d
replace InputFileStream with InputFileStreamNew in some files
2019-10-22 14:35:39 -07:00
Hieu Hoang
61d7ba4d4d
delete file buffer in destructor
2019-10-22 12:27:39 -07:00
Hieu Hoang
8e8aae3cd9
use std::filebuf and InputFileStream derives from istream rather than zstr::ifstream. Optional gzip filter
2019-10-22 11:34:53 -07:00
Frank Seide
0baf0aeeb3
Merged PR 9998: new build option NO_BOOST to disable inclusion of Boost and all features that
...
This PR is to allow Marian to be built on Windows without Boost, since Boost 1.66 causes build errors under Flo.
With this PR, if the preprocessor symbol `NO_BOOST` is defined, Boost will not be included anywhere, and configurations that rely on that cannot be used.
The long-term goal is to remove Boost completely. This `#define` allows to easily locate all relevant locations.
Boost is currently used for two things:
* the timer in the AutoTuner
* when shuffling the corpus to a temp file (not --shuffle-in-ram)
With `NO_BOOST`, attempting to shuffle to file will terminate the program, and auto-tuner will select the first algorithm.
Not tested with Linux.
2019-10-22 17:28:05 +00:00
Hieu Hoang
5896a41fbc
debug
2019-10-21 18:14:59 -07:00
Hieu Hoang
0e8117600b
add OutputFileStreamNew class
2019-10-21 15:02:51 -07:00
Hieu Hoang
01da99fc52
add InputFileStreamNew class
2019-10-21 14:57:08 -07:00
Hieu Hoang
532c0d1771
base arg
2019-10-21 14:15:09 -07:00
Hieu Hoang
6e27d021f4
add new TemporaryFile class
2019-10-21 14:05:48 -07:00
Hieu Hoang
7a838161c6
move code .h -> .cpp
2019-10-21 12:24:08 -07:00
Hieu Hoang
c33074ce54
move code .h -> .cpp
2019-10-21 12:17:35 -07:00
Hieu Hoang
14ae130a6b
move code .h -> .cpp
2019-10-21 11:26:50 -07:00
Hieu Hoang
fef5c301ca
figure out where temp file class is used
2019-10-21 10:48:14 -07:00
Ulrich Germann
1f461c0ea2
Merge branch 'master' into ug-graceful-shutdown2
2019-10-16 22:43:51 +01:00
Frank Seide
2111c28036
Merge pull request #420 from marian-nmt/fix_default_comm
...
reset gradients in default comm.
2019-10-16 08:36:48 -07:00
Hieu Hoang
f8061fc298
file stream without boost /Ken
2019-10-08 18:23:10 -07:00
Hieu Hoang
53f5a60dbc
Merge branch 'hieu/fdstream' of vs-ssh.visualstudio.com:v3/machinetranslation/Marian/marian-dev into hieu/fdstream
2019-10-08 17:37:06 -07:00
Hieu Hoang
0bad3f9a15
test commit
2019-10-08 17:36:57 -07:00
Frank Seide
718327847e
Merged PR 9826: fix .gz support on Windows
...
On Windows, we must explicitly select binary file mode for .gz support to work.
2019-10-09 00:29:37 +00:00
Frank Seide
51f8c41e2a
Merge branch 'master' of https://machinetranslation.visualstudio.com/DefaultCollection/Marian/_git/marian-dev into fseide/fixgzwin
2019-10-08 17:27:21 -07:00
Frank Seide
e897f913b6
bug fix: reading and writing .gz files on Windows should explicitly specify binary mode
2019-10-08 17:27:00 -07:00
Marcin Junczys-Dowmunt
cd18ee0396
safeguard cmake against rogue doxygen
2019-10-08 15:54:59 -07:00
Martin Junczys-Dowmunt
328ec7c1be
Merged PR 9529: FP16 inference
...
Gets FP16 inference to work on Pascal and Volta GPUs
* Extended gpu kernels to work with different types
* Handle compilation problems for different compilers, cuda versions etc. Probably some issues to be discovered.
* Translation can be turned on via --fp16
2019-10-07 23:46:52 +00:00
Marcin Junczys-Dowmunt
12cd08af12
merge with master
2019-10-07 16:35:58 -07:00
Marcin Junczys-Dowmunt
dfbf4f403a
remove wrong abort
2019-10-07 15:52:48 -07:00
Marcin Junczys-Dowmunt
834905411e
address code review comments
2019-10-07 15:49:46 -07:00
Frank Seide
8b2d295e5e
Merged PR 9555: bug fix: workaround for a bug fix in FactoredSegmenter that caused a backcompat
...
bug fix: workaround for a bug fix in FactoredSegmenter that caused a backcompat issue
Also temporarily fixes Windows build issues with the current branch (these will be updated soon by Young or Marcin)
2019-09-18 20:33:20 +00:00
Frank Seide
bc146302f3
bug fix: workaround for a bug fix in FactoredSegmenter that caused a backcompat issue
2019-09-18 13:30:14 -07:00
Marcin Junczys-Dowmunt
efed4edf3d
some more clean-up
2019-09-17 14:15:33 -07:00
Marcin Junczys-Dowmunt
4d10ae9212
small changes to cmake
2019-09-17 13:46:52 -07:00
Marcin Junczys-Dowmunt
16e6fb9fae
small clean up, move fbgemm pointer
2019-09-17 11:30:46 -07:00
Marcin Junczys-Dowmunt
f35886ce8e
resolve conflicts
2019-09-17 09:55:29 -07:00
Marcin Junczys-Dowmunt
b948cd63db
missing part for fp16 inference, some clean-up for types
2019-09-17 09:53:45 -07:00
