This PR enables loading and serving of offline packed models.
* Enable support for FBGEMM offline-packed models
* Added different parameters sets for different parameter element types (currently inference only)
* New types (packed*) are used for the packed memory format.
* Added CPU instruction query into the quicksand API.
Related work items: #102572
This is a helper PR for making Young's packed models cleaner. Parameter objects in Marian are meant to be of one type (say float32) and cannot be correctly mixed for various reasons. This introduces a type -> parameter object map that separates parameters with different types into different spaces. Currently this only works reliably for inference (I hope) and will abort for training if more than one parameter type are specified. With this we will be able to have a parameter object for packed16 and one for float32.
It should also be possible in the future to have the same name for a parameter but differentiated by type, which might solve the tied-embedding problem we discussed a while ago.
* Use ccache only when requested via cmake -DUSE_CCACHE=on
* Add link to https://ccache.dev in comment about using ccache.
* Issue success / missing ccache message when ccache is requested during the CCACHE run
* Issue cmake warning instead of cmake status message when use of ccache is requested but ccache cannot be found.
Make everything compile as far as possible on gcc 5.0 to 8.3 and CUDA 8.0 to 10.1.
Also created this compatibility/bug list. BUG is a bug in GCC or CUDA, not Marian.
```
CUDA
8.0 9.0 9.2 10.0 10.1 comment
gcc <5.0 NO incompatible with c++11 features (codecvt)
5.4 YES YES YES YES YES Any version of gcc 5 smaller than 5.4 should be fine too
5.5 NO/BUG NO/BUG NO/BUG YES YES bugs in GCC and CUDA<9.2
6.5 NO NO/BUG YES YES YES bug in GCC and CUDA 9.0
7.4 NO NO YES YES YES
8.3 NO NO NO NO YES warnings in 3rd-party code
9.2 NO NO NO NO NO not yet supported by CUDA
TC FP16 FP16 FP16
```
Update internal master with changes to public master. Includes a few PRs accepted in public master. Preparation to merge internal changes back to public master.
I don't think there is actually anything to review here.
* Remove hard dependency on Boost
* Now optional:
* Compiling with g++-4.9 still requires boost::regex
* Compiling marian_server requires boost::asio (marian_server is optional now)
* Handled automatically by cmake.
