@@ -10,21 +10,22 @@ Five is right out.
1010
1111💣 Machine learning which might blow up in your face 💣
1212
13−Grenade is a dependently typed, practical, and pretty quick neural network
14−library for concise and precise specifications of complex networks in Haskell.
13+Grenade is a dependently typed, practical, and fast neural network library for
14+concise and precise specifications of complex networks in Haskell.
1515
16−As an example, a network which can achieve less than 1% error on MNIST can be
16+As an example, a network which can achieve ~1.5% error on MNIST can be
1717specified and initialised with random weights in a few lines of code with
1818```haskell
19−randomMnist :: MonadRandom m
20− => m (Network '[ Convolution 1 10 5 5 1 1, Pooling 2 2 2 2, Relu, Convolution 10 16 5 5 1 1, Pooling 2 2 2 2, FlattenLayer, Relu, FullyConnected 256 80, Logit, FullyConnected 80 10, Logit]
21− '[ 'D2 28 28, 'D3 24 24 10, 'D3 12 12 10, 'D3 12 12 10, 'D3 8 8 16, 'D3 4 4 16, 'D1 256, 'D1 256, 'D1 80, 'D1 80, 'D1 10, 'D1 10])
19+type MNIST = Network '[ Convolution 1 10 5 5 1 1, Pooling 2 2 2 2, Relu, Convolution 10 16 5 5 1 1, Pooling 2 2 2 2, FlattenLayer, Relu, FullyConnected 256 80, Logit, FullyConnected 80 10, Logit]
20+ '[ 'D2 28 28, 'D3 24 24 10, 'D3 12 12 10, 'D3 12 12 10, 'D3 8 8 16, 'D3 4 4 16, 'D1 256, 'D1 256, 'D1 80, 'D1 80, 'D1 10, 'D1 10]
21+
22+randomMnist :: MonadRandom m => m MNIST
2223randomMnist = randomNetwork
2324```
2425
25−And that's it. Because the types contain all the information we need, there's
26−no specific term level code required; although it is of course possible and
27−easy to build one oneself.
26+And that's it. Because the types are rich, there's no specific term level code
27+required; although it is of course possible and easy to construct one explicitly
28+oneself.
2829
2930The network can be thought of as a heterogeneous list of layers, where its type
3031includes not only the layers of the network, but also the shapes of data that
@@ -36,6 +37,9 @@ data Network :: [*] -> [Shape] -> * where
3637 (:~>) :: Layer x i h => !x -> !(Network xs (h ': hs)) -> Network (x ': xs) (i ': h ': hs)
3738```
3839
40+The `Layer x i o` constraint ensures that the layer `x` can sensibly perform a
41+transformation between the input and output shapes `i` and `o`.
42+
3943In the above example, the input layer can be seen to be a two dimensional (`D2`),
4044image with 28 by 28 pixels. When the first *Convolution* layer runs, it outputs
4145a three dimensional (`D3`) 24x24x10 image. The last item in the list is one
@@ -96,15 +100,16 @@ needed, and was a great starting point for writing this library.
96100
97101Performance
98102-----------
99−Grenade is backed by hmatrix and blas, and uses a pretty clever convolution
100−trick popularised by Caffe, which is surprisingly effective and fast. So for many
101−small scale problems it should be sufficient.
103+Grenade is backed by hmatrix, BLAS, and LAPACK, with critical functions optimised
104+in C. Using the im2col trick popularised by Caffe, it should be sufficient for
105+many problems.
102106
103−Being purely functional, it's probably pretty easy to parallelise and batch up.
104−My current examples however are single threaded.
107+Being purely functional, it should also be easy to run batches in parallel, which
108+would be appropriate for larger networks, my current examples however are single
109+threaded.
105110
106−Training 15 generations over Kaggle's 42000 sample MNIST training set took under
107−an hour, achieving 0.5% error rate on a 1000 sample holdout set.
111+Training 15 generations over Kaggle's 41000 sample MNIST training set on a single
112+core took around 12 minutes, achieving 1.5% error rate on a 1000 sample holdout set.
108113
109114Contributing
110115------------