@@ -10,11 +10,11 @@ Five is right out.
1010
1111💣 Machine learning which might blow up in your face 💣
1212
13−Grenade is a dependently typed, practical, and pretty quick neural network library for concise and precise
14−specifications of complex networks in Haskell.
13+Grenade is a dependently typed, practical, and pretty quick neural network
14+library for concise and precise specifications of complex networks in Haskell.
1515
16−As an example, a network which can achieve less than 1.5% error on MNIST can be specified and
17−initialised with random weights in a few lines of code with
16+As an example, a network which can achieve less than 1% error on MNIST can be
17+specified and initialised with random weights in a few lines of code with
1818```haskell
1919randomMnist :: MonadRandom m
2020 => m (Network '[ Convolution 1 10 5 5 1 1, Pooling 2 2 2 2, Relu, Convolution 10 16 5 5 1 1, Pooling 2 2 2 2, FlattenLayer, Relu, FullyConnected 256 80, Logit, FullyConnected 80 10, Logit]
@@ -22,22 +22,56 @@ randomMnist :: MonadRandom m
2222randomMnist = randomNetwork
2323```
2424
25−The network can be thought of as a heterogeneous list of layers, and its type signature includes a type
26−level list of the shapes of the data passed between the layers of the network.
25+And that's it. Because the types contain all the information we need, there's
26+no specific term level code required; although it is of course possible and
27+easy to build one oneself.
2728
28−In the above example, the input layer can be seen to be a two dimensional (`D2`) image with 28 by 28 pixels.
29−The last item in the list is one dimensional (`D1`) with 10 values, representing the categories of the mnist data.
29+The network can be thought of as a heterogeneous list of layers, where its type
30+includes not only the layers of the network, but also the shapes of data that
31+are passed between the layers.
3032
31−Layers in Grenade are represented as Haskell classes, so creating one's own is easy in downstream code. If the shapes
32−of a network are not specified correctly and a layer can not sensibly perform the operation between two shapes, then
33−it will result in a compile time error.
33+```haskell
34+data Network :: [*] -> [Shape] -> * where
35+ O :: Layer x i o => !x -> Network '[x] '[i, o]
36+ (:~>) :: Layer x i h => !x -> !(Network xs (h ': hs)) -> Network (x ': xs) (i ': h ': hs)
37+```
38+
39+In the above example, the input layer can be seen to be a two dimensional (`D2`),
40+image with 28 by 28 pixels. When the first *Convolution* layer runs, it outputs
41+a three dimensional (`D3`) 24x24x10 image. The last item in the list is one
42+dimensional (`D1`) with 10 values, representing the categories of the MNIST
43+data.
44+
45+Usage
46+-----
47+
48+To perform back propagation, one can call the eponymous function
49+```haskell
50+backPropagate :: forall input target shapes layers. (Head shapes ~ input, Last shapes ~ target)
51+ => Network layers shapes -> S' input -> S' target -> Gradients layers
52+```
53+which takes a network, appropriate input and target data, and returns the
54+back propagated gradients for the network. The shapes of the gradients are
55+appropriate for each layer, and may be trivial for layers like `Rulu` which
56+have no learnable parameters.
57+
58+The gradients however can always be applied, yielding a new (hopefully better)
59+layer with
60+```haskell
61+applyUpdate :: LearningParameters -> Network ls ss -> Gradients ls -> Network ls ss
62+```
3463
64+Layers in Grenade are represented as Haskell classes, so creating one's own is
65+easy in downstream code. If the shapes of a network are not specified correctly
66+and a layer can not sensibly perform the operation between two shapes, then
67+it will result in a compile time error.
3568
3669Build Instructions
3770------------------
38−Grenade currently only builds with the [mafia](https://github.com/ambiata/mafia) scipt that is located in the
39−repository. You will also need the `lapack` and `blas` libraries and development tools. Once you have all
40−that, Grenade can be build using:
71+Grenade currently only builds with the [mafia](https://github.com/ambiata/mafia)
72+script that is located in the repository. You will also need the `lapack` and
73+`blas` libraries and development tools. Once you have all that, Grenade can be
74+build using:
4175
4276```
4377./mafia build
@@ -54,19 +88,23 @@ Grenade is currently known to build with ghc 7.10 and 8.0.
5488
5589Thanks
5690------
57−Writing a library like this has been on my mind for a while now, but a big shout out must go to [Justin Le](https://github.com/mstksg), whose
58−[dependently typed fully connected network](https://blog.jle.im/entry/practical-dependent-types-in-haskell-1.html) inspired me to get cracking, gave many ideas for the type level tools I
91+Writing a library like this has been on my mind for a while now, but a big shout
92+out must go to [Justin Le](https://github.com/mstksg), whose
93+[dependently typed fully connected network](https://blog.jle.im/entry/practical-dependent-types-in-haskell-1.html)
94+inspired me to get cracking, gave many ideas for the type level tools I
5995needed, and was a great starting point for writing this library.
6096
6197Performance
6298-----------
63−Grenade is backed by hmatrix and blas, and uses a pretty clever convolution trick popularised by Caffe, which
64−is surprisingly effective and fast. So for many small scale problems it should be sufficient.
99+Grenade is backed by hmatrix and blas, and uses a pretty clever convolution
100+trick popularised by Caffe, which is surprisingly effective and fast. So for many
101+small scale problems it should be sufficient.
65102
66−That said, it's currently stuck on a single core and doesn't hit up the GPU, so there's a fair bit of performance
67−sitting there begging.
103+Being purely functional, it's probably pretty easy to parallelise and batch up.
104+My current examples however are single threaded.
68105
69−Training 15 generations over Kaggle's mnist training data took a few hours.
106+Training 15 generations over Kaggle's 42000 sample MNIST training set took under
107+an hour, achieving 0.5% error rate on a 1000 sample holdout set.
70108
71109Contributing
72110------------