Eg (given that my local bin path for the project is set to ./.bin in my stack.yaml):
$ ./install.hs
<lots out output>
$ ls .bin
haskell-code-indexer-8.0.2* haskell-code-indexer-8.4.4*
haskell-code-indexer-8.2.2* haskell-code-indexer-8.6.3*
haskell-code-indexer-8.4.3* haskell-code-server*
Note that it works by installing and then moving the `haskell-code-indexer` executable in your project's bin to `haskell-code-indexer-X-Y-Z`. So if you already have a plain `haskell-code-indexer` executable before the run it will be deleted.
It uses plain IO instead of Shake because Shake runs actions in unpredictable order. One of the goals of the script is that it will install haskell-code-server and the latest haskell-code-indexer as soon as possible, so that it's useful even if you have to cancel the run for some reason. Shake was running the haskell-code-indexer command at the correct time, but then waiting to copy it to its -X-Y-Z final home until near the end of the run.
(Also this is really just a simple install script, we don't need another layer of caching from Shake for it.)
This commit adds an option to create an on-disk key-value store that contains all the data from PackageInfo of each indexed package in a queriable form. The store can be used by haskell-code-server to respond to API requests. The main benefit of using the store, compared to deserializing and loading PackageInfo of each package into memory, is reduced memory usage (approximately 7 times for a set of Haskell packages).
The key-value store on disk consists of two files: 'index' and 'values'. 'index' is a small file that contains a map from strings to locations in a 'values' file. 'index' file should be deserialized and loaded into memory. 'values' is a large file that contains serialized Haskell data structures. 'values' file can be either read directly (without deserializing) or memory-mapped.
Vector.fromListN allocates less memory and allows to remove the call to Vector.force before compaction ('compact $ Vector.fromList [1,2,3]' crashes, 'compact $ Vector.fromListN 3 [1,2,3]' doesn't crash)
Building cabal-helper-0.8.1.2 with Stack failed with 'Dependency cycle detected' error. It seems to be https://github.com/commercialhaskell/stack/issues/4265 Stack bug. As a temporary solution I added source code of cabal-helper package to vendor directory and commented out 'build-tool-depends: cabal-helper:cabal-helper-wrapper' line in the cabal-helper.cabal file.
