git.iko.soy/stackage
on lts help sign in
stackage/CURATORS.md
105 lines 5.7 kB history raw
1This is a collection of instructions for what Stackage curators- the guys who
2maintain the Stackage project itself- should be doing on a regular basis. At
3the time of writing (July 2015), this is handled almost entirely by Michael
4Snoyman. Curation activities are mostly automated, and do not take up a
5significant amount of time. But it's good to spread the knowledge for obvious
6reasons (errant buses, if I ever decide to take a vacation...).
8## Workflow
10This section sketches out at a high level how the entire Stackage build/curation
11process works:
13* [build-constraints.yaml](https://github.com/fpco/stackage/blob/master/build-constraints.yaml) specifies packages to be included in Stackage
14* [stackage-curator](http://www.stackage.org/package/stackage-curator) combines build-constraints.yaml with the current state of Hackage to create a build plan for a Stackage Nightly
15* stackage-curator can check that build plan to ensure all version bounds are consistent
16 * The [Travis job](https://github.com/fpco/stackage/blob/master/.travis.yml) performs these two steps to provide immediate feedback on pull requests
17* Docker Hub [builds](https://github.com/fpco/stackage/blob/master/Dockerfile) a [Docker image](https://registry.hub.docker.com/u/snoyberg/stackage/) for running builds
18* The stackage-build server (described below) is able to run automated builds using the [build.sh script](https://github.com/fpco/stackage/blob/master/automated/build.sh)
19* When a new Nightly build is completed, it is uploaded to [the nightly repo](https://github.com/fpco/stackage-nightly)
20* Once a week, we run an LTS minor bump. Instead of using build-constraints.yaml, that job takes the previous LTS release, turns it into constraints, and then bumps the version numbers to the latest on Hackage, in accordance with the version bounds in the build plan. This plans are uploaded to [the LTS repo](https://github.com/fpco/lts-haskell)
21* Cutting a new LTS major release is essentially just a Stackage Nightly that gets uploaded as an LTS
23## Pull requests
25The typical story on pull requests is: if Travis accepts it, and the author
26only added packages under his/her own name, merge it. If the build later fails
27(see below), then block the package until it's fixed.
29## Fixing bounds issues
31The most common activity you'll deal with in Stackage curation is a version
32bound issue, usually a restrictive upper bound. You fix this by opening an
33issue on the Stackage repo about the problem, and modifying the
34build-constraints.yaml file to work around it in one of the ways below. Be sure
35to refer to the issue for workarounds added to that file.
37* __Temporary upper bounds__ Most common technique, just prevent a new version of a library from being included immediately
38* __Skipping tests and benchmarks__ If the upper bound is only in a test suite or benchmark, you can add the relevant package to skipped-tests or skipped-benchmarks. For example, if conduit had an upper bound on criterion for a benchmark, you could added conduit as a skipped benchmark.
39* __Excluding packages__ In an extreme case of a non-responsive maintainer, you can remove the package entirely from Stackage. We try to avoid that whenever possible
41## Upgrading GHC versions
43The Dockerfile contains information on which GHC versions should be used. You
44can modify it and push it to Github to trigger a DockerHub build. The master
45branch is used for nightlies, and the lts branch for LTS. Once a new Docker
46image is available, you'll need to pull it onto the stackage-build server (see
47below). Instead of pulling an unbounded number of images, I typically just
48delete all of the old images and let the new ones get downloaded:
50```
51docker rm $(docker ps -a -q)
52docker rmi $(docker images -q)
53```
55You should also delete the cache directories on the stackage-build server to
56force all packages to be rebuilt. See: [issue
57#746](https://github.com/fpco/stackage/issues/746).
59## stackage-build server
61You'll need to get your SSH public key added to the machine. ~/.ssh/config info:
63```
64Host stackage-build
65 User ubuntu
66 Hostname ec2-52-5-20-252.compute-1.amazonaws.com
67```
69We currently run the builds manually so make it easy to see when there are
70bounds issues that need to be corrected. Automated this would be even better,
71we're just not there yet.
73```
74# Run a nightly build
75/opt/stackage-build/stackage/automated/build.sh nightly-2015-07-08
77# Run an LTS minor bump
78/opt/stackage-build/stackage/automated/build.sh lts-2.17
80# Run an LTS major bump
81/opt/stackage-build/stackage/automated/build.sh lts-3.0
82```
84Recommended: run these from inside a `screen` session. If you get version bound
85problems on nightly or LTS major, you need to fix build-constraints.yaml (see
86info above). For an LTS minor bump, you'll typically want to use the
87`CONSTRAINTS` environment variable, e.g.:
89```
90CONSTRAINTS='--constraint "conduit < 1.4.5" --constraint "criterion < 1.2.3"' /opt/stackage-build/stackage/automated/build.sh lts-2.17
91```
93If a build fails for bounds reasons, see all of the advice above. If the code
94itself doesn't build, or tests fail, open up an issue and then either put in a
95version bound to avoid that version or something else. It's difficult to give
96universal advice on how to solve things, since each situation is unique. Let's
97develop this advice over time. For now: if you're not sure, ask Michael for
98guidance.
100### Timing
102Nightly builds should be run once a day. A common technique I use is, after a
103build succeeds, write something like `sleep 20h;
104/opt/stackage-build/stackage/automated/build.sh nightly-2015-01-02`. LTS minor
105bumps typically are run on Sundays.