# Maintaining ## Overview We use a three-stage release pipeline. Each stage of the release pipeline has its own dedicated branch and corresponding testing moon. Features and bug fixes progress through each stage--and are subject to testing along the way--until they're eventually released to the live network. This pipeline automates our release process, making it much easier to quickly and reliably ship code. It's also simple to reason about. ## Branches and Moons The branches and their corresponding moons that comprise the stages of the release pipeline are: ``` ---------------------------------------------------------------------------------------------- Branch | Moon | Target audience | Contains ---------------------------------------------------------------------------------------------- `develop` | `~binnec-dozzod-marzod` | Kernel developers | Latest `develop` branch commit `release` | `~marnec-dozzod-marzod` | Early Adopters | Latest `release` branch commit `release` | `~doznec-dozzod-marzod` | App Developers | Latest release candidate `master` | `~zod` | Everyone else | Latest release ``` **WARNING**: If you lack the requisite skills to troubleshoot and fix kernel issues, you should not sync from develop/~binnec. If you're not prepared to breach your ship in response to an issue stemming from an early release, do not use pre-release moons. `develop` is the default branch in the repo, which means that all new pull requests target it by default. The general flow of a new feature or bug fix through the pipeline is: ```console feature branch ----> develop ----> release ---------> master | | | deployed to deployed to deployed to ~binnec moon ~marnec/~doznec moon network ``` If an issue arises in the course of testing the `release` branch (because more people are using `marnec` than `binnec`), a PR can be opened to target `release`. If that's the case, the `master` needs to be merged back into `develop` after `release` merges into `master` to ensure that `develop` gets the fix. ## Release Workflow Developers work on feature branches built against `develop`. While doing this, they continually merge in changes from `develop` to their feature branch. When their feature is ready (and they've tested it), they open a pull request. After code review approval and passing tests, their feature can merge into `develop`. Every merge into `develop` immediately triggers a deploy to the `binnec` moon. If your merge breaks `binnec` it's your responsibility to fix it. Once a week on Tuesday, a `release` branch is cut off of `develop`. This release gets deployed to `marnec` to be tested for the rest of the week. Any fixes that have to go into the release can go straight into the release branch. New work that didn't make the release continues on feature branches against `develop` (eventually merging there). After initial testing on `marnec`, a release candidate is tagged and merges into `~doznec` where early adopters and app developers can pick it up and test or update their apps for a new kelvin. If it's a new kelvin, we also send an email to urbit-dev with instructions for testing the breaking changes. Then on the next Tuesday the release branch merges into master and tagged using the tag instructions below, we create a GitHub release (marked latest) using that tag on `master` which documents the changes that went into the release. In the Github UI you can get the changelog by selecting the tag prior to it from the previous release when creating the new release. Then the release is deployed to the broader network via `zod`. Master is then merged back into `develop` where any fixes that went straight to release get picked up. Lastly, a new release branch is cut from `develop` and the process begins again. ### Tagging When we branch release to deploy to `~marnec`, we need to tag it as a release candidate (RC), like `urbit-os-vx.y-rc1`. Here 'x' is the major version and 'y' is an OTA patch counter. After this any change that goes into release gets a new tag that increments the rc. After we ship a release to the live network, add a tag that is not a release candidate, like `urbit-os-vx.y`, to the master branch, since that's what was released. #### Applying the Tag Locally Use an annotated tag with the `-a` git argument. Make sure to follow the naming convention for RCs and live releases, described above. To add a tag to the local repo, run this: ``` git tag -a ``` This will bring up an editor, where you should add the release notes, which should look like this: ``` This release will be pushed to the network as an over-the-air update. Release notes: [..] Contributions: [..] ``` To fill in the "contributions" section, copy in the shortlog between the last release and this release, obtained by running this command: ``` git shortlog --no-merges LAST_RELEASE.. ``` #### Pushing the Tag to the Main Repo Once you have added a tag, push it to the main repository using the following command: ``` git push origin ``` ## Releases - [ ] Create a pull request from the relevant release branch (with the format `release/urbit-os-vX.XX`) to `master`. - [ ] ssh into `~zod` - [ ] Check to ensure that nobody else is ssh'd into `~zod`, by running `screen -ls` and verifying no sessions are attached. - [ ] Attach to the screen session using `screen -x` - [ ] Ensure that the release candidate was correctly propagated through the prerelease moons - Use `-read %z ~SHIP %DESK da+now /` to check desk hashes - [ ] Run the above check on the following [SHIP DESK] pairs: [~marnec-dozzod-marzod %base], [~marnec-dozzod-marzod %kids], [~doznec-dozzod-marzod %base], [~doznec-dozzod-marzod %kids] — **they should all match** - [ ] Install the contents of the `%kids` desk on `~doznec-dozzod-marzod` into our `%base`: `|merge %base ~doznec-dozzod-marzod %kids, =gem %only-that` - [ ] Check that `~zod` has updated to the latest release. For a Kelvin release, you can run `zuse` in the Dojo. Each non-Kelvin release might its own way of checking whether the update has completed, possibly through checking the `%cz` hash of the `%base` desk matches the hash on `~marnec-dozzod-marzod` by comparing the outputs of `+vat %base` on both ships. - [ ] Merge `~zod`'s `%base` desk into its `%kids` desk to distribute the new code to the network: `|merge %kids our %base, =gem %only-that` - [ ] Before exiting the screen session on `~zod`, make sure the screen session is not left in copy mode for a long period of time, since that will disrupt `~zod`'s operation. ### Release Communications - [ ] Tag the commit that went onto the live network as a release, using GitHub's "Releases" interface. See the "Tagging" section of this document for details. - [ ] Update (add a response) the mailing list post to include the base hash of the new release, and indicate that this has now been deployed to the network. - [ ] Tweet from the `@zodisok` Twitter account linking to the GitHub release. - [ ] Post links to the release in the Urbit Community Development channel and a channel in the UF public group. ### Post-Release Git Cleanup - [ ] Merge `master` back into `develop`. - [ ] Cut a new release branch from `develop`. The branch should have the format `release/urbit-os-vX.XX` ### Release Next Release Candidate We'll now need to **IMMEDIATELY** deploy the new release candidate to the pre-release moon(s). Otherwise PRs merged during this window will bypass the testing period on `~binnec` and go straight to the release candidate. - On `~marnec-dozzod-marzod`: - [ ] `|merge %base ~binnec-dozzod-marzod %kids, =gem %only-that` to update `~marnec` with the contents of the GH release branch - [ ] `|merge %kids our %base, =gem %only-that` to OTA the release candidate to subscribers - When ready to deploy the release candidate to App Developers, on `~doznec-dozzod-marzod`: - [ ] `|merge %base ~marnec-dozzod-marzod %kids, =gem %only-that` to update `~doznec` with the release candidate - [ ] `|merge %kids our %base, =gem %only-that` to OTA the release candidate to subscribers ### Post-Release Checks - [ ] Check that `~marzod` and other distribution stars are receiving the update by running `|ames-sift ~zod` and `|ames-verb %rcv %ges`. You should see lots of packets from `~zod`. Once you have confirmed packets are flowing, run `|ames-sift` and `|ames-verb` with no arguments to reset the verbosity state. - [ ] Check that planets are receiving the update. They should start updating within an hour or so. - [ ] Monitor the Urbit Community Help channel, UF public group channels, and Twitter to make ### Post-Release Artifacts After waiting at least 24 hours after the release to the network, make and distribute a pill. - [ ] Find a ship on the network (for now, use `~halbex-palheb`, which runs the UF public group) whose sources for `%base` and the standard app desks are mainline, not devstream. - [ ] Ensure the `%cz` hashes of all desks match those on the distribution ships. - [ ] Make a pill by running `.multi-vX-XX/pill +solid %base %garden %webterm %landscape %groups %talk` (replacing `X-XX` with the appropriate version numbers, in this and later steps). - [ ] Boot a fakezod off that pill to make sure the pill is viable. - [ ] Upload the pill to `bootstrap.urbit.org` using the Google Cloud SDK by running: `gsutil cp /path/to/pier/.urb/put/multi-vX-XX.pill gs://bootstrap.urbit.org/urbit-vX.XX.pill` -- note that it should be `vX.XX`, not `vX-XX` as in the original pill filename. - [ ] Boot a ship with the latest binary and check that it downloads the pill from `https://bootstrap.urbit.org/urbit-vX.XX.pill` where you just uploaded it.