git.iko.soy/obelisk
on ae@mfix-performable help sign in
obelisk/README.md
425 lines 21 kB history raw
1# Obelisk
3[![Haskell Programming Language](https://img.shields.io/badge/language-Haskell-blue.svg)](http://www.haskell.org)
4[![BSD3 License](http://img.shields.io/badge/license-BSD3-brightgreen.svg)](https://tldrlegal.com/license/bsd-3-clause-license-%28revised%29)
7Obelisk provides an easy way to develop and deploy your [Reflex](https://github.com/reflex-frp/reflex) project as web apps and as mobile apps.
9- [Installing Obelisk](#installing-obelisk)
10- [Developing an Obelisk project](#developing-an-obelisk-project)
11 - [Adding Packages](#adding-packages)
12 - [Adding Package Overrides](#adding-package-overrides)
13 - [Running over HTTPS](#running-over-https)
14 - [IDE Support](#ide-support)
15 - [Terminal-based feedback](#terminal-based-feedback)
16 - [Experimental `ghcide` support](#experimental-ghcide-support)
17- [Deploying](#deploying)
18 - [Locally](#locally)
19 - [Default EC2 Deployment](#default-ec2-deployment)
20 - [Custom Non-EC2 Deployment](#custom-non-ec2-deployment)
21 - [VirtualBox Deployment](#virtualbox-deployment)
22 - [From macOS](#from-macos)
23 - [Deploying an updated version](#deploying-an-updated-version)
24- [Mobile](#mobile)
25 - [iOS](#ios)
26 - [Android](#android)
28## Installing Obelisk
291. [Install Nix](https://nixos.org/nix/).
30 If you already have Nix installed, make sure you have version 2.0 or higher. To check your current version, run `nix-env --version`.
311. Set up nix caches
32 1. If you are running NixOS, add this to `/etc/nixos/configuration.nix`:
33 ```
34 nix.binaryCaches = [ "https://nixcache.reflex-frp.org" ];
35 nix.binaryCachePublicKeys = [ "ryantrinkle.com-1:JJiAKaRv9mWgpVAz8dwewnZe0AzzEAzPkagE9SP5NWI=" ];
36 ```
37 and rebuild your NixOS configuration (e.g. `sudo nixos-rebuild switch`).
38 1. If you are using another operating system or linux distribution, ensure that these lines are present in your Nix configuration file (`/etc/nix/nix.conf` on most systems; [see full list](https://nixos.org/nix/manual/#sec-conf-file)):
39 ```
40 binary-caches = https://cache.nixos.org https://nixcache.reflex-frp.org
41 binary-cache-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= ryantrinkle.com-1:JJiAKaRv9mWgpVAz8dwewnZe0AzzEAzPkagE9SP5NWI=
42 binary-caches-parallel-connections = 40
43 ```
44 * other Linux: enable sandboxing (see these [issue172](https://github.com/obsidiansystems/obelisk/issues/172#issuecomment-411507818) or [issue6](https://github.com/obsidiansystems/obelisk/issues/6) if you run into build problems)
45 ```
46 sandbox = true
47 ```
48 * MacOS: disable sandboxing (there are still some impure dependencies for now)
49 ```
50 sandbox = false
51 ```
52 then restart the nix daemon
53 ```
54 sudo launchctl stop org.nixos.nix-daemon
55 sudo launchctl start org.nixos.nix-daemon
56 ```
571. Install obelisk: `nix-env -f https://github.com/obsidiansystems/obelisk/archive/master.tar.gz -iA command`
59### Contributing to Obelisk
61When developing on obelisk itself you may launch `ghcid` for the corresponding project as follows. For example to launch ghcid for `lib/backend` project:
63```bash
64nix-shell -A obeliskEnvs.obelisk-backend --run "cd lib/backend && ghcid -c 'cabal new-repl'"
65```
67Or to launch ghcid for `lib/command` project:
69```bash
70nix-shell -A obeliskEnvs.obelisk-command --run "cd lib/command && ghcid -c 'cabal new-repl'"
71```
73To re-install `ob` from source do
74```bash
75nix-env -f /path/to/obelisk -iA command
76```
78Note that `ob` will defer to the version found in your project's `.obelisk/impl` directory. To update that version specifically:
80```bash
81ob thunk unpack ./.obelisk/impl
82cd ./.obelisk/impl
83# apply your changes
84```
86If you want to commit your changes, first push them to your fork of obelisk and then
88```bash
89cd /your/project/root
90ob thunk pack .obelisk/impl
91git add .obelisk/impl
92git commit -m "Bump obelisk"
93```
95### Accessing private repositories
97To allow the Nix builder to access private git repositories, you must be set up
98to access them via SSH. Follow these steps depending on the platform you need
99access to:
101- [GitHub](https://help.github.com/articles/connecting-to-github-with-ssh/)
102- [GitLab](https://docs.gitlab.com/ee/gitlab-basics/create-your-ssh-keys.html)
104## Developing an Obelisk project
106To create a new Obelisk project, go to an empty directory and run:
108```bash
109ob init
110```
112Obelisk leverages ghcid to provide a live-reloading server that handles both frontend and backend. To run your Obelisk app and monitor the source for changes:
114```bash
115ob run
116```
118Now, with an appropriate browser, go to http://localhost:8000 (or the address/port specified in `config/common/route`) to access your app.
119Firefox will not be able to properly run the development website due to [issue 460](https://github.com/obsidiansystems/obelisk/issues/460). Fortunately, this problem does not occur on a fully built website.
121Every time you change the Haskell source files in frontend, common or backend, `ob run` will automatically recompile the modified files and reload the server. Furthermore, it will display on screen compilation errors and warnings if any.
123### Adding packages
125In order to add package dependencies, declare them under the build-depends field in the appropriate cabal files (backend, common, and frontend each have their own). The corresponding Nix packages will automatically be selected when building.
127### Adding package overrides
129To add a version override to any Haskell package, or to add a Haskell package that doesn't exist in the nixpkgs used by Obelisk, use the `overrides` attribute in your project's `default.nix`. For example, to use a specific version of the `aeson` package fetched from GitHub and a specific version of the `waargonaut` package fetched from Hackage, your `default.nix` will look like:
131```nix
132# ...
133project ./. ({ pkgs, ... }: {
134# ...
135 overrides = self: super: let
136 aesonSrc = pkgs.fetchFromGitHub {
137 owner = "obsidiansystems";
138 repo = "aeson-gadt-th";
139 rev = "ed573c2cccf54d72aa6279026752a3fecf9c1383";
140 sha256 = "08q6rnz7w9pn76jkrafig6f50yd0f77z48rk2z5iyyl2jbhcbhx3";
141 };
142 in
143 {
144 aeson = self.callCabal2nix "aeson" aesonSrc {};
145 waargonaut = self.callHackageDirect {
146 pkg = "waargonaut";
147 ver = "0.8.0.1";
148 sha256 = "1zv28np3k3hg378vqm89v802xr0g8cwk7gy3mr77xrzy5jbgpa39";
149 } {};
150 };
151# ...
152```
154For further information see [the Haskell section](https://nixos.org/nixpkgs/manual/#users-guide-to-the-haskell-infrastructure) of nixpkgs Contributors Guide.
156### Adding extra local packages
158If the standard packages (`frontend`, `backend`, and `common`) are not
159enough, to add more local Haskell packages, define them with the
160`packages` parameter. The sources of these packages will be
161automatically reloaded by `ob run`.
163```nix
164# ...
165project ./. ({ pkgs, ... }: {
166# ...
167 packages = {
168 another = ./another;
169 };
170# ...
171```
173### Running over HTTPS
175To run your app locally over HTTPS, update the protocol in `config/common/route` to `https`, and then use `ob run` as normal.
177Since Obelisk generates a self-signed certificate for running HTTPS, the browser will issue a warning about using an invalid certificate. On Chrome, you can go to `chrome://flags/#allow-insecure-localhost` to enable invalid certificates for localhost.
179### IDE Support
181#### Terminal-based feedback
183Obelisk officially supports terminal-based feedback (akin to [`ghcid`](https://github.com/ndmitchell/ghcid)) in `ob run` and `ob watch`.
185#### **Experimental** `ghcide` support
187> **NOTE:** `ghcide` support is strictly **experimental** and **not officially supported**. There is no guarantee that it will work for you or that this support will be maintained over time.
189To try out the **experimental** [`ghcide`](https://github.com/digital-asset/ghcide) support:
191 * Install a `ghcide` plugin for your editor.
192 * Add `__withGhcide = true;` to your Obelisk project's `default.nix` (inside the call to `project`). With this setting, `ob shell` will provide `ghcide` as well.
193 * Add `hie-bios.sh` to the root of your project:
194 ```shell
195 cat > hie-bios.sh <<'EOF'
196 #!/usr/bin/env bash
197 source "$HOME/.bash_profile"; # NOTE: Some editors need help finding 'ob' and this assumes that `ob` is made available on your `$PATH` in `.bash_profile`
198 ob internal export-ghci-configuration > "$HIE_BIOS_OUTPUT"
199 EOF
200 chmod +x hie-bios.sh
201 ```
202 * **Note:** In some projects you need to use `ob internal export-ghci-configuration --use-relative-paths` in this script instead. This is known to happen when your project contains symlinks to other packages.
203 * Add `hie.yaml` to the root of your project:
204 ```shell
205 echo 'cradle: { bios: { program: hie-bios.sh } }' > hie.yaml
206 ```
207 * Test the configuration with `ob shell ghcide`. Assuming your project compiles and works with `ghcide` otherwise, this should succeed.
208 * Start your editor in a way that it can access the appropriate `ghcide`:
209 * This can sometimes be achieved with `ob shell 'start my editor'`.
210 * If you can't easily start your editor in a way that it inherits your shell, you may need to customize how your `ghcide` plugin finds `ghcide` or create a custom `ghcide` replacement script that you install globally. For example, for Visual Studio Code on macOS adding a script called `ghcide` to your `PATH` like the one below will cause `ghcide` from each project's `ob shell` to be used:
211 ```shell
212 #!/usr/bin/env bash
213 source "$HOME/.bash_profile" # NOTE: This assumes that `ob` is made available on your `$PATH` in `.bash_profile`
214 ob shell "ghcide $@"
215 ```
217## Deploying
219### Locally
221Build everything:
223```bash
224nix-build -A exe --no-out-link
225```
227Copy the result to a new directory, add configuration, and run!
229```bash
230mkdir test-app
231ln -s $(nix-build -A exe --no-out-link)/* test-app/
232cp -r config test-app
233(cd test-app && ./backend)
234```
236### Default EC2 Deployment
238In this section we will demonstrate how to deploy your Obelisk app to an Amazon EC2 instance. Obelisk deployments are configured for EC2 by default (see [Custom Non-EC2 Deployment](#custom-non-ec2-deployment)).
240First create a new EC2 instance:
2421. Launch a NixOS 19.09 EC2 instance (we recommend [this AMI](https://console.aws.amazon.com/ec2/v2/home?region=us-east-1#LaunchInstanceWizard:ami=ami-00a8eeaf232a74f84))
2431. In the instance configuration wizard ensure that your instance has at least 1GB RAM and 10GB disk space.
2441. When prompted save your AWS private key (`~/myaws.pem`) somewhere safe. We'll need it later during deployment.
2451. Go to "Security Groups", select your instance's security group and under "Inbound" tab add a new rule for HTTP port 80 and 443.
247At this stage your instance should be booting and become accessible shortly. Note down the hostname of your EC2 instance.
249Now go to your Obelisk project directory (`~/code/myapp`), and initialize a deployment config (`~/code/myapp-deploy`):
250Your project directory must be "thunkable", i.e. something on which `ob thunk pack` can be called. Usually it will be a git repository whose current revision has been pushed upstream.
252```
253cd ~/code/myapp
254SERVER=ec2-35-183-22-197.ca-central-1.compute.amazonaws.com
255ROUTE=https://myapp.com # Publicly accessible route to your app
256EMAIL=myname@myapp.com
257ob deploy init \
258 --ssh-key ~/myaws.pem \
259 --hostname $SERVER \
260 --route $ROUTE \
261 --admin-email $EMAIL \
262 ~/code/myapp-deploy
263```
265HTTPS is enabled by default; to disable HTTPS pass `--disable-https` to the `ob deploy init` command above.
267This step will also require that you manually verify the authenticity of the host `$SERVER`. Obelisk will save the fingerprint in a deployment-specific configuration. **Obelisk deployments do *not* rely on the `known_hosts` of your local machine.** This is because, in the event that you need to switch from one deploy machine / bastion host to another, you want to be absolutely sure that you're still connecting to the machines you think you are, even if that deploy machine / bastion host has never connected to them before. Obelisk explicitly avoids a workflow that encourages people to accept host keys without checking them, since that could result in leaking production secrets to anyone who manages to MITM you, e.g. via DNS spoofing or cache poisoning. (Note that an active attack is a circumstance where you may need to quickly switch bastion hosts, e.g. because the attacker has taken one down or you have taken it down in case it was compromised. In this circumstance you might need to deploy to production to fix an exploit or rotate keys, etc.) When you run `ob deploy` later it will rely on the saved verification in this step.
269Next, go to the deployment directory that you just initialized and deploy!
271```
272cd ~/code/myapp-deploy
273ob deploy push
274```
276`ob deploy push` will locally build your app and then transfer it, along with all the Nix package dependencies, via ssh to the EC2 instance. The backend will live in `/var/lib/backend`.
278At this point you are done. Your app will be accessible at `${ROUTE}`. The currently deployed version - the git commit hash of the source repo - can be found at `${ROUTE}/version`.
280### Custom Non-EC2 Deployment
282By default Obelisk deployments are configured for NixOS machines running on AWS EC2. To provide your own configuration, you need to write a custom `module.nix` in the deployment repository. This still requires that your server is running NixOS.
284`module.nix` must contain a Nix *function* that produces a [NixOS module function](https://nixos.org/nixos/manual/index.html#sec-writing-modules). The top-level function takes deployment configuration as arguments: `hostName`, `adminEmail`, `routeHost`, `enableHttps`, `version`, `exe`, `nixosPkgs`. Most of these are the values you specified during `ob deploy init` and are stored in the deployment repository. `version` is a `git` hash for the app that you're deploying. `exe` is the Linux build of the app (as seen in [Deploying Locally](#locally)). `nixosPkgs` is the package set used to construct the NixOS VM.
286The [VirtualBox Deployment](#virtualbox-deployment) section provides an example.
288#### VirtualBox Deployment
290Here's a `module.nix` that is configured for deployment to a VirtualBox VM (running NixOS):
292```nix
293{ nixosPkgs, ... }: {...}: {
294 imports = [ (nixosPkgs.path + /nixos/modules/virtualisation/virtualbox-image.nix) ];
296```
298The `{...}:` and following is the [NixOS module](https://nixos.org/nixos/manual/index.html#sec-writing-modules) definition.
300### From macOS
302Deploying from macOS requires some extra setup:
304- [Install nix-darwin](https://github.com/LnL7/nix-darwin)
305- [Install docker](https://docs.docker.com/)
307Running `ob deploy push` will give you additional setup instructions.
309### Deploying an updated version
311If you'd like to deploy an updated version (with new commits) of your Obelisk app: simply go to the configuration directory, update the source thunk and push:
313```bash
314cd ~/code/myapp-deploy
315ob deploy update
316ob deploy push
317```
319## Mobile
320Until Obelisk offers a `ob deploy` equivalent for mobile apps, you are recommended to do it manually as follows.
322### iOS
324#### First time setup
325Development on iOS requires a computer running macOS and an iOS developer account.
326Your account must also belong to an Apple Developer Team, if you want to access developer portal links (otherwise they'll redirect to your account page).
328##### iPhone
3291. Connect the iPhone on which you'd like to run builds - this will open up iTunes.
3301. Click accept to authorize on both the computer and the iPhone.
332##### Xcode
333Install Xcode 8.2 (contains iOS SDK 10.2) and open it so that it runs its post install tool setup.
335These versions will work out of the box but iOS SDKs prior to 11.3 should also work. You can choose another installed version in `default.nix`
337More recent Xcodes should also work, as long as one of the SDKs mentioned above has been used.
338To add another SDK to your current Xcode, [download](https://developer.apple.com/download/more/) the corresponding Xcode, extract it and copy its SDK folder next to the installed one, e.g.
339```bash
340open -W Xcode_9.2.xip
341sudo cp -R Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS.sdk /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS11.2.sdk
342```
345You can verify that you have correct versions by running
346```bash
347xcodebuild -showsdks
348```
350##### Certificates
351To deploy and/or package apps, you'll need to inform Apple of your development devices and permissions by
352adding credentials to the correct provisioning profile via the Apple Developer portal.
3541. Open up XCode and go to Preferences - Accounts. Select the organization
355Member role, click Manage Certificates, and add an iOS Development
356certificate.
3571. Go to [developer portal - devices](https://developer.apple.com/account/ios/device/) and add your device.
358To find your device's UDID, select it in iTunes and click the serial number.
3591. Go to [developer portal - development profiles](https://developer.apple.com/account/ios/profile/limited).
360Create a development profile and add your certificate and device.
361Click "Generate" and then download and open the profile.
363#### Building
3641. In your project's `default.nix` set values for `ios.bundleIdentifier` and `ios.bundleName`.
365Ensure that `bundleIdentifier` matches the App ID of the development profile, or that you are using a wildcard profile.
3661. Run `nix-build -A ios.frontend -o result-ios` to build the app. Find it at `result-ios/frontend.app`
368#### Deploying
3691. Connect the registered iPhone.
3701. Find your Apple Team ID in the [developer portal](https://developer.apple.com/account/#/membership).
3711. Run the deploy command with your Team ID:
372```bash
373result-ios/bin/deploy [TEAM_ID]
374# or in debug mode via lldb:
375result-ios/bin/deploy [TEAM_ID] -d
376```
378#### Packaging
3791. Go to [developer portal - distribution profiles](https://developer.apple.com/account/ios/profile/production).
380Create and download a distribution profile.
3811. Run the package script with your TEAM ID and your distribution profile to create a `.ipa`:
382```bash
383result-ios/bin/package [TEAM_ID] /path/to/output/.ipa /path/to/profile/file
384```
386#### Debugging
387It's also possible to inspect iOS WkWebView apps once they are installed in the iPhone:
3881. On the desktop, go to Safari > Preferences > Advanced and enable Develop menu.
3891. On the iPhone go to Settings > Safari > Advanced and enable Web Inspector.
3901. Open the app on the iPhone while it is connected to the desktop.
3911. In the desktop's Safari Develop menu, you should see your iPhone. Select the screen under the name of the app.
393### Android
395NOTE: Currently Android builds are only supported on Linux.
3971. In your project's `default.nix` set a suitable value for `android.applicationId` and `android.displayName`.
3981. In your project's `default.nix` pass `config.android_sdk.accept_license = true;` in the arguments to the import of of `.obelisk/impl` to indicate your acceptance of the [Android Software Development Kit License Agreement](https://developer.android.com/studio/terms), which is required to build Android apps.
3991. Run `nix-build -A android.frontend -o result-android` to build the Android app.
4001. A debug version of the app should be generated at `result-android/android-app-debug.apk`
402Now deploy the built apk file to your Android device:
4041. Enable *USB debugging* in your Android device ([instructions here](https://developer.android.com/studio/debug/dev-options))
4051. Connect the device using USB (be sure to confirm any security prompts on the device)
4061. Run the deploy script: `result-android/bin/deploy`
408Alternatively, you can deploy from an obelisk deployment directory (a directory generated post `ob deploy init ...` command) using the `ob deploy test android` command.
409This command will accomplish the following:
4111. Create a key store and apk signing key (`android_keystore.jks`)
4121. Build a Signed Android apk for your application
4131. Deploy the Signed apk to your connected Android device
415In the event that you change your key or keystore password, you will have to update your credentials within the JSON object found in `android_keytool_config.json`.
417Additional documentation on Java key stores can be found [here](https://docs.oracle.com/javase/8/docs/technotes/tools/unix/keytool.html).
419This should copy over and install the application on your device (if you see a "*signatures do not match*" error, simply uninstall the previous app from the device before retrying the deploy). The name of the installed application will be what you have specified for `android.displayName` in the `default.nix`.
421#### Releasing to Play Store
423##### Build a release version
425After having configured signing for your app, you may proceed to build a release version of the app. This is no different to how you build the non-release version, so consult the section [Android](#android) further above for exact instructions on building and deploying to your device.