on master help sign in
Merge remote-tracking branch 'origin/develop' into depend-on-reflex-dom-core
commitf755f08ea7807af3c30ae74697074aab8d10bdcf authorElliot Cameron <eacameron@gmail.com> dateMon 16 Mar 2020, 15:54 (7 years ago) parent3c7fdb7d502479
41 files changed+1558−787
.github/pull_request_template.md+8 ChangeLog.md+41 LICENSE+30 README.md+63−27 all-builds.nix+37−47 all-tests.nix+12−12 default.nix+173−99 dep/gitignore.nix/default.nix+8 dep/gitignore.nix/github.json+8 dep/reflex-platform/default.nix+7−6 dep/reflex-platform/github.json+3−2 dep/snap-core/default.nix+8 dep/snap-core/github.json+8 lib/asset/serve-snap/src/Obelisk/Asset/Accept.hs+6−6 lib/asset/serve-snap/src/Obelisk/Asset/Serve/Snap.hs+14−4 lib/backend/src/Obelisk/Backend.hs+90−29 lib/cliapp/obelisk-cliapp.cabal+2−2 lib/cliapp/src/Obelisk/CliApp.hs+2 lib/cliapp/src/Obelisk/CliApp/Demo.hs−52 lib/cliapp/src/Obelisk/CliApp/Logging.hs+11−10 lib/cliapp/src/Obelisk/CliApp/Process.hs+23−3 lib/command/obelisk-command.cabal+10−10 lib/command/src/Obelisk/Command.hs+58−62 lib/command/src/Obelisk/Command/Deploy.hs+61−60 lib/command/src/Obelisk/Command/Nix.hs+65−24 lib/command/src/Obelisk/Command/Preprocessor.hs+96 lib/command/src/Obelisk/Command/Project.hs+100−36 lib/command/src/Obelisk/Command/Run.hs+260−132 lib/command/src/Obelisk/Command/Thunk.hs+90−31 lib/command/src/Obelisk/Command/Utils.hs+18−7 lib/executable-config/lookup/src-other/Obelisk/ExecutableConfig/Lookup.hs+1−8 lib/frontend/src/Obelisk/Frontend.hs+8−8 lib/frontend/src/Obelisk/Frontend/Cookie.hs+2−1 lib/route/src/Obelisk/Route.hs+11−4 lib/route/src/Obelisk/Route/Frontend.hs+80−16 lib/run/src/Obelisk/Run.hs+11−6 lib/selftest/src/Obelisk/SelfTest.hs+130−78 lib/snap-extras/obelisk-snap-extras.cabal+1−1 lib/snap-extras/src/Obelisk/Snap/Extras.hs+1−1 skeleton/.gitignore+1 skeleton/backend/src/Backend.hs−3
.github/pull_request_template.mdnew file+8
@@ -0,0 +1,8 @@
1+<!-- Provide a clear overview of your changes. -->
2+
3+I have:
4+
5+ - [ ] Based work on latest `develop` branch
6+ - [ ] Looked for lint in my changes with `hlint .` (lint found code you did not write can be left alone)
7+ - [ ] Run the test suite: `$(nix-build -A selftest --no-out-link)`
8+ - [ ] (Optional) Run CI tests locally: `nix-build release.nix -A build.x86_64-linux --no-out-link` (or `x86_64-darwin` on macOS)
ChangeLog.md+41
@@ -4,8 +4,35 @@ This project's release branch is `master`. This log is written from the perspect
44
55## Unreleased
66
7+* Fully support HTTP [Range](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Range) requests on static assets to support Safari. ([#664](https://github.com/obsidiansystems/obelisk/pull/664))
8+* Support non-EC2 deployments. ([#658](https://github.com/obsidiansystems/obelisk/pull/658))
9+* Fix `ob deploy test android` to work. ([#645](https://github.com/obsidiansystems/obelisk/pull/645))
10+* Fix vulnerability where Android deployments would leave signing keys in the nix store which is world readable. ([#645](https://github.com/obsidiansystems/obelisk/pull/645)) (Thanks to [kmicklas](https://github.com/kmicklas) for the report.)
11+* Add `Obelisk.Backend.runBackendWith` to allow customization of how GHCJS resources are loaded in the page. ([#668](https://github.com/obsidiansystems/obelisk/pull/668))
12+* Add `ob profile` command to run Obelisk projects with profiling. `ob profile` works like `ob run`, but instead of using `ghci`, it builds an executable that is built with profiling enabled. ([#654](https://github.com/obsidiansystems/obelisk/pull/654))
713* Obelisk now depends directly on `reflex-dom-core` instead of `reflex-dom`. This avoids requiring a `jsaddle-webkit2gtk` dependency in apps that are only using `jsaddle-warp`. ([#606](https://github.com/obsidiansystems/obelisk/pull/606))
814
15+## v0.6.0.0 - 2020-02-21
16+
17+* Fix a bug in `Obelisk.Route.Frontend` where `routeLink`, `routeLinkDynAttr`, and `dynRouteLink` would not behave exactly like `<a href="...">` when run by JavaScript. These functions now scroll to the top of the page when the link is clicked. ([#540](https://github.com/obsidiansystems/obelisk/pull/540))
18+* Fix a bug in `ob run`/`ob watch`/`ob repl` where nested Obelisk projects would also be loaded into the session. These are now ignored. ([#652](https://github.com/obsidiansystems/obelisk/pull/652))
19+* Improve behavior of `ob run`/`ob watch`/`ob repl` when multiple packages with the same name are encountered. Obelisk now issues a warning and tells you which one it will use. ([#653](https://github.com/obsidiansystems/obelisk/pull/653))
20+* Removed `Obelisk.Backend.mkRouteToUrl` since it is easily written in terms of `Obelisk.Route.renderObeliskRoute`:
21+
22+ mkRouteToUrl validFullEncoder (k :/ v) = renderObeliskRoute validFullEncoder (FullRoute_Frontend (ObeliskRoute_App k) :/ v)
23+
24+* Add `Obelisk.Backend.renderAllJsPath` to expose URL path to `ghcjs/all.js`. ([#545](https://github.com/obsidiansystems/obelisk/pull/545))
25+* Add argument to `serveDefaultObeliskApp`, `serveObeliskApp`, and `serveGhcjsApp` to take the path to `all.js` instead of hard-coding it. ([#545](https://github.com/obsidiansystems/obelisk/pull/545))
26+
27+## v0.5.0.0 - 2020-02-07
28+
29+* Add `Obelisk.Route.(?/)`, a convenience function for constructing routes nested in `Maybe`. ([#457](https://github.com/obsidiansystems/obelisk/pull/457))
30+* Add local unpacked packages to the `ob run`, `ob watch`, and `ob repl` sessions. Any `.cabal` or hpack package inside the current obelisk project will be loaded into the session. For `ob run`/`ob watch` this means the session will automatically reload when you save a source file in any of those packages. For `ob repl` it means that `:r` will reload changes to any of those packages. There are some edge cases where this integration is still rough. Report any issues you encounter. ([#489](https://github.com/obsidiansystems/obelisk/pull/489))
31+* Add `ob hoogle` command to start a local [Hoogle](https://hoogle.haskell.org/) server for the project. ([#628](https://github.com/obsidiansystems/obelisk/pull/628))
32+* `ob thunk pack` will now attempt to automatically detect if the thunk is a private or public repo. To avoid this detection, specify `--private` or `--public` manually. ([#607](https://github.com/obsidiansystems/obelisk/pull/607))
33+* Fix a bug in the plain git thunk loader for thunks marked as 'private' when the revision is not in the default branch. ([#648](https://github.com/obsidiansystems/obelisk/pull/648))
34+* Improve handling of runtime nix dependencies. This may fix some issues encountered particularly by users on systems other than NixOS.
35+
936## v0.4.0.0 - 2020-01-10
1037
1138* Bump reflex-platform which, notably, bumps nixpkgs to 19.09. ([#585](https://github.com/obsidiansystems/obelisk/pull/585))
@@ -19,6 +46,20 @@ This project's release branch is `master`. This log is written from the perspect
1946
2047## v0.3.0.0 - 2019-12-20
2148
49+* Change the structure of Obelisk routes to use a designated
50+ `FullRoute` type. This combines frontend and backend routes into one
51+ structure. This is a **breaking** change which requires Obelisk apps
52+ to take specific migrations. They are:
53+ * Rewrite the implementation of `_backend_routeEncoder` in
54+ `Backend` to use `mkFullRouteEncoder` instead of
55+ `handleEncoder`. Specifically, the backend and frontend cases of
56+ the top-level `pathComponentEncoder` become the second and third
57+ arguments of `mkFullRouteEncoder` respectively, while the
58+ missing route becomes the first argument. An example of how to
59+ do this is available in [a reflex-examples
60+ commit](https://github.com/reflex-frp/reflex-examples/commits/28f566c3e7dc615578dc74297b7c620c1f13683e).
61+ * Replace type constructions of `InL` with `FullRoute_Backend` and
62+ `InR` with `FullRoute_Frontend`.
2263* Generalised `pathSegmentEncoder`, added `pathFieldEncoder`.
2364* Added some `Prism`s to the encoder library for manipulating `DSum`s.
2465* Add `ob doc` command, which lists paths to haddock documentation for specified packages.
LICENSEnew file+30
@@ -0,0 +1,30 @@
1+Copyright (c) 2020 Obsidian Systems
2+
3+All rights reserved.
4+
5+Redistribution and use in source and binary forms, with or without
6+modification, are permitted provided that the following conditions are met:
7+
8+ * Redistributions of source code must retain the above copyright
9+ notice, this list of conditions and the following disclaimer.
10+
11+ * Redistributions in binary form must reproduce the above
12+ copyright notice, this list of conditions and the following
13+ disclaimer in the documentation and/or other materials provided
14+ with the distribution.
15+
16+ * Neither the name of Elliot Cameron nor the names of other
17+ contributors may be used to endorse or promote products derived
18+ from this software without specific prior written permission.
19+
20+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
21+"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
22+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
23+A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
24+OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
25+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
26+LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
27+DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
28+THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
29+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
30+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
README.md+63−27
@@ -1,16 +1,21 @@
11# Obelisk
22
3+[![Haskell Programming Language](https://img.shields.io/badge/language-Haskell-blue.svg)](http://www.haskell.org)
4+[![BSD3 License](http://img.shields.io/badge/license-BSD3-brightgreen.svg)](https://tldrlegal.com/license/bsd-3-clause-license-%28revised%29)
5+
6+
37Obelisk provides an easy way to develop and deploy your [Reflex](https://github.com/reflex-frp/reflex) project as web apps and as mobile apps.
48
59- [Installing Obelisk](#installing-obelisk)
610- [Developing an Obelisk project](#developing-an-obelisk-project)
7− - [Hoogle](#hoogle)
811 - [Adding Packages](#adding-packages)
912 - [Adding Package Overrides](#adding-package-overrides)
1013 - [Running over https](#running-over-https)
1114- [Deploying](#deploying)
1215 - [Locally](#locally)
13− - [EC2](#ec2)
16+ - [Default EC2 Deployment](#default-ec2-deployment)
17+ - [Custom Non-EC2 Deployment](#custom-non-ec2-deployment)
18+ - [VirtualBox Deployment](#virtualbox-deployment)
1419 - [From macOS](#from-macos)
1520 - [Deploying an updated version](#deploying-an-updated-version)
1621- [Mobile](#mobile)
@@ -52,24 +57,24 @@ Obelisk provides an easy way to develop and deploy your [Reflex](https://github.
5257
5358When developing on obelisk itself you may launch `ghcid` for the corresponding project as follows. For example to launch ghcid for `lib/backend` project:
5459
55−```
60+```bash
5661nix-shell -A obeliskEnvs.obelisk-backend --run "cd lib/backend && ghcid -c 'cabal new-repl'"
5762```
5863
5964Or to launch ghcid for `lib/command` project:
6065
61−```
66+```bash
6267nix-shell -A obeliskEnvs.obelisk-command --run "cd lib/command && ghcid -c 'cabal new-repl'"
6368```
6469
6570To re-install `ob` from source do
66−```
71+```bash
6772nix-env -f /path/to/obelisk -iA command
6873```
6974
7075Note that `ob` will defer to the version found in your project's `.obelisk/impl` directory. To update that version specifically:
7176
72−```shell
77+```bash
7378ob thunk unpack ./.obelisk/impl
7479cd ./.obelisk/impl
7580# apply your changes
@@ -77,7 +82,7 @@ cd ./.obelisk/impl
7782
7883If you want to commit your changes, first push them to your fork of obelisk and then
7984
80−```shell
85+```bash
8186cd /your/project/root
8287ob thunk pack .obelisk/impl
8388git add .obelisk/impl
@@ -97,13 +102,13 @@ access to:
97102
98103To create a new Obelisk project, go to an empty directory and run:
99104
100−```
105+```bash
101106ob init
102107```
103108
104109Obelisk leverages ghcid to provide a live-reloading server that handles both frontend and backend. To run your Obelisk app and monitor the source for changes:
105110
106−```
111+```bash
107112ob run
108113```
109114
@@ -112,12 +117,6 @@ Firefox will not be able to properly run the development website due to [issue 4
112117
113118Every time you change the Haskell source files in frontend, common or backend, `ob run` will automatically recompile the modified files and reload the server. Furthermore, it will display on screen compilation errors and warnings if any.
114119
115−### Hoogle
116−
117−To enter a nix-shell from which you can run the hoogle command-line client or a hoogle server for your project:
118−
119−`nix-shell -A shells.ghc --arg withHoogle true`
120−
121120### Adding packages
122121
123122In order to add package dependencies, declare them under the build-depends field in the appropriate cabal files (backend, common, and frontend each have their own). The corresponding Nix packages will automatically be selected when building.
@@ -151,7 +150,24 @@ project ./. ({ pkgs, ... }: {
151150
152151For further information see [the Haskell section](https://nixos.org/nixpkgs/manual/#users-guide-to-the-haskell-infrastructure) of nixpkgs Contributors Guide.
153152
154−### Running over https
153+### Adding extra local packages
154+
155+If the standard packages (`frontend`, `backend`, and `common`) are not
156+enough, to add more local Haskell packages, define them with the
157+`packages` parameter. The sources of these packages will be
158+automatically reloaded by `ob run`.
159+
160+```nix
161+# ...
162+project ./. ({ pkgs, ... }: {
163+# ...
164+ packages = {
165+ another = ./another;
166+ };
167+# ...
168+```
169+
170+### Running over HTTPS
155171
156172To run your app locally over https, update the protocol in `config/common/route` to `https`, and then use `ob run` as normal.
157173
@@ -163,26 +179,26 @@ Since Obelisk generates a self-signed certificate for running https, the browser
163179
164180Build everything:
165181
166−```
182+```bash
167183nix-build -A exe --no-out-link
168184```
169185
170186Copy the result to a new directory, add configuration, and run!
171187
172−```
188+```bash
173189mkdir test-app
174190ln -s $(nix-build -A exe --no-out-link)/* test-app/
175191cp -r config test-app
176192(cd test-app && ./backend)
177193```
178194
179−### EC2
195+### Default EC2 Deployment
180196
181−In this section we will demonstrate how to deploy your Obelisk app to an Amazon EC2 instance.
197+In this section we will demonstrate how to deploy your Obelisk app to an Amazon EC2 instance. Obelisk deployments are configured for EC2 by default (see [Custom Non-EC2 Deployment](#custom-non-ec2-deployment)).
182198
183199First create a new EC2 instance:
184200
185−1. Launch a NixOS 18.09 EC2 instance (we recommend [this AMI](https://console.aws.amazon.com/ec2/v2/home?region=us-east-1#LaunchInstanceWizard:ami=ami-009c9c3f1af480ff3))
201+1. Launch a NixOS 19.09 EC2 instance (we recommend [this AMI](https://console.aws.amazon.com/ec2/v2/home?region=us-east-1#LaunchInstanceWizard:ami=ami-00a8eeaf232a74f84))
1862021. In the instance configuration wizard ensure that your instance has at least 1GB RAM and 10GB disk space.
1872031. When prompted save your AWS private key (`~/myaws.pem`) somewhere safe. We'll need it later during deployment.
1882041. Go to "Security Groups", select your instance's security group and under "Inbound" tab add a new rule for HTTP port 80 and 443.
@@ -205,7 +221,7 @@ ob deploy init \
205221 ~/code/myapp-deploy
206222```
207223
208−HTTPS is enabled by default; to disable https, pass `--disable-https` to the `ob deploy init` command above.
224+HTTPS is enabled by default; to disable HTTPS pass `--disable-https` to the `ob deploy init` command above.
209225
210226This step will also require that you manually verify the authenticity of the host `$SERVER`. Obelisk will save the fingerprint in a deployment-specific configuration. **Obelisk deployments do *not* rely on the `known_hosts` of your local machine.** This is because, in the event that you need to switch from one deploy machine / bastion host to another, you want to be absolutely sure that you're still connecting to the machines you think you are, even if that deploy machine / bastion host has never connected to them before. Obelisk explicitly avoids a workflow that encourages people to accept host keys without checking them, since that could result in leaking production secrets to anyone who manages to MITM you, e.g. via DNS spoofing or cache poisoning. (Note that an active attack is a circumstance where you may need to quickly switch bastion hosts, e.g. because the attacker has taken one down or you have taken it down in case it was compromised. In this circumstance you might need to deploy to production to fix an exploit or rotate keys, etc.) When you run `ob deploy` later it will rely on the saved verification in this step.
211227
@@ -220,6 +236,26 @@ ob deploy push
220236
221237At this point you are done. Your app will be accessible at `${ROUTE}`. The currently deployed version - the git commit hash of the source repo - can be found at `${ROUTE}/version`.
222238
239+### Custom Non-EC2 Deployment
240+
241+By default Obelisk deployments are configured for NixOS machines running on AWS EC2. To provide your own configuration, you need to write a custom `module.nix` in the deployment repository. This still requires that your server is running NixOS.
242+
243+`module.nix` must contain a Nix *function* that produces a [NixOS module function](https://nixos.org/nixos/manual/index.html#sec-writing-modules). The top-level function takes deployment configuration as arguments: `hostName`, `adminEmail`, `routeHost`, `enableHttps`, `version`, `exe`, `nixosPkgs`. Most of these are the values you specified during `ob deploy init` and are stored in the deployment repository. `version` is a `git` hash for the app that you're deploying. `exe` is the Linux build of the app (as seen in [Deploying Locally](#locally)). `nixosPkgs` is the package set used to construct the NixOS VM.
244+
245+The [VirtualBox Deployment](#virtualbox-deployment) section provides an example.
246+
247+#### VirtualBox Deployment
248+
249+Here's a `module.nix` that is configured for deployment to a VirtualBox VM (running NixOS):
250+
251+```nix
252+{ nixosPkgs, ... }: {...}: {
253+ imports = [ (nixosPkgs.path + /nixos/modules/virtualisation/virtualbox-image.nix) ];
254+}
255+```
256+
257+The `{...}:` and following is the [NixOS module](https://nixos.org/nixos/manual/index.html#sec-writing-modules) definition.
258+
223259### From macOS
224260
225261Deploying from macOS requires some extra setup:
@@ -233,7 +269,7 @@ Running `ob deploy push` will give you additional setup instructions.
233269
234270If you'd like to deploy an updated version (with new commits) of your Obelisk app: simply go to the configuration directory, update the source thunk and push:
235271
236−```
272+```bash
237273cd ~/code/myapp-deploy
238274ob deploy update
239275ob deploy push
@@ -259,14 +295,14 @@ These versions will work out of the box but iOS SDKs prior to 11.3 should also w
259295
260296More recent Xcodes should also work, as long as one of the SDKs mentioned above has been used.
261297To add another SDK to your current Xcode, [download](https://developer.apple.com/download/more/) the corresponding Xcode, extract it and copy its SDK folder next to the installed one, e.g.
262−```
298+```bash
263299open -W Xcode_9.2.xip
264300sudo cp -R Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS.sdk /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS11.2.sdk
265301```
266302
267303
268304You can verify that you have correct versions by running
269−```
305+```bash
270306xcodebuild -showsdks
271307```
272308
@@ -292,7 +328,7 @@ Ensure that `bundleIdentifier` matches the App ID of the development profile, or
2923281. Connect the registered iPhone.
2933291. Find your Apple Team ID in the [developer portal](https://developer.apple.com/account/#/membership).
2943301. Run the deploy command with your Team ID:
295−```
331+```bash
296332result-ios/bin/deploy [TEAM_ID]
297333# or in debug mode via lldb:
298334result-ios/bin/deploy [TEAM_ID] -d
@@ -302,7 +338,7 @@ result-ios/bin/deploy [TEAM_ID] -d
3023381. Go to [developer portal - distribution profiles](https://developer.apple.com/account/ios/profile/production).
303339Create and download a distribution profile.
3043401. Run the package script with your TEAM ID and your distribution profile to create a `.ipa`:
305−```
341+```bash
306342result-ios/bin/package [TEAM_ID] /path/to/output/.ipa /path/to/profile/file
307343```
308344
all-builds.nix+37−47
@@ -38,21 +38,16 @@ let
3838 (name: { inherit name; value = pkgsSet.${name}; })
3939 pnames);
4040
41− concatDepends = let
42− extractDeps = x: (x.override {
43− mkDerivation = drv: {
44− out = builtins.concatLists [
45− (drv.buildDepends or [])
46− (drv.libraryHaskellDepends or [])
47− (drv.executableHaskellDepends or [])
48− ];
49− };
50− }).out;
51− in pkgAttrs: builtins.concatLists (map extractDeps (builtins.attrValues pkgAttrs));
41+ collect = v:
42+ if lib.isDerivation v then [v]
43+ else if lib.isAttrs v then lib.concatMap collect (builtins.attrValues v)
44+ else if lib.isList v then lib.concatMap collect v
45+ else [];
5246
5347 perPlatform = lib.genAttrs cacheBuildSystems (system: let
5448 reflex-platform = import ./dep/reflex-platform { inherit system; };
55− perProfiling = profiling: let
49+
50+ mkPerProfiling = profiling: let
5651 obelisk = import ./. (self-args // { inherit system profiling; });
5752 ghc = pnameToAttrs
5853 obelisk.haskellPackageSets.ghc
@@ -60,19 +55,6 @@ let
6055 ghcjs = pnameToAttrs
6156 obelisk.haskellPackageSets.ghcjs
6257 obeliskPackagesCommon;
63− cachePackages = builtins.concatLists [
64− (builtins.attrValues ghc)
65− (lib.optionals (!profiling) (builtins.attrValues ghcjs))
66− (concatDepends ghc)
67− (lib.optionals (!profiling) (concatDepends ghcjs))
68− (lib.optional reflex-platform.androidSupport androidSkeleton)
69− (lib.optional reflex-platform.iosSupport iosSkeleton)
70− (lib.optionals (!profiling) [
71− command
72− serverSkeletonExe
73− serverSkeletonShell
74− ])
75− ];
7658 command = obelisk.command;
7759 skeleton = import ./skeleton { inherit obelisk; };
7860 serverSkeletonExe = skeleton.exe;
@@ -85,29 +67,37 @@ let
8567 androidSkeleton = (import ./skeleton { inherit obelisk; }).android.frontend;
8668 iosSkeleton = (import ./skeleton { inherit obelisk; }).ios.frontend;
8769 nameSuffix = if profiling then "profiled" else "unprofiled";
88− in {
89− inherit
90− ghc
91− ;
92− cache = reflex-platform.pinBuildInputs "obelisk-${system}-${nameSuffix}" cachePackages;
93− } // lib.optionalAttrs (!profiling) {
94− inherit
95− command
96− ghcjs
97− serverSkeletonExe
98− serverSkeletonShell
99− ;
100− } // lib.optionalAttrs reflex-platform.androidSupport {
101− inherit androidSkeleton;
102− } // lib.optionalAttrs reflex-platform.iosSupport {
103− inherit iosSkeleton;
70+ packages = {
71+ skeletonProfiledObRun = skeleton.__unstable__.profiledObRun;
72+ inherit
73+ command
74+ serverSkeletonShell
75+ ghc
76+ ;
77+ } // lib.optionalAttrs (!profiling) {
78+ inherit
79+ ghcjs
80+ serverSkeletonExe
81+ ;
82+ } // lib.optionalAttrs reflex-platform.androidSupport {
83+ inherit androidSkeleton;
84+ } // lib.optionalAttrs reflex-platform.iosSupport {
85+ inherit iosSkeleton;
86+ };
87+ in packages // {
88+ cache = reflex-platform.pinBuildInputs
89+ "obelisk-${system}-${nameSuffix}"
90+ (collect packages);
91+ };
92+
93+ perProfiling = {
94+ profiled = mkPerProfiling true;
95+ unprofiled = mkPerProfiling false;
10496 };
105− profiled = perProfiling true;
106− unprofiled = perProfiling false;
107− cachePackages = map (p: p.cache) [ profiled unprofiled ];
108− in {
109− inherit profiled unprofiled;
110− cache = reflex-platform.pinBuildInputs "obelisk-${system}" cachePackages;
97+ in perProfiling // {
98+ cache = reflex-platform.pinBuildInputs
99+ "obelisk-${system}"
100+ (map (p: p.cache) (builtins.attrValues perProfiling));
111101 });
112102
113103 metaCache = local-self.reflex-platform.pinBuildInputs
all-tests.nix+12−12
@@ -1,7 +1,7 @@
11{
22 supportedSystems ? [ builtins.currentSystem ]
33}:
4−let
4+let
55 nginxRoot = "/run/nginx";
66 obelisk = import ./default.nix {};
77 # Get NixOS a pre-release 20.03 that contains the python based tests and recursive nix
@@ -21,7 +21,7 @@ in
2121 enable = true;
2222 };
2323 environment.systemPackages = [
24− pkgs.git
24+ pkgs.git
2525 ];
2626 users.users.root.openssh.authorizedKeys.keys = [
2727 snakeOilPublicKey
@@ -38,7 +38,7 @@ in
3838 obelisk.command
3939 obelisk.shell
4040 obelisk-everywhere
41− pkgs.git
41+ pkgs.git
4242 ];
4343 };
4444 };
@@ -64,10 +64,10 @@ in
6464 in ''
6565 start_all()
6666 githost.wait_for_open_port("22")
67−
67+
6868 with subtest("test obelisk is installed"):
6969 client.succeed("ob --help")
70−
70+
7171 with subtest("test the client can access the server via ssh"):
7272 client.succeed("mkdir -p ~/.ssh/")
7373 client.succeed(
@@ -81,11 +81,11 @@ in
8181 "cp ${sshConfigFile} ~/.ssh/config"
8282 )
8383 client.wait_until_succeeds("ssh githost true")
84−
84+
8585 with subtest("test a remote bare repo can be started"):
8686 githost.succeed("mkdir -p ~/myorg/myapp.git")
8787 githost.succeed("cd ~/myorg/myapp.git && git init --bare")
88−
88+
8989 with subtest("test a git project can be configured with a remote using ssh"):
9090 client.succeed("mkdir -p ~/code/myapp")
9191 client.succeed("cd ~/code/myapp && git init")
@@ -99,18 +99,18 @@ in
9999 client.succeed(
100100 "cd ~/code/myapp && git remote add origin root@githost:/root/myorg/myapp.git"
101101 )
102−
103−
102+
103+
104104 with subtest("test pushing code to the remote"):
105105 client.succeed("cd ~/code/myapp && git push -u origin master")
106106 client.succeed("cd ~/code/myapp && git status")
107−
107+
108108 with subtest("test obelisk can pack"):
109109 client.succeed("ob -v thunk pack ~/code/myapp")
110110 client.succeed("grep -qF 'git' ~/code/myapp/default.nix")
111111 client.succeed("grep -qF 'myorg' ~/code/myapp/git.json")
112112 client.succeed("ob -v thunk unpack ~/code/myapp")
113−
113+
114114 with subtest("test obelisk can set the public / private flag"):
115115 client.succeed("ob -v thunk pack ~/code/myapp --private")
116116 client.fail("""grep -qF '"private": practice' ~/code/myapp/git.json""")
@@ -139,7 +139,7 @@ in
139139
140140 with subtest("test obelisk can detect private repos"):
141141 client.succeed("ob -v thunk pack ~/code/myapp")
142− client.succeed("""grep -qF '"private": false' ~/code/myapp/git.json""")
142+ client.succeed("""grep -qF '"private": true' ~/code/myapp/git.json""")
143143 client.succeed("ob -v thunk unpack ~/code/myapp")
144144 '';
145145 }) {}
default.nix+173−99
@@ -5,10 +5,18 @@
55, reflex-platform-func ? import ./dep/reflex-platform
66}:
77let
8− cleanSource = builtins.filterSource (name: _: let baseName = builtins.baseNameOf name; in !(
9− builtins.match "^\\.ghc\\.environment.*" baseName != null ||
10− baseName == "cabal.project.local"
11− ));
8+ reflex-platform = getReflexPlatform { inherit system; };
9+ inherit (reflex-platform) hackGet nixpkgs;
10+ pkgs = nixpkgs;
11+
12+ inherit (import dep/gitignore.nix { inherit (nixpkgs) lib; }) gitignoreSource;
13+
14+ cleanSource = src:
15+ # WARNING: The order of application here seems to matter a great deal to
16+ # how quickly `ghcid` is able to reload changes. As a rule of thumb,
17+ # always apply `gitignoreSource` first.
18+ # See https://github.com/obsidiansystems/obelisk/pull/666 and related.
19+ pkgs.lib.cleanSource (gitignoreSource src);
1220
1321 commandRuntimeDeps = pkgs: with pkgs; [
1422 coreutils
@@ -17,10 +25,8 @@ let
1725 openssh
1826 ];
1927
20− getReflexPlatform = sys: reflex-platform-func {
21− inherit iosSdkVersion config;
22− system = sys;
23− enableLibraryProfiling = profiling;
28+ getReflexPlatform = { system, enableLibraryProfiling ? profiling }: reflex-platform-func {
29+ inherit iosSdkVersion config system enableLibraryProfiling;
2430
2531 nixpkgsOverlays = [
2632 (self: super: {
@@ -39,6 +45,8 @@ let
3945 hnix-store-core = self.callHackage "hnix-store-core" "0.1.0.0" {};
4046
4147 ghcid = self.callCabal2nix "ghcid" (hackGet ./dep/ghcid) {};
48+ # Exports more internals
49+ snap-core = haskellLib.dontCheck (self.callCabal2nix "snap-core" (hackGet ./dep/snap-core) {});
4250 })
4351
4452 pkgs.obeliskExecutableConfig.haskellOverlay
@@ -58,6 +66,7 @@ let
5866 obelisk-cliapp = self.callCabal2nix "obelisk-cliapp" (cleanSource ./lib/cliapp) {};
5967 obelisk-command = haskellLib.overrideCabal (self.callCabal2nix "obelisk-command" (cleanSource ./lib/command) {}) {
6068 librarySystemDepends = [
69+ pkgs.jre
6170 pkgs.nix
6271 (haskellLib.justStaticExecutables self.ghcid)
6372 ];
@@ -71,6 +80,8 @@ let
7180 librarySystemDepends = [
7281 pkgs.cabal-install
7382 pkgs.coreutils
83+ pkgs.git
84+ pkgs.nix
7485 ];
7586 };
7687 obelisk-snap-extras = self.callCabal2nix "obelisk-snap-extras" (cleanSource ./lib/snap-extras) {};
@@ -100,10 +111,6 @@ let
100111 ];
101112 };
102113
103− reflex-platform = getReflexPlatform system;
104− inherit (reflex-platform) hackGet nixpkgs;
105− pkgs = nixpkgs;
106−
107114 # The haskell environment used to build Obelisk itself, e.g. the 'ob' command
108115 ghcObelisk = reflex-platform.ghc;
109116
@@ -126,12 +133,11 @@ in rec {
126133 shell = pinBuildInputs "obelisk-shell" ([command] ++ commandRuntimeDeps pkgs);
127134
128135 selftest = pkgs.writeScript "selftest" ''
129− #!/usr/bin/env bash
136+ #!${pkgs.runtimeShell}
130137 set -euo pipefail
131138
132139 PATH="${command}/bin:$PATH"
133− export OBELISK_IMPL="$(mktemp -d)"
134− git clone file://${pathGit} $OBELISK_IMPL
140+ cd ${./.}
135141 "${ghcObelisk.obelisk-selftest}/bin/obelisk-selftest" +RTS -N -RTS "$@"
136142 '';
137143 skeleton = pkgs.runCommand "skeleton" {
@@ -166,18 +172,28 @@ in rec {
166172 '';
167173
168174 serverModules = {
169− mkBaseEc2 = { hostName, routeHost, enableHttps, adminEmail, ... }: {...}: {
175+ mkBaseEc2 = { nixosPkgs, ... }: {...}: {
170176 imports = [
171− (pkgs.path + /nixos/modules/virtualisation/amazon-image.nix)
177+ (nixosPkgs.path + /nixos/modules/virtualisation/amazon-image.nix)
172178 ];
179+ ec2.hvm = true;
180+ };
181+
182+ mkDefaultNetworking = { adminEmail, enableHttps, hostName, routeHost, ... }: {...}: {
173183 networking = {
174184 inherit hostName;
175185 firewall.allowedTCPPorts = if enableHttps then [ 80 443 ] else [ 80 ];
176186 };
187+
188+ # `amazon-image.nix` already sets these but if the user provides their own module then
189+ # forgetting these can cause them to lose access to the server!
190+ # https://github.com/NixOS/nixpkgs/blob/fab05f17d15e4e125def4fd4e708d205b41d8d74/nixos/modules/virtualisation/amazon-image.nix#L133-L136
191+ services.openssh.enable = true;
192+ services.openssh.permitRootLogin = "prohibit-password";
193+
177194 security.acme.certs = if enableHttps then {
178195 "${routeHost}".email = adminEmail;
179196 } else {};
180− ec2.hvm = true;
181197 };
182198
183199 mkObeliskApp =
@@ -244,114 +260,172 @@ in rec {
244260 echo ${version} > $out/version
245261 '';
246262
247− server = { exe, hostName, adminEmail, routeHost, enableHttps, version }@args:
263+ server = { exe, hostName, adminEmail, routeHost, enableHttps, version, module ? serverModules.mkBaseEc2 }@args:
248264 let
249265 nixos = import (pkgs.path + /nixos);
250266 in nixos {
251267 system = "x86_64-linux";
252268 configuration = {
253269 imports = [
254− (serverModules.mkBaseEc2 args)
270+ (module { inherit exe hostName adminEmail routeHost enableHttps version; nixosPkgs = pkgs; })
271+ (serverModules.mkDefaultNetworking args)
255272 (serverModules.mkObeliskApp args)
256273 ];
257274 };
258275 };
259276
260−
261277 # An Obelisk project is a reflex-platform project with a predefined layout and role for each component
262− project = base: projectDefinition:
263− let projectOut = sys: (getReflexPlatform sys).project (args@{ nixpkgs, ... }:
264− let mkProject = { android ? null #TODO: Better error when missing
265− , ios ? null #TODO: Better error when missing
266− , packages ? {}
267− , overrides ? _: _: {}
268− , staticFiles ? base + /static
269− , tools ? _: []
270− , shellToolOverrides ? _: _: {}
271− , withHoogle ? false # Setting this to `true` makes shell reloading far slower
272− , __closureCompilerOptimizationLevel ? "ADVANCED" # Set this to `null` to skip the closure-compiler step
273− }:
274− let frontendName = "frontend";
275− backendName = "backend";
276− commonName = "common";
277− staticName = "obelisk-generated-static";
278− staticFilesImpure = if lib.isDerivation staticFiles then staticFiles else toString staticFiles;
279− processedStatic = processAssets { src = staticFiles; };
280− # The packages whose names and roles are defined by this package
281− predefinedPackages = lib.filterAttrs (_: x: x != null) {
282− ${frontendName} = nullIfAbsent (base + "/frontend");
283− ${commonName} = nullIfAbsent (base + "/common");
284− ${backendName} = nullIfAbsent (base + "/backend");
285− };
286− combinedPackages = predefinedPackages // packages;
287− projectOverrides = self: super: {
288− ${staticName} = haskellLib.dontHaddock (self.callCabal2nix staticName processedStatic.haskellManifest {});
289− ${backendName} = haskellLib.addBuildDepend super.${backendName} self.obelisk-run;
290− };
291− totalOverrides = lib.composeExtensions projectOverrides overrides;
292− inherit (lib.strings) hasPrefix;
293− privateConfigDirs = ["config/backend"];
294− injectableConfig = builtins.filterSource (path: _:
295− !(lib.lists.any (x: hasPrefix (toString base + "/" + toString x) (toString path)) privateConfigDirs)
296− );
297− __androidWithConfig = configPath: {
298− ${if android == null then null else frontendName} = {
299− executableName = "frontend";
300− ${if builtins.pathExists staticFiles then "assets" else null} =
301− nixpkgs.obeliskExecutableConfig.platforms.android.inject
302− (injectableConfig configPath)
303− processedStatic.symlinked;
304− } // android;
278+ project = base': projectDefinition:
279+ let
280+ projectOut = { system, enableLibraryProfiling ? profiling }: let reflexPlatformProject = (getReflexPlatform { inherit system enableLibraryProfiling; }).project; in reflexPlatformProject (args@{ nixpkgs, ... }:
281+ let
282+ inherit (lib.strings) hasPrefix;
283+ mkProject =
284+ { android ? null #TODO: Better error when missing
285+ , ios ? null #TODO: Better error when missing
286+ , packages ? {}
287+ , overrides ? _: _: {}
288+ , staticFiles ? null
289+ , tools ? _: []
290+ , shellToolOverrides ? _: _: {}
291+ , withHoogle ? false # Setting this to `true` makes shell reloading far slower
292+ , __closureCompilerOptimizationLevel ? "ADVANCED" # Set this to `null` to skip the closure-compiler step
293+ }:
294+ let
295+ allConfig = nixpkgs.lib.makeExtensible (self: {
296+ base = base';
297+ inherit args;
298+ userSettings = {
299+ inherit android ios packages overrides tools shellToolOverrides withHoogle __closureCompilerOptimizationLevel;
300+ staticFiles = if staticFiles == null then self.base + /static else staticFiles;
301+ };
302+ frontendName = "frontend";
303+ backendName = "backend";
304+ commonName = "common";
305+ staticName = "obelisk-generated-static";
306+ staticFilesImpure = let fs = self.userSettings.staticFiles; in if lib.isDerivation fs then fs else toString fs;
307+ processedStatic = processAssets { src = self.userSettings.staticFiles; };
308+ # The packages whose names and roles are defined by this package
309+ predefinedPackages = lib.filterAttrs (_: x: x != null) {
310+ ${self.frontendName} = nullIfAbsent (self.base + "/frontend");
311+ ${self.commonName} = nullIfAbsent (self.base + "/common");
312+ ${self.backendName} = nullIfAbsent (self.base + "/backend");
313+ };
314+ shellPackages = {};
315+ combinedPackages = self.predefinedPackages // self.userSettings.packages // self.shellPackages;
316+ projectOverrides = self': super': {
317+ ${self.staticName} = haskellLib.dontHaddock (self'.callCabal2nix self.staticName self.processedStatic.haskellManifest {});
318+ ${self.backendName} = haskellLib.addBuildDepend super'.${self.backendName} self'.obelisk-run;
319+ };
320+ totalOverrides = lib.composeExtensions self.projectOverrides self.userSettings.overrides;
321+ privateConfigDirs = ["config/backend"];
322+ injectableConfig = builtins.filterSource (path: _:
323+ !(lib.lists.any (x: hasPrefix (toString self.base + "/" + toString x) (toString path)) self.privateConfigDirs)
324+ );
325+ __androidWithConfig = configPath: {
326+ ${if self.userSettings.android == null then null else self.frontendName} = {
327+ executableName = "frontend";
328+ ${if builtins.pathExists self.userSettings.staticFiles then "assets" else null} =
329+ nixpkgs.obeliskExecutableConfig.platforms.android.inject
330+ (self.injectableConfig configPath)
331+ self.processedStatic.symlinked;
332+ } // self.userSettings.android;
333+ };
334+ __iosWithConfig = configPath: {
335+ ${if self.userSettings.ios == null then null else self.frontendName} = {
336+ executableName = "frontend";
337+ ${if builtins.pathExists self.userSettings.staticFiles then "staticSrc" else null} =
338+ nixpkgs.obeliskExecutableConfig.platforms.ios.inject
339+ (self.injectableConfig configPath)
340+ self.processedStatic.symlinked;
341+ } // self.userSettings.ios;
342+ };
343+
344+ shells-ghc = builtins.attrNames (self.predefinedPackages // self.shellPackages);
345+
346+ shells-ghcjs = [
347+ self.frontendName
348+ self.commonName
349+ ];
350+
351+ shells-ghcSavedSplices = [
352+ self.commonName
353+ self.frontendName
354+ ];
355+
356+ project = reflexPlatformProject ({...}: self.projectConfig);
357+ projectConfig = {
358+ inherit (self.userSettings) shellToolOverrides tools withHoogle;
359+ overrides = self.totalOverrides;
360+ packages = self.combinedPackages;
361+ shells = {
362+ ${if self.userSettings.android == null && self.userSettings.ios == null then null else "ghcSavedSplices"} =
363+ lib.filter (x: lib.hasAttr x self.combinedPackages) self.shells-ghcSavedSplices;
364+ ghc = lib.filter (x: lib.hasAttr x self.combinedPackages) self.shells-ghc;
365+ ghcjs = lib.filter (x: lib.hasAttr x self.combinedPackages) self.shells-ghcjs;
305366 };
306− __iosWithConfig = configPath: {
307− ${if ios == null then null else frontendName} = {
308− executableName = "frontend";
309− ${if builtins.pathExists staticFiles then "staticSrc" else null} =
310− nixpkgs.obeliskExecutableConfig.platforms.ios.inject
311− (injectableConfig configPath)
312− processedStatic.symlinked;
313− } // ios;
367+ android = self.__androidWithConfig (self.base + "/config");
368+ ios = self.__iosWithConfig (self.base + "/config");
369+
370+ passthru = {
371+ __unstable__.self = allConfig;
372+ inherit (self)
373+ staticFilesImpure processedStatic
374+ __iosWithConfig __androidWithConfig
375+ ;
376+ inherit (self.userSettings)
377+ android ios overrides packages shellToolOverrides staticFiles tools withHoogle
378+ __closureCompilerOptimizationLevel
379+ ;
314380 };
315− in {
316− inherit shellToolOverrides tools withHoogle;
317− overrides = totalOverrides;
318− packages = combinedPackages;
319− shells = {
320− ${if android == null && ios == null then null else "ghcSavedSplices"} = (lib.filter (x: lib.hasAttr x combinedPackages) [
321− commonName
322− frontendName
323− ]);
324− ghc = (lib.filter (x: lib.hasAttr x combinedPackages) [
325− backendName
326− commonName
327− frontendName
328− ]);
329− ghcjs = lib.filter (x: lib.hasAttr x combinedPackages) [
330− frontendName
331− commonName
332− ];
333381 };
334− android = __androidWithConfig (base + "/config");
335− ios = __iosWithConfig (base + "/config");
336− passthru = { inherit android ios packages overrides tools shellToolOverrides withHoogle staticFiles staticFilesImpure __closureCompilerOptimizationLevel processedStatic __iosWithConfig __androidWithConfig; };
337− };
338− in mkProject (projectDefinition args));
339− mainProjectOut = projectOut system;
382+ });
383+ in allConfig;
384+ in (mkProject (projectDefinition args)).projectConfig);
385+ mainProjectOut = projectOut { inherit system; };
340386 serverOn = projectInst: version: serverExe
341387 projectInst.ghc.backend
342388 mainProjectOut.ghcjs.frontend
343389 projectInst.passthru.staticFiles
344390 projectInst.passthru.__closureCompilerOptimizationLevel
345391 version;
346− linuxExe = serverOn (projectOut "x86_64-linux");
392+ linuxExe = serverOn (projectOut { system = "x86_64-linux"; });
347393 dummyVersion = "Version number is only available for deployments";
348394 in mainProjectOut // {
395+ __unstable__.profiledObRun = let
396+ profiled = projectOut { inherit system; enableLibraryProfiling = true; };
397+ exeSource = builtins.toFile "ob-run.hs" ''
398+ module Main where
399+
400+ import Control.Exception
401+ import Reflex.Profiled
402+ import System.Environment
403+
404+ import qualified Obelisk.Run
405+ import qualified Frontend
406+ import qualified Backend
407+
408+ main :: IO ()
409+ main = do
410+ args <- getArgs
411+ let port = read $ args !! 0
412+ assets = args !! 1
413+ profileFile = (args !! 2) <> ".rprof"
414+ Obelisk.Run.run port (Obelisk.Run.runServeAsset assets) Backend.backend Frontend.frontend `finally` writeProfilingData profileFile
415+ '';
416+ in nixpkgs.runCommand "ob-run" {
417+ buildInputs = [ (profiled.ghc.ghcWithPackages (p: [ p.backend p.frontend])) ];
418+ } ''
419+ mkdir -p $out/bin/
420+ ghc -x hs -prof -fno-prof-auto -threaded ${exeSource} -o $out/bin/ob-run
421+ '';
422+
349423 linuxExeConfigurable = linuxExe;
350424 linuxExe = linuxExe dummyVersion;
351425 exe = serverOn mainProjectOut dummyVersion;
352− server = args@{ hostName, adminEmail, routeHost, enableHttps, version }:
426+ server = args@{ hostName, adminEmail, routeHost, enableHttps, version, module ? serverModules.mkBaseEc2 }:
353427 server (args // { exe = linuxExe version; });
354− obelisk = import (base + "/.obelisk/impl") {};
428+ obelisk = import (base' + "/.obelisk/impl") {};
355429 };
356430 haskellPackageSets = {
357431 inherit (reflex-platform) ghc ghcjs;
dep/gitignore.nix/default.nixnew file+8
@@ -0,0 +1,8 @@
1+# DO NOT HAND-EDIT THIS FILE
2+let fetch = { private ? false, fetchSubmodules ? false, owner, repo, rev, sha256, ... }:
3+ if !fetchSubmodules && !private then builtins.fetchTarball {
4+ url = "https://github.com/${owner}/${repo}/archive/${rev}.tar.gz"; inherit sha256;
5+ } else (import <nixpkgs> {}).fetchFromGitHub {
6+ inherit owner repo rev sha256 fetchSubmodules private;
7+ };
8+in import (fetch (builtins.fromJSON (builtins.readFile ./github.json)))
dep/gitignore.nix/github.jsonnew file+8
@@ -0,0 +1,8 @@
1+{
2+ "owner": "hercules-ci",
3+ "repo": "gitignore.nix",
4+ "branch": "master",
5+ "private": false,
6+ "rev": "7415c4feb127845553943a3856cbc5cb967ee5e0",
7+ "sha256": "1zd1ylgkndbb5szji32ivfhwh04mr1sbgrnvbrqpmfb67g2g3r9i"
8+}
dep/reflex-platform/default.nix+7−6
@@ -1,7 +1,8 @@
11# DO NOT HAND-EDIT THIS FILE
2−import ((import <nixpkgs> {}).fetchFromGitHub (
3− let json = builtins.fromJSON (builtins.readFile ./github.json);
4− in { inherit (json) owner repo rev sha256;
5− private = json.private or false;
6− }
7−))
2+let fetch = { private ? false, fetchSubmodules ? false, owner, repo, rev, sha256, ... }:
3+ if !fetchSubmodules && !private then builtins.fetchTarball {
4+ url = "https://github.com/${owner}/${repo}/archive/${rev}.tar.gz"; inherit sha256;
5+ } else (import <nixpkgs> {}).fetchFromGitHub {
6+ inherit owner repo rev sha256 fetchSubmodules private;
7+ };
8+in import (fetch (builtins.fromJSON (builtins.readFile ./github.json)))
dep/reflex-platform/github.json+3−2
@@ -2,6 +2,7 @@
22 "owner": "reflex-frp",
33 "repo": "reflex-platform",
44 "branch": "develop",
5− "rev": "90351bdb2c99d416ea95fd5b8b474e8f9fab2942",
6− "sha256": "0bbzz4ckdx0ilhhpd3drfbsc0zdb2rnhw2m3nls9villbs3lz3lh"
5+ "private": false,
6+ "rev": "156ebb7c391ec909d19e794e591f107462862543",
7+ "sha256": "0rvbaimcvcica80538i6vxckv8si1gbd1ynvgxv7zr5jd1gk6vi5"
78}
dep/snap-core/default.nixnew file+8
@@ -0,0 +1,8 @@
1+# DO NOT HAND-EDIT THIS FILE
2+let fetch = { private ? false, fetchSubmodules ? false, owner, repo, rev, sha256, ... }:
3+ if !fetchSubmodules && !private then builtins.fetchTarball {
4+ url = "https://github.com/${owner}/${repo}/archive/${rev}.tar.gz"; inherit sha256;
5+ } else (import <nixpkgs> {}).fetchFromGitHub {
6+ inherit owner repo rev sha256 fetchSubmodules private;
7+ };
8+in import (fetch (builtins.fromJSON (builtins.readFile ./github.json)))
dep/snap-core/github.jsonnew file+8
@@ -0,0 +1,8 @@
1+{
2+ "owner": "obsidiansystems",
3+ "repo": "snap-core",
4+ "branch": "ts-expose-fileserve-internals",
5+ "private": false,
6+ "rev": "e015fc484e38dd9389c58b277b1884395e553bdf",
7+ "sha256": "1793x1h4lw9vnrii5akhfxx1x9yrgxf3052jf6bs8f1mhaic6mxx"
8+}
lib/asset/serve-snap/src/Obelisk/Asset/Accept.hs+6−6
@@ -136,16 +136,16 @@ qvalue = do
136136 skipMany lws
137137 q0 <|> q1
138138 where q0 = do
139− char '0'
139+ _ <- char '0'
140140 decimals <- option [] $ do
141− char '.'
141+ _ <- char '.'
142142 starRule (Just 0) (Just numAllowedDigits) digit
143143 return $ QValue $ MkFixed $ fromIntegral $ (read ('0' : decimals) :: Int) * 10 ^ (numAllowedDigits - length decimals)
144144 q1 = do
145− char '1'
145+ _ <- char '1'
146146 option () $ do
147− char '.'
148− starRule (Just 0) (Just numAllowedDigits) $ char '0'
147+ _ <- char '.'
148+ _ <- starRule (Just 0) (Just numAllowedDigits) $ char '0'
149149 return ()
150150 return $ QValue 1
151151 numAllowedDigits :: Int
@@ -232,7 +232,7 @@ hashRule minNum maxNum element = do
232232lws :: Parser ()
233233lws = do
234234 option () $ cr >> lf
235− starRule (Just 1) Nothing $ sp <|> ht
235+ _ <- starRule (Just 1) Nothing $ sp <|> ht
236236 return ()
237237
238238-- | See http://www.w3.org/Protocols/rfc2616/rfc2616-sec2.html#sec2.2
lib/asset/serve-snap/src/Obelisk/Asset/Serve/Snap.hs+14−4
@@ -12,12 +12,13 @@ import Obelisk.Asset.Accept (Encoding (..), acceptEncodingBody, chooseEncoding,
1212import Obelisk.Snap.Extras
1313
1414import Snap
15− (MonadSnap, getHeader, getsRequest, modifyResponse, pass, redirect, sendFile, setContentLength, setContentType, setHeader, setResponseCode)
15+ (MonadSnap, getHeader, getRequest, getsRequest, modifyResponse, pass, redirect, sendFile, setContentLength, setContentType, setHeader, setResponseCode)
1616import Snap.Util.FileServe (defaultMimeTypes, fileType, getSafePath, serveFile)
17+import Snap.Internal.Util.FileServe (checkRangeReq)
1718
1819import Control.Applicative ((<|>))
1920import Control.Exception (handleJust, try, throwIO)
20−import Control.Monad (forM, liftM)
21+import Control.Monad (forM, liftM, unless)
2122import Control.Monad.IO.Class (liftIO)
2223import Control.Monad.Fail (MonadFail)
2324import Data.Attoparsec.ByteString (parseOnly, endOfInput)
@@ -94,8 +95,17 @@ serveAsset' doRedirect base fallback p = do
9495 let finalFilename = base </> p </> "encodings" </> T.unpack (decodeUtf8 e)
9596 stat <- liftIO $ getFileStatus finalFilename
9697 modifyResponse $ setHeader "Last-Modified" "Thu, 1 Jan 1970 00:00:00 GMT"
97− modifyResponse $ setResponseCode 200 . setContentLength (fromIntegral $ fileSize stat) . setContentType (fileType defaultMimeTypes p)
98− sendFile finalFilename
98+ . setHeader "Accept-Ranges" "bytes"
99+ . setContentType (fileType defaultMimeTypes p)
100+ let size = fromIntegral $ fileSize stat
101+ req <- getRequest
102+ -- Despite the name, this function actually does all of the work for
103+ -- responding to range requests. We only need to handle *none* range
104+ -- requests ourselves.
105+ wasRange <- checkRangeReq req finalFilename size
106+ unless wasRange $ do
107+ modifyResponse $ setResponseCode 200 . setContentLength size
108+ sendFile finalFilename
99109 Just _ -> cachePermanently >> modifyResponse (setResponseCode 304)
100110 Right "redirect" -> do
101111 mtarget <- liftIO $ getAssetTarget $ base </> p
lib/backend/src/Obelisk/Backend.hs+90−29
@@ -1,3 +1,4 @@
1+{-# LANGUAGE DeriveFunctor #-}
12{-# LANGUAGE FlexibleContexts #-}
23{-# LANGUAGE FlexibleInstances #-}
34{-# LANGUAGE GADTs #-}
@@ -9,6 +10,17 @@
910{-# OPTIONS_GHC -fno-warn-orphans #-}
1011module Obelisk.Backend
1112 ( Backend (..)
13+ -- * Running a backend
14+ , runBackend
15+ , runBackendWith
16+ -- * Configuration of backend
17+ , GhcjsWidgets(..)
18+ , defaultGhcjsWidgets
19+ -- * all.js script loading functions
20+ , deferredGhcjsScript
21+ , delayedGhcjsScript
22+ -- * all.js preload functions
23+ , preloadGhcjs
1224 -- * Re-exports
1325 , Default (def)
1426 , getPageName
@@ -16,18 +28,14 @@ module Obelisk.Backend
1628 , runSnapWithCommandLineArgs
1729 , serveDefaultObeliskApp
1830 , prettifyOutput
19− , runBackend
31+ , renderAllJsPath
2032 , staticRenderContentType
21− , mkRouteToUrl
2233 , getPublicConfigs
2334 ) where
2435
25−import Prelude hiding (id, (.))
26−import Control.Category
2736import Control.Monad
2837import Control.Monad.Except
2938import Control.Monad.Fail (MonadFail)
30−import Control.Categorical.Bifunctor
3139import Data.ByteString (ByteString)
3240import qualified Data.ByteString.Char8 as BSC8
3341import Data.Default (Default (..))
@@ -38,7 +46,6 @@ import qualified Data.Map as Map
3846import Data.Monoid ((<>))
3947import Data.Text (Text)
4048import qualified Data.Text as T
41−import Data.Text.Encoding
4249import Obelisk.Asset.Serve.Snap (serveAsset)
4350import qualified Obelisk.ExecutableConfig.Lookup as Lookup
4451import Obelisk.Frontend
@@ -62,19 +69,35 @@ data GhcjsApp route = GhcjsApp
6269 , _ghcjsApp_value :: !(Frontend route)
6370 }
6471
72+-- | Widgets used to load all.js on the frontend
73+data GhcjsWidgets a = GhcjsWidgets
74+ { _ghcjsWidgets_preload :: a
75+ -- ^ A preload widget, placed in the document head
76+ , _ghcjsWidgets_script :: a
77+ -- ^ A script widget, placed in the document body
78+ } deriving Functor
79+
80+-- | Given the URL of all.js, return the widgets which are responsible for
81+-- loading the script. Defaults to 'preloadGhcjs' and 'deferredGhcjsScript'.
82+defaultGhcjsWidgets :: GhcjsWidgets (Text -> FrontendWidgetT r ())
83+defaultGhcjsWidgets = GhcjsWidgets
84+ { _ghcjsWidgets_preload = preloadGhcjs
85+ , _ghcjsWidgets_script = deferredGhcjsScript
86+ }
87+
6588-- | Serve a frontend, which must be the same frontend that Obelisk has built and placed in the default location
6689--TODO: The frontend should be provided together with the asset paths so that this isn't so easily breakable; that will probably make this function obsolete
6790serveDefaultObeliskApp
6891 :: (MonadSnap m, HasCookies m, MonadFail m)
69− => (R appRoute
70− -> Text)
71− -> ([Text]
72− -> m ())
92+ => (R appRoute -> Text)
93+ -> GhcjsWidgets (FrontendWidgetT (R appRoute) ())
94+ -> ([Text] -> m ())
7395 -> Frontend (R appRoute)
7496 -> Map Text ByteString
7597 -> R (ObeliskRoute appRoute)
7698 -> m ()
77−serveDefaultObeliskApp urlEnc serveStaticAsset frontend configs = serveObeliskApp urlEnc serveStaticAsset frontendApp configs
99+serveDefaultObeliskApp urlEnc ghcjsWidgets serveStaticAsset frontend =
100+ serveObeliskApp urlEnc ghcjsWidgets serveStaticAsset frontendApp
78101 where frontendApp = GhcjsApp
79102 { _ghcjsApp_compiled = defaultFrontendGhcjsAssets
80103 , _ghcjsApp_value = frontend
@@ -121,19 +144,24 @@ getRouteWith e = do
121144 pageName <- getPageName
122145 return $ tryDecode e pageName
123146
147+renderAllJsPath :: Encoder Identity Identity (R (FullRoute a b)) PageName -> Text
148+renderAllJsPath validFullEncoder =
149+ renderObeliskRoute validFullEncoder $ FullRoute_Frontend (ObeliskRoute_Resource ResourceRoute_Ghcjs) :/ ["all.js"]
150+
124151serveObeliskApp
125152 :: (MonadSnap m, HasCookies m, MonadFail m)
126153 => (R appRoute -> Text)
154+ -> GhcjsWidgets (FrontendWidgetT (R appRoute) ())
127155 -> ([Text] -> m ())
128156 -> GhcjsApp (R appRoute)
129157 -> Map Text ByteString
130158 -> R (ObeliskRoute appRoute)
131159 -> m ()
132−serveObeliskApp urlEnc serveStaticAsset frontendApp config = \case
133− ObeliskRoute_App appRouteComponent :=> Identity appRouteRest -> serveGhcjsApp urlEnc frontendApp config $ GhcjsAppRoute_App appRouteComponent :/ appRouteRest
160+serveObeliskApp urlEnc ghcjsWidgets serveStaticAsset frontendApp config = \case
161+ ObeliskRoute_App appRouteComponent :=> Identity appRouteRest -> serveGhcjsApp urlEnc ghcjsWidgets frontendApp config $ GhcjsAppRoute_App appRouteComponent :/ appRouteRest
134162 ObeliskRoute_Resource resComponent :=> Identity resRest -> case resComponent :=> Identity resRest of
135163 ResourceRoute_Static :=> Identity pathSegments -> serveStaticAsset pathSegments
136− ResourceRoute_Ghcjs :=> Identity pathSegments -> serveGhcjsApp urlEnc frontendApp config $ GhcjsAppRoute_Resource :/ pathSegments
164+ ResourceRoute_Ghcjs :=> Identity pathSegments -> serveGhcjsApp urlEnc ghcjsWidgets frontendApp config $ GhcjsAppRoute_Resource :/ pathSegments
137165 ResourceRoute_JSaddleWarp :=> Identity _ -> do
138166 let msg = "Error: Obelisk.Backend received jsaddle request"
139167 liftIO $ putStrLn $ T.unpack msg
@@ -160,48 +188,81 @@ staticRenderContentType = "text/html; charset=utf-8"
160188serveGhcjsApp
161189 :: (MonadSnap m, HasCookies m, MonadFail m)
162190 => (R appRouteComponent -> Text)
191+ -> GhcjsWidgets (FrontendWidgetT (R appRouteComponent) ())
163192 -> GhcjsApp (R appRouteComponent)
164193 -> Map Text ByteString
165194 -> R (GhcjsAppRoute appRouteComponent)
166195 -> m ()
167−serveGhcjsApp urlEnc app config = \case
196+serveGhcjsApp urlEnc ghcjsWidgets app config = \case
168197 GhcjsAppRoute_App appRouteComponent :=> Identity appRouteRest -> do
169198 modifyResponse $ setContentType staticRenderContentType
170199 modifyResponse $ setHeader "Cache-Control" "no-store private"
171− writeBS <=< renderGhcjsFrontend urlEnc (appRouteComponent :/ appRouteRest) config $ _ghcjsApp_value app
200+ writeBS <=< renderGhcjsFrontend urlEnc ghcjsWidgets (appRouteComponent :/ appRouteRest) config $ _ghcjsApp_value app
172201 GhcjsAppRoute_Resource :=> Identity pathSegments -> serveStaticAssets (_ghcjsApp_compiled app) pathSegments
173202
174203runBackend :: Backend backendRoute frontendRoute -> Frontend (R frontendRoute) -> IO ()
175−runBackend backend frontend = case checkEncoder $ _backend_routeEncoder backend of
204+runBackend = runBackendWith defaultGhcjsWidgets
205+
206+runBackendWith
207+ :: GhcjsWidgets (Text -> FrontendWidgetT (R frontendRoute) ())
208+ -- ^ Given the URL of all.js, return the widgets which are responsible for
209+ -- loading the script.
210+ -> Backend backendRoute frontendRoute
211+ -> Frontend (R frontendRoute)
212+ -> IO ()
213+runBackendWith ghcjsWidgets backend frontend = case checkEncoder $ _backend_routeEncoder backend of
176214 Left e -> fail $ "backend error:\n" <> T.unpack e
177215 Right validFullEncoder -> do
178216 publicConfigs <- getPublicConfigs
179− _backend_run backend $ \serveRoute -> do
180− runSnapWithCommandLineArgs $ do
217+ _backend_run backend $ \serveRoute ->
218+ runSnapWithCommandLineArgs $
181219 getRouteWith validFullEncoder >>= \case
182220 Identity r -> case r of
183221 FullRoute_Backend backendRoute :/ a -> serveRoute $ backendRoute :/ a
184222 FullRoute_Frontend obeliskRoute :/ a ->
185− serveDefaultObeliskApp (mkRouteToUrl validFullEncoder) (serveStaticAssets defaultStaticAssets) frontend publicConfigs $
223+ serveDefaultObeliskApp routeToUrl (($ allJsUrl) <$> ghcjsWidgets) (serveStaticAssets defaultStaticAssets) frontend publicConfigs $
186224 obeliskRoute :/ a
187−
188−mkRouteToUrl :: Encoder Identity parse (R (FullRoute br fr)) PageName -> R fr -> Text
189−mkRouteToUrl validFullEncoder =
190− let pageNameEncoder' :: Encoder Identity (Either Text) PageName PathQuery = pageNameEncoder
191− in \(k :/ v) -> T.pack . uncurry (<>) . encode pageNameEncoder' . encode validFullEncoder $ (FullRoute_Frontend $ ObeliskRoute_App k) :/ v
225+ where
226+ routeToUrl (k :/ v) = renderObeliskRoute validFullEncoder $ FullRoute_Frontend (ObeliskRoute_App k) :/ v
227+ allJsUrl = renderAllJsPath validFullEncoder
192228
193229renderGhcjsFrontend
194230 :: (MonadSnap m, HasCookies m)
195231 => (route -> Text)
232+ -> GhcjsWidgets (FrontendWidgetT route ())
196233 -> route
197234 -> Map Text ByteString
198235 -> Frontend route
199236 -> m ByteString
200−renderGhcjsFrontend urlEnc route configs f = do
201− let ghcjsPreload = elAttr "link" ("rel" =: "preload" <> "as" =: "script" <> "href" =: "ghcjs/all.js") blank
202− ghcjsScript = elAttr "script" ("language" =: "javascript" <> "src" =: "ghcjs/all.js" <> "defer" =: "defer") blank
237+renderGhcjsFrontend urlEnc ghcjsWidgets route configs f = do
203238 cookies <- askCookies
204− renderFrontendHtml configs cookies urlEnc route f ghcjsPreload ghcjsScript
239+ renderFrontendHtml configs cookies urlEnc route f (_ghcjsWidgets_preload ghcjsWidgets) (_ghcjsWidgets_script ghcjsWidgets)
240+
241+-- | Preload all.js in a link tag.
242+-- This is the default preload method.
243+preloadGhcjs :: Text -> FrontendWidgetT r ()
244+preloadGhcjs allJsUrl = elAttr "link" ("rel" =: "preload" <> "as" =: "script" <> "href" =: allJsUrl) blank
245+
246+-- | Load the script from the given URL in a deferred script tag.
247+-- This is the default method.
248+deferredGhcjsScript :: Text -> FrontendWidgetT r ()
249+deferredGhcjsScript allJsUrl = elAttr "script" ("type" =: "text/javascript" <> "src" =: allJsUrl <> "defer" =: "defer") blank
250+
251+-- | An all.js script which is loaded after waiting for some time to pass. This
252+-- is useful to ensure any CSS animations on the page can play smoothly before
253+-- blocking the UI thread by running all.js.
254+delayedGhcjsScript
255+ :: Int -- ^ The number of milliseconds to delay loading by
256+ -> Text -- ^ URL to GHCJS app JavaScript
257+ -> FrontendWidgetT r ()
258+delayedGhcjsScript n allJsUrl = elAttr "script" ("type" =: "text/javascript") $ text $ T.unlines
259+ [ "setTimeout(function() {"
260+ , " var all_js_script = document.createElement('script');"
261+ , " all_js_script.type = 'text/javascript';"
262+ , " all_js_script.src = '" <> allJsUrl <> "';"
263+ , " document.body.appendChild(all_js_script);"
264+ , "}, " <> T.pack (show n) <> ");"
265+ ]
205266
206267instance HasCookies Snap where
207268 askCookies = map (\c -> (cookieName c, cookieValue c)) <$> getsRequest rqCookies
lib/cliapp/obelisk-cliapp.cabal+2−2
@@ -7,7 +7,8 @@ cabal-version: >=1.2
77library
88 hs-source-dirs: src
99 build-depends:
10− ansi-terminal
10+ aeson
11+ , ansi-terminal
1112 , base
1213 , bytestring
1314 , containers
@@ -24,7 +25,6 @@ library
2425 , transformers
2526 exposed-modules:
2627 Obelisk.CliApp
27− Obelisk.CliApp.Demo
2828 other-modules:
2929 Obelisk.CliApp.Logging
3030 Obelisk.CliApp.Process
lib/cliapp/src/Obelisk/CliApp.hs+2
@@ -45,11 +45,13 @@ module Obelisk.CliApp
4545 , readCreateProcessWithExitCode
4646 , readProcessAndLogOutput
4747 , readProcessAndLogStderr
48+ , readProcessJSONAndLogStderr
4849 , reconstructCommand
4950 , setCwd
5051 , setDelegateCtlc
5152 , setEnvOverride
5253 , shell
54+ , waitForProcess
5355 ) where
5456
5557import Control.Monad.Log (Severity (..))
lib/cliapp/src/Obelisk/CliApp/Demo.hsdeleted−52
@@ -1,52 +0,0 @@
1−{-# LANGUAGE FlexibleContexts #-}
2−{-# LANGUAGE OverloadedStrings #-}
3−{-# LANGUAGE ScopedTypeVariables #-}
4−module Obelisk.CliApp.Demo where
5−
6−import Control.Concurrent (threadDelay)
7−import Control.Monad.Fail
8−import Control.Monad.IO.Class
9−import Data.Semigroup ((<>))
10−import qualified Data.Text as T
11−
12−import Control.Monad.Catch (MonadMask)
13−
14−import Obelisk.CliApp
15−
16−cliDemo
17− :: ( MonadIO m, MonadMask m, MonadFail m
18− , CliLog m, HasCliConfig e m, CliThrow e m, AsProcessFailure e, AsUnstructuredError e)
19− => m ()
20−cliDemo = withSpinner "CLI Demo" $ do
21− putLog Notice "This demo will showcase the CLI library functionality"
22− _ <- withSpinner' "Searching long for something" (Just $ \c -> "Discovered " <> T.pack (show c) <> " objects") $ do
23− delay
24− withSpinner "Nested task" $ do
25− delay
26− putLog Notice "In nested task"
27− -- This is a 'no trail' spinner that won't leave a trail (even its sub spinners)
28− withSpinnerNoTrail "In between, doing something temporary" $ do
29− withSpinner "Runnning something deep inside" $ do
30− delay
31− putLog Notice "You can still log from inside no-trail spinners"
32− delay
33− delay
34− putLog Error "Top nested task finished"
35− delay
36− putLog Error "Some user error while spinning"
37− delay
38− putLog Notice "This is some info mesage"
39− putLog Warning "And now a warning as well"
40− delay
41− return (42 :: Integer)
42− putLog Notice "Now we start a 2nd spinner, run a couple of process, the last of which fails:"
43− withSpinner "Looking around" $ do
44− delay
45− output <- readProcessAndLogStderr Notice $ proc "ls" ["-l", "/"]
46− putLog Notice $ "Output was: " <> output
47− delay
48− callProcessAndLogOutput (Notice, Error) $ proc "ls" ["-l", "/does-not-exist"]
49− delay
50− failWith "Something dangerous happened"
51− where
52− delay = liftIO $ threadDelay 1000000
lib/cliapp/src/Obelisk/CliApp/Logging.hs+11−10
@@ -39,7 +39,6 @@ import Control.Monad.Log (Severity (..), WithSeverity (..), logMessage, runLoggi
3939import Control.Monad.Loops (iterateUntil)
4040import Control.Monad.Reader (MonadIO, ReaderT (..))
4141import Data.IORef (atomicModifyIORef', newIORef, readIORef, writeIORef)
42−import Data.Maybe
4342import Data.Semigroup ((<>))
4443import Data.Text (Text)
4544import qualified Data.Text as T
@@ -67,7 +66,7 @@ newCliConfig sev noColor noSpinner errorLogExitCode = do
6766 tipDisplayed <- newIORef False
6867 stack <- newIORef ([], [])
6968 textEncoding <- hGetEncoding stdout
70− let theme = if fromMaybe False $ supportsUnicode <$> textEncoding
69+ let theme = if maybe False supportsUnicode textEncoding
7170 then unicodeTheme
7271 else noUnicodeTheme
7372 return $ CliConfig level noColor noSpinner lock tipDisplayed stack errorLogExitCode theme
@@ -133,7 +132,7 @@ handleLog' noColor output = do
133132 hFlush stdout
134133 Output_Overwrite ts -> liftIO $ do
135134 width <- TS.getTerminalWidth
136− T.putStr $ "\r" <> (TS.render (not noColor) width ts)
135+ T.putStr $ "\r" <> TS.render (not noColor) width ts
137136 hFlush stdout
138137 Output_ClearLine -> liftIO $ do
139138 -- Go to the first column and clear the whole line
@@ -179,14 +178,16 @@ withExitFailMessage msg f = f `catch` \(e :: ExitCode) -> do
179178
180179-- | Write log to stdout, with colors (unless `noColor`)
181180writeLog :: (MonadIO m, MonadMask m) => Bool -> Bool -> WithSeverity Text -> m ()
182−writeLog withNewLine noColor (WithSeverity severity s)
183− | noColor && severity <= Warning = liftIO $ putFn $ T.pack (show severity) <> ": " <> s
184− | not noColor && severity <= Error = TS.putStrWithSGR errorColors h withNewLine s
185− | not noColor && severity <= Warning = TS.putStrWithSGR warningColors h withNewLine s
186− | not noColor && severity >= Debug = TS.putStrWithSGR debugColors h withNewLine s
187− | otherwise = liftIO $ putFn s
181+writeLog withNewLine noColor (WithSeverity severity s) = if T.null s then pure () else write
188182 where
189− putFn = if withNewLine then (T.hPutStrLn h) else (T.hPutStr h)
183+ write
184+ | noColor && severity <= Warning = liftIO $ putFn $ T.pack (show severity) <> ": " <> s
185+ | not noColor && severity <= Error = TS.putStrWithSGR errorColors h withNewLine s
186+ | not noColor && severity <= Warning = TS.putStrWithSGR warningColors h withNewLine s
187+ | not noColor && severity >= Debug = TS.putStrWithSGR debugColors h withNewLine s
188+ | otherwise = liftIO $ putFn s
189+
190+ putFn = if withNewLine then T.hPutStrLn h else T.hPutStr h
190191 h = if severity <= Error then stderr else stdout
191192 errorColors = [SetColor Foreground Vivid Red]
192193 warningColors = [SetColor Foreground Vivid Yellow]
lib/cliapp/src/Obelisk/CliApp/Process.hs+23−3
@@ -22,11 +22,13 @@ module Obelisk.CliApp.Process
2222 , readCreateProcessWithExitCode
2323 , readProcessAndLogOutput
2424 , readProcessAndLogStderr
25+ , readProcessJSONAndLogStderr
2526 , reconstructCommand
2627 , setCwd
2728 , setDelegateCtlc
2829 , setEnvOverride
2930 , shell
31+ , waitForProcess
3032 ) where
3133
3234import Control.Monad ((<=<), join, void)
@@ -50,8 +52,9 @@ import System.IO (Handle)
5052import System.IO.Streams (InputStream, handleToInputStream)
5153import qualified System.IO.Streams as Streams
5254import System.IO.Streams.Concurrent (concurrentMerge)
53−import System.Process (CreateProcess, ProcessHandle, StdStream (CreatePipe), std_err, std_out, waitForProcess)
55+import System.Process (CreateProcess, ProcessHandle, StdStream (CreatePipe), std_err, std_out)
5456import qualified System.Process as Process
57+import qualified Data.Aeson as Aeson
5558
5659import Control.Monad.Log (Severity (..))
5760import Obelisk.CliApp.Logging (putLog, putLogRaw)
@@ -101,6 +104,19 @@ readProcessAndLogStderr sev process = do
101104 streamToLog =<< liftIO (streamHandle sev err)
102105 liftIO $ T.decodeUtf8With lenientDecode <$> BS.hGetContents out
103106
107+readProcessJSONAndLogStderr
108+ :: (Aeson.FromJSON a, MonadIO m, CliLog m, CliThrow e m, AsProcessFailure e, MonadFail m)
109+ => Severity -> ProcessSpec -> m a
110+readProcessJSONAndLogStderr sev process = do
111+ (out, _err) <- withProcess process $ \_out err -> do
112+ streamToLog =<< liftIO (streamHandle sev err)
113+ json <- liftIO $ BS.hGetContents out
114+ case Aeson.eitherDecodeStrict json of
115+ Right a -> pure a
116+ Left err -> do
117+ putLog Error $ "Could not decode process output as JSON: " <> T.pack err
118+ throwError $ review asProcessFailure $ ProcessFailure (Process.cmdspec $ _processSpec_createProcess process) 0
119+
104120readCreateProcessWithExitCode
105121 :: (MonadIO m, CliLog m, CliThrow e m, AsProcessFailure e)
106122 => ProcessSpec -> m (ExitCode, String, String)
@@ -128,7 +144,7 @@ readProcessAndLogOutput (sev_out, sev_err) process = do
128144 outText <- liftIO $ T.decodeUtf8With lenientDecode <$> BS.hGetContents out
129145 putLogRaw sev_out outText
130146
131− liftIO (waitForProcess p) >>= \case
147+ waitForProcess p >>= \case
132148 ExitSuccess -> pure outText
133149 ExitFailure code -> throwError $ review asProcessFailure $ ProcessFailure (Process.cmdspec $ _processSpec_createProcess process) code
134150
@@ -203,7 +219,7 @@ withProcess process f = do -- TODO: Use bracket.
203219
204220 f out err -- Pass the handles to the passed function
205221
206− liftIO (waitForProcess p) >>= \case
222+ waitForProcess p >>= \case
207223 ExitSuccess -> return (out, err)
208224 ExitFailure code -> throwError $ review asProcessFailure $ ProcessFailure (Process.cmdspec $ _processSpec_createProcess process) code
209225
@@ -220,6 +236,10 @@ streamToLog stream = fix $ \loop -> do
220236 Nothing -> return ()
221237 Just (sev, line) -> putLogRaw sev (T.decodeUtf8With lenientDecode line) >> loop
222238
239+-- | Wrapper around `System.Process.waitForProcess`
240+waitForProcess :: MonadIO m => ProcessHandle -> m ExitCode
241+waitForProcess = liftIO . Process.waitForProcess
242+
223243-- | Pretty print a 'CmdSpec'
224244reconstructCommand :: Process.CmdSpec -> Text
225245reconstructCommand p = case p of
lib/command/obelisk-command.cabal+10−10
@@ -1,5 +1,5 @@
11name: obelisk-command
2−version: 0.4
2+version: 0.6
33cabal-version: >= 1.8
44build-type: Simple
55
@@ -7,35 +7,35 @@ library
77 hs-source-dirs: src
88 build-depends:
99 aeson
10− , ansi-terminal
11− , Cabal
1210 , aeson-pretty
11+ , ansi-terminal
1312 , base
1413 , base16-bytestring
1514 , binary
1615 , bytestring
17− , obelisk-cliapp
16+ , Cabal
1817 , containers
1918 , data-default
2019 , directory
2120 , either
2221 , errors
22+ , exceptions
2323 , filepath
24+ , git
2425 , github
2526 , here
26− , git
2727 , hnix
2828 , hpack
2929 , io-streams
30− , exceptions
31− , logging-effect
3230 , lens
31+ , logging-effect
32+ , megaparsec
3333 , modern-uri
3434 , monad-loops
3535 , mtl
36− , megaparsec
3736 , network
3837 , network-uri
38+ , obelisk-cliapp
3939 , optparse-applicative
4040 , placeholders
4141 , process
@@ -58,8 +58,8 @@ library
5858 Obelisk.Command.Thunk
5959 Obelisk.Command.Utils
6060 Obelisk.Command.VmBuilder
61− -- -fobject-code is so that the StaticPointers extension can work in ghci
62− ghc-options: -Wall -fobject-code
61+ Obelisk.Command.Preprocessor
62+ ghc-options: -Wall
6363
6464executable ob
6565 main-is: src-bin/ob.hs
lib/command/src/Obelisk/Command.hs+58−62
@@ -3,8 +3,6 @@
33{-# LANGUAGE MultiParamTypeClasses #-}
44{-# LANGUAGE OverloadedStrings #-}
55{-# LANGUAGE ScopedTypeVariables #-}
6−{-# LANGUAGE StaticPointers #-}
7−{-# LANGUAGE TypeApplications #-}
86module Obelisk.Command where
97
108import Control.Monad
@@ -15,7 +13,6 @@ import qualified Data.ByteString.Base16 as Base16
1513import qualified Data.ByteString.Lazy as LBS
1614import Data.Foldable (for_)
1715import Data.List
18−import Data.Maybe (catMaybes)
1916import qualified Data.Text as T
2017import Data.Text.Encoding
2118import Data.Text.Encoding.Error (lenientDecode)
@@ -31,13 +28,12 @@ import System.Posix.Process (executeFile)
3128
3229import Obelisk.App
3330import Obelisk.CliApp
34−import Obelisk.CliApp.Demo (cliDemo)
3531import Obelisk.Command.Deploy
3632import Obelisk.Command.Project
3733import Obelisk.Command.Run
3834import Obelisk.Command.Thunk
39−import Obelisk.Command.Utils
4035import qualified Obelisk.Command.VmBuilder as VmBuilder
36+import qualified Obelisk.Command.Preprocessor as Preprocessor
4137
4238
4339data Args = Args
@@ -91,58 +87,40 @@ data ObCommand
9187 = ObCommand_Init InitSource Bool
9288 | ObCommand_Deploy DeployCommand
9389 | ObCommand_Run
90+ | ObCommand_Profile String [String]
9491 | ObCommand_Thunk ThunkCommand
9592 | ObCommand_Repl
9693 | ObCommand_Watch
9794 | ObCommand_Shell ShellOpts
9895 | ObCommand_Doc String [String] -- shell and list of packages
96+ | ObCommand_Hoogle String Int -- shell and port
9997 | ObCommand_Internal ObInternal
10098 deriving Show
10199
102100data ObInternal
103− = ObInternal_RunStaticIO StaticKey
104− | ObInternal_CLIDemo
101+ -- the preprocessor argument syntax is also handled outside
102+ -- optparse-applicative, but it shouldn't ever conflict with another syntax
103+ = ObInternal_ApplyPackages String String String [String]
105104 deriving Show
106105
107−inNixShell' :: MonadObelisk m => StaticPtr (ObeliskT IO ()) -> m ()
108−inNixShell' p = withProjectRoot "." $ \root -> do
109− cmd <- liftIO $ unwords <$> mkCmd -- TODO: shell escape instead of unwords
110− projectShell root True "ghc" (Just cmd)
111− where
112− mkCmd = do
113− argsCfg <- getArgsConfig
114− myArgs <- getArgs
115− obArgs <- parseCLIArgs argsCfg myArgs
116− progName <- getObeliskExe
117− return $ progName : catMaybes
118− [ Just "--no-handoff"
119− , bool Nothing (Just "--verbose") $ _args_verbose obArgs
120− , Just "internal"
121− , Just "run-static-io"
122− , Just $ encodeStaticKey $ staticKey p
123− ]
124−
125106obCommand :: ArgsConfig -> Parser ObCommand
126107obCommand cfg = hsubparser
127− (mconcat
128− [ command "init" $ info (ObCommand_Init <$> initSource <*> initForce) $ progDesc "Initialize an Obelisk project"
129− , command "deploy" $ info (ObCommand_Deploy <$> deployCommand cfg) $ progDesc "Prepare a deployment for an Obelisk project"
130− , command "run" $ info (pure ObCommand_Run) $ progDesc "Run current project in development mode"
131− , command "thunk" $ info (ObCommand_Thunk <$> thunkCommand) $ progDesc "Manipulate thunk directories"
132− , command "repl" $ info (pure ObCommand_Repl) $ progDesc "Open an interactive interpreter"
133− , command "watch" $ info (pure ObCommand_Watch) $ progDesc "Watch current project for errors and warnings"
134− , command "shell" $ info (ObCommand_Shell <$> shellOpts) $ progDesc "Enter a shell with project dependencies"
135− , command "doc" $ info (ObCommand_Doc <$> shellFlags <*> packageNames) $
136− progDesc "List paths to haddock documentation for specified packages"
137− <> footerDoc (Just $
138− text "Hint: To open the documentation you can pipe the output of this command like"
139− <$$> text "ob doc reflex reflex-dom-core | xargs -n1 xdg-open")
140− ])
141− <|> subparser
142− (mconcat
143− [ internal
144− , command "internal" (info (ObCommand_Internal <$> internalCommand) mempty)
145− ])
108+ (mconcat
109+ [ command "init" $ info (ObCommand_Init <$> initSource <*> initForce) $ progDesc "Initialize an Obelisk project"
110+ , command "deploy" $ info (ObCommand_Deploy <$> deployCommand cfg) $ progDesc "Prepare a deployment for an Obelisk project"
111+ , command "run" $ info (pure ObCommand_Run) $ progDesc "Run current project in development mode"
112+ , command "profile" $ info (uncurry ObCommand_Profile <$> profileCommand) $ progDesc "Run current project with profiling enabled"
113+ , command "thunk" $ info (ObCommand_Thunk <$> thunkCommand) $ progDesc "Manipulate thunk directories"
114+ , command "repl" $ info (pure ObCommand_Repl) $ progDesc "Open an interactive interpreter"
115+ , command "watch" $ info (pure ObCommand_Watch) $ progDesc "Watch current project for errors and warnings"
116+ , command "shell" $ info (ObCommand_Shell <$> shellOpts) $ progDesc "Enter a shell with project dependencies"
117+ , command "doc" $ info (ObCommand_Doc <$> shellFlags <*> packageNames) $
118+ progDesc "List paths to haddock documentation for specified packages"
119+ <> footerDoc (Just $
120+ text "Hint: To open the documentation you can pipe the output of this command like"
121+ <$$> text "ob doc reflex reflex-dom-core | xargs -n1 xdg-open")
122+ , command "hoogle" $ info (ObCommand_Hoogle <$> shellFlags <*> portOpt 8080) $ progDesc "Run a hoogle server locally for your project's dependency tree"
123+ ])
146124
147125packageNames :: Parser [String]
148126packageNames = some (strArgument (metavar "PACKAGE-NAME..."))
@@ -157,7 +135,7 @@ deployCommand cfg = hsubparser $ mconcat
157135 where
158136 platformP = hsubparser $ mconcat
159137 [ command "android" $ info (pure (Android, [])) mempty
160− , command "ios" $ info ((,) <$> pure IOS <*> (fmap pure $ strArgument (metavar "TEAMID" <> help "Your Team ID - found in the Apple developer portal"))) mempty
138+ , command "ios" $ info ((,) <$> pure IOS <*> fmap pure (strArgument (metavar "TEAMID" <> help "Your Team ID - found in the Apple developer portal"))) mempty
161139 ]
162140
163141 remoteBuilderParser :: Parser (Maybe RemoteBuilder)
@@ -179,8 +157,8 @@ deployCommand cfg = hsubparser $ mconcat
179157
180158deployInitOpts :: Parser DeployInitOpts
181159deployInitOpts = DeployInitOpts
182− <$> strArgument (action "directory" <> metavar "DEPLOYDIR" <> help "Path to a directory that it will create")
183− <*> strOption (long "ssh-key" <> action "file" <> metavar "SSHKEY" <> help "Path to an ssh key that it will symlink to")
160+ <$> strArgument (action "directory" <> metavar "DEPLOYDIR" <> help "Path to a directory where the deployment repository will be initialized")
161+ <*> strOption (long "ssh-key" <> action "file" <> metavar "SSHKEY" <> help "Path to an SSH key that will be *copied* to the deployment repository")
184162 <*> some (strOption (long "hostname" <> metavar "HOSTNAME" <> help "hostname of the deployment target"))
185163 <*> strOption (long "route" <> metavar "PUBLICROUTE" <> help "Publicly accessible URL of your app")
186164 <*> strOption (long "admin-email" <> metavar "ADMINEMAIL" <> help "Email address where administrative alerts will be sent")
@@ -207,11 +185,24 @@ data DeployInitOpts = DeployInitOpts
207185 }
208186 deriving Show
209187
210−internalCommand :: Parser ObInternal
211−internalCommand = subparser $ mconcat
212− [ command "run-static-io" $ info (ObInternal_RunStaticIO <$> argument (eitherReader decodeStaticKey) (action "static-key")) mempty
213− , command "clidemo" $ info (pure ObInternal_CLIDemo) mempty
214− ]
188+profileCommand :: Parser (String, [String])
189+profileCommand = (,)
190+ <$> strOption
191+ ( long "output"
192+ <> short 'o'
193+ <> help "Base output to use for profiling output. Suffixes are added to this based on the profiling type. Defaults to a timestamped path in the profile/ directory in the project's root."
194+ <> metavar "PATH"
195+ <> value "profile/%Y-%m-%dT%H:%M:%S"
196+ <> showDefault
197+ )
198+ <*> (words <$> strOption
199+ ( long "rts-flags"
200+ <> help "RTS Flags to pass to the executable."
201+ <> value "-p -hc"
202+ <> metavar "FLAGS"
203+ <> showDefault
204+ ))
205+
215206
216207--TODO: Result should provide normalised path and also original user input for error reporting.
217208thunkDirectoryParser :: Parser FilePath
@@ -270,13 +261,16 @@ shellOpts = ShellOpts
270261 <$> shellFlags
271262 <*> optional (strArgument (metavar "COMMAND"))
272263
264+portOpt :: Int -> Parser Int
265+portOpt dfault = option auto (long "port" <> short 'p' <> help "Port number for server" <> showDefault <> value dfault <> metavar "INT")
266+
273267parserPrefs :: ParserPrefs
274268parserPrefs = defaultPrefs
275269 { prefShowHelpOnEmpty = True
276270 }
277271
278272parseCLIArgs :: ArgsConfig -> [String] -> IO Args
279−parseCLIArgs cfg as = pure as >>= handleParseResult . execParserPure parserPrefs (argsInfo cfg)
273+parseCLIArgs cfg = handleParseResult . execParserPure parserPrefs (argsInfo cfg)
280274
281275-- | Create an Obelisk config for the current process.
282276mkObeliskConfig :: IO Obelisk
@@ -349,6 +343,9 @@ main' argsCfg = do
349343 liftIO $ executeFile impl False ("--no-handoff" : myArgs) Nothing
350344 case myArgs of
351345 "--no-handoff" : as -> go as -- If we've been told not to hand off, don't hand off
346+ origPath:inPath:outPath:preprocessorName:packagePaths
347+ | preprocessorName == preprocessorIdentifier && any (\c -> c == '.' || c == pathSeparator) origPath ->
348+ ob $ ObCommand_Internal $ ObInternal_ApplyPackages origPath inPath outPath packagePaths
352349 a:as -- Otherwise bash completion would always hand-off even if the user isn't trying to
353350 | "--bash-completion" `isPrefixOf` a
354351 && "--no-handoff" `elem` as -> go (a:as)
@@ -375,7 +372,7 @@ ob = \case
375372 Right (ThunkData_Packed ptr) -> return ptr
376373 Right (ThunkData_Checkout (Just ptr)) -> return ptr
377374 Right (ThunkData_Checkout Nothing) ->
378− getThunkPtr False root
375+ getThunkPtr False root Nothing
379376 let sshKeyPath = _deployInitOpts_sshKey deployOpts
380377 hostname = _deployInitOpts_hostname deployOpts
381378 route = _deployInitOpts_route deployOpts
@@ -389,24 +386,23 @@ ob = \case
389386 Just RemoteBuilder_ObeliskVM -> (:[]) <$> VmBuilder.getNixBuildersArg
390387 DeployCommand_Update -> deployUpdate "."
391388 DeployCommand_Test (platform, extraArgs) -> deployMobile platform extraArgs
392− ObCommand_Run -> inNixShell' $ static run
389+ ObCommand_Run -> run
390+ ObCommand_Profile basePath rtsFlags -> profile basePath rtsFlags
393391 ObCommand_Thunk tc -> case tc of
394392 ThunkCommand_Update thunks config -> for_ thunks (updateThunkToLatest config)
395393 ThunkCommand_Unpack thunks -> for_ thunks unpackThunk
396394 ThunkCommand_Pack thunks config -> for_ thunks (packThunk config)
397395 ObCommand_Repl -> runRepl
398− ObCommand_Watch -> inNixShell' $ static runWatch
396+ ObCommand_Watch -> runWatch
399397 ObCommand_Shell so -> withProjectRoot "." $ \root ->
400398 projectShell root False (_shellOpts_shell so) (_shellOpts_command so)
401399 ObCommand_Doc shell' pkgs -> withProjectRoot "." $ \root ->
402400 projectShell root False shell' (Just $ haddockCommand pkgs)
401+ ObCommand_Hoogle shell' port -> withProjectRoot "." $ \root -> do
402+ nixShellWithHoogle root True shell' $ Just $ "hoogle server -p " <> show port <> " --local"
403403 ObCommand_Internal icmd -> case icmd of
404− ObInternal_RunStaticIO k -> liftIO (unsafeLookupStaticPtr @(ObeliskT IO ()) k) >>= \case
405− Nothing -> failWith $ "ObInternal_RunStaticIO: no such StaticKey: " <> T.pack (show k)
406− Just p -> do
407− c <- getObelisk
408− liftIO $ runObelisk c $ deRefStaticPtr p
409− ObInternal_CLIDemo -> cliDemo
404+ ObInternal_ApplyPackages origPath inPath outPath packagePaths -> do
405+ liftIO $ Preprocessor.applyPackages origPath inPath outPath packagePaths
410406
411407haddockCommand :: [String] -> String
412408haddockCommand pkgs = unwords
lib/command/src/Obelisk/Command/Deploy.hs+61−60
@@ -8,22 +8,17 @@ module Obelisk.Command.Deploy where
88
99import Control.Lens
1010import Control.Monad
11−import Control.Monad.Catch (Exception (displayException), MonadThrow, throwM, try, bracket_)
11+import Control.Monad.Catch (Exception (displayException), MonadThrow, bracket, throwM, try)
1212import Control.Monad.IO.Class (liftIO)
1313import Data.Aeson (FromJSON, ToJSON, encode, eitherDecode)
1414import Data.Bits
1515import qualified Data.ByteString.Lazy as BSL
1616import Data.Default
17−import qualified Data.HashMap.Strict as HM
1817import qualified Data.Map as Map
19−import Data.Text (Text)
2018import qualified Data.Set as Set
2119import qualified Data.Text as T
2220import qualified Data.Text.IO as T
2321import GHC.Generics
24−import Nix.Pretty
25−import Nix.String (principledMakeNixStringWithoutContext)
26−import Nix.Value
2722import System.Directory
2823import System.FilePath
2924import System.IO
@@ -53,6 +48,7 @@ deployInit
5348 -> Bool -- ^ enable https
5449 -> m ()
5550deployInit thunkPtr deployDir sshKeyPath hostnames route adminEmail enableHttps = do
51+ liftIO $ createDirectoryIfMissing True deployDir
5652 localKey <- withSpinner ("Preparing " <> T.pack deployDir) $ do
5753 localKey <- liftIO (doesFileExist sshKeyPath) >>= \case
5854 False -> failWith $ T.pack $ "ob deploy init: file does not exist: " <> sshKeyPath
@@ -79,22 +75,30 @@ deployInit thunkPtr deployDir sshKeyPath hostnames route adminEmail enableHttps
7975 , deployDir </> "config" </> "common"
8076 , deployDir </> "config" </> "frontend"
8177 ]
78+
79+ let srcDir = deployDir </> "src"
80+ withSpinner ("Creating source thunk (" <> T.pack (makeRelative deployDir srcDir) <> ")") $ liftIO $ do
81+ createThunk srcDir thunkPtr
82+ setupObeliskImpl deployDir
83+
8284 withSpinner "Writing deployment configuration" $ do
8385 writeDeployConfig deployDir "backend_hosts" $ unlines hostnames
8486 writeDeployConfig deployDir "enable_https" $ show enableHttps
8587 writeDeployConfig deployDir "admin_email" adminEmail
8688 writeDeployConfig deployDir ("config" </> "common" </> "route") route
87− withSpinner "Creating source thunk (./src)" $ liftIO $ do
88− createThunk (deployDir </> "src") thunkPtr
89− setupObeliskImpl deployDir
89+ writeDeployConfig deployDir "module.nix" $
90+ "(import " <> toNixPath (makeRelative deployDir srcDir) <> " {}).obelisk.serverModules.mkBaseEc2"
91+
9092 withSpinner ("Initializing git repository (" <> T.pack deployDir <> ")") $
9193 initGit deployDir
9294
9395setupObeliskImpl :: FilePath -> IO ()
9496setupObeliskImpl deployDir = do
95− let implDir = toImplDir deployDir
97+ let
98+ implDir = toImplDir deployDir
99+ goBackUp = foldr (</>) "" $ (".." <$) $ splitPath $ makeRelative deployDir implDir
96100 createDirectoryIfMissing True implDir
97− writeFile (implDir </> "default.nix") "(import ../../src {}).obelisk"
101+ writeFile (implDir </> "default.nix") $ "(import " <> toNixPath (goBackUp </> "src") <> " {}).obelisk"
98102
99103deployPush :: MonadObelisk m => FilePath -> m [String] -> m ()
100104deployPush deployPath getNixBuilders = do
@@ -113,6 +117,8 @@ deployPush deployPath getNixBuilders = do
113117 Left err -> failWith $ "ob deploy push: couldn't read src thunk: " <> T.pack (show err)
114118 let version = show . _thunkRev_commit $ _thunkPtr_rev thunkPtr
115119 builders <- getNixBuilders
120+ let moduleFile = deployPath </> "module.nix"
121+ moduleFileExists <- liftIO $ doesFileExist moduleFile
116122 buildOutputByHost <- ifor (Map.fromSet (const ()) hosts) $ \host () -> do
117123 --TODO: What does it mean if this returns more or less than 1 line of output?
118124 [result] <- fmap lines $ nixCmd $ NixCmd_Build $ def
@@ -122,13 +128,13 @@ deployPush deployPath getNixBuilders = do
122128 , _target_expr = Nothing
123129 }
124130 & nixBuildConfig_outLink .~ OutLink_None
125− & nixCmdConfig_args .~
131+ & nixCmdConfig_args .~ (
126132 [ strArg "hostName" host
127133 , strArg "adminEmail" adminEmail
128134 , strArg "routeHost" routeHost
129135 , strArg "version" version
130136 , boolArg "enableHttps" enableHttps
131− ]
137+ ] <> [rawArg "module" ("import " <> toNixPath moduleFile) | moduleFileExists ])
132138 & nixCmdConfig_builders .~ builders
133139 pure result
134140 let knownHostsPath = deployPath </> "backend_known_hosts"
@@ -172,20 +178,6 @@ deployPush deployPath getNixBuilders = do
172178deployUpdate :: MonadObelisk m => FilePath -> m ()
173179deployUpdate deployPath = updateThunkToLatest (ThunkUpdateConfig Nothing (ThunkConfig Nothing)) (deployPath </> "src")
174180
175−keytoolToAndroidConfig :: KeytoolConfig -> HM.HashMap Text (NValue t Identity m)
176−keytoolToAndroidConfig conf =
177− let path = nvPath $ _keytoolConfig_keystore conf
178− fromStr = nvStr . principledMakeNixStringWithoutContext . T.pack
179− storepass = fromStr $ _keytoolConfig_storepass conf
180− alias = fromStr $ _keytoolConfig_alias conf
181− keypass = fromStr $ _keytoolConfig_keypass conf
182− in HM.fromList
183− [ ("storeFile", path)
184− , ("storePassword", storepass)
185− , ("keyAlias", alias)
186− , ("keyPassword", keypass)
187− ]
188−
189181data PlatformDeployment = Android | IOS
190182 deriving (Show, Eq)
191183
@@ -200,7 +192,7 @@ deployMobile platform mobileArgs = withProjectRoot "." $ \root -> do
200192 configDir = root </> "config"
201193 exists <- liftIO $ doesDirectoryExist srcDir
202194 unless exists $ failWith "ob test should be run inside of a deploy directory"
203− nixBuildTarget <- case platform of
195+ (nixBuildTarget, extraArgs) <- case platform of
204196 Android -> do
205197 let keystorePath = root </> "android_keystore.jks"
206198 keytoolConfPath = root </> "android_keytool_config.json"
@@ -224,43 +216,55 @@ deployMobile platform mobileArgs = withProjectRoot "." $ \root -> do
224216 checkKeytoolConfExist <- liftIO $ doesFileExist keytoolConfPath
225217 unless checkKeytoolConfExist $ failWith "Missing android KeytoolConfig"
226218 keytoolConfContents <- liftIO $ BSL.readFile keytoolConfPath
227− liftIO $ print keytoolConfContents
228− releaseKey :: String <- case eitherDecode keytoolConfContents :: Either String KeytoolConfig of
219+ keyArgs <- case eitherDecode keytoolConfContents :: Either String KeytoolConfig of
229220 Left err -> failWith $ T.pack err
230− Right conf -> return $
231− (printNix :: MonadObelisk m => NValue t Identity m -> String) $
232− nvSet (keytoolToAndroidConfig conf) HM.empty
221+ Right conf -> pure
222+ [ "--sign"
223+ , "--store-file", _keytoolConfig_keystore conf
224+ , "--store-password", _keytoolConfig_storepass conf
225+ , "--key-alias", _keytoolConfig_alias conf
226+ , "--key-password", _keytoolConfig_keypass conf
227+ ]
233228 let expr = mconcat
234− [ "with (import ", srcDir, " {});"
235− , "android.frontend.override (drv: { "
236− , "releaseKey = (if builtins.isNull drv.releaseKey then {} else drv.releaseKey) // " <> releaseKey <> "; "
229+ [ "with (import ", toNixPath srcDir, " {});"
230+ , "android.frontend.override (drv: {"
231+ , "isRelease = true;"
237232 , "staticSrc = (passthru.__androidWithConfig ", configDir, ").frontend.staticSrc;"
233+ , "assets = (passthru.__androidWithConfig ", configDir, ").frontend.assets;"
238234 , "})"
239235 ]
240− return $ Target
236+ return (Target
241237 { _target_path = Nothing
242238 , _target_attr = Nothing
243239 , _target_expr = Just expr
244− }
240+ }, keyArgs)
245241 IOS -> do
246242 let expr = mconcat
247− [ "with (import ", srcDir, " {});"
248− , "ios.frontend.override (_: { staticSrc = (passthru.__iosWithConfig ", configDir, ").frontend.staticSrc; })"
243+ [ "with (import ", toNixPath srcDir, " {});"
244+ , "ios.frontend.override (_: { staticSrc = (passthru.__iosWithConfig ", toNixPath configDir, ").frontend.staticSrc; })"
249245 ]
250− return $ Target
246+ return (Target
251247 { _target_path = Nothing
252248 , _target_attr = Nothing
253249 , _target_expr = Just expr
254− }
250+ }, [])
255251 result <- nixCmd $ NixCmd_Build $ def
256252 & nixBuildConfig_outLink .~ OutLink_None
257253 & nixCmdConfig_target .~ nixBuildTarget
258− putLog Notice $ T.pack $ "Your recently built android apk can be found at the following path: " <> show result
259− callProcessAndLogOutput (Notice, Error) $ proc (result </> "bin" </> "deploy") mobileArgs
254+ let mobileArtifact = case platform of
255+ IOS -> "iOS App"
256+ Android -> "Android APK"
257+ putLog Notice $ T.pack $ unwords ["Your recently built", mobileArtifact, "can be found at the following path:", show result]
258+ callProcessAndLogOutput (Notice, Error) $ proc (result </> "bin" </> "deploy") (mobileArgs ++ extraArgs)
260259 where
261− withEcho showEcho f = do
262− prevEcho <- hGetEcho stdin
263− bracket_ (hSetEcho stdin showEcho) (hSetEcho stdin prevEcho) f
260+ withEcho showEcho f = bracket
261+ (do
262+ prevEcho <- hGetEcho stdin
263+ hSetEcho stdin showEcho
264+ pure prevEcho
265+ )
266+ (hSetEcho stdin)
267+ (const f)
264268
265269data KeytoolConfig = KeytoolConfig
266270 { _keytoolConfig_keystore :: FilePath
@@ -273,19 +277,16 @@ instance FromJSON KeytoolConfig
273277instance ToJSON KeytoolConfig
274278
275279createKeystore :: MonadObelisk m => FilePath -> KeytoolConfig -> m ()
276−createKeystore root config = do
277− let expr = "with (import " <> toImplDir root <> ").reflex-platform.nixpkgs; pkgs.mkShell { buildInputs = [ pkgs.jdk ]; }"
278− callProcessAndLogOutput (Notice,Notice) $ setCwd (Just root) $ proc "nix-shell" ["default.nix", "-E" , expr, "--run" , keytoolCmd]
279− where
280− keytoolCmd = processToShellString "keytool"
281− [ "-genkeypair", "-noprompt"
282− , "-keystore", _keytoolConfig_keystore config
283− , "-keyalg", "RSA", "-keysize", "2048"
284− , "-validity", "1000000"
285− , "-storepass", _keytoolConfig_storepass config
286− , "-alias", _keytoolConfig_alias config
287− , "-keypass", _keytoolConfig_keypass config
288− ]
280+createKeystore root config =
281+ callProcessAndLogOutput (Notice, Notice) $ setCwd (Just root) $ proc jreKeyToolPath
282+ [ "-genkeypair", "-noprompt"
283+ , "-keystore", _keytoolConfig_keystore config
284+ , "-keyalg", "RSA", "-keysize", "2048"
285+ , "-validity", "1000000"
286+ , "-storepass", _keytoolConfig_storepass config
287+ , "-alias", _keytoolConfig_alias config
288+ , "-keypass", _keytoolConfig_keypass config
289+ ]
289290
290291-- | Simplified deployment configuration mechanism. At one point we may revisit this.
291292writeDeployConfig :: MonadObelisk m => FilePath -> FilePath -> String -> m ()
lib/command/src/Obelisk/Command/Nix.hs+65−24
@@ -9,22 +9,33 @@
99{-# LANGUAGE TemplateHaskell #-}
1010{-# LANGUAGE TypeApplications #-}
1111module Obelisk.Command.Nix
12− ( nixCmd
13− , NixCmd (..)
14− , nixCmdConfig_target
15− , nixCmdConfig_args
16− , nixCmdConfig_builders
12+ ( Arg (..)
1713 , NixBuildConfig (..)
1814 , nixBuildConfig_common
1915 , nixBuildConfig_outLink
16+ , NixCmd (..)
17+ , nixCmdConfig_args
18+ , nixCmdConfig_builders
19+ , nixCmdConfig_target
20+ , NixCommonConfig (..)
2021 , NixInstantiateConfig (..)
2122 , nixInstantiateConfig_eval
22− , NixCommonConfig (..)
23− , Target (..)
23+ , NixShellConfig (..)
24+ , nixShellConfig_common
25+ , nixShellConfig_pure
26+ , nixShellConfig_run
2427 , OutLink (..)
25− , Arg (..)
28+ , Target (..)
29+ , target_attr
30+ , target_expr
31+ , target_path
32+
2633 , boolArg
34+ , nixCmd
35+ , nixCmdProc
36+ , nixCmdProc'
2737 , rawArg
38+ , runNixShellConfig
2839 , strArg
2940 ) where
3041
@@ -62,10 +73,10 @@ data Arg
6273 deriving (Eq, Show)
6374
6475strArg :: String -> String -> Arg
65−strArg k = Arg_Str k
76+strArg = Arg_Str
6677
6778rawArg :: String -> String -> Arg
68−rawArg k = Arg_Expr k
79+rawArg = Arg_Expr
6980
7081boolArg :: String -> Bool -> Arg
7182boolArg k = Arg_Expr k . bool "false" "true"
@@ -85,7 +96,7 @@ makeClassy ''NixCommonConfig
8596instance Default NixCommonConfig where
8697 def = NixCommonConfig def mempty mempty
8798
88−runNixCommonConfig :: NixCommonConfig -> [FilePath]
99+runNixCommonConfig :: NixCommonConfig -> [String]
89100runNixCommonConfig cfg = mconcat [maybeToList path, attrArg, exprArg, args, buildersArg]
90101 where
91102 path = _target_path $ _nixCmdConfig_target cfg
@@ -123,7 +134,7 @@ instance HasNixCommonConfig NixBuildConfig where
123134instance Default NixBuildConfig where
124135 def = NixBuildConfig def def
125136
126−runNixBuildConfig :: NixBuildConfig -> [FilePath]
137+runNixBuildConfig :: NixBuildConfig -> [String]
127138runNixBuildConfig cfg = mconcat
128139 [ runNixCommonConfig $ cfg ^. nixCommonConfig
129140 , case _nixBuildConfig_outLink cfg of
@@ -144,12 +155,26 @@ instance HasNixCommonConfig NixInstantiateConfig where
144155instance Default NixInstantiateConfig where
145156 def = NixInstantiateConfig def False
146157
147−runNixInstantiateConfig :: NixInstantiateConfig -> [FilePath]
158+runNixInstantiateConfig :: NixInstantiateConfig -> [String]
148159runNixInstantiateConfig cfg = mconcat
149160 [ runNixCommonConfig $ cfg ^. nixCommonConfig
150161 , "--eval" <$ guard (_nixInstantiateConfig_eval cfg)
151162 ]
152163
164+data NixShellConfig = NixShellConfig
165+ { _nixShellConfig_common :: NixCommonConfig
166+ , _nixShellConfig_pure :: Bool
167+ , _nixShellConfig_run :: Maybe String
168+ }
169+
170+makeLenses ''NixShellConfig
171+
172+instance HasNixCommonConfig NixShellConfig where
173+ nixCommonConfig = nixShellConfig_common
174+
175+instance Default NixShellConfig where
176+ def = NixShellConfig def False Nothing
177+
153178data NixCmd
154179 = NixCmd_Build NixBuildConfig
155180 | NixCmd_Instantiate NixInstantiateConfig
@@ -157,27 +182,43 @@ data NixCmd
157182instance Default NixCmd where
158183 def = NixCmd_Build def
159184
160−nixCmd :: MonadObelisk m => NixCmd -> m FilePath
161−nixCmd cmdCfg = withSpinner' ("Running " <> cmd <> desc) (Just $ const $ "Built " <> desc) $ do
162− output <- readProcessAndLogStderr Debug $ proc (T.unpack cmd) options
163− -- Remove final newline that Nix appends
164− Just (outPath, '\n') <- pure $ T.unsnoc output
165− pure $ T.unpack outPath
185+runNixShellConfig :: NixShellConfig -> [String]
186+runNixShellConfig cfg = mconcat
187+ [ runNixCommonConfig $ cfg ^. nixCommonConfig
188+ , [ "--pure" | cfg ^. nixShellConfig_pure ]
189+ ] ++ mconcat [
190+ ["--run", run] | run <- maybeToList $ cfg ^. nixShellConfig_run
191+ ]
192+
193+nixCmdProc :: NixCmd -> ProcessSpec
194+nixCmdProc = fst . nixCmdProc'
195+
196+nixCmdProc' :: NixCmd -> (ProcessSpec, T.Text)
197+nixCmdProc' cmdCfg = (proc (T.unpack cmd) options, cmd)
166198 where
167− (cmd, options, commonCfg) = case cmdCfg of
199+ (cmd, options) = case cmdCfg of
168200 NixCmd_Build cfg' ->
169201 ( "nix-build"
170202 , runNixBuildConfig cfg'
171− , cfg' ^. nixCommonConfig
172203 )
173204 NixCmd_Instantiate cfg' ->
174205 ( "nix-instantiate"
175206 , runNixInstantiateConfig cfg'
176− , cfg' ^. nixCommonConfig
177207 )
208+
209+nixCmd :: MonadObelisk m => NixCmd -> m FilePath
210+nixCmd cmdCfg = withSpinner' ("Running " <> cmd <> desc) (Just $ const $ "Built " <> desc) $ do
211+ output <- readProcessAndLogStderr Debug cmdProc
212+ -- Remove final newline that Nix appends
213+ Just (outPath, '\n') <- pure $ T.unsnoc output
214+ pure $ T.unpack outPath
215+ where
216+ (cmdProc, cmd) = nixCmdProc' cmdCfg
217+ commonCfg = case cmdCfg of
218+ NixCmd_Build cfg' -> cfg' ^. nixCommonConfig
219+ NixCmd_Instantiate cfg' -> cfg' ^. nixCommonConfig
178220 path = commonCfg ^. nixCmdConfig_target . target_path
179221 desc = T.pack $ mconcat $ catMaybes
180− [ (" on " <>) <$> path
222+ [ ("on " <>) <$> path
181223 , (\a -> " [" <> a <> "]") <$> (commonCfg ^. nixCmdConfig_target . target_attr)
182224 ]
183−
lib/command/src/Obelisk/Command/Preprocessor.hsnew file+96
@@ -0,0 +1,96 @@
1+{-# LANGUAGE LambdaCase #-}
2+{-# LANGUAGE OverloadedStrings #-}
3+
4+module Obelisk.Command.Preprocessor where
5+
6+import qualified Data.ByteString.Lazy as BL
7+import qualified Data.ByteString.Lazy.Builder as BU
8+import Data.List (intersperse, isPrefixOf, sortOn)
9+import Data.Maybe (fromMaybe)
10+import qualified Data.Text.Lazy.Builder as TL
11+import qualified Data.Text.Lazy.Encoding as TL
12+import Distribution.Compiler (CompilerFlavor (..))
13+import Language.Haskell.Extension (Extension (..), Language(..))
14+import System.IO (IOMode (..), hClose, hPutStrLn, openFile, stderr)
15+import System.FilePath (hasTrailingPathSeparator, joinPath, normalise, splitPath)
16+
17+import Obelisk.Command.Run (CabalPackageInfo (..), parseCabalPackage')
18+
19+applyPackages :: FilePath -> FilePath -> FilePath -> [FilePath] -> IO ()
20+applyPackages origPath inPath outPath packagePaths = do
21+ -- This code is intended to be executed via ghci's -pgmF preprocessor option
22+ -- The command line arguments are passed in via ghc, which dictates the first three options and meanings
23+ -- In order for this code to execute, origPath must contain either a '.' character or a '/' character.
24+ -- (This is to avoid the possibility of the command line syntax conflicting with another ob command)
25+ -- We do have control over the remaining arguments, but they must be the same for all files.
26+ -- Thus, the fourth command line argument must be "apply-packages", which has already been handled.
27+ -- We assume all the remaining arguments passed in are paths to cabal or hpack package specifications.
28+
29+ outFile <- openFile outPath WriteMode
30+ let hPutTextBuilder h = BU.hPutBuilder h . TL.encodeUtf8Builder . TL.toLazyText
31+
32+ -- putStr "--------------------------------------------------------------------------------\n"
33+ -- print args
34+ -- putStr "--------------------------------------------------------------------------------\n"
35+
36+ -- Thus we must select among the packagePaths for the file we are going to parse.
37+
38+ let takeDirs = takeWhile hasTrailingPathSeparator
39+ packageDirs = sortOn (negate . length . takeDirs) $ map (splitPath . normalise) packagePaths
40+ origDir = splitPath $ normalise origPath
41+ matches = [ joinPath d | d <- packageDirs, takeDirs d `isPrefixOf` origDir ]
42+
43+ -- So the first element of matches is going to be the deepest path to a package spec that contains
44+ -- our file as a subdirectory.
45+
46+ case matches of
47+ [] -> hPutTextBuilder outFile (lineNumberPragma origPath) -- TODO: probably should produce a warning
48+ (packagePath:_) -> do
49+ parseCabalPackage' packagePath >>= \case
50+ Left err ->
51+ hPutStrLn stderr $ "Error: Unable to parse cabal package " <> packagePath <> "; Skipping preprocessor on " <> origPath <> ". Error was " <> show err
52+ Right (_warnings, packageInfo) ->
53+ hPutTextBuilder outFile (generateHeader origPath packageInfo)
54+
55+ BL.readFile inPath >>= BL.hPut outFile
56+ hClose outFile
57+
58+
59+ -- I'm pretty sure there's a certain amount of oversimplification in CabalPackageInfo, so I doubt this is fully robust.
60+
61+generateHeader :: FilePath -> CabalPackageInfo -> TL.Builder
62+generateHeader origPath packageInfo =
63+ hsExtensions <> ghcOptions <> lineNumberPragma origPath
64+ where
65+ hsExtensions =
66+ if not (null extList)
67+ then TL.fromText "{-# LANGUAGE "
68+ <> mconcat (intersperse (TL.fromText ", ") extList)
69+ <> TL.fromText " #-}\n"
70+ else mempty
71+ extList = addDefaultLanguage $ concatMap showExt $ _cabalPackageInfo_defaultExtensions packageInfo
72+ addDefaultLanguage =
73+ case _cabalPackageInfo_defaultLanguage packageInfo of
74+ Nothing -> id
75+ Just x -> case x of
76+ UnknownLanguage ext -> ( TL.fromString ext :)
77+ ext -> ( TL.fromString (show ext) :)
78+ showExt = \case
79+ EnableExtension ext -> [TL.fromString (show ext)]
80+ DisableExtension _ -> []
81+ UnknownExtension ext -> [TL.fromString ext]
82+
83+ ghcOptions =
84+ if not (null optList)
85+ then TL.fromText "{-# OPTIONS_GHC "
86+ <> mconcat (intersperse (TL.fromText " ") optList)
87+ <> TL.fromText " #-}\n"
88+ else mempty
89+ optList = map TL.fromString
90+ $ filter (\x -> not (isPrefixOf "-O" x))
91+ $ fromMaybe []
92+ $ lookup GHC (_cabalPackageInfo_compilerOptions packageInfo)
93+
94+lineNumberPragma :: FilePath -> TL.Builder
95+lineNumberPragma origPath =
96+ TL.fromText "{-# LINE 1 \"" <> TL.fromString origPath <> TL.fromText "\" #-}\n"
lib/command/src/Obelisk/Command/Project.hs+100−36
@@ -1,28 +1,40 @@
11{-# LANGUAGE FlexibleContexts #-}
22{-# LANGUAGE LambdaCase #-}
33{-# LANGUAGE OverloadedStrings #-}
4−{-# LANGUAGE TemplateHaskell #-}
54module Obelisk.Command.Project
65 ( InitSource (..)
7− , initProject
86 , findProjectObeliskCommand
97 , findProjectRoot
10− , withProjectRoot
11− , inProjectShell
12− , inImpureProjectShell
8+ , findProjectAssets
9+ , initProject
10+ , nixShellRunConfig
11+ , nixShellRunProc
12+ , nixShellWithHoogle
13+ , nixShellWithPkgs
14+ , obeliskDirName
1315 , projectShell
14−
15− , toObeliskDir
1616 , toImplDir
17+ , toNixPath
18+ , toObeliskDir
19+ , withProjectRoot
1720 ) where
1821
1922import Control.Concurrent.MVar (MVar, newMVar, withMVarMasked)
23+import Control.Lens ((.~), (?~), (<&>))
2024import Control.Monad
2125import Control.Monad.IO.Class (liftIO)
2226import Control.Monad.State
27+import qualified Data.Aeson as Json
2328import Data.Bits
24−import Data.Function (on)
29+import qualified Data.ByteString.Lazy as BSL
30+import Data.Default (def)
31+import Data.Function ((&), on)
32+import Data.List (isInfixOf)
33+import Data.Map (Map)
34+import Data.Text (Text)
2535import qualified Data.Text as T
36+import Data.Text.Encoding (decodeUtf8)
37+import Data.Traversable (for)
2638import System.Directory
2739import System.Environment (lookupEnv)
2840import System.FilePath
@@ -30,15 +42,15 @@ import System.IO.Temp
3042import System.IO.Unsafe (unsafePerformIO)
3143import System.Posix (FileStatus, FileMode, CMode (..), UserID, deviceID, fileID, fileMode, fileOwner, getFileStatus, getRealUserID)
3244import System.Posix.Files
33−import System.Process (waitForProcess)
34−import System.Which (staticWhich)
3545
3646import GitHub.Data.GitData (Branch)
3747import GitHub.Data.Name (Name)
3848
3949import Obelisk.App (MonadObelisk)
4050import Obelisk.CliApp
51+import Obelisk.Command.Nix
4152import Obelisk.Command.Thunk
53+import Obelisk.Command.Utils (nixExePath, nixBuildExePath)
4254
4355--TODO: Make this module resilient to random exceptions
4456
@@ -62,9 +74,12 @@ data InitSource
6274 | InitSource_Symlink FilePath
6375 deriving Show
6476
77+obeliskDirName :: FilePath
78+obeliskDirName = ".obelisk"
79+
6580-- | Path to obelisk directory in given path
6681toObeliskDir :: FilePath -> FilePath
67−toObeliskDir p = p </> ".obelisk"
82+toObeliskDir p = p </> obeliskDirName
6883
6984-- | Path to impl file in given path
7085toImplDir :: FilePath -> FilePath
@@ -171,7 +186,7 @@ findProjectRoot target = do
171186 targetStat <- liftIO $ getFileStatus target
172187 umask <- liftIO getUmask
173188 (result, _) <- liftIO $ runStateT (walkToProjectRoot target targetStat umask myUid) []
174− return result
189+ return $ makeRelative "." <$> result
175190
176191withProjectRoot :: MonadObelisk m => FilePath -> (FilePath -> m a) -> m a
177192withProjectRoot target f = findProjectRoot target >>= \case
@@ -234,32 +249,81 @@ filePermissionIsSafe s umask = not fileWorldWritable && fileGroupWritable <= uma
234249 fileGroupWritable = fileMode s .&. 0o020 == 0o020
235250 umaskGroupWritable = umask .&. 0o020 == 0
236251
237−-- | Run a command in the given shell for the current project
238−inProjectShell :: MonadObelisk m => String -> String -> m ()
239−inProjectShell shellName command = withProjectRoot "." $ \root ->
240− projectShell root True shellName (Just command)
252+-- | Nix syntax requires relative paths to be prefixed by @./@ or
253+-- @../@. This will make a 'FilePath' that can be embedded in a Nix
254+-- expression.
255+toNixPath :: FilePath -> FilePath
256+toNixPath root | "/" `isInfixOf` root = root
257+ | otherwise = "./" <> root
241258
242−inImpureProjectShell :: MonadObelisk m => String -> String -> m ()
243−inImpureProjectShell shellName command = withProjectRoot "." $ \root ->
244− projectShell root False shellName (Just command)
259+nixShellRunConfig :: MonadObelisk m => FilePath -> Bool -> Maybe String -> m NixShellConfig
260+nixShellRunConfig root isPure command = do
261+ nixpkgsPath <- fmap T.strip $ readProcessAndLogStderr Debug $ setCwd (Just root) $
262+ proc nixExePath ["eval", "(import .obelisk/impl {}).nixpkgs.path"]
263+ nixRemote <- liftIO $ lookupEnv "NIX_REMOTE"
264+ pure $ def
265+ & nixShellConfig_pure .~ isPure
266+ & nixShellConfig_common . nixCmdConfig_target .~ (def & target_path .~ Nothing)
267+ & nixShellConfig_run .~ (command <&> \c -> mconcat
268+ [ "export NIX_PATH=nixpkgs=", T.unpack nixpkgsPath, "; "
269+ , maybe "" (\v -> "export NIX_REMOTE=" <> v <> "; ") nixRemote
270+ , c
271+ ])
272+
273+nixShellRunProc :: NixShellConfig -> ProcessSpec
274+nixShellRunProc cfg = setDelegateCtlc True $ proc "nix-shell" $ runNixShellConfig cfg
275+
276+nixShellWithPkgs :: MonadObelisk m => FilePath -> Bool -> Bool -> Map Text FilePath -> Maybe String -> m ()
277+nixShellWithPkgs root isPure chdirToRoot packageNamesAndPaths command = do
278+ packageNamesAndAbsPaths <- liftIO $ for packageNamesAndPaths makeAbsolute
279+ defShellConfig <- nixShellRunConfig root isPure command
280+ let setCwd_ = if chdirToRoot then setCwd (Just root) else id
281+ (_, _, _, ph) <- createProcess_ "nixShellWithPkgs" $ setCwd_ $ nixShellRunProc $ defShellConfig
282+ & nixShellConfig_common . nixCmdConfig_target . target_expr ?~
283+ "{root, pkgs}: ((import root {}).passthru.__unstable__.self.extend (_: _: {\
284+ \shellPackages = builtins.fromJSON pkgs;\
285+ \})).project.shells.ghc"
286+ & nixShellConfig_common . nixCmdConfig_args .~
287+ [ rawArg "root" $ toNixPath $ if chdirToRoot then "." else root
288+ , strArg "pkgs" (T.unpack $ decodeUtf8 $ BSL.toStrict $ Json.encode packageNamesAndAbsPaths)
289+ ]
290+ void $ waitForProcess ph
291+
292+nixShellWithHoogle :: MonadObelisk m => FilePath -> Bool -> String -> Maybe String -> m ()
293+nixShellWithHoogle root isPure shell' command = do
294+ defShellConfig <- nixShellRunConfig root isPure command
295+ (_, _, _, ph) <- createProcess_ "nixShellWithHoogle" $ setCwd (Just root) $ nixShellRunProc $ defShellConfig
296+ & nixShellConfig_common . nixCmdConfig_target . target_expr ?~
297+ "{shell}: ((import ./. {}).passthru.__unstable__.self.extend (_: super: {\
298+ \userSettings = super.userSettings // { withHoogle = true; };\
299+ \})).project.shells.${shell}"
300+ & nixShellConfig_common . nixCmdConfig_args .~ [ strArg "shell" shell' ]
301+ void $ waitForProcess ph
245302
246303projectShell :: MonadObelisk m => FilePath -> Bool -> String -> Maybe String -> m ()
247304projectShell root isPure shellName command = do
248− let nixPath = $(staticWhich "nix")
249− nixpkgsPath <- fmap T.strip $ readProcessAndLogStderr Debug $ setCwd (Just root) $ proc nixPath ["eval", "(import .obelisk/impl {}).nixpkgs.path"]
250− nixRemote <- liftIO $ lookupEnv "NIX_REMOTE"
251− (_, _, _, ph) <- createProcess_ "runNixShellAttr" $ setDelegateCtlc True $ setCwd (Just root) $ proc "nix-shell" $
252− [ "default.nix"] <>
253− ["--pure" | isPure] <>
254− [ "-A"
255− , "shells." <> shellName
256− ] <> case command of
257− Nothing -> []
258− Just c ->
259− -- TODO: Escape nixpkgsPath and nixRemote
260− [ "--run"
261− , "export NIX_PATH=nixpkgs=" <> T.unpack nixpkgsPath <> "; " <>
262− maybe "" (\v -> "export NIX_REMOTE=" <> v <> "; ") nixRemote <>
263− c
305+ defShellConfig <- nixShellRunConfig root isPure command
306+ (_, _, _, ph) <- createProcess_ "runNixShellAttr" $ setCwd (Just root) $ nixShellRunProc $ defShellConfig
307+ & nixShellConfig_common . nixCmdConfig_target . target_path ?~ "default.nix"
308+ & nixShellConfig_common . nixCmdConfig_target . target_attr ?~ ("shells." <> shellName)
309+ void $ waitForProcess ph
310+
311+findProjectAssets :: MonadObelisk m => FilePath -> m Text
312+findProjectAssets root = do
313+ isDerivation <- readProcessAndLogStderr Debug $ setCwd (Just root) $
314+ proc nixExePath
315+ [ "eval"
316+ , "(let a = import ./. {}; in toString (a.reflex.nixpkgs.lib.isDerivation a.passthru.staticFilesImpure))"
317+ , "--raw"
318+ -- `--raw` is not available with old nix-instantiate. It drops quotation
319+ -- marks and trailing newline, so is very convenient for shelling out.
320+ ]
321+ -- Check whether the impure static files are a derivation (and so must be built)
322+ if isDerivation == "1"
323+ then fmap T.strip $ readProcessAndLogStderr Debug $ setCwd (Just root) $ -- Strip whitespace here because nix-build has no --raw option
324+ proc nixBuildExePath
325+ [ "--no-out-link"
326+ , "-E", "(import ./. {}).passthru.staticFilesImpure"
264327 ]
265− void $ liftIO $ waitForProcess ph
328+ else readProcessAndLogStderr Debug $ setCwd (Just root) $
329+ proc nixExePath ["eval", "-f", ".", "passthru.staticFilesImpure", "--raw"]
lib/command/src/Obelisk/Command/Run.hs+260−132
@@ -3,47 +3,77 @@
33{-# LANGUAGE MultiParamTypeClasses #-}
44{-# LANGUAGE OverloadedStrings #-}
55{-# LANGUAGE ScopedTypeVariables #-}
6−{-# LANGUAGE TemplateHaskell #-}
76{-# LANGUAGE TupleSections #-}
7+{-# LANGUAGE QuasiQuotes #-}
88module Obelisk.Command.Run where
99
10−import Control.Applicative (liftA2)
10+import Control.Arrow ((&&&))
1111import Control.Exception (Exception, bracket)
12−import Control.Monad
12+import Control.Lens (ifor, (.~), (&))
13+import Control.Monad (filterM, unless, void)
1314import Control.Monad.Except (runExceptT, throwError)
1415import Control.Monad.IO.Class (liftIO)
1516import Control.Monad.Reader (MonadIO)
17+import Data.Coerce (coerce)
18+import Data.Default (def)
1619import Data.Either
17−import Data.Foldable (for_)
18−import qualified Data.List as L
20+import Data.Foldable (for_, toList)
1921import qualified Data.List.NonEmpty as NE
22+import Data.Map (Map)
23+import qualified Data.Map as Map
2024import Data.Maybe
25+import qualified Data.Set as Set
26+import Data.Text (Text)
2127import qualified Data.Text as T
28+import Data.Time.Clock (getCurrentTime)
29+import Data.Time.Format (formatTime, defaultTimeLocale)
30+import Data.Traversable (for)
31+import Debug.Trace (trace)
2232import Distribution.Compiler (CompilerFlavor(..))
2333import Distribution.PackageDescription.Parsec (parseGenericPackageDescription)
2434import Distribution.Parsec.ParseResult (runParseResult)
25−import Distribution.Pretty
35+import qualified Distribution.System as Dist
2636import Distribution.Types.BuildInfo
2737import Distribution.Types.CondTree
2838import Distribution.Types.GenericPackageDescription
2939import Distribution.Types.Library
3040import Distribution.Utils.Generic
41+import qualified Distribution.Parsec.Common as Dist
3142import qualified Hpack.Config as Hpack
3243import qualified Hpack.Render as Hpack
3344import qualified Hpack.Yaml as Hpack
3445import Language.Haskell.Extension
3546import Network.Socket hiding (Debug)
3647import System.Directory
48+import System.Environment (getExecutablePath)
3749import System.FilePath
50+import qualified System.Info
3851import System.IO.Temp (withSystemTempDirectory)
39−import System.Which (staticWhich)
4052
41−import Obelisk.App (MonadObelisk, ObeliskT)
42−import Obelisk.CliApp (Severity (..) , callCommand, failWith, putLog, proc, readProcessAndLogStderr)
43−import Obelisk.Command.Project (inProjectShell, withProjectRoot)
53+import Obelisk.App (MonadObelisk)
54+import Obelisk.CliApp (
55+ Severity (..),
56+ createProcess_,
57+ failWith,
58+ proc,
59+ putLog,
60+ readCreateProcessWithExitCode,
61+ readProcessAndLogStderr,
62+ setCwd,
63+ setDelegateCtlc,
64+ waitForProcess,
65+ withSpinner,
66+ )
67+import Obelisk.Command.Nix
68+import Obelisk.Command.Project (nixShellWithPkgs, toImplDir, withProjectRoot, findProjectAssets)
69+import Obelisk.Command.Thunk (attrCacheFileName)
70+import Obelisk.Command.Utils (findExePath, ghcidExePath)
4471
4572data CabalPackageInfo = CabalPackageInfo
46− { _cabalPackageInfo_packageRoot :: FilePath
73+ { _cabalPackageInfo_packageFile :: FilePath
74+ , _cabalPackageInfo_packageName :: T.Text
75+ , _cabalPackageInfo_packageRoot :: FilePath
76+ , _cabalPackageInfo_buildable :: Bool
4777 , _cabalPackageInfo_sourceDirs :: NE.NonEmpty FilePath
4878 -- ^ List of hs src dirs of the library component
4979 , _cabalPackageInfo_defaultExtensions :: [Extension]
@@ -54,108 +84,182 @@ data CabalPackageInfo = CabalPackageInfo
5484 -- ^ List of compiler-specific options (e.g., the "ghc-options" field of the cabal file)
5585 }
5686
57−-- NOTE: `run` is not polymorphic like the rest because we use StaticPtr to invoke it.
58−run :: ObeliskT IO ()
59−run = do
60− let nixPath = $(staticWhich "nix")
61− nixBuildPath = $(staticWhich "nix-build")
62− pkgs <- getLocalPkgs
63− withGhciScript pkgs $ \dotGhciPath -> do
87+-- | Used to signal to obelisk that it's being invoked as a preprocessor
88+preprocessorIdentifier :: String
89+preprocessorIdentifier = "__preprocessor-apply-packages"
90+
91+profile
92+ :: MonadObelisk m
93+ => String
94+ -> [String]
95+ -> m ()
96+profile profileBasePattern rtsFlags = withProjectRoot "." $ \root -> do
97+ putLog Debug "Using profiled build of project."
98+
99+ outPath <- withSpinner "Building profiled executable" $
100+ fmap (T.unpack . T.strip) $ readProcessAndLogStderr Debug $ setCwd (Just root) $ nixCmdProc $
101+ NixCmd_Build $ def
102+ & nixBuildConfig_outLink .~ OutLink_None
103+ & nixCmdConfig_target .~ Target
104+ { _target_path = Just "."
105+ , _target_attr = Just "__unstable__.profiledObRun"
106+ , _target_expr = Nothing
107+ }
108+ assets <- findProjectAssets root
109+ putLog Debug $ "Assets impurely loaded from: " <> assets
110+ time <- liftIO getCurrentTime
111+ let profileBaseName = formatTime defaultTimeLocale profileBasePattern time
112+ liftIO $ createDirectoryIfMissing True $ takeDirectory $ root </> profileBaseName
113+ putLog Debug $ "Storing profiled data under base name of " <> T.pack (root </> profileBaseName)
114+ freePort <- getFreePort
115+ (_, _, _, ph) <- createProcess_ "runProfExe" $ setCwd (Just root) $ setDelegateCtlc True $ proc (outPath </> "bin" </> "ob-run") $
116+ [ show freePort
117+ , T.unpack assets
118+ , profileBaseName
119+ , "+RTS"
120+ , "-po" <> profileBaseName
121+ ] <> rtsFlags
122+ <> [ "-RTS" ]
123+ void $ waitForProcess ph
124+
125+run
126+ :: MonadObelisk m
127+ => m ()
128+run = withProjectRoot "." $ \root -> do
129+ pkgs <- fmap toList . parsePackagesOrFail =<< getLocalPkgs root
130+ withGhciScript pkgs root $ \dotGhciPath -> do
64131 freePort <- getFreePort
65− assets <- withProjectRoot "." $ \root -> do
66− let importableRoot = if "/" `isInfixOf` root
67− then root
68− else "./" <> root
69− isDerivation <- readProcessAndLogStderr Debug $
70− proc nixPath
71− [ "eval"
72− , "-f"
73− , root
74− , "(let a = import " <> importableRoot <> " {}; in toString (a.reflex.nixpkgs.lib.isDerivation a.passthru.staticFilesImpure))"
75− , "--raw"
76− -- `--raw` is not available with old nix-instantiate. It drops quotation
77− -- marks and trailing newline, so is very convenient for shelling out.
78− ]
79− -- Check whether the impure static files are a derivation (and so must be built)
80− if isDerivation == "1"
81− then fmap T.strip $ readProcessAndLogStderr Debug $ -- Strip whitespace here because nix-build has no --raw option
82− proc nixBuildPath
83− [ "--no-out-link"
84− , "-E", "(import " <> importableRoot <> "{}).passthru.staticFilesImpure"
85− ]
86− else readProcessAndLogStderr Debug $
87− proc nixPath ["eval", "-f", root, "passthru.staticFilesImpure", "--raw"]
132+ assets <- findProjectAssets root
88133 putLog Debug $ "Assets impurely loaded from: " <> assets
89− runGhcid dotGhciPath $ Just $ unwords
134+ runGhcid root True dotGhciPath pkgs $ Just $ unwords
90135 [ "Obelisk.Run.run"
91136 , show freePort
92− , "(runServeAsset " ++ show assets ++ ")"
137+ , "(Obelisk.Run.runServeAsset " ++ show assets ++ ")"
93138 , "Backend.backend"
94139 , "Frontend.frontend"
95140 ]
96141
97142runRepl :: MonadObelisk m => m ()
98−runRepl = do
99− pkgs <- getLocalPkgs
100− withGhciScript pkgs $ \dotGhciPath -> do
101− runGhciRepl dotGhciPath
143+runRepl = withProjectRoot "." $ \root -> do
144+ pkgs <- fmap toList . parsePackagesOrFail =<< getLocalPkgs root
145+ withGhciScript pkgs "." $ \dotGhciPath ->
146+ runGhciRepl root pkgs dotGhciPath
102147
103148runWatch :: MonadObelisk m => m ()
104−runWatch = do
105− pkgs <- getLocalPkgs
106− withGhciScript pkgs $ \dotGhciPath -> runGhcid dotGhciPath Nothing
149+runWatch = withProjectRoot "." $ \root -> do
150+ pkgs <- fmap toList . parsePackagesOrFail =<< getLocalPkgs root
151+ withGhciScript pkgs root $ \dotGhciPath ->
152+ runGhcid root True dotGhciPath pkgs Nothing
153+
154+-- | Relative paths to local packages of an obelisk project.
155+--
156+-- These are a combination of the obelisk predefined local packages,
157+-- and any packages that the user has set with the @packages@ argument
158+-- to the Nix @project@ function.
159+getLocalPkgs :: MonadObelisk m => FilePath -> m [FilePath]
160+getLocalPkgs root = do
161+ obeliskPaths <- runFind ["-L", root, "-name", ".obelisk", "-type", "d"]
162+
163+ -- We do not want to find packages that are embedded inside other obelisk projects, unless that
164+ -- obelisk project is our own.
165+ let exclusions = filter (/= root) $ map takeDirectory obeliskPaths
166+ fmap (map (makeRelative ".")) $ runFind $
167+ ["-L", root, "(", "-name", "*.cabal", "-o", "-name", Hpack.packageConfig, ")", "-a", "-type", "f"]
168+ <> concat [["-not", "-path", p </> "*"] | p <- (toImplDir "*" </> attrCacheFileName) : exclusions]
169+ where
170+ runFind args = do
171+ (_exitCode, out, err) <- readCreateProcessWithExitCode $ proc findExePath args
172+ putLog Debug $ T.strip $ T.pack err
173+ pure $ map T.unpack $ T.lines $ T.strip $ T.pack out
107174
108175data GuessPackageFileError = GuessPackageFileError_Ambiguous [FilePath] | GuessPackageFileError_NotFound
109176 deriving (Eq, Ord, Show)
110177instance Exception GuessPackageFileError
111178
179+newtype HPackFilePath = HPackFilePath { unHPackFilePath :: FilePath } deriving (Eq, Ord, Show)
180+newtype CabalFilePath = CabalFilePath { unCabalFilePath :: FilePath } deriving (Eq, Ord, Show)
181+
112182-- | Given a directory, try to guess what the appropriate @.cabal@ or @package.yaml@ file is for the package.
113183guessCabalPackageFile
114− :: MonadIO m
115− => FilePath -- ^ Directory to search for cabal package
116− -> m (Either GuessPackageFileError (Either FilePath FilePath))
117− -- ^ 'Right' 'Left' for hpack package, 'Right' 'Right' for cabal package; 'FilePath' is relative to given directory.
118−guessCabalPackageFile dir = do
119− candidates <- liftIO $
120− filterM (doesFileExist . (dir </>))
121− =<< filter (liftA2 (||) (== Hpack.packageConfig) (".cabal" `L.isSuffixOf`))
122− <$> listDirectory dir
123− pure $ case L.partition (== Hpack.packageConfig) candidates of
124− ([hpack], _) -> Right $ Left hpack
125− ([], [cabal]) -> Right $ Right cabal
126− ([], []) -> Left GuessPackageFileError_NotFound
127− (hpacks, cabals) -> Left $ GuessPackageFileError_Ambiguous $ hpacks <> cabals
184+ :: (MonadIO m)
185+ => FilePath -- ^ Directory or path to search for cabal package
186+ -> m (Either GuessPackageFileError (Either CabalFilePath HPackFilePath))
187+guessCabalPackageFile pkg = do
188+ liftIO (doesDirectoryExist pkg) >>= \case
189+ False -> case cabalOrHpackFile pkg of
190+ (Just hpack@(Right _)) -> pure $ Right hpack
191+ (Just cabal@(Left (CabalFilePath cabalFilePath))) -> do
192+ -- If the cabal file has a sibling hpack file, we use that instead
193+ -- since running hpack often generates a sibling cabal file
194+ let possibleHpackSibling = takeDirectory cabalFilePath </> Hpack.packageConfig
195+ hasHpackSibling <- liftIO $ doesFileExist possibleHpackSibling
196+ pure $ Right $ if hasHpackSibling then Right (HPackFilePath possibleHpackSibling) else cabal
197+ Nothing -> pure $ Left GuessPackageFileError_NotFound
198+ True -> do
199+ candidates <- liftIO $
200+ filterM (doesFileExist . either unCabalFilePath unHPackFilePath)
201+ =<< mapMaybe (cabalOrHpackFile . (pkg </>)) <$> listDirectory pkg
202+ pure $ case partitionEithers candidates of
203+ ([hpack], _) -> Right $ Left hpack
204+ ([], [cabal]) -> Right $ Right cabal
205+ ([], []) -> Left GuessPackageFileError_NotFound
206+ (hpacks, cabals) -> Left $ GuessPackageFileError_Ambiguous $ coerce hpacks <> coerce cabals
128207
129−-- | Relative paths to local packages of an obelisk project
130−-- TODO a way to query this
131−getLocalPkgs :: Applicative f => f [FilePath]
132−getLocalPkgs = pure ["backend", "common", "frontend"]
208+cabalOrHpackFile :: FilePath -> Maybe (Either CabalFilePath HPackFilePath)
209+cabalOrHpackFile = \case
210+ x | takeExtension x == ".cabal" -> Just (Left $ CabalFilePath x)
211+ | takeFileName x == Hpack.packageConfig -> Just (Right $ HPackFilePath x)
212+ | otherwise -> Nothing
133213
214+-- | Parses the cabal package in a given directory.
215+-- This automatically figures out which .cabal file or package.yaml (hpack) file to use in the given directory.
134216parseCabalPackage
135− :: (MonadObelisk m)
136− => FilePath -- ^ package directory
217+ :: MonadObelisk m
218+ => FilePath -- ^ Package directory
137219 -> m (Maybe CabalPackageInfo)
138−parseCabalPackage dir = either ((Nothing <$) . putLog Error) (pure . Just) <=< runExceptT $ do
139− (cabalContents, packageFile) <- guessCabalPackageFile dir >>= \case
140− Left GuessPackageFileError_NotFound -> throwError $ "No .cabal or package.yaml file found in " <> T.pack dir
141− Left (GuessPackageFileError_Ambiguous _) -> throwError $ "Unable to determine which .cabal file to use in " <> T.pack dir
142− Right (Right cabalFileName) -> let file = dir </> cabalFileName in (, file) <$> liftIO (readUTF8File file)
143− Right (Left hpackFileName) -> do
220+parseCabalPackage dir = parseCabalPackage' dir >>= \case
221+ Left err -> Nothing <$ putLog Error err
222+ Right (warnings, pkgInfo) -> do
223+ for_ warnings $ putLog Warning . T.pack . show
224+ pure $ Just pkgInfo
225+
226+-- | Like 'parseCabalPackage' but returns errors and warnings directly so as to avoid 'MonadObelisk'.
227+parseCabalPackage'
228+ :: (MonadIO m)
229+ => FilePath -- ^ Package directory
230+ -> m (Either T.Text ([Dist.PWarning], CabalPackageInfo))
231+parseCabalPackage' pkg = runExceptT $ do
232+ (cabalContents, packageFile, packageName) <- guessCabalPackageFile pkg >>= \case
233+ Left GuessPackageFileError_NotFound -> throwError $ "No .cabal or package.yaml file found in " <> T.pack pkg
234+ Left (GuessPackageFileError_Ambiguous _) -> throwError $ "Unable to determine which .cabal file to use in " <> T.pack pkg
235+ Right (Left (CabalFilePath file)) -> (, file, takeBaseName file) <$> liftIO (readUTF8File file)
236+ Right (Right (HPackFilePath file)) -> do
144237 let
145− file = dir </> hpackFileName
146238 decodeOptions = Hpack.DecodeOptions (Hpack.ProgramName "ob") file Nothing Hpack.decodeYaml
147239 liftIO (Hpack.readPackageConfig decodeOptions) >>= \case
148240 Left err -> throwError $ T.pack $ "Failed to parse " <> file <> ": " <> err
149− Right (Hpack.DecodeResult hpackPackage _ _ _) -> pure (Hpack.renderPackage [] hpackPackage, file)
241+ Right (Hpack.DecodeResult hpackPackage _ _ _) -> pure (Hpack.renderPackage [] hpackPackage, file, Hpack.packageName hpackPackage)
150242
151− let (warnings, result) = runParseResult $ parseGenericPackageDescription $
152− toUTF8BS cabalContents
153− for_ warnings $ putLog Warning . T.pack . show
243+ let
244+ (warnings, result) = runParseResult $ parseGenericPackageDescription $ toUTF8BS cabalContents
245+ osConfVar = case System.Info.os of
246+ "linux" -> Just Dist.Linux
247+ "darwin" -> Just Dist.OSX
248+ _ -> trace "Unrecgonized System.Info.os" Nothing
249+ archConfVar = Just Dist.X86_64 -- TODO: Actually infer this
250+ evalConfVar v = Right $ case v of
251+ OS osVar -> Just osVar == osConfVar
252+ Arch archVar -> Just archVar == archConfVar
253+ Impl GHC _ -> True -- TODO: Actually check version range
254+ _ -> False
154255 case condLibrary <$> result of
155256 Right (Just condLib) -> do
156− let (_, lib) = simplifyCondTree (const $ pure True) condLib
157− pure $ CabalPackageInfo
158− { _cabalPackageInfo_packageRoot = dir
257+ let (_, lib) = simplifyCondTree evalConfVar condLib
258+ pure $ (warnings,) $ CabalPackageInfo
259+ { _cabalPackageInfo_packageName = T.pack packageName
260+ , _cabalPackageInfo_packageFile = packageFile
261+ , _cabalPackageInfo_packageRoot = takeDirectory packageFile
262+ , _cabalPackageInfo_buildable = buildable $ libBuildInfo lib
159263 , _cabalPackageInfo_sourceDirs =
160264 fromMaybe (pure ".") $ NE.nonEmpty $ hsSourceDirs $ libBuildInfo lib
161265 , _cabalPackageInfo_defaultExtensions =
@@ -168,45 +272,60 @@ parseCabalPackage dir = either ((Nothing <$) . putLog Error) (pure . Just) <=< r
168272 Left (_, errors) ->
169273 throwError $ T.pack $ "Failed to parse " <> packageFile <> ":\n" <> unlines (map show errors)
170274
275+parsePackagesOrFail :: MonadObelisk m => [FilePath] -> m (NE.NonEmpty CabalPackageInfo)
276+parsePackagesOrFail dirs = do
277+ (pkgDirErrs, packageInfos') <- fmap partitionEithers $ for dirs $ \dir -> do
278+ flip fmap (parseCabalPackage dir) $ \case
279+ Just packageInfo
280+ | _cabalPackageInfo_buildable packageInfo -> Right packageInfo
281+ _ -> Left dir
282+
283+ let packagesByName = Map.fromListWith (<>) [(_cabalPackageInfo_packageName p, p NE.:| []) | p <- packageInfos']
284+ unambiguous <- ifor packagesByName $ \packageName ps -> case ps of
285+ p NE.:| [] -> pure p -- No ambiguity here
286+ p NE.:| _ -> do
287+ putLog Warning $ T.pack $
288+ "Packages named '" <> T.unpack packageName <> "' appear in " <> show (length ps) <> " different locations: "
289+ <> intercalate ", " (map _cabalPackageInfo_packageFile $ toList ps)
290+ <> "; Picking " <> _cabalPackageInfo_packageFile p
291+ pure p
292+
293+ packageInfos <- case NE.nonEmpty $ toList unambiguous of
294+ Nothing -> failWith $ T.pack $ "No valid, buildable packages found in " <> intercalate ", " dirs
295+ Just xs -> pure xs
296+
297+ unless (null pkgDirErrs) $
298+ putLog Warning $ T.pack $ "Failed to find buildable packages in " <> intercalate ", " pkgDirErrs
299+
300+ pure packageInfos
301+
302+packageInfoToNamePathMap :: [CabalPackageInfo] -> Map Text FilePath
303+packageInfoToNamePathMap = Map.fromList . map (_cabalPackageInfo_packageName &&& _cabalPackageInfo_packageRoot)
304+
171305-- | Create ghci configuration to load the given packages
172306withGhciScript
173307 :: MonadObelisk m
174− => [FilePath] -- ^ List of packages to load into ghci
308+ => [CabalPackageInfo] -- ^ List of packages to load into ghci
309+ -> FilePath -- ^ All paths written to the .ghci file will be relative to this path
175310 -> (FilePath -> m ()) -- ^ Action to run with the path to generated temporary .ghci
176311 -> m ()
177−withGhciScript dirs f = do
178− (pkgDirErrs, packageInfos) <- fmap partitionEithers $ forM dirs $ \dir -> do
179− flip fmap (parseCabalPackage dir) $ \case
180− Nothing -> Left dir
181− Just packageInfo -> Right packageInfo
182−
183− when (null packageInfos) $
184− failWith $ T.pack $ "No valid pkgs found in " <> intercalate ", " dirs
185− unless (null pkgDirErrs) $
186− putLog Warning $ T.pack $ "Failed to find pkgs in " <> intercalate ", " pkgDirErrs
187−
188− let extensions = packageInfos >>= _cabalPackageInfo_defaultExtensions
189− languageFromPkgs = L.nub $ mapMaybe _cabalPackageInfo_defaultLanguage packageInfos
190− -- NOTE when no default-language is present cabal sets Haskell98
191− language = NE.toList $ fromMaybe (Haskell98 NE.:| []) $ NE.nonEmpty languageFromPkgs
192− extensionsLine = if extensions == mempty
193− then ""
194− else ":set " <> unwords (("-X" <>) . prettyShow <$> extensions)
195− ghcOptions = concat $ mapMaybe (\case (GHC, xs) -> Just xs; _ -> Nothing) $
196− packageInfos >>= _cabalPackageInfo_compilerOptions
197− dotGhci = unlines
198− [ ":set -i" <> intercalate ":" (packageInfos >>= rootedSourceDirs)
199− , case ghcOptions of
200− [] -> ""
201− xs -> ":set " <> unwords xs
202− , extensionsLine
203− , ":set " <> unwords (("-X" <>) . prettyShow <$> language)
204− , ":load Backend Frontend"
205− , "import Obelisk.Run"
206− , "import qualified Frontend"
207− , "import qualified Backend"
208− ]
209− warnDifferentLanguages language
312+withGhciScript packageInfos pathBase f = do
313+ selfExe <- liftIO getExecutablePath
314+ let
315+ packageNames = Set.fromList $ map _cabalPackageInfo_packageName packageInfos
316+ modulesToLoad = mconcat
317+ [ [ "Obelisk.Run" | "obelisk-run" `Set.member` packageNames ]
318+ , [ "Backend" | "backend" `Set.member` packageNames ]
319+ , [ "Frontend" | "frontend" `Set.member` packageNames ]
320+ ]
321+ dotGhci = unlines
322+ -- TODO: Shell escape
323+ [ ":set -F -pgmF " <> selfExe <> " -optF " <> preprocessorIdentifier <> " " <> unwords (map (("-optF " <>) . makeRelative pathBase . _cabalPackageInfo_packageFile) packageInfos)
324+ , ":set -i" <> intercalate ":" (packageInfos >>= rootedSourceDirs)
325+ , if null modulesToLoad then "" else ":load " <> unwords modulesToLoad
326+ , "import qualified Obelisk.Run"
327+ , "import qualified Frontend"
328+ , "import qualified Backend" ]
210329 withSystemTempDirectory "ob-ghci" $ \fp -> do
211330 let dotGhciPath = fp </> ".ghci"
212331 liftIO $ writeFile dotGhciPath dotGhci
@@ -214,35 +333,44 @@ withGhciScript dirs f = do
214333
215334 where
216335 rootedSourceDirs pkg = NE.toList $
217− (_cabalPackageInfo_packageRoot pkg </>) <$> _cabalPackageInfo_sourceDirs pkg
218−
219−warnDifferentLanguages :: MonadObelisk m => [Language] -> m ()
220−warnDifferentLanguages (_:_:_) = putLog Warning "Different languages detected across packages which may result in errors when loading the repl"
221−warnDifferentLanguages _ = return ()
336+ makeRelative pathBase . (_cabalPackageInfo_packageRoot pkg </>) <$> _cabalPackageInfo_sourceDirs pkg
222337
223338-- | Run ghci repl
224339runGhciRepl
225340 :: MonadObelisk m
226− => FilePath -- ^ Path to .ghci
341+ => FilePath -- ^ Path to project root
342+ -> [CabalPackageInfo]
343+ -> FilePath -- ^ Path to .ghci
227344 -> m ()
228−runGhciRepl dotGhci = inProjectShell "ghc" $ "ghci " <> makeBaseGhciOptions dotGhci
345+runGhciRepl root packages dotGhci =
346+ -- NOTE: We do *not* want to use $(staticWhich "ghci") here because we need the
347+ -- ghc that is provided by the shell in the user's project.
348+ nixShellWithPkgs root True False (packageInfoToNamePathMap packages) $ Just $ "ghci " <> makeBaseGhciOptions dotGhci -- TODO: Shell escape
229349
230350-- | Run ghcid
231351runGhcid
232352 :: MonadObelisk m
233− => FilePath -- ^ Path to .ghci
353+ => FilePath -- ^ Path to project root
354+ -> Bool -- ^ Should we chdir to root when running this process?
355+ -> FilePath -- ^ Path to .ghci
356+ -> [CabalPackageInfo]
234357 -> Maybe String -- ^ Optional command to run at every reload
235358 -> m ()
236−runGhcid dotGhci mcmd = callCommand $ unwords $ $(staticWhich "ghcid") : opts
359+runGhcid root chdirToRoot dotGhci packages mcmd =
360+ nixShellWithPkgs root True chdirToRoot (packageInfoToNamePathMap packages) (Just $ unwords $ ghcidExePath : opts) -- TODO: Shell escape
237361 where
238362 opts =
239363 [ "-W"
240− --TODO: The decision of whether to use -fwarn-redundant-constraints should probably be made by the user
241− , "--command='ghci -Wall -ignore-dot-ghci -fwarn-redundant-constraints " <> makeBaseGhciOptions dotGhci <> "' "
242− , "--reload=config"
364+ , "--command='ghci -ignore-dot-ghci " <> makeBaseGhciOptions dotGhci <> "' "
243365 , "--outputfile=ghcid-output.txt"
244− ] <> testCmd
245− testCmd = maybeToList (flip fmap mcmd $ \cmd -> "--test='" <> cmd <> "'")
366+ ] <> map (\x -> "--reload='" <> x <> "'") reloadFiles
367+ <> map (\x -> "--restart='" <> x <> "'") restartFiles
368+ <> testCmd
369+ testCmd = maybeToList (flip fmap mcmd $ \cmd -> "--test='" <> cmd <> "'") -- TODO: Shell escape
370+
371+ adjustRoot x = if chdirToRoot then makeRelative root x else x
372+ reloadFiles = map adjustRoot [root </> "config"]
373+ restartFiles = map (adjustRoot . _cabalPackageInfo_packageFile) packages
246374
247375makeBaseGhciOptions :: FilePath -> String
248376makeBaseGhciOptions dotGhci =
lib/command/src/Obelisk/Command/Thunk.hs+90−31
@@ -18,6 +18,7 @@ module Obelisk.Command.Thunk
1818 , ThunkRev (..)
1919 , ThunkSource (..)
2020 , ThunkUpdateConfig (..)
21+ , attrCacheFileName
2122 , createThunk
2223 , createThunkWithLatest
2324 , getLatestRev
@@ -47,6 +48,7 @@ import qualified Data.Aeson as Aeson
4748import Data.Aeson.Encode.Pretty
4849import qualified Data.Aeson.Types as Aeson
4950import qualified Data.ByteString.Lazy as LBS
51+import Data.Containers.ListUtils (nubOrd)
5052import Data.Default
5153import Data.Either.Combinators (fromRight', rightToMaybe)
5254import Data.Git.Ref (Ref)
@@ -69,6 +71,7 @@ import GitHub
6971import GitHub.Data.Name
7072import Obelisk.Command.Nix
7173import System.Directory
74+import System.Exit
7275import System.FilePath
7376import System.IO.Error
7477import System.IO.Temp
@@ -119,7 +122,7 @@ data GitHubSource = GitHubSource
119122 }
120123 deriving (Show, Eq, Ord)
121124
122−newtype GitUri = GitUri URI.URI deriving (Eq, Ord, Show)
125+newtype GitUri = GitUri { unGitUri :: URI.URI } deriving (Eq, Ord, Show)
123126
124127gitUriToText :: GitUri -> Text
125128gitUriToText (GitUri uri)
@@ -229,6 +232,9 @@ readThunk thunkDir = do
229232 False -> return $ return Nothing
230233 False -> liftIO $ fmap ThunkData_Packed <$> readPackedThunk thunkDir
231234
235+attrCacheFileName :: FilePath
236+attrCacheFileName = ".attr-cache"
237+
232238data ThunkType = ThunkType
233239 { _thunkType_loader :: FilePath
234240 , _thunkType_json :: FilePath
@@ -241,7 +247,7 @@ gitHubThunkType :: ThunkType
241247gitHubThunkType = ThunkType
242248 { _thunkType_loader = "default.nix"
243249 , _thunkType_json = "github.json"
244− , _thunkType_optional = Set.fromList [".attr-cache"]
250+ , _thunkType_optional = Set.fromList [attrCacheFileName]
245251 , _thunkType_loaderVersions = gitHubStandaloneLoaders
246252 , _thunkType_parser = parseThunkPtr $ \v ->
247253 ThunkSource_GitHub <$> parseGitHubSource v <|> ThunkSource_Git <$> parseGitSource v
@@ -251,7 +257,7 @@ gitThunkType :: ThunkType
251257gitThunkType = ThunkType
252258 { _thunkType_loader = "default.nix"
253259 , _thunkType_json = "git.json"
254− , _thunkType_optional = Set.fromList [".attr-cache"]
260+ , _thunkType_optional = Set.fromList [attrCacheFileName]
255261 , _thunkType_loaderVersions = plainGitStandaloneLoaders
256262 , _thunkType_parser = parseThunkPtr $ fmap ThunkSource_Git . parseGitSource
257263 }
@@ -398,7 +404,7 @@ encodeThunkPtrData (ThunkPtr rev src) = case src of
398404
399405createThunk :: MonadIO m => FilePath -> ThunkPtr -> m ()
400406createThunk target thunk = liftIO $ do
401− createDirectoryIfMissing True (target </> ".attr-cache")
407+ createDirectoryIfMissing True (target </> attrCacheFileName)
402408 T.writeFile (target </> "default.nix") (thunkPtrLoader thunk)
403409 let
404410 jsonFileName = case _thunkPtr_source thunk of
@@ -443,7 +449,7 @@ updateThunkToLatest (ThunkUpdateConfig mBranch thunkConfig) target = withSpinner
443449
444450setThunk :: MonadObelisk m => ThunkConfig -> FilePath -> GitSource -> String -> m ()
445451setThunk thunkConfig target gs branch = do
446− newThunkPtr <- uriThunkPtr (_gitSource_url gs) (Just $ T.pack branch) Nothing
452+ newThunkPtr <- uriThunkPtr (_gitSource_url gs) (_thunkConfig_private thunkConfig) (Just $ T.pack branch) Nothing
447453 overwriteThunk target newThunkPtr
448454 updateThunkToLatest (ThunkUpdateConfig Nothing thunkConfig) target
449455
@@ -507,9 +513,10 @@ gitHubStandaloneLoaderV4 = T.unlines
507513
508514plainGitStandaloneLoaders :: NonEmpty Text
509515plainGitStandaloneLoaders =
510− plainGitStandaloneLoaderV3 :|
516+ plainGitStandaloneLoaderV4 :|
511517 [ plainGitStandaloneLoaderV1
512518 , plainGitStandaloneLoaderV2
519+ , plainGitStandaloneLoaderV3
513520 ]
514521
515522plainGitStandaloneLoaderV1 :: Text
@@ -530,6 +537,8 @@ plainGitStandaloneLoaderV2 = T.unlines
530537 , "in import (fetchGit (builtins.fromJSON (builtins.readFile ./git.json)))"
531538 ]
532539
540+-- This loader has a bug because @builtins.fetchGit@ is not given a @ref@
541+-- and will fail to find commits without this because it does shallow clones.
533542plainGitStandaloneLoaderV3 :: Text
534543plainGitStandaloneLoaderV3 = T.unlines
535544 [ "# DO NOT HAND-EDIT THIS FILE"
@@ -546,6 +555,23 @@ plainGitStandaloneLoaderV3 = T.unlines
546555 , "in import (fetch (builtins.fromJSON (builtins.readFile ./git.json)))"
547556 ]
548557
558+plainGitStandaloneLoaderV4 :: Text
559+plainGitStandaloneLoaderV4 = T.unlines
560+ [ "# DO NOT HAND-EDIT THIS FILE"
561+ , "let fetch = {url, rev, branch ? null, sha256 ? null, fetchSubmodules ? false, private ? false, ...}:"
562+ , " let realUrl = let firstChar = builtins.substring 0 1 url; in"
563+ , " if firstChar == \"/\" then /. + url"
564+ , " else if firstChar == \".\" then ./. + url"
565+ , " else url;"
566+ , " in if !fetchSubmodules && private then builtins.fetchGit {"
567+ , " url = realUrl; inherit rev;"
568+ , " ${if branch == null then null else \"ref\"} = branch;"
569+ , " } else (import <nixpkgs> {}).fetchgit {"
570+ , " url = realUrl; inherit rev sha256;"
571+ , " };"
572+ , "in import (fetch (builtins.fromJSON (builtins.readFile ./git.json)))"
573+ ]
574+
549575thunkFileNames :: [FilePath]
550576thunkFileNames = ["default.nix", "github.json", "git.json"]
551577
@@ -572,7 +598,7 @@ nixBuildThunkAttrWithCache thunkDir attr = do
572598 let cacheErrHandler e
573599 | isDoesNotExistError e = return Nothing -- expected from a cache miss
574600 | otherwise = putLog Error (T.pack $ displayException e) >> return Nothing
575− cacheDir = thunkDir </> ".attr-cache"
601+ cacheDir = thunkDir </> attrCacheFileName
576602 cachePath = cacheDir </> attr <.> "out"
577603 latestChange <- liftIO $ do
578604 createDirectoryIfMissing False cacheDir
@@ -699,7 +725,7 @@ packThunk' noTrail (ThunkPackConfig force thunkConfig) thunkDir = checkThunkDire
699725 Right (ThunkData_Checkout _) -> do
700726 withSpinner' ("Packing thunk " <> T.pack thunkDir)
701727 (finalMsg noTrail $ const $ "Packed thunk " <> T.pack thunkDir) $ do
702− thunkPtr <- modifyThunkPtrByConfig thunkConfig <$> getThunkPtr (not force) thunkDir
728+ thunkPtr <- modifyThunkPtrByConfig thunkConfig <$> getThunkPtr (not force) thunkDir (_thunkConfig_private thunkConfig)
703729 callProcessAndLogOutput (Debug, Error) $ proc "rm" ["-rf", thunkDir]
704730 liftIO $ createThunk thunkDir thunkPtr
705731 pure thunkPtr
@@ -712,8 +738,8 @@ modifyThunkPtrByConfig (ThunkConfig markPrivate') ptr = case markPrivate' of
712738 ThunkSource_GitHub s -> ThunkSource_GitHub $ s { _gitHubSource_private = markPrivate }
713739 }
714740
715−getThunkPtr :: forall m. MonadObelisk m => Bool -> FilePath -> m ThunkPtr
716−getThunkPtr checkClean thunkDir = do
741+getThunkPtr :: forall m. MonadObelisk m => Bool -> FilePath -> Maybe Bool -> m ThunkPtr
742+getThunkPtr checkClean thunkDir mPrivate = do
717743 when checkClean $ ensureCleanGitRepo thunkDir True
718744 "thunk pack: thunk checkout contains unsaved modifications"
719745
@@ -727,7 +753,7 @@ getThunkPtr checkClean thunkDir = do
727753 ] ++ T.lines stashOutput
728754 True -> return ()
729755
730− -- Get current branch ``
756+ -- Get current branch
731757 (mCurrentBranch, mCurrentCommit) <- do
732758 b <- listToMaybe
733759 <$> T.lines
@@ -829,7 +855,7 @@ getThunkPtr checkClean thunkDir = do
829855 remoteUri <- case parseGitUri remoteUri' of
830856 Nothing -> failWith $ "Could not identify git remote: " <> remoteUri'
831857 Just uri -> pure uri
832− uriThunkPtr remoteUri mCurrentBranch mCurrentCommit
858+ uriThunkPtr remoteUri mPrivate mCurrentBranch mCurrentCommit
833859
834860-- | Get the latest revision available from the given source
835861getLatestRev :: MonadObelisk m => ThunkSource -> m ThunkRev
@@ -848,12 +874,12 @@ getLatestRev os = do
848874-- performance. If that doesn't work (e.g. authentication issue), we fall back
849875-- on just doing things the normal way for git repos in general, and save it as
850876-- a regular git thunk.
851−uriThunkPtr :: MonadObelisk m => GitUri -> Maybe Text -> Maybe Text -> m ThunkPtr
852−uriThunkPtr uri mbranch mcommit = do
877+uriThunkPtr :: MonadObelisk m => GitUri -> Maybe Bool -> Maybe Text -> Maybe Text -> m ThunkPtr
878+uriThunkPtr uri mPrivate mbranch mcommit = do
853879 commit <- case mcommit of
854880 Nothing -> gitGetCommitBranch uri mbranch >>= return . snd
855881 (Just c) -> return c
856− (src, rev) <- case uriToThunkSource uri mbranch of
882+ (src, rev) <- uriToThunkSource uri mPrivate mbranch >>= \case
857883 ThunkSource_GitHub s -> do
858884 rev <- runExceptT $ githubThunkRev s commit
859885 case rev of
@@ -876,8 +902,8 @@ uriThunkPtr uri mbranch mcommit = do
876902-- If the thunk is a GitHub thunk and fails, we do *not* fall back like with
877903-- `uriThunkPtr`. Unlike a plain URL, a thunk src explicitly states which method
878904-- should be employed, and so we respect that.
879−uriToThunkSource :: GitUri -> Maybe Text -> ThunkSource
880−uriToThunkSource (GitUri u)
905+uriToThunkSource :: MonadObelisk m => GitUri -> Maybe Bool -> Maybe Text -> m ThunkSource
906+uriToThunkSource (GitUri u) mPrivate
881907 | Right uriAuth <- URI.uriAuthority u
882908 , Just scheme <- URI.unRText <$> URI.uriScheme u
883909 , case scheme of
@@ -889,21 +915,53 @@ uriToThunkSource (GitUri u)
889915 s -> s `L.elem` [ "git", "https", "http" ] -- "http:" just redirects to "https:"
890916 && URI.unRText (URI.authHost uriAuth) == "github.com"
891917 , Just (_, owner :| [repoish]) <- URI.uriPath u
892− = \mbranch -> ThunkSource_GitHub $ GitHubSource
893− { _gitHubSource_owner = N $ URI.unRText owner
894− , _gitHubSource_repo = N $ let
895− repoish' = URI.unRText repoish
896− in fromMaybe repoish' $ T.stripSuffix ".git" repoish'
897− , _gitHubSource_branch = N <$> mbranch
898− , _gitHubSource_private = False -- TODO: Can we try something to infer this?
899− }
918+ = \mbranch -> do
919+ isPrivate <- getIsPrivate
920+ pure $ ThunkSource_GitHub $ GitHubSource
921+ { _gitHubSource_owner = N $ URI.unRText owner
922+ , _gitHubSource_repo = N $ let
923+ repoish' = URI.unRText repoish
924+ in fromMaybe repoish' $ T.stripSuffix ".git" repoish'
925+ , _gitHubSource_branch = N <$> mbranch
926+ , _gitHubSource_private = isPrivate
927+ }
900928
901− | otherwise = \mbranch -> ThunkSource_Git $ GitSource
902− { _gitSource_url = GitUri u
903− , _gitSource_branch = N <$> mbranch
904− , _gitSource_fetchSubmodules = False -- TODO: How do we determine if this should be true?
905− , _gitSource_private = False -- TODO: Can we try something to infer this?
906− }
929+ | otherwise = \mbranch -> do
930+ isPrivate <- getIsPrivate
931+ pure $ ThunkSource_Git $ GitSource
932+ { _gitSource_url = GitUri u
933+ , _gitSource_branch = N <$> mbranch
934+ , _gitSource_fetchSubmodules = False -- TODO: How do we determine if this should be true?
935+ , _gitSource_private = isPrivate
936+ }
937+ where
938+ getIsPrivate = maybe (guessGitRepoIsPrivate $ GitUri u) pure mPrivate
939+
940+guessGitRepoIsPrivate :: MonadObelisk m => GitUri -> m Bool
941+guessGitRepoIsPrivate uri = flip fix urisToTry $ \loop -> \case
942+ [] -> pure True
943+ uriAttempt:xs -> do
944+ result <- readCreateProcessWithExitCode $
945+ isolateGitProc $
946+ gitProcNoRepo
947+ [ "ls-remote"
948+ , "--quiet"
949+ , "--exit-code"
950+ , "--symref"
951+ , T.unpack $ gitUriToText uriAttempt
952+ ]
953+ case result of
954+ (ExitSuccess, _, _) -> pure False -- Must be a public repo
955+ _ -> loop xs
956+ where
957+ urisToTry = nubOrd $
958+ -- Include the original URI if it isn't using SSH because SSH will certainly fail.
959+ [uri | fmap URI.unRText (URI.uriScheme (unGitUri uri)) /= Just "ssh"] <>
960+ [changeScheme "https" uri, changeScheme "http" uri]
961+ changeScheme scheme (GitUri u) = GitUri $ u
962+ { URI.uriScheme = URI.mkScheme scheme
963+ , URI.uriAuthority = (\x -> x { URI.authUserInfo = Nothing }) <$> URI.uriAuthority u
964+ }
907965
908966-- Funny signature indicates no effects depend on the optional branch name.
909967githubThunkRev
@@ -963,6 +1021,7 @@ gitThunkRev s commit = do
9631021--
9641022-- If the branch name is passed in, it is returned exactly as-is. If it is not
9651023-- passed it, the default branch of the repo is used instead.
1024+
9661025gitGetCommitBranch
9671026 :: MonadObelisk m => GitUri -> Maybe Text -> m (Text, CommitId)
9681027gitGetCommitBranch uri mbranch = withExitFailMessage ("Failure for git remote " <> uriMsg) $ do
lib/command/src/Obelisk/Command/Utils.hs+18−7
@@ -21,8 +21,6 @@ import Data.Maybe (maybeToList)
2121import Data.Text (Text)
2222import qualified Data.Text as T
2323import Data.Void (Void)
24−import System.Directory (canonicalizePath)
25−import System.Environment (getExecutablePath)
2624import System.Exit (ExitCode)
2725import System.Which (staticWhich)
2826import Text.Megaparsec as MP
@@ -31,12 +29,24 @@ import Text.Megaparsec.Char as MP
3129import Obelisk.App (MonadObelisk)
3230import Obelisk.CliApp
3331
34−getObeliskExe :: IO FilePath
35−getObeliskExe = getExecutablePath >>= canonicalizePath
36−
3732cp :: FilePath
3833cp = $(staticWhich "cp")
3934
35+ghcidExePath :: FilePath
36+ghcidExePath = $(staticWhich "ghcid")
37+
38+findExePath :: FilePath
39+findExePath = $(staticWhich "find")
40+
41+nixExePath :: FilePath
42+nixExePath = $(staticWhich "nix")
43+
44+nixBuildExePath :: FilePath
45+nixBuildExePath = $(staticWhich "nix-build")
46+
47+jreKeyToolPath :: FilePath
48+jreKeyToolPath = $(staticWhich "keytool")
49+
4050-- Check whether the working directory is clean
4151checkGitCleanStatus :: MonadObelisk m => FilePath -> Bool -> m Bool
4252checkGitCleanStatus repo withIgnored = do
@@ -81,8 +91,9 @@ isolateGitProc = setEnvOverride (overrides <>)
8191 overrides = M.fromList
8292 [ ("HOME", "/dev/null")
8393 , ("GIT_CONFIG_NOSYSTEM", "1")
84− , ("GIT_TERMINAL_PROMPT", "0")
85− , ("GIT_SSH_COMMAND", "ssh -o PreferredAuthentications=password -o PubkeyAuthentication=no -o GSSAPIAuthentication=no")
94+ , ("GIT_TERMINAL_PROMPT", "0") -- git 2.3+
95+ , ("GIT_ASKPASS", "echo") -- pre git 2.3 to just use empty password
96+ , ("GIT_SSH_COMMAND", "ssh -o PreferredAuthentications password -o PubkeyAuthentication no -o GSSAPIAuthentication no")
8697 ]
8798
8899-- | Recursively copy a directory using `cp -a` -- TODO: Should use -rT instead of -a
lib/executable-config/lookup/src-other/Obelisk/ExecutableConfig/Lookup.hs+1−8
@@ -1,18 +1,11 @@
1−{-# LANGUAGE LambdaCase #-}
21{-# LANGUAGE OverloadedStrings #-}
32module Obelisk.ExecutableConfig.Lookup where
43
54import Data.Map (Map)
6−import qualified Data.Map as Map
75import Data.Text (Text)
8−import qualified Data.Text as T
96import Data.ByteString (ByteString)
10−import qualified Data.ByteString as BS
11−import System.Directory
12−import System.FilePath
13−import Control.Monad
147
15−import Obelisk.Configs.Internal.Directory
8+import Obelisk.Configs.Internal.Directory (getConfigsFromDirectory)
169
1710getConfigs :: IO (Map Text ByteString)
1811getConfigs = getConfigsFromDirectory "config"
lib/frontend/src/Obelisk/Frontend.hs+8−8
@@ -5,6 +5,7 @@
55{-# LANGUAGE LambdaCase #-}
66{-# LANGUAGE MultiParamTypeClasses #-}
77{-# LANGUAGE OverloadedStrings #-}
8+{-# LANGUAGE PackageImports #-}
89{-# LANGUAGE RankNTypes #-}
910{-# LANGUAGE RecursiveDo #-}
1011{-# LANGUAGE ScopedTypeVariables #-}
@@ -19,6 +20,7 @@ module Obelisk.Frontend
1920 , renderFrontendHtml
2021 , removeHTMLConfigs
2122 , FrontendMode (..)
23+ , FrontendWidgetT
2224 , module Obelisk.Frontend.Cookie
2325 ) where
2426
@@ -45,7 +47,7 @@ import qualified GHCJS.DOM.History as DOM
4547import qualified GHCJS.DOM.Window as DOM
4648import Language.Javascript.JSaddle (MonadJSM, JSM, jsNull)
4749import GHCJS.DOM (currentDocument)
48−import GHCJS.DOM.Document (getHead)
50+import "ghcjs-dom" GHCJS.DOM.Document (getHead)
4951import GHCJS.DOM.Node (Node, removeChild_)
5052import GHCJS.DOM.NodeList (IsNodeList, item, getLength)
5153import GHCJS.DOM.ParentNode (querySelectorAll)
@@ -197,7 +199,6 @@ runFrontendWithConfigsAndCurrentRoute mode configs validFullEncoder frontend = d
197199 , PrimMonad m
198200 , MonadSample DomTimeline (Performable m)
199201 , DOM.MonadJSM m
200− , Monad (Performable (Client (HydrationDomBuilderT s DomTimeline m)))
201202 , MonadFix (Client (HydrationDomBuilderT s DomTimeline m))
202203 , MonadFix (Performable m)
203204 , MonadFix m
@@ -226,18 +227,17 @@ runFrontendWithConfigsAndCurrentRoute mode configs validFullEncoder frontend = d
226227 then runHydrationWidgetWithHeadAndBody (pure ()) w
227228 else runImmediateWidgetWithHeadAndBody w
228229
230+type FrontendWidgetT r = RoutedT DomTimeline r (SetRouteT DomTimeline r (RouteToUrlT r (ConfigsT (CookiesT (HydratableT (PostBuildT DomTimeline (StaticDomBuilderT DomTimeline (PerformEventT DomTimeline DomHost))))))))
231+
229232renderFrontendHtml
230− :: ( t ~ DomTimeline
231− , MonadIO m
232− , widget ~ RoutedT t r (SetRouteT t r (RouteToUrlT r (ConfigsT (CookiesT (HydratableT (PostBuildT t (StaticDomBuilderT t (PerformEventT t DomHost))))))))
233− )
233+ :: MonadIO m
234234 => Map Text ByteString
235235 -> Cookies
236236 -> (r -> Text)
237237 -> r
238238 -> Frontend r
239− -> widget ()
240− -> widget ()
239+ -> FrontendWidgetT r ()
240+ -> FrontendWidgetT r ()
241241 -> m ByteString
242242renderFrontendHtml configs cookies urlEnc route frontend headExtra bodyExtra = do
243243 --TODO: We should probably have a "NullEventWriterT" or a frozen reflex timeline
lib/frontend/src/Obelisk/Frontend/Cookie.hs+2−1
@@ -6,6 +6,7 @@
66{-# LANGUAGE FlexibleInstances #-}
77{-# LANGUAGE GeneralizedNewtypeDeriving #-}
88{-# LANGUAGE MultiParamTypeClasses #-}
9+{-# LANGUAGE PackageImports #-}
910{-# LANGUAGE RankNTypes #-}
1011{-# LANGUAGE TypeFamilies #-}
1112{-# LANGUAGE UndecidableInstances #-}
@@ -22,7 +23,7 @@ import Data.Text.Encoding (encodeUtf8)
2223import Reflex
2324import Reflex.Host.Class
2425import Reflex.Dom.Core
25−import GHCJS.DOM.Document (getCookie, Document)
26+import "ghcjs-dom" GHCJS.DOM.Document (getCookie, Document)
2627import GHCJS.DOM.Types (MonadJSM)
2728import Web.Cookie
2829
lib/route/src/Obelisk/Route.hs+11−4
@@ -20,10 +20,11 @@
2020{-# LANGUAGE ViewPatterns #-}
2121module Obelisk.Route
2222 ( R
23− , pattern (:/)
2423 , (:.)
25− , pattern (:.)
24+ , (?/)
2625 , hoistR
26+ , pattern (:.)
27+ , pattern (:/)
2728 , PageName
2829 , PathQuery
2930 , Encoder
@@ -194,10 +195,16 @@ import Data.Aeson (FromJSON, ToJSON)
194195
195196type R f = DSum f Identity --TODO: Better name
196197
197−{-# COMPLETE (:/) #-}
198−infixr 5 :/
198+-- | Convenience builder for an 'R' using 'Identity' for the functor.
199199pattern (:/) :: f a -> a -> R f
200200pattern a :/ b = a :=> Identity b
201+{-# COMPLETE (:/) #-}
202+infixr 5 :/
203+
204+-- | Like '(:/)' but adds a 'Just' wrapper around the right-hand side.
205+(?/) :: f (Maybe a) -> a -> R f
206+r ?/ a = r :/ Just a
207+infixr 5 ?/
201208
202209mapSome :: (forall a. f a -> g a) -> Some f -> Some g
203210mapSome f (Some a) = Some $ f a
lib/route/src/Obelisk/Route/Frontend.hs+80−16
@@ -7,15 +7,11 @@
77{-# LANGUAGE FlexibleContexts #-}
88{-# LANGUAGE FlexibleInstances #-}
99{-# LANGUAGE GADTs #-}
10−{-# LANGUAGE KindSignatures #-}
1110{-# LANGUAGE LambdaCase #-}
12−{-# LANGUAGE MultiParamTypeClasses #-}
1311{-# LANGUAGE OverloadedStrings #-}
1412{-# LANGUAGE PatternSynonyms #-}
1513{-# LANGUAGE RankNTypes #-}
1614{-# LANGUAGE ScopedTypeVariables #-}
17−{-# LANGUAGE TemplateHaskell #-}
18−{-# LANGUAGE TupleSections #-}
1915{-# LANGUAGE TypeFamilies #-}
2016{-# LANGUAGE TypeOperators #-}
2117{-# LANGUAGE ViewPatterns #-}
@@ -64,7 +60,7 @@ import Prelude hiding ((.), id)
6460import Obelisk.Route
6561
6662import Control.Category (Category (..), (.))
67−import Control.Category.Cartesian
63+import Control.Category.Cartesian ((&&&))
6864import Control.Lens hiding (Bifunctor, bimap, universe, element)
6965import Control.Monad ((<=<))
7066import Control.Monad.Fix
@@ -94,7 +90,9 @@ import Reflex.Dom.Builder.Class
9490import Data.Type.Coercion
9591import Language.Javascript.JSaddle --TODO: Get rid of this - other platforms can also be routed
9692import Reflex.Dom.Core
93+import qualified GHCJS.DOM as DOM
9794import qualified GHCJS.DOM.Types as DOM
95+import qualified GHCJS.DOM.Window as Window
9896import Network.URI
9997import Data.Maybe (fromMaybe)
10098import qualified Data.List as L
@@ -511,59 +509,125 @@ runRouteViewT routeEncoder switchover useHash a = do
511509-- | A link widget that, when clicked, sets the route to the provided route. In non-javascript
512510-- contexts, this widget falls back to using @href@s to control navigation
513511routeLink
514− :: forall t m a route.
512+ :: forall t m a route js.
515513 ( DomBuilder t m
516514 , RouteToUrl route m
517515 , SetRoute t route m
516+ , Prerender js t m
518517 )
519518 => route -- ^ Target route
520519 -> m a -- ^ Child widget
521520 -> m a
522521routeLink r w = do
522+ (e, a) <- routeLinkImpl r w
523+ scrollToTop e
524+ return a
525+
526+-- | Raw implementation of 'routeLink'. Does not scroll to the top of the page on clicks.
527+routeLinkImpl
528+ :: forall t m a route.
529+ ( DomBuilder t m
530+ , RouteToUrl route m
531+ , SetRoute t route m
532+ )
533+ => route -- ^ Target route
534+ -> m a -- ^ Child widget
535+ -> m (Event t (), a)
536+routeLinkImpl r w = do
523537 enc <- askRouteToUrl
524538 let cfg = (def :: ElementConfig EventResult t (DomBuilderSpace m))
525539 & elementConfig_eventSpec %~ addEventSpecFlags (Proxy :: Proxy (DomBuilderSpace m)) Click (\_ -> preventDefault)
526540 & elementConfig_initialAttributes .~ "href" =: enc r
527541 (e, a) <- element "a" cfg w
528542 setRoute $ r <$ domEvent Click e
529− return a
543+ return (domEvent Click e, a)
544+
545+scrollToTop :: forall m t js. (Prerender js t m, Monad m) => Event t () -> m ()
546+scrollToTop e = prerender_ blank $ performEvent_ $ ffor e $ \_ -> liftJSM $ DOM.currentWindow >>= \case
547+ Nothing -> pure ()
548+ Just win -> Window.scrollTo win 0 0
530549
531550-- | Like 'routeLinkDynAttr' but without custom attributes.
532551dynRouteLink
533− :: forall t m a route.
552+ :: forall t m a route js.
534553 ( DomBuilder t m
535554 , PostBuild t m
536− , RouteToUrl (R route) m
537− , SetRoute t (R route) m
555+ , RouteToUrl route m
556+ , SetRoute t route m
557+ , Prerender js t m
538558 )
539− => Dynamic t (R route) -- ^ Target route
540− -> m a -- ^ Child widget of the @a@ element
559+ => Dynamic t route -- ^ Target route
560+ -> m a -- ^ Child widget
541561 -> m a
542−dynRouteLink = routeLinkDynAttr mempty
562+dynRouteLink r w = do
563+ (e, a) <- dynRouteLinkImpl r w
564+ scrollToTop e
565+ return a
566+
567+-- | Raw implementation of 'dynRouteLink'. Does not scroll to the top of the page on clicks.
568+dynRouteLinkImpl
569+ :: forall t m a route.
570+ ( DomBuilder t m
571+ , PostBuild t m
572+ , RouteToUrl route m
573+ , SetRoute t route m
574+ )
575+ => Dynamic t route -- ^ Target route
576+ -> m a -- ^ Child widget
577+ -> m (Event t (), a)
578+dynRouteLinkImpl dr w = do
579+ enc <- askRouteToUrl
580+ er <- dynamicAttributesToModifyAttributes $ ("href" =:) . enc <$> dr
581+ let cfg = (def :: ElementConfig EventResult t (DomBuilderSpace m))
582+ & elementConfig_eventSpec %~ addEventSpecFlags (Proxy :: Proxy (DomBuilderSpace m)) Click (\_ -> preventDefault)
583+ & elementConfig_modifyAttributes .~ er
584+ (e, a) <- element "a" cfg w
585+ let clk = domEvent Click e
586+ setRoute $ tag (current dr) clk
587+ return (clk, a)
543588
544589-- | An @a@-tag link widget that, when clicked, sets the route to current value of the
545590-- provided dynamic route. In non-JavaScript contexts the value of the dynamic post
546591-- build is used so the link still works like 'routeLink'.
547592routeLinkDynAttr
548− :: forall t m a route.
593+ :: forall t m a route js.
549594 ( DomBuilder t m
550595 , PostBuild t m
551596 , RouteToUrl (R route) m
552597 , SetRoute t (R route) m
598+ , Prerender js t m
553599 )
554600 => Dynamic t (Map AttributeName Text) -- ^ Attributes for @a@ element. Note that if @href@ is present it will be ignored
555601 -> Dynamic t (R route) -- ^ Target route
556602 -> m a -- ^ Child widget of the @a@ element
557603 -> m a
558604routeLinkDynAttr dAttr dr w = do
605+ (e, a) <- routeLinkDynAttrImpl dAttr dr w
606+ scrollToTop e
607+ return a
608+
609+-- | Raw implementation of 'routeLinkDynAttr'. Does not scroll to the top of the page on clicks.
610+routeLinkDynAttrImpl
611+ :: forall t m a route.
612+ ( DomBuilder t m
613+ , PostBuild t m
614+ , RouteToUrl (R route) m
615+ , SetRoute t (R route) m
616+ )
617+ => Dynamic t (Map AttributeName Text) -- ^ Attributes for @a@ element. Note that if @href@ is present it will be ignored
618+ -> Dynamic t (R route) -- ^ Target route
619+ -> m a -- ^ Child widget of the @a@ element
620+ -> m (Event t (), a)
621+routeLinkDynAttrImpl dAttr dr w = do
559622 enc <- askRouteToUrl
560623 er <- dynamicAttributesToModifyAttributes $ zipDynWith (<>) (("href" =:) . enc <$> dr) dAttr
561624 let cfg = (def :: ElementConfig EventResult t (DomBuilderSpace m))
562625 & elementConfig_eventSpec %~ addEventSpecFlags (Proxy :: Proxy (DomBuilderSpace m)) Click (\_ -> preventDefault)
563626 & elementConfig_modifyAttributes .~ er
564627 (e, a) <- element "a" cfg w
565− setRoute $ tag (current dr) $ domEvent Click e
566− return a
628+ let clk = domEvent Click e
629+ setRoute $ tag (current dr) clk
630+ return (clk, a)
567631
568632-- On ios due to sandboxing when loading the page from a file adapt the
569633-- path to be based on the hash.
lib/run/src/Obelisk/Run.hs+11−6
@@ -79,7 +79,7 @@ import qualified System.Which
7979run
8080 :: Int -- ^ Port to run the backend
8181 -> ([Text] -> Snap ()) -- ^ Static asset handler
82− -> Backend fullRoute frontendRoute -- ^ Backend
82+ -> Backend backendRoute frontendRoute -- ^ Backend
8383 -> Frontend (R frontendRoute) -- ^ Frontend
8484 -> IO ()
8585run port serveStaticAsset backend frontend = do
@@ -90,14 +90,18 @@ run port serveStaticAsset backend frontend = do
9090 Left e -> hPutStrLn stderr $ "backend error:\n" <> T.unpack e
9191 Right validFullEncoder -> do
9292 publicConfigs <- getPublicConfigs
93− backendTid <- forkIO $ handle handleBackendErr $ withArgs ["--quiet", "--port", show port] $ do
94− _backend_run backend $ \serveRoute -> do
95− runSnapWithCommandLineArgs $ do
93+ backendTid <- forkIO $ handle handleBackendErr $ withArgs ["--quiet", "--port", show port] $
94+ _backend_run backend $ \serveRoute ->
95+ runSnapWithCommandLineArgs $
9696 getRouteWith validFullEncoder >>= \case
9797 Identity r -> case r of
9898 FullRoute_Backend backendRoute :/ a -> serveRoute $ backendRoute :/ a
9999 FullRoute_Frontend obeliskRoute :/ a ->
100− serveDefaultObeliskApp (mkRouteToUrl validFullEncoder) serveStaticAsset frontend publicConfigs $ obeliskRoute :/ a
100+ serveDefaultObeliskApp appRouteToUrl (($ allJsUrl) <$> defaultGhcjsWidgets) serveStaticAsset frontend publicConfigs $ obeliskRoute :/ a
101+ where
102+ appRouteToUrl (k :/ v) = renderObeliskRoute validFullEncoder (FullRoute_Frontend (ObeliskRoute_App k) :/ v)
103+ allJsUrl = renderAllJsPath validFullEncoder
104+
101105 let conf = defRunConfig { _runConfig_redirectPort = port }
102106 runWidget conf publicConfigs frontend validFullEncoder `finally` killThread backendTid
103107
@@ -189,7 +193,8 @@ obeliskApp configs opts frontend validFullEncoder uri backend = do
189193 }
190194 FullRoute_Frontend (ObeliskRoute_App appRouteComponent) :/ appRouteRest -> do
191195 let cookies = maybe [] parseCookies $ lookup (fromString "Cookie") (W.requestHeaders req)
192− html <- renderJsaddleFrontend configs cookies (mkRouteToUrl validFullEncoder) (appRouteComponent :/ appRouteRest) frontend
196+ routeToUrl (k :/ v) = renderObeliskRoute validFullEncoder $ FullRoute_Frontend (ObeliskRoute_App k) :/ v
197+ html <- renderJsaddleFrontend configs cookies routeToUrl (appRouteComponent :/ appRouteRest) frontend
193198 sendResponse $ W.responseLBS H.status200 [("Content-Type", staticRenderContentType)] $ BSLC.fromStrict html
194199 _ -> backend req sendResponse
195200
lib/selftest/src/Obelisk/SelfTest.hs+130−78
@@ -1,3 +1,4 @@
1+{-# LANGUAGE DoAndIfThenElse #-}
12{-# LANGUAGE EmptyCase #-}
23{-# LANGUAGE LambdaCase #-}
34{-# LANGUAGE OverloadedStrings #-}
@@ -23,13 +24,12 @@ import qualified Network.HTTP.Client as HTTP
2324import qualified Network.HTTP.Types as HTTP
2425import qualified Network.Socket as Socket
2526import Shelly hiding (cp)
26−import System.Directory (withCurrentDirectory, getDirectoryContents)
27+import System.Directory (getCurrentDirectory, withCurrentDirectory, getDirectoryContents)
2728import System.Environment
2829import System.Exit (ExitCode (..))
29−import System.FilePath (replaceBaseName, takeBaseName)
3030import System.Which (staticWhich)
3131import qualified System.Info
32−import System.IO (Handle, hClose)
32+import System.IO (Handle, hClose, hIsEOF, hGetContents)
3333import System.IO.Temp
3434import System.Process (readProcessWithExitCode, CreateProcess(cwd), readCreateProcessWithExitCode, proc)
3535import Test.Hspec
@@ -52,11 +52,32 @@ cp = $(staticWhich "cp")
5252cabalPath :: FilePath
5353cabalPath = $(staticWhich "cabal")
5454
55+gitPath :: FilePath
56+gitPath = $(staticWhich "git")
57+
58+chownPath :: FilePath
59+chownPath = $(staticWhich "chown")
60+
61+chmodPath :: FilePath
62+chmodPath = $(staticWhich "chmod")
63+
64+whoamiPath :: FilePath
65+whoamiPath = $(staticWhich "whoami")
66+
67+nixBuildPath :: FilePath
68+nixBuildPath = $(staticWhich "nix-build")
69+
70+lnPath :: FilePath
71+lnPath = $(staticWhich "ln")
72+
73+rmPath :: FilePath
74+rmPath = $(staticWhich "rm")
75+
5576gitUserConfig :: [Text]
5677gitUserConfig = ["-c", "user.name=Obelisk Selftest", "-c", "user.email=noreply@example.com"]
5778
5879commit :: Text -> Sh ()
59−commit msg = void $ run "git" $ gitUserConfig <> [ "commit"
80+commit msg = void $ run gitPath $ gitUserConfig <> [ "commit"
6081 , "--no-gpg-sign"
6182 , "--allow-empty"
6283 , "-m"
@@ -76,6 +97,14 @@ tshow = T.pack . show
7697shellyOb :: MonadIO m => (Sh a -> Sh a) -> Sh a -> m a
7798shellyOb f obTest = shelly $ f obTest
7899
100+-- Set "ob" in a single place so it can be replaced with a
101+-- link to obelisk in the nix store in the future,
102+-- and avoid PATH hacking before calling this script.
103+ob :: FilePath
104+ob = "ob"
105+
106+augmentWithVerbosity :: (String -> [Text] -> a) -> String -> Bool -> [Text] -> a
107+augmentWithVerbosity runner executable isVerbose args = runner executable $ (if isVerbose then ("-v" :) else id) args
79108
80109main :: IO ()
81110main = do
@@ -84,33 +113,43 @@ main = do
84113 unless isVerbose $
85114 putStrLn "Tests may take longer to run if there are unbuilt derivations: use -v for verbose output"
86115 let verbosity = bool silently verbosely isVerbose
87− nixBuild args = run "nix-build" ("--no-out-link" : args)
88− obeliskImplRaw <- fromString <$> getEnv "OBELISK_IMPL"
116+ nixBuild args = run nixBuildPath ("--no-out-link" : args)
117+ obeliskImplDirtyReadOnly <- fromString <$> getCurrentDirectory
118+ let runOb_ = augmentWithVerbosity run_ ob isVerbose
119+ let runOb = augmentWithVerbosity run ob isVerbose
120+ let testObRunInDir' = augmentWithVerbosity testObRunInDir ob isVerbose ["run"]
121+ let testThunkPack' = augmentWithVerbosity testThunkPack ob isVerbose []
89122 let
90− withObeliskImpl f =
123+ withObeliskImplClean f =
124+ withSystemTempDirectory "obelisk-impl-clean" $ \(fromString -> obeliskImpl) -> do
125+ void . shellyOb verbosity $ chdir obeliskImpl $ do
126+ run_ gitPath ["clone", "file://" <> toTextIgnore obeliskImplDirtyReadOnly, toTextIgnore obeliskImpl]
127+ f obeliskImpl
128+
129+ withObeliskImplDirty f =
91130 withSystemTempDirectory "obelisk-impl-copy" $ \(fromString -> obeliskImpl) -> do
92131 void . shellyOb verbosity $ chdir obeliskImpl $ do
93− user <- T.strip <$> run "whoami" []
94− run_ cp ["-rT", toTextIgnore obeliskImplRaw, toTextIgnore obeliskImpl]
95− run_ "chown" ["-R", user, toTextIgnore obeliskImpl]
96− run_ "chmod" ["-R", "g-rw,o-rw", toTextIgnore obeliskImpl]
132+ user <- T.strip <$> run whoamiPath []
133+ run_ cp ["-rT", "--no-preserve=mode", toTextIgnore obeliskImplDirtyReadOnly, toTextIgnore obeliskImpl]
134+ run_ chownPath ["-R", user, toTextIgnore obeliskImpl]
135+ run_ chmodPath ["-R", "g-rw,o-rw", toTextIgnore obeliskImpl]
97136 f obeliskImpl
98137
99138 withInitCache f obeliskImpl =
100139 withSystemTempDirectory "init Cache λ" $ \initCache -> do
101140 -- Setup the ob init cache
102141 void . shellyOb verbosity $ chdir initCache $ do
103− run_ "ob" ["init", "--symlink", toTextIgnore obeliskImpl]
104− run_ "git" ["init"]
142+ runOb_ ["init", "--symlink", toTextIgnore obeliskImpl]
143+ run_ gitPath ["init"]
105144
106145 f initCache
107146
108− withObeliskImplAndInitCache f =
109− withObeliskImpl $ \impl -> withInitCache (f impl) impl
147+ withObeliskImplDirtyAndInitCache f =
148+ withObeliskImplDirty $ \impl -> withInitCache (f impl) impl
110149
111150 httpManager <- HTTP.newManager HTTP.defaultManagerSettings
112151
113− withObeliskImplAndInitCache $ \obeliskImpl initCache ->
152+ withObeliskImplDirtyAndInitCache $ \obeliskImplDirty initCache ->
114153 hspec $ parallel $ do
115154 let shelly_ = void . shellyOb verbosity
116155
@@ -130,41 +169,41 @@ main = do
130169
131170 -- To be used in tests that change the obelisk impl directory
132171 inTmpObInitWithImplCopy f = inTmpObInit $ \dir ->
133− withObeliskImpl $ \(fromString -> implCopy) -> do
134− run_ "rm" [thunk]
135− run_ "ln" ["-s", implCopy, thunk]
172+ withObeliskImplClean $ \(fromString -> implClean) -> do
173+ run_ rmPath [thunk]
174+ run_ lnPath ["-s", implClean, thunk]
136175 f dir
137176
138177 assertRevEQ a b = liftIO . assertEqual "" "" =<< diff a b
139178 assertRevNE a b = liftIO . assertBool "" . (/= "") =<< diff a b
140179
141− revParseHead = T.strip <$> run "git" ["rev-parse", "HEAD"]
180+ revParseHead = T.strip <$> run gitPath ["rev-parse", "HEAD"]
142181
143182 commitAll = do
144− run_ "git" ["add", "."]
183+ run_ gitPath ["add", "."]
145184 commit "checkpoint"
146185 revParseHead
147186
148187 thunk = ".obelisk/impl"
149− update = run "ob" ["thunk", "update", thunk] >> commitAll
150− pack = run "ob" ["thunk", "pack", thunk] >> commitAll
151− unpack = run "ob" ["thunk", "unpack", thunk] >> commitAll
188+ update = runOb ["thunk", "update", thunk] >> commitAll
189+ pack = runOb ["thunk", "pack", thunk] >> commitAll
190+ unpack = runOb ["thunk", "unpack", thunk] >> commitAll
152191
153− diff a b = run "git" ["diff", a, b]
192+ diff a b = run gitPath ["diff", a, b]
154193
155194 describe "ob init" $ parallel $ do
156− it "works with default impl" $ inTmp $ \_ -> run "ob" ["init"]
157− it "works with master branch impl" $ inTmp $ \_ -> run "ob" ["init", "--branch", "master"]
158− it "works with symlink" $ inTmp $ \_ -> run "ob" ["init", "--symlink", toTextIgnore obeliskImpl]
195+ it "works with default impl" $ inTmp $ \_ -> runOb ["init"]
196+ it "works with master branch impl" $ inTmp $ \_ -> runOb ["init", "--branch", "master"]
197+ it "works with symlink" $ inTmp $ \_ -> runOb ["init", "--symlink", toTextIgnore obeliskImplDirty]
159198 it "doesn't silently overwrite existing files" $ withSystemTempDirectory "ob-init λ" $ \dir -> do
160− let p force = (System.Process.proc "ob" $ "--no-handoff" : "init" : ["--force"|force]) { cwd = Just dir }
199+ let p force = (System.Process.proc ob $ "--no-handoff" : "-v" : "init" : ["--force"|force]) { cwd = Just dir }
161200 (ExitSuccess, _, _) <- readCreateProcessWithExitCode (p False) ""
162201 (ExitFailure _, _, _) <- readCreateProcessWithExitCode (p False) ""
163202 (ExitSuccess, _, _) <- readCreateProcessWithExitCode (p True) ""
164203 pure ()
165204
166205 it "doesn't create anything when given an invalid impl" $ inTmp $ \tmp -> do
167− void $ errExit False $ run "ob" ["init", "--symlink", "/dev/null"]
206+ void $ errExit False $ runOb ["init", "--symlink", "/dev/null"]
168207 ls tmp >>= liftIO . assertEqual "" []
169208
170209 it "produces a valid route config" $ inTmpObInit $ \tmp -> liftIO $
@@ -175,17 +214,14 @@ main = do
175214 -- These tests fail with "Could not find module 'Obelisk.Generated.Static'"
176215 -- when not run by 'nix-build --attr selftest'
177216 describe "ob run" $ {- NOT parallel $ -} do
178− it "works in root directory" $ inTmpObInit $ \_ -> testObRunInDir Nothing httpManager
179− it "works in sub directory" $ inTmpObInit $ \_ -> testObRunInDir (Just "frontend") httpManager
180− it "works with differently named cabal files" $ inTmpObInit $ \_ -> do
181− changeCabalPackageName "backend/backend.cabal" "new-backend"
182− testObRunInDir Nothing httpManager
217+ it "works in root directory" $ inTmpObInit $ \_ -> testObRunInDir' Nothing httpManager
218+ it "works in sub directory" $ inTmpObInit $ \_ -> testObRunInDir' (Just "frontend") httpManager
183219
184220 describe "obelisk project" $ parallel $ do
185− it "can build obelisk command" $ inTmpObInit $ \_ -> nixBuild ["-A", "command" , toTextIgnore obeliskImpl]
186− it "can build obelisk skeleton" $ inTmpObInit $ \_ -> nixBuild ["-A", "skeleton", toTextIgnore obeliskImpl]
187− it "can build obelisk shell" $ inTmpObInit $ \_ -> nixBuild ["-A", "shell", toTextIgnore obeliskImpl]
188− it "can build everything" $ inTmpObInit $ \_ -> nixBuild [toTextIgnore obeliskImpl]
221+ it "can build obelisk command" $ inTmpObInit $ \_ -> nixBuild ["-A", "command" , toTextIgnore obeliskImplDirty]
222+ it "can build obelisk skeleton" $ inTmpObInit $ \_ -> nixBuild ["-A", "skeleton", toTextIgnore obeliskImplDirty]
223+ it "can build obelisk shell" $ inTmpObInit $ \_ -> nixBuild ["-A", "shell", toTextIgnore obeliskImplDirty]
224+ it "can build everything" $ inTmpObInit $ \_ -> nixBuild [toTextIgnore obeliskImplDirty]
189225
190226 describe "blank initialized project" $ parallel $ do
191227
@@ -235,76 +271,93 @@ main = do
235271
236272 it "unpacks the correct branch" $ withTmp $ \dir -> do
237273 let branch = "master"
238− run_ "git" ["clone", "https://github.com/reflex-frp/reflex.git", toTextIgnore dir, "--branch", branch]
239− run_ "ob" ["thunk", "pack", toTextIgnore dir]
240− run_ "ob" ["thunk", "unpack", toTextIgnore dir]
241− branch' <- chdir dir $ run "git" ["rev-parse", "--abbrev-ref", "HEAD"]
274+ run_ gitPath ["clone", "https://github.com/reflex-frp/reflex.git", toTextIgnore dir, "--branch", branch]
275+ runOb_ ["thunk", "pack", toTextIgnore dir]
276+ runOb_ ["thunk", "unpack", toTextIgnore dir]
277+ branch' <- chdir dir $ run gitPath ["rev-parse", "--abbrev-ref", "HEAD"]
242278 liftIO $ assertEqual "" branch (T.strip branch')
243279
244280 it "can pack and unpack plain git repos" $
245281 shelly_ $ withSystemTempDirectory "git-repo" $ \dir -> do
246282 let repo = toTextIgnore $ dir </> ("repo" :: String)
247− run_ "git" ["clone", "https://github.com/haskell/process.git", repo]
283+ run_ gitPath ["clone", "https://github.com/haskell/process.git", repo]
248284 origHash <- chdir (fromText repo) revParseHead
249285
250− run_ "ob" ["thunk", "pack", repo]
286+ runOb_ ["thunk", "pack", repo]
251287 packedFiles <- Set.fromList <$> ls (fromText repo)
252288 liftIO $ assertEqual "" packedFiles $ Set.fromList $ (repo </>) <$>
253289 ["default.nix", "github.json", ".attr-cache" :: String]
254290
255− run_ "ob" ["thunk", "unpack", repo]
291+ runOb_ ["thunk", "unpack", repo]
256292 chdir (fromText repo) $ do
257293 unpackHash <- revParseHead
258294 assertRevEQ origHash unpackHash
259295
260− testThunkPack $ fromText repo
296+ testThunkPack' $ fromText repo
261297
262− it "aborts thunk pack when there are uncommitted files" $ inTmpObInit $ \dir -> do
263− testThunkPack (dir </> thunk)
298+ it "aborts thunk pack when there are uncommitted files" $ inTmpObInitWithImplCopy $ \dir -> do
299+ testThunkPack' (dir </> thunk)
264300
265301 describe "ob thunk update --branch" $ parallel $ do
266302 it "can change a thunk to the latest version of a desired branch" $ withTmp $ \dir -> do
267303 let branch1 = "master"
268304 branch2 = "develop"
269− run_ "git" ["clone", "https://github.com/reflex-frp/reflex.git", toTextIgnore dir, "--branch", branch1]
270− run_ "ob" ["thunk" , "pack", toTextIgnore dir]
271− run_ "ob" ["thunk", "update", toTextIgnore dir, "--branch", branch2]
305+ run_ gitPath ["clone", "https://github.com/reflex-frp/reflex.git", toTextIgnore dir, "--branch", branch1]
306+ runOb_ ["thunk" , "pack", toTextIgnore dir]
307+ runOb_ ["thunk", "update", toTextIgnore dir, "--branch", branch2]
272308
273309 it "doesn't create anything when given an invalid branch" $ withTmp $ \dir -> do
274310 let checkDir dir' = liftIO $ getDirectoryContents $ T.unpack $ toTextIgnore dir'
275− run_ "git" ["clone", "https://github.com/reflex-frp/reflex.git", toTextIgnore dir, "--branch", "master"]
276− run_ "ob" ["thunk" , "pack", toTextIgnore dir]
311+ run_ gitPath ["clone", "https://github.com/reflex-frp/reflex.git", toTextIgnore dir, "--branch", "master"]
312+ runOb_ ["thunk" , "pack", toTextIgnore dir]
277313 startingContents <- checkDir dir
278− void $ errExit False $ run "ob" ["thunk", "update", toTextIgnore dir, "--branch", "dumble-palooza"]
314+ void $ errExit False $ runOb ["thunk", "update", toTextIgnore dir, "--branch", "dumble-palooza"]
279315 checkDir dir >>= liftIO . assertEqual "" startingContents
280316
317+ describe "ob hoogle" $ {- NOT parallel -} do
318+ it "starts a hoogle server on the given port" $ inTmpObInit $ \_ -> do
319+ [p0] <- liftIO $ getFreePorts 1
320+ maskExitSuccess $ runHandle "ob" ["hoogle", "--port", T.pack (show p0)] $ \stdout -> fix $ \loop -> do
321+ ln <- liftIO $ T.hGetLine stdout
322+ echo ln
323+ let search = "Server starting on port " <> T.pack (show p0)
324+ case search `T.isInfixOf` ln of
325+ False -> loop -- keep waiting
326+ True -> do
327+ let req uri = liftIO $ HTTP.parseRequest uri >>= flip HTTP.httpLbs httpManager
328+ req ("http://127.0.0.1:" <> show p0) >>= \r -> case HTTP.responseStatus r == HTTP.ok200 of
329+ False -> errorExit $ "Request to hoogle server failed: " <> T.pack (show r)
330+ True -> exit 0
331+
332+maskExitSuccess :: Sh () -> Sh ()
333+maskExitSuccess = handle_sh (\case ExitSuccess -> pure (); e -> throw e)
281334
282335-- | Run `ob run` in the given directory (maximum of one level deep)
283−testObRunInDir :: Maybe FilePath -> HTTP.Manager -> Sh ()
284−testObRunInDir mdir httpManager = handle_sh (\case ExitSuccess -> pure (); e -> throw e) $ do
336+testObRunInDir :: String -> [Text] -> Maybe FilePath -> HTTP.Manager -> Sh ()
337+testObRunInDir executable extraArgs mdir httpManager = maskExitSuccess $ do
285338 [p0, p1] <- liftIO $ getFreePorts 2
286339 let uri p = "http://localhost:" <> T.pack (show p) <> "/" -- trailing slash required for comparison
287340 writefile "config/common/route" $ uri p0
288− maybe id chdir mdir $ runHandle "ob" ["run", "-v"] $ \stdout -> do
289− firstUri <- handleObRunStdout httpManager stdout
341+ maybe id chdir mdir $ runHandles executable extraArgs [] $ \_stdin stdout stderr -> do
342+ firstUri <- handleObRunStdout httpManager stdout stderr
290343 let newUri = uri p1
291344 when (firstUri == newUri) $ errorExit $
292345 "Startup URI (" <> firstUri <> ") is the same as test URI (" <> newUri <> ")"
293346 maybe id (\_ -> chdir "..") mdir $ alterRouteTo newUri stdout
294− runningUri <- handleObRunStdout httpManager stdout
347+ runningUri <- handleObRunStdout httpManager stdout stderr
295348 if runningUri /= newUri
296349 then errorExit $ "Reloading failed: expected " <> newUri <> " but got " <> runningUri
297350 else exit 0
298351
299−testThunkPack :: FilePath -> Sh ()
300−testThunkPack path' = withTempFile (T.unpack $ toTextIgnore path') "test-file" $ \file handle -> do
301− let pack' = readProcessWithExitCode "ob" ["thunk", "pack", T.unpack $ toTextIgnore path'] ""
352+testThunkPack :: String -> [Text] -> FilePath -> Sh ()
353+testThunkPack executable args path' = withTempFile (T.unpack $ toTextIgnore path') "test-file" $ \file handle -> do
354+ let pack' = readProcessWithExitCode executable (T.unpack <$> ["thunk", "pack", toTextIgnore path'] ++ args) ""
302355 ensureThunkPackFails q = liftIO $ pack' >>= \case
303356 (code, out, err)
304− | code == ExitSuccess -> fail "ob thunk pack succeeded when it should have failed"
357+ | code == ExitSuccess -> fail $ "ob thunk pack succeeded when it should have failed with error '" <> show q <> "'"
305358 | q `T.isInfixOf` T.pack (out <> err) -> pure ()
306− | otherwise -> fail $ "ob thunk pack failed for an unexpected reason: " <> show out <> "\nstderr: " <> err
307− git = chdir path' . run "git"
359+ | otherwise -> fail $ "ob thunk pack failed for an unexpected reason, expecting '" <> show q <> "', received: " <> show out <> "\nstderr: " <> err
360+ git = chdir path' . run gitPath
308361 -- Untracked files
309362 ensureThunkPackFails "Untracked files"
310363 void $ git ["add", T.pack file]
@@ -338,9 +391,12 @@ alterRouteTo uri stdout = do
338391 "Reloading failed: " <> T.pack (show t)
339392
340393-- | Handle stdout of `ob run`: check that the frontend and backend servers are started correctly
341−handleObRunStdout :: HTTP.Manager -> Handle -> Sh Text
342−handleObRunStdout httpManager stdout = flip fix (ObRunState_Init, []) $ \loop (state, msgs) ->
343− liftIO (T.hGetLine stdout) >>= \t -> case state of
394+handleObRunStdout :: HTTP.Manager -> Handle -> Handle -> Sh Text
395+handleObRunStdout httpManager stdout stderr = flip fix (ObRunState_Init, []) $ \loop (state, msgs) -> do
396+ isEOF <- liftIO $ hIsEOF stdout
397+ if isEOF
398+ then handleObRunError msgs
399+ else liftIO (T.hGetLine stdout) >>= \t -> case state of
344400 ObRunState_Init
345401 | "Running test..." `T.isPrefixOf` t -> loop (ObRunState_BackendStarted, msgs)
346402 ObRunState_Startup
@@ -352,8 +408,12 @@ handleObRunStdout httpManager stdout = flip fix (ObRunState_Init, []) $ \loop (s
352408 obRunCheck httpManager stdout uri
353409 pure uri
354410 | not (T.null t) -> errorExit $ "Started: " <> t -- If theres any other output here, startup failed
355− _ | "Failed" `T.isPrefixOf` t -> errorExit $ "ob run failed: " <> T.unlines (reverse $ t : msgs)
411+ _ | "Failed" `T.isPrefixOf` t -> handleObRunError (t : msgs)
356412 | otherwise -> loop (state, t : msgs)
413+ where
414+ handleObRunError msgs = do
415+ stderrContent <- liftIO $ hGetContents stderr
416+ errorExit $ "ob run failed: " <> T.unlines (reverse msgs) <> " stderr: " <> T.pack stderrContent
357417
358418-- | Make requests to frontend/backend servers to check they are working properly
359419obRunCheck :: HTTP.Manager -> Handle -> Text -> Sh ()
@@ -362,14 +422,6 @@ obRunCheck httpManager _stdout frontendUri = do
362422 req frontendUri >>= \r -> when (HTTP.responseStatus r /= HTTP.ok200) $ errorExit $
363423 "Request to frontend server failed: " <> T.pack (show r)
364424
365−-- | Rename a cabal file and do a really dumb, brittle search/replace in its content to update the name.
366−changeCabalPackageName :: FilePath -> Text -> Sh ()
367−changeCabalPackageName cabalFile newName = do
368− contents <- readfile cabalFile
369− writefile (replaceBaseName cabalFile (T.unpack newName)) $
370− T.replace (" " <> T.pack (takeBaseName cabalFile)) (" " <> newName) contents -- WARNING: Super brittle
371− rm cabalFile
372−
373425getFreePorts :: Int -> IO [Socket.PortNumber]
374426getFreePorts 0 = pure []
375427getFreePorts n = Socket.withSocketsDo $ do
lib/snap-extras/obelisk-snap-extras.cabal+1−1
@@ -1,6 +1,6 @@
11name: obelisk-snap-extras
22version: 0.1
3−synopsys: Extra functionality for Snap that should be considered for upstreaming
3+synopsis: Extra functionality for Snap that should be considered for upstreaming
44license: BSD3
55build-type: Simple
66cabal-version: >=1.2
lib/snap-extras/src/Obelisk/Snap/Extras.hs+1−1
@@ -2,13 +2,13 @@
22module Obelisk.Snap.Extras
33 ( cachePermanently
44 , doNotCache
5+ , ensureSecure
56 , serveFileIfExists
67 , serveFileIfExistsAs
78 ) where
89
910import Control.Monad.IO.Class
1011import Data.ByteString (ByteString)
11−import Data.Monoid
1212import Data.String
1313import Snap.Core
1414import Snap.Util.FileServe
skeleton/.gitignore+1
@@ -6,3 +6,4 @@ result-ios
66result-exe
77.attr-cache
88ghcid-output.txt
9+profile/
skeleton/backend/src/Backend.hs−3
@@ -1,6 +1,3 @@
1−{-# LANGUAGE FlexibleContexts #-}
2−{-# LANGUAGE RankNTypes #-}
3−{-# LANGUAGE TypeFamilies #-}
41module Backend where
52
63import Common.Route