0cd2804 · obelisk · git.iko.soy

git.iko.soy/obelisk

on master help sign in

Merge branch 'master' into routes

0cd2804Ryan Trinkle8 years ago

README.md+24−5

compare with master

unifiedsplit

@@ -5,6 +5,8 @@ Obelisk provides an easy way to develop and deploy your [Reflex](https://github.

5- [Installing Obelisk](#installing-obelisk)

6- [Developing an Obelisk project](#developing-an-obelisk-project)

7- [Deploying](#deploying)

8+ - [Locally](#locally)

9+ - [EC2](#ec2)

8- [Mobile](#mobile)

10- [Mobile](#mobile)

9 - [iOS](#ios)

11 - [iOS](#ios)

10 - [Android](#android)

12 - [Android](#android)

@@ -90,11 +92,28 @@ Every time you change the Haskell source files in frontend, common or backend, `

91## Deploying

93## Deploying

95+### Locally

96+

97+Build everything:

98+

99+```

100+nix-build -A exe -o result-exe

101+```

102+

103+Run the server:

104+

105+```

106+cd result-exe

107+./backend

108+```

109+

110+### EC2

111+

93In this section we will demonstrate how to deploy your Obelisk app to an Amazon EC2 instance.

112In this section we will demonstrate how to deploy your Obelisk app to an Amazon EC2 instance.

113

95First create a new EC2 instance:

114First create a new EC2 instance:

115

97−1. Launch a NixOS 17.09 EC2 instance (we recommend [this AMI](https://console.aws.amazon.com/ec2/v2/home?region=us-east-1#LaunchInstanceWizard:ami=ami-40bee63a))

116+1. Launch a NixOS 17.09 EC2 instance (we recommend [this AMI](https://console.aws.amazon.com/ec2/v2/home?region=us-east-1#LaunchInstanceWizard:ami=ami-40bee63a))

981. In the instance configuration wizard ensure that your instance has at least 1GB RAM and 10GB disk space.

1171. In the instance configuration wizard ensure that your instance has at least 1GB RAM and 10GB disk space.

991. When prompted save your AWS private key (`~/myaws.pem`) somewhere safe. We'll need it later during deployment.

1181. When prompted save your AWS private key (`~/myaws.pem`) somewhere safe. We'll need it later during deployment.

1001. Go to "Security Groups", select your instance's security group and under "Inbound" tab add a new rule for HTTP port 80 and 443.

1191. Go to "Security Groups", select your instance's security group and under "Inbound" tab add a new rule for HTTP port 80 and 443.

@@ -159,7 +178,7 @@ Install Xcode 8.2 (contains iOS SDK 10.2) and open it so that it runs its post i

159These versions will work out of the box but iOS SDKs prior to 11.3 should also work. You can choose another installed version in `default.nix`

178These versions will work out of the box but iOS SDKs prior to 11.3 should also work. You can choose another installed version in `default.nix`

160

179

161More recent Xcodes should also work, as long as one of the SDKs mentioned above has been used.

180More recent Xcodes should also work, as long as one of the SDKs mentioned above has been used.

162−To add another SDK to your current Xcode, [download](https://developer.apple.com/download/more/) the corresponding Xcode, extract it and copy its SDK folder next to the installed one, e.g.

181+To add another SDK to your current Xcode, [download](https://developer.apple.com/download/more/) the corresponding Xcode, extract it and copy its SDK folder next to the installed one, e.g.

163```

182```

164open -W Xcode_9.2.xip

183open -W Xcode_9.2.xip

165sudo cp -R Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS.sdk /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS11.2.sdk

184sudo cp -R Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS.sdk /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS11.2.sdk

@@ -173,7 +192,7 @@ xcodebuild -showsdks

173

192

174##### Certificates

193##### Certificates

175To deploy and/or package apps, you'll need to inform Apple of your development devices and permissions by

194To deploy and/or package apps, you'll need to inform Apple of your development devices and permissions by

176−adding credentials to the correct provisioning profile via the Apple Developer portal.

195+adding credentials to the correct provisioning profile via the Apple Developer portal.

177

196

1781. Open up XCode and go to Preferences - Accounts. Select the organization

1971. Open up XCode and go to Preferences - Accounts. Select the organization

179Member role, click Manage Certificates, and add an iOS Development

198Member role, click Manage Certificates, and add an iOS Development

@@ -250,8 +269,8 @@ Now edit your project's `default.nix` and tell Obelisk of your app's keystore fi

250 ...

269 ...

251 android.applicationId = "com.example.myapp";

270 android.applicationId = "com.example.myapp";

252 android.displayName = "My App";

271 android.displayName = "My App";

253− android.releaseKey =

272+ android.releaseKey =

254− { storeFile = /path/to/myandroidkey.jks;

273+ { storeFile = /path/to/myandroidkey.jks;

255 storePassword = "abcd1234";

274 storePassword = "abcd1234";

256 keyAlias = "myandroidalias";

275 keyAlias = "myandroidalias";

257 keyPassword = "abcd1234";

276 keyPassword = "abcd1234";

lib/cliapp/src/Obelisk/CliApp.hs+1

@@ -29,6 +29,7 @@ module Obelisk.CliApp

29 , Severity (..)

31 -- .Process

32+ , ProcessFailed (..)

32 , readProcessAndLogStderr

33 , readProcessAndLogStderr

33 , callProcessAndLogOutput

34 , callProcessAndLogOutput

34 , createProcess_

35 , createProcess_

lib/cliapp/src/Obelisk/CliApp/Logging.hs+6−5

@@ -26,7 +26,7 @@ import System.Console.ANSI (Color (Red, White, Yellow), ColorIntensity (Vivid),

26 ConsoleIntensity (FaintIntensity), ConsoleLayer (Foreground),

27 SGR (SetColor, SetConsoleIntensity), clearLine)

28import System.Exit (ExitCode (..), exitWith)

29−import System.IO (BufferMode (NoBuffering), hFlush, hReady, hSetBuffering, stdin, stdout)

29+import System.IO (BufferMode (NoBuffering), hFlush, hReady, hSetBuffering, stderr, stdin, stdout)

31import qualified Obelisk.CliApp.TerminalString as TS

32import Obelisk.CliApp.Types

@@ -137,12 +137,13 @@ withExitFailMessage msg f = f `catch` \(e :: ExitCode) -> do

137writeLog:: (MonadIO m, MonadMask m) => Bool -> Bool -> WithSeverity Text -> m ()

138writeLog withNewLine noColor (WithSeverity severity s)

139 | noColor && severity <= Warning = liftIO $ putFn $ T.pack (show severity) <> ": " <> s

140− | not noColor && severity <= Error = TS.putStrWithSGR errorColors withNewLine s

140+ | not noColor && severity <= Error = TS.putStrWithSGR errorColors h withNewLine s

141− | not noColor && severity <= Warning = TS.putStrWithSGR warningColors withNewLine s

141+ | not noColor && severity <= Warning = TS.putStrWithSGR warningColors h withNewLine s

142− | not noColor && severity >= Debug = TS.putStrWithSGR debugColors withNewLine s

142+ | not noColor && severity >= Debug = TS.putStrWithSGR debugColors h withNewLine s

143 | otherwise = liftIO $ putFn s

144 where

145− putFn = if withNewLine then T.putStrLn else T.putStr

145+ putFn = if withNewLine then (T.hPutStrLn h) else (T.hPutStr h)

146+ h = if severity <= Error then stderr else stdout

146 errorColors = [SetColor Foreground Vivid Red]

147 errorColors = [SetColor Foreground Vivid Red]

147 warningColors = [SetColor Foreground Vivid Yellow]

148 warningColors = [SetColor Foreground Vivid Yellow]

148 debugColors = [SetColor Foreground Vivid White, SetConsoleIntensity FaintIntensity]

149 debugColors = [SetColor Foreground Vivid White, SetConsoleIntensity FaintIntensity]

lib/cliapp/src/Obelisk/CliApp/Process.hs+12−13

@@ -9,7 +9,8 @@

9-- | An extension of `System.Process` that integrates with logging (`Obelisk.CLI.Logging`)

10-- and is thus spinner friendly.

11module Obelisk.CliApp.Process

12+ ( ProcessFailed (..)

12− ( readProcessAndLogStderr

13+ , readProcessAndLogStderr

13 , callProcessAndLogOutput

14 , callProcessAndLogOutput

14 , createProcess

15 , createProcess

15 , createProcess_

16 , createProcess_

@@ -19,9 +20,8 @@ module Obelisk.CliApp.Process

20import Control.Applicative (liftA2)

21import Control.Applicative (liftA2)

21import Control.Monad (void, (<=<))

22import Control.Monad (void, (<=<))

22−import Control.Monad.Catch (MonadMask)

23+import Control.Monad.Catch (Exception, MonadMask, throwM)

23−import Control.Monad.IO.Class (liftIO)

24+import Control.Monad.IO.Class (MonadIO, liftIO)

24−import Control.Monad.Reader (MonadIO)

25import qualified Data.ByteString.Char8 as BSC

26import Data.Function (fix)

27import Data.Monoid ((<>))

@@ -38,10 +38,14 @@ import System.Process (CreateProcess, ProcessHandle, StdStream (CreatePipe), cmd

38import qualified System.Process as Process

40import Control.Monad.Log (Severity (..))

41−import Obelisk.CliApp.Logging (failWith, putLog, putLogRaw)

41+import Obelisk.CliApp.Logging (putLog, putLogRaw)

42import Obelisk.CliApp.Types (Cli)

44−-- | Like `System.Process.readProcess` but logs the stderr instead of letting the external process inherit it.

44+data ProcessFailed = ProcessFailed Process.CmdSpec Int -- exit code

45+ deriving Show

46+

47+instance Exception ProcessFailed

48+

45readProcessAndLogStderr

49readProcessAndLogStderr

46 :: (MonadIO m, MonadMask m, Cli m)

50 :: (MonadIO m, MonadMask m, Cli m)

47 => Severity -> CreateProcess -> m String

51 => Severity -> CreateProcess -> m String

@@ -104,7 +108,6 @@ withProcess

104 :: (MonadIO m, MonadMask m, Cli m)

108 :: (MonadIO m, MonadMask m, Cli m)

105 => CreateProcess -> (Handle -> Handle -> m ()) -> m (Handle, Handle)

109 => CreateProcess -> (Handle -> Handle -> m ()) -> m (Handle, Handle)

106withProcess process f = do -- TODO: Use bracket.

110withProcess process f = do -- TODO: Use bracket.

107− let procTitle = T.pack $ show $ cmdspec process

108 -- FIXME: Using `withCreateProcess` here leads to something operating illegally on closed handles.

111 -- FIXME: Using `withCreateProcess` here leads to something operating illegally on closed handles.

109 (_, Just out, Just err, p) <- createProcess $ process

112 (_, Just out, Just err, p) <- createProcess $ process

110 { std_out = CreatePipe , std_err = CreatePipe }

113 { std_out = CreatePipe , std_err = CreatePipe }

@@ -112,12 +115,8 @@ withProcess process f = do -- TODO: Use bracket.

112 f out err -- Pass the handles to the passed function

115 f out err -- Pass the handles to the passed function

113

116

114 liftIO (waitForProcess p) >>= \case

117 liftIO (waitForProcess p) >>= \case

115− ExitSuccess -> return ()

118+ ExitSuccess -> return (out, err)

116− ExitFailure code -> do

119+ ExitFailure code -> throwM $ ProcessFailed (cmdspec process) code

117− -- Log an error. We also fail immediately; however we should probably let the caller control that.

118− failWith $ procTitle <> " failed with exit code: " <> T.pack (show code)

119− return (out, err) -- Return the handles

120−

121

120

122-- Create an input stream from the file handle, associating each item with the given severity.

121-- Create an input stream from the file handle, associating each item with the given severity.

123streamHandle :: Severity -> Handle -> IO (InputStream (Severity, BSC.ByteString))

122streamHandle :: Severity -> Handle -> IO (InputStream (Severity, BSC.ByteString))

lib/cliapp/src/Obelisk/CliApp/TerminalString.hs+6−6

@@ -18,9 +18,9 @@ import Data.Semigroup ((<>))

18import Data.Text (Text)

19import qualified Data.Text as T

20import qualified Data.Text.IO as T

21−import System.Console.ANSI (Color, ColorIntensity (Vivid), ConsoleLayer (Foreground), SGR (Reset, SetColor),

21+import System.Console.ANSI

22− SGR, setSGR, setSGRCode)

23import qualified System.Console.Terminal.Size as TerminalSize

22import qualified System.Console.Terminal.Size as TerminalSize

23+import System.IO (Handle)

25-- | Printable text on terminals

26--

@@ -65,11 +65,11 @@ colorizeText color s = mconcat

65 ]

67-- | Safely print the string with the given ANSI control codes, resetting in the end.

68−putStrWithSGR :: MonadIO m => [SGR] -> Bool -> Text -> m ()

68+putStrWithSGR :: MonadIO m => [SGR] -> Handle -> Bool -> Text -> m ()

69−putStrWithSGR sgr withNewLine s = liftIO $ bracket_ (setSGR sgr) reset $ T.putStr s

69+putStrWithSGR sgr h withNewLine s = liftIO $ bracket_ (hSetSGR h sgr) reset $ T.hPutStr h s

70 where

71− reset = setSGR [Reset] >> newline -- New line should come *after* reset (to reset cursor color).

71+ reset = hSetSGR h [Reset] >> newline -- New line should come *after* reset (to reset cursor color).

72− newline = when withNewLine $ T.putStrLn ""

72+ newline = when withNewLine $ T.hPutStrLn h ""

74-- | Code for https://en.wikipedia.org/wiki/Enquiry_character

75enquiryCode :: String

lib/command/src/Obelisk/Command.hs+4−1

@@ -8,6 +8,7 @@

8module Obelisk.Command where

10import Control.Monad

11+import Control.Monad.Catch (catch)

11import Control.Monad.IO.Class (liftIO)

12import Control.Monad.IO.Class (liftIO)

12import qualified Data.Binary as Binary

13import qualified Data.Binary as Binary

13import Data.Bool (bool)

14import Data.Bool (bool)

@@ -37,7 +38,7 @@ import Obelisk.Command.Run

37import Obelisk.Command.Thunk

38import Obelisk.Command.Thunk

38import Obelisk.Command.Upgrade

39import Obelisk.Command.Upgrade

39import Obelisk.Command.Upgrade.Hash (getDirectoryHash, getHashAtGitRevision)

40import Obelisk.Command.Upgrade.Hash (getDirectoryHash, getHashAtGitRevision)

40−import Obelisk.Command.Utils (getObeliskExe)

41+import Obelisk.Command.Utils

41import qualified Obelisk.Command.VmBuilder as VmBuilder

42import qualified Obelisk.Command.VmBuilder as VmBuilder

42import Obelisk.Migration (Hash)

43import Obelisk.Migration (Hash)

@@ -297,6 +298,8 @@ main' argsCfg = do

297 ]

298 ]

298

299

299 (mainWithHandOff argsCfg <=< parseHandoff) =<< liftIO getArgs

300 (mainWithHandOff argsCfg <=< parseHandoff) =<< liftIO getArgs

301+ `catch`

302+ \(ProcessFailed p code) -> failWith $ "Process exited with code " <> tshow code <> "; " <> tshow p

300

303

301-- Type representing the result of a handoff calculation.

304-- Type representing the result of a handoff calculation.

302data HandOff m

305data HandOff m

lib/command/src/Obelisk/Command/Nix.hs+27−1

@@ -5,6 +5,7 @@

5{-# LANGUAGE ScopedTypeVariables #-}

6module Obelisk.Command.Nix

7 ( nixBuild

8+ , withNixRemoteCheck

8 , NixBuildConfig (..)

9 , NixBuildConfig (..)

9 , Target (..)

10 , Target (..)

10 , OutLink (..)

11 , OutLink (..)

@@ -13,11 +14,15 @@ module Obelisk.Command.Nix

13 , strArg

14 , strArg

14 ) where

15 ) where

17+import Control.Monad.Catch (catch, throwM)

18+import Control.Monad.IO.Class (liftIO)

16import Data.Bool (bool)

19import Data.Bool (bool)

17import Data.Default

20import Data.Default

18import Data.List (intercalate)

21import Data.List (intercalate)

19import Data.Monoid ((<>))

22import Data.Monoid ((<>))

20import qualified Data.Text as T

23import qualified Data.Text as T

24+import System.Directory

25+import System.Environment

21import System.Process (proc)

26import System.Process (proc)

23import Obelisk.App (MonadObelisk)

28import Obelisk.App (MonadObelisk)

@@ -72,7 +77,7 @@ instance Default NixBuildConfig where

73nixBuild :: MonadObelisk m => NixBuildConfig -> m FilePath

78nixBuild :: MonadObelisk m => NixBuildConfig -> m FilePath

74nixBuild cfg = withSpinner' ("Running nix-build on " <> desc) (Just $ const $ "Built " <> desc) $ do

79nixBuild cfg = withSpinner' ("Running nix-build on " <> desc) (Just $ const $ "Built " <> desc) $ do

75− readProcessAndLogStderr Debug $ proc "nix-build" $ mconcat

80+ withNixRemoteCheck $ readProcessAndLogStderr Debug $ proc "nix-build" $ mconcat

76 [[path], attrArg, args, outLink, buildersArg]

81 [[path], attrArg, args, outLink, buildersArg]

77 where

82 where

78 path = _target_path $ _nixBuildConfig_target cfg

83 path = _target_path $ _nixBuildConfig_target cfg

@@ -89,3 +94,24 @@ nixBuild cfg = withSpinner' ("Running nix-build on " <> desc) (Just $ const $ "B

89 buildersArg = case _nixBuildConfig_builders cfg of

94 buildersArg = case _nixBuildConfig_builders cfg of

90 [] -> []

95 [] -> []

91 builders -> ["--builders", intercalate ";" builders]

96 builders -> ["--builders", intercalate ";" builders]

97+

98+-- | If a nix command fails, and this may be related to the NIX_REMOTE issue,

99+-- tell the user what to do.

100+--

101+-- Added: June, 2018. Consider removing this eventually.

102+withNixRemoteCheck :: MonadObelisk m => m a -> m a

103+withNixRemoteCheck f = f `catch` \e@(ProcessFailed _ _) -> do

104+ liftIO (lookupEnv "NIX_REMOTE") >>= \case

105+ Just _ -> throwM e

106+ Nothing -> liftIO (writable <$> getPermissions "/nix/var/nix/db") >>= \case

107+ True -> throwM e

108+ False -> do

109+ putLog Error "!!! "

110+ putLog Error "!!! A nix command failed to run. You might need to set the NIX_REMOTE environment variable"

111+ putLog Error "!!! to `daemon`. To do this, run the following before running obelisk:"

112+ putLog Error "!!! "

113+ putLog Error "!!! export NIX_REMOTE=daemon"

114+ putLog Error "!!! "

115+ putLog Error "!!! For details, see https://github.com/NixOS/nixpkgs/issues/5713"

116+ putLog Error "!!! "

117+ throwM e

lib/command/src/Obelisk/Command/Project.hs+2−1

@@ -35,6 +35,7 @@ import GitHub.Data.Name (Name)

35import Obelisk.App (MonadObelisk)

36import Obelisk.CliApp

37import Obelisk.Command.Thunk

38+import Obelisk.Command.Nix (withNixRemoteCheck)

38--TODO: Make this module resilient to random exceptions

39--TODO: Make this module resilient to random exceptions

40--TODO: Don't hardcode this

41--TODO: Don't hardcode this

@@ -201,7 +202,7 @@ inImpureProjectShell shellName command = withProjectRoot "." $ \root ->

201 projectShell root False shellName command

202 projectShell root False shellName command

202

203

203projectShell :: MonadObelisk m => FilePath -> Bool -> String -> String -> m ()

204projectShell :: MonadObelisk m => FilePath -> Bool -> String -> String -> m ()

204−projectShell root isPure shellName command = do

205+projectShell root isPure shellName command = withNixRemoteCheck $ do

205 (_, _, _, ph) <- createProcess_ "runNixShellAttr" $ setCwd (Just root) $ proc "nix-shell" $

206 (_, _, _, ph) <- createProcess_ "runNixShellAttr" $ setCwd (Just root) $ proc "nix-shell" $

206 [ "--pure" | isPure ] <>

207 [ "--pure" | isPure ] <>

207 [ "-A"

208 [ "-A"

lib/command/src/Obelisk/Command/Thunk.hs+3−2

@@ -74,6 +74,7 @@ import System.Process (proc)

75import Obelisk.App (MonadObelisk)

76import Obelisk.CliApp

77+import Obelisk.Command.Nix (withNixRemoteCheck)

77import Obelisk.Command.Utils

78import Obelisk.Command.Utils

79--TODO: Support symlinked thunk data

80--TODO: Support symlinked thunk data

@@ -135,13 +136,13 @@ commitNameToRef (N c) = Ref.fromHex $ encodeUtf8 c

135getNixSha256ForUriUnpacked :: MonadObelisk m => URI -> m NixSha256

136getNixSha256ForUriUnpacked :: MonadObelisk m => URI -> m NixSha256

136getNixSha256ForUriUnpacked uri =

137getNixSha256ForUriUnpacked uri =

137 withExitFailMessage ("nix-prefetch-url: Failed to determine sha256 hash of URL " <> T.pack (show uri)) $ do

138 withExitFailMessage ("nix-prefetch-url: Failed to determine sha256 hash of URL " <> T.pack (show uri)) $ do

138− fmap T.pack $ readProcessAndLogStderr Debug $

139+ fmap T.pack $ withNixRemoteCheck $ readProcessAndLogStderr Debug $

139 proc "nix-prefetch-url" ["--unpack" , "--type" , "sha256" , show uri]

140 proc "nix-prefetch-url" ["--unpack" , "--type" , "sha256" , show uri]

140

141

141nixPrefetchGit :: MonadObelisk m => URI -> Text -> Bool -> m NixSha256

142nixPrefetchGit :: MonadObelisk m => URI -> Text -> Bool -> m NixSha256

142nixPrefetchGit uri rev fetchSubmodules =

143nixPrefetchGit uri rev fetchSubmodules =

143 withExitFailMessage ("nix-prefetch-git: Failed to determine sha256 hash of Git repo " <> T.pack (show uri) <> " at " <> rev) $ do

144 withExitFailMessage ("nix-prefetch-git: Failed to determine sha256 hash of Git repo " <> T.pack (show uri) <> " at " <> rev) $ do

144− out <- readProcessAndLogStderr Debug $

145+ out <- withNixRemoteCheck $ readProcessAndLogStderr Debug $

145 proc "nix-prefetch-git" $ filter (/="")

146 proc "nix-prefetch-git" $ filter (/="")

146 [ "--url", show uri

147 [ "--url", show uri

147 , "--rev", T.unpack rev

148 , "--rev", T.unpack rev

lib/command/src/Obelisk/Command/Upgrade/Hash.hs+2−1

@@ -23,6 +23,7 @@ import System.Process (cwd, proc)

24import Obelisk.App (MonadObelisk)

25import Obelisk.CliApp

26+import Obelisk.Command.Nix (withNixRemoteCheck)

26import Obelisk.Command.Utils

27import Obelisk.Command.Utils

28import Obelisk.Migration

29import Obelisk.Migration

@@ -87,7 +88,7 @@ getHashAtGitRevision revs excludes dir = withSystemTempDirectory "obelisk-hashre

87 return result

88 return result

89nixHash :: MonadObelisk m => FilePath -> m Hash

90nixHash :: MonadObelisk m => FilePath -> m Hash

90−nixHash dir = withSpinnerNoTrail "Running `nix-hash`" $

91+nixHash dir = withSpinnerNoTrail "Running `nix-hash`" $ withNixRemoteCheck $

91 readProc $ proc "nix-hash" [dir]

92 readProc $ proc "nix-hash" [dir]

93-- | Clean up the following files in the git working copy

94-- | Clean up the following files in the git working copy

lib/command/src/Obelisk/Command/VmBuilder.hs+4−4

@@ -16,13 +16,13 @@ import qualified Data.Text as T

16import qualified Data.Text.IO as T

17import System.Directory (createDirectoryIfMissing)

18import System.Exit (ExitCode)

19−import System.FilePath ((</>), (<.>))

19+import System.FilePath ((<.>), (</>))

20import qualified System.Info

21import System.Process (proc)

23import Obelisk.App (MonadObelisk, getObeliskUserStateDir)

24−import Obelisk.CliApp (Severity (..), callProcessAndLogOutput, failWith, readProcessAndLogStderr,

24+import Obelisk.CliApp

25− withExitFailMessage, withSpinner)

25+import Obelisk.Command.Nix (withNixRemoteCheck)

27-- | Generate the `--builders` argument string to enable the VM builder after ensuring it is available.

28getNixBuildersArg :: MonadObelisk m => m String

@@ -131,7 +131,7 @@ getDockerBuilderStateDir = liftA2 (</>) getObeliskUserStateDir (pure "nix-docker

131-- | Run a test build to see if a Linux build can finish successfully.

132testLinuxBuild :: MonadObelisk m => FilePath -> m Bool

133testLinuxBuild stateDir =

134− fmap (isRight :: Either ExitCode () -> Bool) $ try $

134+ fmap (isRight :: Either ExitCode () -> Bool) $ try $ withNixRemoteCheck $

135 callProcessAndLogOutput (Debug, Debug) $ proc "nix-build"

136 [ "-E", "(import <nixpkgs> { system = \"x86_64-linux\"; }).writeText \"test\" builtins.currentTime"

137 , "--builders", nixBuildersArgString stateDir

lib/selftest/obelisk-selftest.cabal+2−1

@@ -6,7 +6,6 @@ build-type: Simple

6cabal-version: >=1.8

8library

9− default-language: Haskell2010

10 hs-source-dirs: src

9 hs-source-dirs: src

11 build-depends:

10 build-depends:

12 base

11 base

@@ -16,6 +15,7 @@ library

16 , http-client

15 , http-client

17 , http-types

16 , http-types

18 , HUnit

17 , HUnit

18+ , obelisk-cliapp

19 , obelisk-run

20 , process

21 , network

@@ -24,6 +24,7 @@ library

24 , text

25 exposed-modules:

26 Obelisk.SelfTest

27+ ghc-options: -Wall

28executable obelisk-selftest

29executable obelisk-selftest

29 main-is: src-bin/obelisk-selftest.hs

30 main-is: src-bin/obelisk-selftest.hs

lib/selftest/src/Obelisk/SelfTest.hs+68−40

@@ -1,14 +1,15 @@

1{-# LANGUAGE LambdaCase #-}

2{-# LANGUAGE OverloadedStrings #-}

3+{-# LANGUAGE PatternGuards #-}

4+{-# LANGUAGE ScopedTypeVariables #-}

3{-# LANGUAGE TypeApplications #-}

5{-# LANGUAGE TypeApplications #-}

4module Obelisk.SelfTest where

6module Obelisk.SelfTest where

6−import Control.Exception (Exception, throw, bracket)

8+import Control.Exception (bracket, throw)

7import Control.Monad

9import Control.Monad

8import Control.Monad.IO.Class

10import Control.Monad.IO.Class

9import Data.Bool (bool)

11import Data.Bool (bool)

10import Data.Function (fix)

12import Data.Function (fix)

11−import Data.Maybe (isJust)

12import Data.Semigroup (Semigroup, (<>))

13import Data.Semigroup (Semigroup, (<>))

13import qualified Data.Set as Set

14import qualified Data.Set as Set

14import Data.String

15import Data.String

@@ -21,15 +22,17 @@ import qualified Network.Socket as Socket

21import Shelly

22import Shelly

22import System.Directory (withCurrentDirectory)

23import System.Directory (withCurrentDirectory)

23import System.Environment

24import System.Environment

24−import System.Exit (ExitCode (..), exitWith)

25+import System.Exit (ExitCode (..))

25import System.Info

26import System.Info

26import System.IO (Handle, hClose)

27import System.IO (Handle, hClose)

27import System.IO.Temp

28import System.IO.Temp

29+import System.Process (readProcessWithExitCode)

28import System.Timeout

30import System.Timeout

29import Test.Hspec

31import Test.Hspec

30import Test.HUnit.Base

32import Test.HUnit.Base

31−import System.Process (readProcessWithExitCode)

34+import Obelisk.CliApp hiding (runCli)

35+import qualified Obelisk.CliApp as CliApp

33import Obelisk.Run (getConfigRoute)

36import Obelisk.Run (getConfigRoute)

35data ObRunState

38data ObRunState

@@ -50,18 +53,44 @@ commit msg = void $ run "git"

50 , doubleQuotes msg

53 , doubleQuotes msg

51 ]

54 ]

56+runCli :: MonadIO m => CliT IO a -> m a

57+runCli f = liftIO $ do

58+ c <- newCliConfig Notice False False

59+ CliApp.runCli c f

60+

61+tshow :: Show a => a -> Text

62+tshow = T.pack . show

63+

64+-- | Like `shelly` but used when running `ob` commands

65+--

66+-- This tests the NIX_REMOTE workaround, whether that gets printed

67+-- on failure, and whether retrying with the suggested workaround

68+-- leads to the test succeeding.

69+shellyOb :: MonadIO m => (Sh a -> Sh a) -> Sh a -> m a

70+shellyOb f obTest = shelly $ f $ do

71+ obTest `catch_sh` \(e :: RunFailed) -> do

72+ obStderr <- lastStderr

73+ if T.isInfixOf "export NIX_REMOTE" obStderr

74+ then do

75+ runCli $ do

76+ putLog Notice "Detected NIX_REMOTE suggestion in stderr of failed ob"

77+ putLog Warning "Retrying this test with NIX_REMOTE=daemon"

78+ setenv "NIX_REMOTE" "daemon"

79+ f obTest

80+ else throw e

81+

53main :: IO ()

82main :: IO ()

54main = do

83main = do

84+ -- Note: you can pass hspec arguments as well, eg: `-m <pattern>`

55− isVerbose <- getArgs >>= \case

85+ isVerbose <- (elem "-v") <$> getArgs

56− ["-v"] -> pure True

86+ unless isVerbose $

57− [] -> putStrLn "Tests may take longer to run if there are unbuilt derivations: use -v for verbose output" *> pure False

87+ putStrLn "Tests may take longer to run if there are unbuilt derivations: use -v for verbose output"

58− _ -> putStrLn "Usage: selftest [-v]" *> exitWith (ExitFailure 2)

59 let verbosity = bool silently verbosely isVerbose

88 let verbosity = bool silently verbosely isVerbose

60 obeliskImpl <- fromString <$> getEnv "OBELISK_IMPL"

89 obeliskImpl <- fromString <$> getEnv "OBELISK_IMPL"

61 httpManager <- HTTP.newManager HTTP.defaultManagerSettings

90 httpManager <- HTTP.newManager HTTP.defaultManagerSettings

62 withSystemTempDirectory "blank-project" $ \blankProject ->

91 withSystemTempDirectory "blank-project" $ \blankProject ->

63 hspec $ do

92 hspec $ do

64− let shelly_ = void . shelly . verbosity

93+ let shelly_ = void . shellyOb verbosity

66 inProj :: Sh a -> IO ()

95 inProj :: Sh a -> IO ()

67 inProj = shelly_ . chdir (fromString blankProject)

96 inProj = shelly_ . chdir (fromString blankProject)

@@ -75,7 +104,7 @@ main = do

75 revParseHead = T.strip <$> run "git" ["rev-parse", "HEAD"]

104 revParseHead = T.strip <$> run "git" ["rev-parse", "HEAD"]

105

77 commitAll = do

106 commitAll = do

78− run "git" ["add", "."]

107+ run_ "git" ["add", "."]

79 commit "checkpoint"

108 commit "checkpoint"

80 revParseHead

109 revParseHead

110

@@ -92,31 +121,32 @@ main = do

92 it "works with symlink" $ inTmp $ \_ -> run "ob" ["init", "--symlink", obeliskImpl]

121 it "works with symlink" $ inTmp $ \_ -> run "ob" ["init", "--symlink", obeliskImpl]

122

94 it "doesn't create anything when given an invalid impl" $ inTmp $ \tmp -> do

123 it "doesn't create anything when given an invalid impl" $ inTmp $ \tmp -> do

95− errExit False $ run "ob" ["init", "--symlink", "/dev/null"]

124+ void $ errExit False $ run "ob" ["init", "--symlink", "/dev/null"]

96 ls tmp >>= liftIO . assertEqual "" []

125 ls tmp >>= liftIO . assertEqual "" []

126

98 it "produces a valid route config" $ inTmp $ \tmp -> do

127 it "produces a valid route config" $ inTmp $ \tmp -> do

99− run "ob" ["init"]

128+ run_ "ob" ["init"]

100 liftIO $ withCurrentDirectory (T.unpack $ toTextIgnore tmp) $ getConfigRoute `shouldNotReturn` Nothing

129 liftIO $ withCurrentDirectory (T.unpack $ toTextIgnore tmp) $ getConfigRoute `shouldNotReturn` Nothing

101

130

102 describe "ob run" $ do

131 describe "ob run" $ do

103 it "works in root directory" $ inTmp $ \_ -> do

132 it "works in root directory" $ inTmp $ \_ -> do

104− run "ob" ["init"]

133+ run_ "ob" ["init"]

105 testObRunInDir Nothing httpManager

134 testObRunInDir Nothing httpManager

106 it "works in sub directory" $ inTmp $ \_ -> do

135 it "works in sub directory" $ inTmp $ \_ -> do

107− run "ob" ["init"]

136+ run_ "ob" ["init"]

108 testObRunInDir (Just "frontend") httpManager

137 testObRunInDir (Just "frontend") httpManager

109

138

110 describe "obelisk project" $ parallel $ do

139 describe "obelisk project" $ parallel $ do

111 it "can build obelisk command" $ shelly_ $ run "nix-build" ["-A", "command" , obeliskImpl]

140 it "can build obelisk command" $ shelly_ $ run "nix-build" ["-A", "command" , obeliskImpl]

112 it "can build obelisk skeleton" $ shelly_ $ run "nix-build" ["-A", "skeleton", obeliskImpl]

141 it "can build obelisk skeleton" $ shelly_ $ run "nix-build" ["-A", "skeleton", obeliskImpl]

113 it "can build obelisk shell" $ shelly_ $ run "nix-build" ["-A", "shell", obeliskImpl]

142 it "can build obelisk shell" $ shelly_ $ run "nix-build" ["-A", "shell", obeliskImpl]

143+ -- See https://github.com/obsidiansystems/obelisk/issues/101

114− it "can build everything" $ shelly_ $ run "nix-build" [obeliskImpl]

144+ -- it "can build everything" $ shelly_ $ run "nix-build" [obeliskImpl]

115

145

116 describe "blank initialized project" $ do

146 describe "blank initialized project" $ do

117 it "can be created" $ inProj $ do

147 it "can be created" $ inProj $ do

118− run "ob" ["init"]

148+ run_ "ob" ["init"]

119− run "git" ["init"]

149+ run_ "git" ["init"]

120 commitAll

150 commitAll

121

151

122 it "can build ghc.backend" $ inProj $ do

152 it "can build ghc.backend" $ inProj $ do

@@ -131,7 +161,7 @@ main = do

131 forM_ ["ghc", "ghcjs"] $ \compiler -> do

161 forM_ ["ghc", "ghcjs"] $ \compiler -> do

132 let

162 let

133 shell = "shells." <> compiler

163 shell = "shells." <> compiler

134− inShell cmd = run "nix-shell" ["-A", fromString shell, "--run", cmd]

164+ inShell cmd' = run "nix-shell" ["-A", fromString shell, "--run", cmd']

135 it ("can enter " <> shell) $ inProj $ inShell "exit"

165 it ("can enter " <> shell) $ inProj $ inShell "exit"

136 it ("can build in " <> shell) $ inProj $ inShell $ "cabal new-build --" <> fromString compiler <> " all"

166 it ("can build in " <> shell) $ inProj $ inShell $ "cabal new-build --" <> fromString compiler <> " all"

137

167

@@ -156,15 +186,15 @@ main = do

156 assertRevNE e eu

186 assertRevNE e eu

157

187

158 it "can pack and unpack plain git repos" $ do

188 it "can pack and unpack plain git repos" $ do

159− withSystemTempDirectory "git-repo" $ \dir -> shelly_ $ do

189+ shelly_ $ withSystemTempDirectory "git-repo" $ \dir -> do

160 let repo = toTextIgnore $ dir </> ("repo" :: String)

190 let repo = toTextIgnore $ dir </> ("repo" :: String)

161− run_ "git" ["clone", "https://git.haskell.org/packages/primitive.git", repo]

191+ run_ "git" ["clone", "https://github.com/haskell/process.git", repo]

162 origHash <- chdir (fromText repo) revParseHead

192 origHash <- chdir (fromText repo) revParseHead

163

193

164 run_ "ob" ["thunk", "pack", repo]

194 run_ "ob" ["thunk", "pack", repo]

165 packedFiles <- Set.fromList <$> ls (fromText repo)

195 packedFiles <- Set.fromList <$> ls (fromText repo)

166− liftIO $ assertEqual "" packedFiles $

196+ liftIO $ assertEqual "" packedFiles $ Set.fromList $ (repo </>) <$>

167− Set.fromList $ (repo </>) <$> ["default.nix", "git.json", ".attr-cache" :: String]

197+ ["default.nix", "github.json", ".attr-cache" :: String]

168

198

169 run_ "ob" ["thunk", "unpack", repo]

199 run_ "ob" ["thunk", "unpack", repo]

170 chdir (fromText repo) $ do

200 chdir (fromText repo) $ do

@@ -174,7 +204,7 @@ main = do

174 testThunkPack $ fromText repo

204 testThunkPack $ fromText repo

175

205

176 it "aborts thunk pack when there are uncommitted files" $ inProj $ do

206 it "aborts thunk pack when there are uncommitted files" $ inProj $ do

177− unpack

207+ void $ unpack

178 testThunkPack (blankProject </> thunk)

208 testThunkPack (blankProject </> thunk)

179

209

180-- | Run `ob run` in the given directory (maximum of one level deep)

210-- | Run `ob run` in the given directory (maximum of one level deep)

@@ -191,33 +221,31 @@ testObRunInDir mdir httpManager = handle_sh (\case ExitSuccess -> pure (); e ->

191 maybe id (\_ -> chdir "..") mdir $ alterRouteTo newUri stdout

221 maybe id (\_ -> chdir "..") mdir $ alterRouteTo newUri stdout

192 (_, runningUri) <- handleObRunStdout httpManager stdout

222 (_, runningUri) <- handleObRunStdout httpManager stdout

193 if runningUri /= newUri

223 if runningUri /= newUri

194− then errorExit $ "Reloading failed: expected " <> newUri <> " but got " <> runningUri

224+ then errorExit $ "Reloading failed: expected " <> newUri <> " but got " <> runningUri

195− else exit 0

225+ else exit 0

196

226

197testThunkPack :: Shelly.FilePath -> Sh ()

227testThunkPack :: Shelly.FilePath -> Sh ()

198−testThunkPack path = withTempFile (T.unpack $ toTextIgnore path) "test-file" $ \file handle -> do

228+testThunkPack path' = withTempFile (T.unpack $ toTextIgnore path') "test-file" $ \file handle -> do

199− let try_sh :: Exception e => Sh a -> Sh (Either e a)

200− try_sh a = catch_sh (a >>= pure . Right) (pure . Left)

201− pack' = readProcessWithExitCode "ob" ["thunk", "pack", T.unpack $ toTextIgnore path] ""

229+ let pack' = readProcessWithExitCode "ob" ["thunk", "pack", T.unpack $ toTextIgnore path'] ""

202− ensureThunkPackFails err = liftIO $ pack' >>= \case

230+ ensureThunkPackFails q = liftIO $ pack' >>= \case

203− (code, out, _err)

231+ (code, out, err)

204 | code == ExitSuccess -> fail "ob thunk pack succeeded when it should have failed"

232 | code == ExitSuccess -> fail "ob thunk pack succeeded when it should have failed"

205− | err `T.isInfixOf` T.pack out -> pure ()

233+ | q `T.isInfixOf` T.pack (out <> err) -> pure ()

206− | otherwise -> fail $ "ob thunk pack failed for an unexpected reason: " <> show out

234+ | otherwise -> fail $ "ob thunk pack failed for an unexpected reason: " <> show out <> "\nstderr: " <> err

207− git = chdir path . run "git"

235+ git = chdir path' . run "git"

208 -- Untracked files

236 -- Untracked files

209− ensureThunkPackFails "unsaved modifications"

237+ ensureThunkPackFails "Untracked files"

210− git ["add", T.pack file]

238+ void $ git ["add", T.pack file]

211 -- Uncommitted files (staged)

239 -- Uncommitted files (staged)

212− ensureThunkPackFails "unsaved modifications"

240+ ensureThunkPackFails "unsaved"

213− chdir path $ commit "test commit"

241+ chdir path' $ commit "test commit"

214 -- Non-pushed commits in any branch

242 -- Non-pushed commits in any branch

215 ensureThunkPackFails "not been pushed"

243 ensureThunkPackFails "not been pushed"

216 -- Uncommitted files (unstaged)

244 -- Uncommitted files (unstaged)

217 liftIO $ T.hPutStrLn handle "test file" >> hClose handle

245 liftIO $ T.hPutStrLn handle "test file" >> hClose handle

218− ensureThunkPackFails "unsaved modifications"

246+ ensureThunkPackFails "modified"

219 -- Existing stashes

247 -- Existing stashes

220− git ["stash"]

248+ void $ git ["stash"]

221 ensureThunkPackFails "has stashes"

249 ensureThunkPackFails "has stashes"

222

250

223-- | Blocks until a non-empty line is available

251-- | Blocks until a non-empty line is available

migration/07900e6c41ab9a78917f66fcf7e0d9b8-554c459ff2cced3e1b11241ed403890d/obelisk-handoffnew file

No changes to the content.

migration/07900e6c41ab9a78917f66fcf7e0d9b8-554c459ff2cced3e1b11241ed403890d/obelisk-upgradenew file

No changes to the content.

migration/554c459ff2cced3e1b11241ed403890d-6a18939264224d6ce24c7426249edfea/obelisk-handoffnew file

No changes to the content.

migration/554c459ff2cced3e1b11241ed403890d-6a18939264224d6ce24c7426249edfea/obelisk-upgradenew file

No changes to the content.

migration/6a18939264224d6ce24c7426249edfea-e66afc8befac4d76b5780a960f768c31/obelisk-handoffnew file

No changes to the content.

migration/6a18939264224d6ce24c7426249edfea-e66afc8befac4d76b5780a960f768c31/obelisk-upgradenew file+15

@@ -0,0 +1,15 @@

1+# ob deploy: HTTPs support

3+Obelisk deployments now support HTTPs. You must upgrade the deployment repo

4+associated with the obelisk project as follows:

6+```

7+cd /your/project's/deployment/repo

8+echo True > enable_https

9+echo admin@yoursite.com > admin_email

10+echo https://yoursite.com/ > config/common/route

11+git add ... && git commit ...

12+ob deploy push

13+```

14+

15+If you do not wish to enable HTTPs, write `False` to `enable_https` instead.

release.nix+5−3

@@ -26,14 +26,16 @@ with pkgs;

26 nix # obelisk uses `nix-hash`

27 ];

28 } ''

29− set -xe

29+ set -e

30 # Fix permissions so `removePathForcibly` works on copied-to-tmp paths

31 cp -a ${self.pathGit} $TMPDIR/obelisk

32 cd $TMPDIR/obelisk

33 chmod -R u+w .

35+ (set -x;

35− ob internal hash

36+ ob internal hash;

36− ob internal verify-migration

37+ ob internal verify-migration;

38+ )

38 mkdir -p $out

40 mkdir -p $out

39 touch $out/done

41 touch $out/done

skeleton/.gitignore+2−1

@@ -2,5 +2,6 @@ dist-newstyle

2result

3result-android

4result-ios

5+result-exe

5.attr-cache

6.attr-cache

6−ghcid-output.txt

7+ghcid-output.txt