1ac00ca · squeal · git.iko.soy

git.iko.soy/squeal

on dev help sign in

Merge pull request #23 from morphismtech/0.2

1ac00caEitan Chatav9 years ago

.circleci/config.ymlnew file+23

compare with dev

unifiedsplit

@@ -0,0 +1,23 @@

1+version: 2

2+jobs:

3+ build:

4+ docker:

5+ - image: haskell:latest

6+ - image: circleci/postgres:latest

7+ environment:

8+ POSTGRES_DB: exampledb

9+ steps:

10+ - checkout

11+ - restore_cache:

12+ keys:

13+ - dependency-cache

14+ - run: apt-get update && apt-get install -y libpq-dev xz-utils make

15+ - run: stack upgrade && stack update

16+ - run: stack build

17+ - run: stack test

18+ - run: stack haddock

19+ - save_cache:

20+ key: dependency-cache

21+ paths:

22+ - ".stack-work"

23+ - "/root/.stack/"

.gitignore+1

@@ -17,3 +17,4 @@ cabal.sandbox.config

17*.eventlog

18.stack-work/

19cabal.project.local

20+.DS_Store

README.md+162−97

@@ -2,7 +2,7 @@

3![squeal-icon](http://www.emoticonswallpapers.com/emotion/cute-big-pig/cute-pig-smiley-046.gif)

5−[![CircleCI](https://circleci.com/gh/echatav/squeal.svg?style=svg&circle-token=a699a654ef50db2c3744fb039cf2087c484d1226)](https://circleci.com/gh/echatav/squeal)

5+[![CircleCI](https://circleci.com/gh/echatav/squeal.svg?style=svg&circle-token=a699a654ef50db2c3744fb039cf2087c484d1226)](https://circleci.com/gh/morphismtech/squeal)

7[Github](https://github.com/morphismtech/squeal)

@@ -17,148 +17,213 @@

17## usage

19Squeal is a deep embedding of PostgreSQL in Haskell. Let's see an example!

20−

21−First, we need some language extensions because Squeal uses modern GHC features.

20+First, we need some language extensions because Squeal uses modern GHC

22−

21+features.

23```haskell

22```haskell

24−{-# LANGUAGE

23+>>> :set -XDataKinds -XDeriveGeneric -XOverloadedLabels

25− DataKinds

24+>>> :set -XOverloadedStrings -XTypeApplications -XTypeOperators

26− , DeriveGeneric

27− , OverloadedLabels

28− , OverloadedStrings

29− , TypeApplications

30− , TypeOperators

31−#-}

32```

25```

34−Here comes the Main module and imports.

27+We'll need some imports.

36```haskell

29```haskell

37−module Main (main) where

30+>>> import Control.Monad (void)

38−

39−import Control.Monad.Base (liftBase)

31+>>> import Control.Monad.Base (liftBase)

40−import Data.Int (Int32)

32+>>> import Data.Int (Int32)

41−import Data.Text (Text)

33+>>> import Data.Text (Text)

42−

43−import Squeal.PostgreSQL

34+>>> import Squeal.PostgreSQL

44```

35```

46We'll use generics to easily convert between Haskell and PostgreSQL values.

37We'll use generics to easily convert between Haskell and PostgreSQL values.

48```haskell

39```haskell

49−import qualified Generics.SOP as SOP

40+>>> import qualified Generics.SOP as SOP

50−import qualified GHC.Generics as GHC

41+>>> import qualified GHC.Generics as GHC

51```

42```

53−The first step is to define the schema of our database. This is where we use DataKinds and TypeOperators. The schema consists of a type-level list of tables, a `:::` pairing of a type level string or `Symbol` and a list a columns, itself a `:::` pairing of a `Symbol` and a `ColumnType`. The `ColumnType` describes the PostgreSQL type of the column as well as whether or not it may contain `NULL` and whether or not inserts and updates can use a `DEFAULT`. For our schema, we'll define two tables, a users table and an emails table.

44+The first step is to define the schema of our database. This is where

45+we use `DataKinds` and `TypeOperators`. The schema consists of a type-level

46+list of tables, a `:::` pairing of a type level string or

47+`Symbol` and a list a columns, itself a `:::` pairing of a

48+`Symbol` and a `ColumnType`. The `ColumnType` describes the

49+PostgreSQL type of the column as well as whether or not it may contain

50+`NULL` and whether or not inserts and updates can use a `DEFAULT`. For our

51+schema, we'll define two tables, a users table and an emails table.

55```haskell

53```haskell

54+>>> :{

56type Schema =

55type Schema =

57− '[ "users" :::

56+ '[ "users" :::

57+ '[ "pk_users" ::: 'PrimaryKey '["id"] ] :=>

58− '[ "id" ::: 'Optional ('NotNull 'PGint4)

58+ '[ "id" ::: 'Def :=> 'NotNull 'PGint4

59− , "name" ::: 'Required ('NotNull 'PGtext)

59+ , "name" ::: 'NoDef :=> 'NotNull 'PGtext

60 ]

61 , "emails" :::

62+ '[ "pk_emails" ::: 'PrimaryKey '["id"]

63+ , "fk_user_id" ::: 'ForeignKey '["user_id"] "users" '["id"]

64+ ] :=>

62− '[ "id" ::: 'Optional ('NotNull 'PGint4)

65+ '[ "id" ::: 'Def :=> 'NotNull 'PGint4

63− , "user_id" ::: 'Required ('NotNull 'PGint4)

66+ , "user_id" ::: 'NoDef :=> 'NotNull 'PGint4

64− , "email" ::: 'Required ('Null 'PGtext)

67+ , "email" ::: 'NoDef :=> 'Null 'PGtext

65 ]

68 ]

66 ]

69 ]

70+:}

67```

71```

69−Next, we'll write Definitions to set up and tear down the schema. In Squeal, a Definition is a `createTable`, `alterTable` or `dropTable` command and has two type parameters, corresponding to the schema before being run and the schema after. We can compose definitions using `>>>`. Here and in the rest of our commands we make use of overloaded labels to refer to named tables and columns in our schema.

73+Next, we'll write `Definition`s to set up and tear down the schema. In

74+Squeal, a `Definition` is a `createTable`, `alterTable` or `dropTable`

75+command and has two type parameters, corresponding to the schema

76+before being run and the schema after. We can compose definitions using

77+`>>>`. Here and in the rest of our commands we make use of overloaded

78+labels to refer to named tables and columns in our schema.

71```haskell

80```haskell

81+>>> :{

82+let

72−setup :: Definition '[] Schema

83+ setup :: Definition '[] Schema

73−setup =

84+ setup =

74− createTable #users

85+ createTable #users

75− ( serial `As` #id :*

86+ ( serial `As` #id :*

76− (text & notNull) `As` #name :* Nil )

87+ (text & notNull) `As` #name :* Nil )

77− [ primaryKey (Column #id :* Nil) ]

88+ ( primaryKey (Column #id :* Nil) `As` #pk_users :* Nil ) >>>

78− >>>

79− createTable #emails

89+ createTable #emails

80− ( serial `As` #id :*

90+ ( serial `As` #id :*

81− (int & notNull) `As` #user_id :*

91+ (int & notNull) `As` #user_id :*

82− text `As` #email :* Nil )

92+ text `As` #email :* Nil )

83− [ primaryKey (Column #id :* Nil)

93+ ( primaryKey (Column #id :* Nil) `As` #pk_emails :*

84− , foreignKey (Column #user_id :* Nil) #users (Column #id :* Nil)

94+ foreignKey (Column #user_id :* Nil) #users (Column #id :* Nil)

85− OnDeleteCascade OnUpdateCascade ]

95+ OnDeleteCascade OnUpdateCascade `As` #fk_user_id :* Nil )

96+:}

86```

97```

88−Notice that setup starts with an empty schema `'[]` and produces `Schema`. In our `createTable` commands we included `TableConstraint`s to define primary and foreign keys, making them somewhat complex. Our tear down `Definition` is simpler.

99+We can easily see the generated SQL is unsuprising looking.

100

90```haskell

101```haskell

91−teardown :: Definition Schema '[]

102+>>> renderDefinition setup

92−teardown = dropTable #emails >>> dropTable #users

103+"CREATE TABLE users (id serial, name text NOT NULL, CONSTRAINT pk_users PRIMARY KEY (id)); CREATE TABLE emails (id serial, user_id int NOT NULL, email text, CONSTRAINT pk_emails PRIMARY KEY (id), CONSTRAINT fk_user_id FOREIGN KEY (user_id) REFERENCES rs (id) ON DELETE CASCADE ON UPDATE CASCADE);"

93```

104```

105

95−Next, we'll write `Manipulation`s to insert data into our two tables. A `Manipulation` is an `insertInto`, `update` or `deleteFrom` command and has three type parameters, the schema it refers to, a list of parameters it can take as input, and a list of columns it produces as output. When we insert into the users table, we will need a parameter for the name field but not for the id field. Since it's optional, we can use a default value. However, since the emails table refers to the users table, we will need to retrieve the user id that the insert generates and insert it into the emails table. Take a careful look at the type and definition of both of our inserts.

106+Notice that `setup` starts with an empty schema `'[]` and produces `Schema`.

107+In our `createTable` commands we included `TableConstraint`s to define

108+primary and foreign keys, making them somewhat complex. Our tear down

109+`Definition` is simpler.

110

97```haskell

111```haskell

98−insertUser :: Manipulation Schema

112+>>> :{

99− '[ 'Required ('NotNull 'PGtext)]

113+let

100− '[ "fromOnly" ::: 'Required ('NotNull 'PGint4) ]

114+ teardown :: Definition Schema '[]

101−insertUser = insertInto #users

115+ teardown = dropTable #emails >>> dropTable #users

102− ( Values (def `As` #id :* param @1 `As` #name :* Nil) [] )

116+:}

103− OnConflictDoNothing (Returning (#id `As` #fromOnly :* Nil))

117+>>> renderDefinition teardown

104−

118+"DROP TABLE emails; DROP TABLE users;"

105−insertEmail :: Manipulation Schema

106− '[ 'Required ('NotNull 'PGint4), 'Required ('Null 'PGtext)] '[]

107−insertEmail = insertInto #emails ( Values

108− ( def `As` #id :*

109− param @1 `As` #user_id :*

110− param @2 `As` #email :* Nil) [] )

111− OnConflictDoNothing (Returning Nil)

112```

119```

113

120

114−Next we write a `Query` to retrieve users from the database. We're not interested in the ids here, just the usernames and email addresses. We need to use an inner join to get the right result. A `Query` is like a `Manipulation` with the same kind of type parameters.

121+Next, we'll write `Manipulation`s to insert data into our two tables.

122+A `Manipulation` is a `insertInto`, `update` or `deleteFrom` command and

123+has three type parameters, the schema it refers to, a list of parameters

124+it can take as input, and a list of columns it produces as output. When

125+we insert into the users table, we will need a parameter for the `name`

126+field but not for the `id` field. Since it's optional, we can use a default

127+value. However, since the emails table refers to the users table, we will

128+need to retrieve the user id that the insert generates and insert it into

129+the emails table. Take a careful look at the type and definition of both

130+of our inserts.

115

131

116```haskell

132```haskell

117−getUsers :: Query Schema '[]

133+>>> :{

118− '[ "userName" ::: 'Required ('NotNull 'PGtext)

134+let

119− , "userEmail" ::: 'Required ('Null 'PGtext) ]

135+ insertUser :: Manipulation Schema '[ 'NotNull 'PGtext ]

120−getUsers = select

136+ '[ "fromOnly" ::: 'NotNull 'PGint4 ]

121− (#u ! #name `As` #userName :* #e ! #email `As` #userEmail :* Nil)

137+ insertUser = insertRow #users

122− ( from (Table (#users `As` #u)

138+ (Default `As` #id :* Set (param `1) `As` #name :* Nil)

123− & InnerJoin (Table (#emails `As` #e))

139+ OnConflictDoNothing (Returning (#id `As` #fromOnly :* Nil))

124− (#u ! #id .== #e ! #user_id)) )

140+:}

141+>>> :{

142+let

143+ insertEmail :: Manipulation Schema '[ 'NotNull 'PGint4, 'Null 'PGtext] '[]

144+ insertEmail = insertRow #emails

145+ ( Default `As` #id :*

146+ Set (param `1) `As` #user_id :*

147+ Set (param `2) `As` #email :* Nil )

148+ OnConflictDoNothing (Returning Nil)

149+:}

150+>>> renderManipulation insertUser

151+"INSERT INTO users (id, name) VALUES (DEFAULT, ($1 :: text)) ON CONFLICT DO NOTHING URNING id AS fromOnly;"

152+>>> renderManipulation insertEmail

153+"INSERT INTO emails (id, user_id, email) VALUES (DEFAULT, ($1 :: int4), ($2 :: text)N CONFLICT DO NOTHING;"

125```

154```

126

155

127−Now that we've defined the SQL side of things, we'll need a Haskell type for users. We give the type `Generic` and `HasDatatypeInfo` instances so that we can decode the rows we receive when we run getUsers. Notice that the record fields of the `User` type match the column names of `getUsers`.

156+Next we write a `Query` to retrieve users from the database. We're not

157+interested in the ids here, just the usernames and email addresses. We

158+need to use an inner join to get the right result. A `Query` is like a

159+`Manipulation` with the same kind of type parameters.

160+

161+```haskell

162+>>> :{

163+let

164+ getUsers :: Query Schema '[]

165+ '[ "userName" ::: 'NotNull 'PGtext

166+ , "userEmail" ::: 'Null 'PGtext ]

167+ getUsers = select

168+ (#u ! #name `As` #userName :* #e ! #email `As` #userEmail :* Nil)

169+ ( from (table (#users `As` #u)

170+ & innerJoin (table (#emails `As` #e))

171+ (#u ! #id .== #e ! #user_id)) )

172+:}

173+>>> renderQuery getUsers

174+"SELECT u.name AS userName, e.email AS userEmail FROM users AS u INNER JOIN emails e ON (u.id = e.user_id)"

175+```

176+Now that we've defined the SQL side of things, we'll need a Haskell type

177+for users. We give the type `Generics.SOP.Generic` and

178+`Generics.SOP.HasDatatypeInfo` instances so that we can decode the rows

179+we receive when we run `getUsers`. Notice that the record fields of the

180+`User` type match the column names of `getUsers`.

128

181

129```haskell

182```haskell

130−data User = User { userName :: Text, userEmail :: Maybe Text }

183+>>> data User = User { userName :: Text, userEmail :: Maybe Text } deriving (Show, .Generic)

131− deriving (Show, GHC.Generic)

132−instance SOP.Generic User

184+>>> instance SOP.Generic User

133−instance SOP.HasDatatypeInfo User

185+>>> instance SOP.HasDatatypeInfo User

134```

186```

135

187

136Let's also create some users to add to the database.

188Let's also create some users to add to the database.

137

189

138```haskell

190```haskell

191+>>> :{

192+let

139−users :: [User]

193+ users :: [User]

140−users =

194+ users =

141− [ User "Alice" (Just "alice@gmail.com")

195+ [ User "Alice" (Just "alice`gmail.com")

142− , User "Bob" Nothing

196+ , User "Bob" Nothing

143− , User "Carole" (Just "carole@hotmail.com")

197+ , User "Carole" (Just "carole`hotmail.com")

144− ]

198+ ]

199+:}

145```

200```

146

201

147−Now we can put together all the pieces into a program. The program connects to the database, sets up the schema, inserts the user data (using prepared statements as an optimization), queries the user data and prints it out and finally closes the connection. We can thread the changing schema information through by using the indexed `PQ` monad transformer and when the schema doesn't change we can use `Monad` and `MonadPQ` functionality.

202+Now we can put together all the pieces into a program. The program

203+connects to the database, sets up the schema, inserts the user data

204+(using prepared statements as an optimization), queries the user

205+data and prints it out and finally closes the connection. We can thread

206+the changing schema information through by using the indexed `PQ` monad

207+transformer and when the schema doesn't change we can use `Monad` and

208+`MonadPQ` functionality.

148

209

149```haskell

210```haskell

150−main :: IO ()

211+>>> :{

151−main = void $

212+let

152− withConnection "host=localhost port=5432 dbname=exampledb" . runPQ $

213+ session :: PQ Schema Schema IO ()

153− define setup

154− & pqThen session

155− & thenDefine teardown

156− where

157− session = do

214+ session = do

158− idResults <- traversePrepared insertUser (Only . userName <$> users)

215+ idResults <- traversePrepared insertUser (Only . userName <$> users)

159− ids <- traverse (fmap fromOnly . getRow (RowNumber 0)) idResults

216+ ids <- traverse (fmap fromOnly . getRow (RowNumber 0)) idResults

160− traversePrepared_ insertEmail (zip (ids :: [Int32]) (userEmail <$> users))

217+ traversePrepared_ insertEmail (zip (ids :: [Int32]) (userEmail <$> users))

161− usersResult <- runQuery getUsers

218+ usersResult <- runQuery getUsers

162− usersRows <- getRows usersResult

219+ usersRows <- getRows usersResult

163− liftBase $ print (usersRows :: [User])

220+ liftBase $ print (usersRows :: [User])

221+:}

222+>>> :{

223+void . withConnection "host=localhost port=5432 dbname=exampledb" $

224+ define setup

225+ & pqThen session

226+ & thenDefine teardown

227+:}

228+[User {userName = "Alice", userEmail = Just "alice`gmail.com"},User {userName = "Bob", userEmail = Nothing},User {userName = "Carole", userEmail = Just role`hotmail.com"}]

164```

229```

RELEASE NOTES.mdnew file+134

@@ -0,0 +1,134 @@

1+## RELEASE NOTES

3+### Version 0.2 - March 24, 2018

5+**Changes**

6+- **Constraints** - Type level table constraints like primary and foreign keys and column constraints like having `DEFAULT`.

7+- **Migrations** - Support for linear, invertible migrations tracked in an auxiliary table

8+- **Arrays** - Support for fixed- and variable-length arrays

9+- **Aliases** - Generalized `Has` constraint

10+- **Pools** - Support for pooled connections

11+- **Transactions** - Support for transaction control language

12+- **Queries, Manipulations, Definitions** - Small and large changes to Squeal's DSL

13+

14+Well, a lot of things changed!

15+

16+**Constraints**

17+

18+An important request was to bring constraints to the type level.

19+This means that more of the schema is statically known. In `0.1` column constraints - which boil down

20+to having `DEFAULT` or not - were at the type level, but they were confusingly named.

21+

22+```haskell

23+0.1: 'Optional ('NotNull 'PGInt4)

24+0.2: 'Def :=> 'NotNull 'PGInt4

25+0.1: 'Required ('NotNull 'PGInt4)

26+0.2: 'NoDef :=> 'NotNull 'PGInt4

27+```

28+

29+The `:=>` type operator is intended to helpfully connote a constraint relation.

30+It's also used for table constraints which are new in `0.2`.

31+

32+```haskell

33+"emails" :::

34+ '[ "pk_emails" ::: 'PrimaryKey '["id"]

35+ , "fk_user_id" ::: 'ForeignKey '["user_id"] "users" '["id"]

36+ ] :=>

37+ '[ "id" ::: 'Def :=> 'NotNull 'PGint4

38+ , "user_id" ::: 'NoDef :=> 'NotNull 'PGint4

39+ , "email" ::: 'NoDef :=> 'Null 'PGtext

40+ ]

41+```

42+

43+Another change in the constraint system was the removal of column constraints from

44+query and manipulation results, as result columns don't support a notion of `DEFAULT`.

45+This necessitates a distinction between `TableType`s which have both column and table

46+constraints and `RelationType`s which have neither.

47+

48+**Migrations**

49+

50+Migrations are a hot topic and many people have requested this feature. Squeal `0.2`

51+adds support for linear, invertible migrations. That is, a migration is a named,

52+invertible, schema-tracking computation:

53+

54+```haskell

55+data Migration io schema0 schema1 = Migration

56+ { name :: Text -- ^ The `name` of a `Migration`.

57+ -- Each `name` in a `Migration` should be unique.

58+ , up :: PQ schema0 schema1 io () -- ^ The `up` instruction of a `Migration`.

59+ , down :: PQ schema1 schema0 io () -- ^ The `down` instruction of a `Migration`.

60+ }

61+```

62+

63+And, `Migration`s can be put together in an "aligned" list:

64+

65+```haskell

66+data AlignedList p x0 x1 where

67+ Done :: AlignedList p x x

68+ (:>>) :: p x0 x1 -> AlignedList p x1 x2 -> AlignedList p x0 x2

69+```

70+

71+These aligned lists are free categories and might look familiar from

72+the [reflections without remorse](http://okmij.org/ftp/Haskell/zseq.pdf) technique,

73+which uses their cousins, aligned sequences.

74+

75+In the context of migration, they allow one to chain new migrations as a

76+schema evolves over time. `Migration`s' execution is tracked in an auxiliary

77+migrations table. Migration lists can then be run or rewinded.

78+

79+```haskell

80+migrateUp

81+ :: MonadBaseControl IO io

82+ => AlignedList (Migration io) schema0 schema1 -- ^ migrations to run

83+ -> PQ

84+ ("schema_migrations" ::: MigrationsTable ': schema0)

85+ ("schema_migrations" ::: MigrationsTable ': schema1)

86+ io ()

87+

88+migrateDown

89+ :: MonadBaseControl IO io

90+ => AlignedList (Migration io) schema0 schema1 -- ^ migrations to rewind

91+ -> PQ

92+ ("schema_migrations" ::: MigrationsTable ': schema1)

93+ ("schema_migrations" ::: MigrationsTable ': schema0)

94+ io ()

95+```

96+

97+**Aliases**

98+

99+In Squeal `0.1`, we had different typeclasses `HasColumn` and `HasTable` to indicate

100+that a table has a column or that a schema has a table. In Squeal `0.2` this has been

101+unified to a single typeclass,

102+

103+```haskell

104+class KnownSymbol alias =>

105+ Has (alias :: Symbol) (fields :: [(Symbol,kind)]) (field :: kind)

106+ | alias fields -> field where

107+```

108+

109+**Arrays**

110+

111+Support for array types has been added to Squeal `0.2` through

112+the `'PGfixarray` and `'PGvararray` `PGType`s. Array values can be

113+constructed using the `array` function and can be encoded from and decoded to

114+Haskell `Vector`s.

115+

116+**Pools**

117+

118+Squeal `0.2` provides a monad transformer `PoolPQ` that's an instance of `MonadPQ`.

119+The `resource-pool` library is leveraged to provide striped pools of `Connection`s.

120+`PoolPQ` should be a drop in replacement for running `Manipulation`s and `Query`s with

121+`PQ`.

122+

123+**Transactions**

124+

125+Squeal `0.2` supports a simple transaction control language. A computation in

126+`MonadPQ` can be called `transactionally` with different levels of isolation.

127+Additionally, a schema changing computation, a data definition, can be run in a

128+transaction. Running a computation in a transaction means that all SQL statements

129+will be rolled back if an exception is encountered.

130+

131+**Queries, Manipulations and Definitions**

132+

133+The above changes required major and minor changes to Squeal DSL functions.

134+Please consult the documentation.

circle.ymldeleted−12

@@ -1,12 +0,0 @@

1−dependencies:

2− cache_directories:

3− - "~/.stack"

4− - ".stack-work"

5− override:

6− - stack upgrade

7− - stack update

8− - stack setup

9− - stack test --no-run-tests

10−test:

11− override:

12− - stack test

squeal-postgresql/README.md+162−97