git.iko.soy/squeal
on dev help sign in
Merge pull request #66 from Raveline/add-a-presentation-with-examples
07fd591Eitan Chatav
squeal-presentation-2018-09-12.htmlnew file+876
compare with dev
unifiedsplit
@@ -0,0 +1,876 @@
1+<!DOCTYPE html>
2+<html>
3+<head>
4+ <title>Title</title>
5+ <meta charset="utf-8">
6+ <style>
7+ @import url(https://fonts.googleapis.com/css?family=PT+Sans+Narrow);
8+ @import url(https://fonts.googleapis.com/css?family=Montserrat:400,700,400italic);
9+ @import url(https://fonts.googleapis.com/css?family=Ubuntu+Mono:400,700,400italic);
10+
11+ body { font-family: 'Montserrat'; }
12+ h1, h2, h3 {
13+ font-family: 'PT Sans Narrow';
14+ font-weight: normal;
15+ }
16+
17+ .note {
18+ background-color: #ccc;
19+ padding: 1rem;
20+ border-radius: 4px;
21+ }
22+ .remark-code, .remark-inline-code { font-family: 'Ubuntu Mono'; }
23+
24+ .signature {
25+ position: absolute;
26+ top: 1%;
27+ right: 1%;
28+ font-size:.6em;
29+ opacity: .5;
30+ }
31+ </style>
32+ </head>
33+ <body>
34+ <textarea id="source">
35+
36+class: center, middle
37+count: false
38+
39+# Squeal
40+
41+_A bridge between SQL and Haskell_
42+
43+.signature[2018/09/12 - by @Raveline]
44+
45+---
46+
47+# Rationale
48+
49+Using `postgresql-simple` you have a typical "Trial, error, despair" workflow:
50+
51+- Write query manually or semi-automatically
52+
53+- Write haskell code using the query
54+
55+- Pray that the query syntax is correct
56+
57+- Pray that it returns what you want
58+
59+- Realize the inefficiency of prayer when it comes to SQL
60+
61+- Iterate till it works
62+
63+---
64+
65+class: center, middle
66+count: false
67+
68+![Can't take this anymore](http://gif.eraveline.eu/static/img/0x16e.gif)
69+
70+---
71+
72+# Use your best friend: GHC
73+
74+Squeal provides several eDSL to make your SQL typesafe:
75+
76+- Type level eDSL to express schema;
77+
78+- Value level eDSL to manipulate schema (plus migration, yeehaw !);
79+
80+- Type level eDSL to express queries;
81+
82+- Value level eDSL to perform queries
83+
84+However, it's not an ORM. There's no caching, lazy loading - you retain control over your memory. Also, joins - and mostly aggregation after joins - have to be handled manually.
85+
86+---
87+
88+class: center, middle
89+count: false
90+
91+# Part I. Schema & migration
92+
93+---
94+
95+# The Schema
96+
97+- The schema will be used to validate _everything_: migration, queries, etc.
98+
99+- A schema is mostly a collection of tables.
100+
101+- In this presentation, we will create a very basic database modeling a parliament. We will store _members of parliament_ and _parliamentary groups_.
102+
103+- A simple Schema to represent a Parliament could be:
104+
105+```haskell
106+type Schema = '[ "mp" ::: 'Table MemberOfParliament
107+ , "groupp" ::: 'Table ParliamentGroup ]
108+```
109+
110+--
111+
112+.note[*
113+We need the `DataKinds` extension to be able to express heterogenous lists containing
114+specific types like this one.
115+
116+*
117+You can perfectly call your table "group" and not "groupp" even though it is a
118+keyword in SQL - Squeal queries will be properly escaped.
119+]
120+
121+---
122+
123+# A small example
124+
125+```haskell
126+type ParliamentaryGroup =
127+ '[ "pk_groupp" ::: 'PrimaryKey '["groupp_id"]]
128+ :=>
129+ '[ "groupp_id" ::: 'NoDef :=> 'NotNull 'PGuuid
130+ , "name" ::: 'NoDef :=> 'NotNull 'PGtext
131+ ]
132+```
133+
134+- That's a whole simple table defined in one go.
135+
136+- `:::` lets us define a column or a constraint.
137+
138+- `:=>` associates the constraints to the column.
139+
140+- Let's split constraints and column to study the syntax a bit more.
141+
142+--
143+
144+.note[*
145+We are using `:::` and `:=>` to quickly express associations when writing
146+our schema. We need the `TypeOperators` extension.]
147+
148+---
149+
150+class: center, middle
151+count: false
152+
153+![What does this means ?!](http://gif.eraveline.eu/static/img/0x27e.gif)
154+
155+---
156+
157+# Defining a table
158+
159+```haskell
160+type MemberOfParliament =
161+ '[ "pk_mp" ::: 'PrimaryKey '["mp_id"]
162+ , "fk_mp_groupp" ::: 'ForeignKey '["mp_group"] "groupp" '["groupp_id"]
163+ ] :=> MpCols
164+```
165+
166+- A table is a collection of constraints associated to a collection of columns.
167+
168+- We defined a table named "mp".
169+
170+- It has a constraint named "pk_mp", defining its primary key on column "mp_id".
171+
172+- It has a constraint named "fk_mp_groupp" defining a foreign key on column "mg_group", connected to the column "groupp_id" of table "groupp".
173+
174+- These constraints will be associated to the columns defined in `MpCols`.
175+
176+---
177+
178+# Defining columns
179+
180+```haskell
181+type MpCols =
182+ '[ "mp_id" ::: 'NoDef :=> 'NotNull 'PGuuid
183+ , "first_name" ::: 'NoDef :=> 'NotNull 'PGtext
184+ , "last_name" ::: 'NoDef :=> 'NotNull 'PGtext
185+ ]
186+```
187+
188+- A listing of column associate, for each element:
189+
190+ * a name;
191+
192+ * the mention of an eventual default value;
193+
194+ * the nullability;
195+
196+ * the type (obviously).
197+
198+--
199+
200+.note[*
201+GHC is already helping. If I named the "mp_id" column differently, GHC would yell because I promised a primary key constraint on a column named `mp_id`, so there must be one.]
202+
203+---
204+
205+# Implementating the schema
206+
207+- We will carry this schema type pretty much everywhere.
208+
209+- But before we play with this schema, we need to implement it.
210+
211+--
212+
213+```haskell
214+setup :: Definition '[] Schema
215+setup =
216+ createTable #groupp
217+ ( notNullable uuid `as` #groupp_id
218+ :* notNullable text `as` #name)
219+ ( primaryKey #groupp_id `as` #pk_groupp )
220+ >>> createTable #mp
221+ ( notNullable uuid `as` #mp_id
222+ :* notNullable text `as` #first_name
223+ :* notNullable text `as` #last_name
224+ :* notNullable text `as` #mp_group)
225+ ( primaryKey #mp_id `as` #pk_mp
226+ :* foreignKey #mp_group #groupp #groupp_id
227+ OnDeleteCascade OnUpdateCascade
228+ `as` #fk_mp_groupp)
229+```
230+
231+---
232+
233+# Damn, that's verbose
234+
235+- Yes. But you need verbosity to get type safety.
236+
237+- On the plus side, it's fairly straightforward.
238+
239+- You use `:*` to compose the element of the heterogenous list of columns.
240+
241+- You use `>>>` to compose table creation.
242+
243+- Note that the compiler will catch any mistype between Schema and
244+definition; wrong nullability, wrong type, wrong name, etc.
245+
246+--
247+
248+.note[*
249+You'll also need `OverloadedLabels`, for naming stuff.
250+This is mostly to avoid having to write manual proxies all
251+the time and for convenience.]
252+
253+---
254+
255+# Setting up the schema
256+
257+- Squeal comes with a very good migration manager, handling _upgrades_ AND
258+_downgrades_.
259+
260+- A migration is a simple type:
261+
262+```haskell
263+Migration io schema0 schema1
264+```
265+
266+- The first parameter is a BaseMonad (typically, IO).
267+
268+- The second parameter is "the current schema of your DB".
269+
270+- The last one is "what you will migrate to".
271+
272+- The `Definition` type used to define upgrade and downgrade functions use the
273+ same logic (from one schema to the other).
274+
275+---
276+
277+# Defining our first migration
278+
279+- We have the setup bit, we need the teardown:
280+
281+```haskell
282+tearDown :: Definition Schema '[]
283+tearDown = dropTable #mp >>> dropTable #groupp
284+```
285+
286+- Note that GHC will also detect the _proper_ order of what you typed in
287+ downgrade and upgrade should there be any conflict (with foreign keys).
288+
289+```haskell
290+initDB :: Migration IO '[] Schema
291+initDB =
292+ Migration { name = "Schema creation"
293+ , up = void $ define setup
294+ , down = void $ define tearDown }
295+```
296+
297+---
298+
299+# Simple migrator example
300+
301+```haskell
302+main :: IO ()
303+main = do
304+ printSQL setup
305+ void $ withConnection connectionString $
306+ migrateUp $ single initDB
307+```
308+
309+- We print the migration query; you can do this for all queries generated through Squeal.
310+
311+- We use `migrateUp` to perform the migration.
312+
313+- You can run several migration at the same time, to run only one use `single`.
314+
315+---
316+
317+class: center, middle
318+count: false
319+
320+![That was easy](http://gif.eraveline.eu/static/img/0x47f.gif)
321+
322+---
323+
324+class: center, middle
325+count: false
326+
327+# Part II. Insertions
328+
329+---
330+
331+# Manipulations & Queries
332+
333+Besides the specific case of migration, you will mostly perform:
334+
335+- `Manipulation` : inserting, updating and deleting data.
336+
337+- `Query` : fetching data.
338+
339+- Both types are parametric over the same things:
340+
341+ * A schema
342+
343+ * Input parameters
344+
345+ * Output
346+
347+- Simplified, they look like this:
348+
349+```haskell
350+Manipulation schema params columns
351+
352+Query schema params columns
353+```
354+
355+
356+---
357+
358+# Our Haskell model
359+
360+```haskell
361+type Parliament = [Group]
362+
363+data Group =
364+ Group { name :: Text
365+ , members :: [MemberOfParliament] }
366+
367+data MemberOfParliament =
368+ MemberOfParliament { firstName :: Text
369+ , lastName :: Text }
370+```
371+
372+- You'll note that we didn't use anything from Squeal.
373+
374+- The model can be entirely separated from the persistence layer.
375+
376+---
377+
378+# Inserting a Parliamentary Group
379+
380+- A group is very simple: it's a uuid and a name. Let's define our params:
381+
382+```haskell
383+type GroupInsertionParams = '[ 'NotNull 'PGuuid
384+ , 'NotNull 'PGtext ]
385+```
386+
387+- Params are not named, but they are indexed. You just need nullability and type.
388+
389+--
390+
391+```haskell
392+groupInsertion :: Manipulation Schema GroupInsertionParams '[]
393+groupInsertion =
394+ insertRow_ #groupp ( Set (param @1) `as` #groupp_id
395+ :* Set (param @2) `as` #name )
396+```
397+
398+--
399+
400+- `TypeApplication` lets us use the index of parameters (counting from 1).
401+
402+- Once again: all this is checked by the compiler. Wrong name, wrong type... you'll get a compile error.
403+
404+- `insertRow_` is a simplified version of `insertRow`. `insertRow` lets you express a `RETURNING` clause and the expected behaviour in case of conflict.
405+
406+---
407+
408+_Inserting a Member of Parliament_
409+# Doing an INSERT INTO ... SELECT
410+
411+- We could create a naive query that takes MP uuid, first name, last name and group uuid...
412+
413+--
414+
415+- But that's boring. So let's use the `INSERT INTO ... SELECT`.
416+
417+--
418+
419+- We will build a query that will return as constants our MP's uuid, first name and last name...
420+
421+--
422+
423+- ... and fetch the uuid of a group given the name of the group.
424+
425+--
426+
427+- There's a `insertQuery` utility function for that. All we have to do is write the `select` !
428+
429+```haskell
430+mpInsertion :: Manipulation Schema MpParams '[]
431+mpInsertion =
432+ insertQuery_ #mp selectGroup -- we just have to write selectGroup !
433+```
434+
435+---
436+
437+_Inserting a Member of Parliament_
438+# The query type
439+
440+- A query looks just like a `Manipulation`:
441+
442+```haskell
443+Query schema params returns
444+```
445+
446+- Returns use a special syntax, that demands names, nullability and types:
447+
448+```haskell
449+type ExampleReturnType = '[ "some_column" ::: 'NotNull 'PGuuid
450+ , "other_column" ::: 'Null 'PGtext ]
451+```
452+
453+- It is not the same as the column definition we used in the schema. That one also expects that you specify, for each column, an eventual default:
454+
455+```haskell
456+type ExampleSchemaCols =
457+ '[ "some_column" ::: 'NoDef :=> 'NotNull 'PGuuid
458+ , "other_column" ::: 'NoDef :=> 'Null 'PGtext ]
459+```
460+
461+---
462+
463+_Inserting a Member of Parliament_
464+# Don't rewrite column definitions
465+
466+- We want our return type to be "all columns from table MP expressed as return type".
467+
468+- But we cannot reuse our neat `MpCols` alias, since the type do not match as we've just seen.
469+
470+- Hopefully, there's a neat Type Family that will let you convert any table you defined to the type of the equivalent row: `TableToRow`.
471+
472+- We finally have the signature for our SELECT query.
473+
474+```haskell
475+selectGroup :: Query Schema MpParams (TableToRow MemberOfParliament)
476+```
477+
478+---
479+
480+_Inserting a Member of Parliament_
481+# Our final Select query
482+
483+- Our intermediate query looks like this:
484+
485+```haskell
486+selectGroup =
487+ select
488+ ( param @1 `as` #mp_id
489+ :* param @2 `as` #first_name
490+ :* param @3 `as` #last_name
491+ :* #groupp ! #groupp_id `as` #mp_group
492+ )
493+ ( from (table #groupp)
494+ & where_ ( #groupp ! #name .== param @4 )
495+ )
496+```
497+
498+- The first three columns are constant defined through our params;
499+
500+- ... we alias the column name using `as`...
501+
502+- ... and when we need the result from the table we use the `#table ! #column` syntax.
503+
504+- We'll go back to the "from" block later.
505+
506+---
507+
508+# Dealing with the connection context
509+
510+- Queries are runned in a connection context:
511+
512+ * When dealing with a single-connection context, use the type `PQ`.
513+
514+ * When dealing with a pool of connection, use the type `PoolPQ`.
515+
516+- Or get rid of the context and use `mtl` style, with the typeclass `MonadPQ`.
517+
518+- We'll use that to demonstrate how to actually run our queries.
519+
520+---
521+
522+# Inserting a whole group
523+
524+```haskell
525+insertGroup :: (MonadPQ Schema m, MonadBaseControl IO m) => Group -> m ()
526+insertGroup g@(Group name _) = do
527+ uuid' <- liftBase nextRandom
528+ void $ manipulateParams groupInsertion (uuid', name)
529+ insertMps g
530+```
531+
532+- Remember that we defined a group as a name and a list of MPs.
533+
534+- We are in `MonadBaseControl`, so we cannot use `liftIO`, we need `liftBase`.
535+
536+- To perform a simple insertion, use `manipulateParams`. It takes into parameter
537+instances of `ToParam`, but you will typically use tuples or Generic-SOP.
538+
539+---
540+
541+# Inserting a bunch of MPs
542+
543+```haskell
544+insertMps :: (MonadPQ Schema m, MonadBaseControl IO m) => Group -> m ()
545+insertMps (Group groupName mps) =
546+ let tuplify Mp{..} = (, firstName, lastName, groupName) <$> nextRandom
547+ params = traverse tuplify mps
548+ in void $ liftBase params >>= traversePrepared mpInsertion
549+```
550+
551+- This time, we want to do a `preparedStatement`. We'll use `traversePrepare`
552+which behaves like `manipulateParams`, but expect a list of `ToParam`
553+instances.
554+
555+- We build our tuple manually again, mostly because we want to generate UUID on
556+the fly and we don't want to have them in our model.
557+
558+---
559+
560+# Inserting a whole parliament
561+
562+```haskell
563+insertParliament ::
564+ (MonadPQ Schema m, MonadBaseControl IO m) => Parliament -> m ()
565+insertParliament = traverse_ insertGroup
566+```
567+
568+- That's simple enough !
569+
570+- If we want to be a bit safer, though, we can wrap this call in a
571+`transactionally_` function, which will put all that in a transaction.
572+
573+---
574+
575+class: center, middle
576+count: false
577+
578+![Very simple](https://media1.tenor.com/images/0188c63209aced59f1583e1ca94e509e/tenor.gif?itemid=3550689)
579+
580+
581+---
582+
583+class: center, middle
584+
585+# Part III. Selects
586+
587+---
588+
589+# Composable queries
590+
591+- Building basic queries is easy, and is well documented.
592+
593+- However, the real interest of a tool like Squeal is in DRYness, and the documentation is still lacking in "how-to" related to composability.
594+
595+- I'll build an example showing how column selections and from clause can be factorized.
596+
597+- We want to write two queries:
598+
599+ * One to get all members of a specific group.
600+
601+ * One to get all the parliament.
602+
603+---
604+
605+# Decomposing a `Query`
606+
607+- A query typically stars with `select` (variants are available).
608+
609+- It then takes:
610+
611+ * A heterogenous list of fields with a scary signature;
612+
613+ * A virtual table (the from clause and filter clauses), called a `TableExpression`.
614+
615+- And return fields. Ours will look like this:
616+
617+```haskell
618+type GroupRowResult =
619+ '[ "groupName" ::: 'NotNull 'PGtext
620+ , "firstName" ::: 'NotNull 'PGtext
621+ , "lastName" ::: 'NotNull 'PGtext ]
622+```
623+
624+---
625+
626+# The Table Expression
627+
628+- It necessarily contains a `fromClause` (table, view or subquery, plus optional joins).
629+
630+- You can add where, groups, "HAVING" clause, order, etc.
631+
632+- `from` creates a basic `TableExpression` that you extend through various function to add clauses.
633+
634+- For our queries, we will share a common `FromClause`.
635+
636+---
637+
638+# The From Clause
639+
640+```
641+FromClause schema params from
642+```
643+
644+- We need Schema, input parameters and a `FromType` giving the available fields of the expression.
645+
646+- In our case, we'll join the table `mp` and the table `groupp`, meaning the `from` will be all fields of these tables.
647+
648+- We can use `TableToRow` to be more DRY:
649+
650+```haskell
651+type BaseParliamentSelection =
652+ '[ "g" ::: TableToRow GroupCols
653+ , "m" ::: TableToRow MpCols ]
654+```
655+
656+- Note that we've also put everything with table aliases: "g" and "m".
657+
658+---
659+
660+# Writing our from clause
661+
662+```haskell
663+baseParliamentTables ::
664+ FromClause Schema (param :: [NullityType]) BaseParliamentSelection
665+baseParliamentTables =
666+ table
667+ (#groupp `as` #g)
668+ & innerJoin
669+ (table (#mp `as` #m))
670+ (#m ! #mp_group .== #g ! #groupp_id)
671+```
672+
673+- When picking "from" something, you need to specify the type with a function: `table` for a table, `view` for a view, etc.
674+
675+- All joins are available. `innerJoin` is the most basic one. It takes the joined table and the joining condition.
676+
677+- We have to specifiy the type of `param` (current limitation of the lib), even though _any_ params will be compatible with this, so we can plug `where` clauses depending on params should we need to !
678+
679+---
680+
681+# Typing the common selection
682+
683+- Both our querie will need the same fields. This has to be factorized too.
684+
685+- The scary signature of selection fields:
686+
687+```haskell
688+NP (Aliased (Expression schema from grouping params)) cols
689+```
690+
691+- `NP` is for heterogenous lists ("n-ary product").
692+
693+- `from` is the virtual table type, so our `BaseParliamentSelection`.
694+
695+- `grouping` is there to make a distinction between aggregated / unaggregated queries.
696+
697+- And finally, cols is the return type.
698+
699+---
700+
701+# Our common selection
702+
703+```haskell
704+groupSelection ::
705+ NP (Aliased (Expression Schema BaseParliamentSelection 'Ungrouped param))
706+ GroupRowResult
707+groupSelection =
708+ #g ! #name `as` #groupName
709+ :* #m ! #first_name `as` #firstName
710+ :* #m ! #last_name `as` #lastName
711+```
712+
713+- I hope you like big signatures.
714+
715+- But it's what let GHC checks that all alias and columns are available.
716+
717+- And that you're returning what you really intend to return.
718+
719+---
720+
721+# Putting it all together
722+
723+```haskell
724+selectParliament :: Query Schema '[] GroupRowResult
725+selectParliament =
726+ select groupSelection (from baseParliamentTables)
727+```
728+
729+```haskell
730+selectGroupMembers :: Query Schema '[ 'NotNull 'PGtext] GroupRowResult
731+selectGroupMembers =
732+ select groupSelection
733+ (from baseParliamentTables
734+ & where_ (#g ! #name .== param @1))
735+```
736+
737+- That is fairly DRY. And _entirely typesafe_.
738+
739+---
740+
741+# Actually fetching the data
742+
743+- We'll build an intermediary datatype representing our rows.
744+
745+- We'll make it match our row.
746+
747+- And we'll add generic-SOP so that we can build them from query results.
748+
749+```haskell
750+type GroupRowResult =
751+ '[ "groupName" ::: 'NotNull 'PGtext
752+ , "firstName" ::: 'NotNull 'PGtext
753+ , "lastName" ::: 'NotNull 'PGtext ]
754+```
755+
756+```haskell
757+data GroupRow =
758+ GroupRow { groupName :: Text
759+ , firstName :: Text
760+ , lastName :: Text }
761+ deriving (Generic)
762+
763+instance SOP.Generic GroupRow
764+instance SOP.HasDatatypeInfo GroupRow
765+```
766+
767+---
768+
769+# Aggregate result logic
770+
771+- We will get tabular, SQL data.
772+
773+- Our results will be `[GroupRow]`.
774+
775+- We'll build `Groups` from this:
776+
777+```haskell
778+buildGroup :: NE.NonEmpty GroupRow -> Group
779+buildGroup grs =
780+ let buildMP (GroupRow _ f l) = Mp f l
781+ name = groupName . NE.head $ grs
782+ members = NE.toList (buildMP <$> grs)
783+ in Group{..}
784+
785+rowToGroups :: [GroupRow] -> [Group]
786+rowToGroups =
787+ let grouped = fmap NE.fromList . L.groupBy ((==) `on` groupName)
788+ in fmap buildGroup . grouped
789+```
790+
791+---
792+
793+# Finally calling our queries
794+
795+- Query with params will use `runQueryParams`.
796+
797+- Query without params will use `runQuery`.
798+
799+- Result is inside a `MonadPQ` and of type `K` from generic-sop.
800+
801+- You get your actual result using `getRows`. Exemple:
802+
803+```haskell
804+getParliament :: (MonadPQ Schema m, MonadBaseControl IO m) => m [Group]
805+getParliament = do
806+ res <- runQuery selectParliament
807+ rows <- getRows res
808+ pure $ rowToGroups rows
809+```
810+
811+- In real life of course you'll write:
812+
813+```haskell
814+getParliament =
815+ rowToGroups <$> (runQuery selectParliament >>= getRows)
816+```
817+
818+---
819+
820+# Our query with params
821+
822+```haskell
823+getGroupMembers ::
824+ (MonadPQ Schema m, MonadBaseControl IO m) => Text -> m (Maybe Group)
825+getGroupMembers =
826+ listToMaybe . rowToGroups <$>
827+ (runQueryParams selectGroupMembers (Only t) >>= getRows)
828+```
829+
830+- And that's it.
831+
832+---
833+
834+class: center, middle
835+count: false
836+
837+![Eazy](https://media.tenor.com/images/8fc7c4077efe11b4a3a3b9ae4e643e87/tenor.gif)
838+
839+---
840+
841+class: center, middle
842+count: false
843+
844+# Conclusion
845+
846+---
847+
848+# Pros & Cons
849+
850+- Not everything is included: some cool stuff like array_agg and window functions are not available yet.
851+
852+- But development is _very_ active. `IN` clauses were missing but were added in the 4.0 released recently.
853+
854+- It's bleeding edge. You need latest LTS to be comfortable and psql >= 9.5.
855+
856+- It's Postgres-only but I would say that's a feature. Multi-DB tools are even more complex.
857+
858+- The author & maintainer is <3.
859+
860+- Typesafe. 'nuff said.
861+
862+---
863+class: center, middle
864+count: false
865+
866+# Thank you !
867+
868+
869+ </textarea>
870+ <script src="https://remarkjs.com/downloads/remark-latest.min.js">
871+ </script>
872+ <script>
873+ var slideshow = remark.create({countIncrementalSlides: false});
874+ </script>
875+ </body>
876+</html>