git.iko.soy/project-m36
on master help sign in
Merge branch 'master' into 165_more_static_optimizations
commit60228c54070b7621b11332192b3712a7bd2d0691 authorAgentM <agentm@themactionfaction.com> dateSun 26 Nov 2017, 17:53 (9 years ago) parentd6658314b4b595
38 files changed+1380−100
compare with master
unifiedsplit
.gitignore+5−1 Changelog.markdown+6 README.markdown+7−2 docs/filesystem-persistence.markdown+128 docs/jupyter_kernel.markdown+49 docs/reaching_out_of_the_tarpit.markdown+6−6 jupyter/TutorialD Notebook Walkthrough.ipynb+794 jupyter/itutd/ITutorialDKernel.py+179 jupyter/itutd/__init__.py+1 jupyter/itutd/__main__.py+4 jupyter/kernel.json+5 jupyter/setup.py+19 project-m36.cabal+2−1 src/bin/ProjectM36/Server/WebSocket.hs+54−21 src/bin/ProjectM36/Server/WebSocket/project-m36.js+3−3 src/bin/ProjectM36/Server/WebSocket/websocket-client.html+1−1 src/bin/TutorialD/Interpreter.hs+2−2 src/bin/TutorialD/Interpreter/DatabaseContextExpr.hs+2−1 src/bin/TutorialD/Interpreter/TransactionGraphOperator.hs+2−1 src/lib/ProjectM36/AtomType.hs+2−2 src/lib/ProjectM36/Attribute.hs+6−3 src/lib/ProjectM36/Client.hs+1−1 src/lib/ProjectM36/Client/Simple.hs+1−1 src/lib/ProjectM36/Error.hs+1 src/lib/ProjectM36/FileLock.hs+3−4 src/lib/ProjectM36/IsomorphicSchema.hs+8−8 src/lib/ProjectM36/MiscUtils.hs+6 src/lib/ProjectM36/Relation.hs+13−6 src/lib/ProjectM36/Relation/Show/HTML.hs+33−10 src/lib/ProjectM36/Relation/Show/Term.hs+2−9 src/lib/ProjectM36/RelationalExpression.hs+1−1 src/lib/ProjectM36/ScriptSession.hs+1−1 src/lib/ProjectM36/StaticOptimizer.hs+1−1 src/lib/ProjectM36/TransactionGraph.hs+1−1 src/lib/ProjectM36/Tuple.hs+8−4 test/Relation/Basic.hs+12−1 test/Server/WebSocket.hs+10−7 test/TransactionGraph/Merge.hs+1−1
.gitignore+5−1
@@ -14,4 +14,8 @@ nextsteps
1414\#*\#
1515view-system-notes
1616persistence-notes
17−*.pyc
17+*.pyc
18+jupyter/itutd.egg-info
19+jupyter/build
20+juyter/dist
21+.ipynb_checkpoints
Changelog.markdown+6
@@ -1,3 +1,9 @@
1+# 2017-11-14
2+
3+* alter websocket server API to allow for multiple representations (JSON, text, or HTML) to be selected and returned simultaneously
4+* add jupyter kernel for TutorialD interpreter
5+* fix warnings suggested by new hlint 2.0.10
6+
17# 2017-10-08 (v0.3)
28
39* replaced overuse of `undefined` with `Proxy` in `Tupleable` and `Atomable` typeclasses
README.markdown+7−2
@@ -50,6 +50,7 @@ Project:M36 supports multiple frontends which target different audiences.
5050
51511. [Installation and Introduction to Project:M36](docs/introduction_to_projectm36.markdown)
52521. [Introduction to the Relational Algebra](docs/introduction_to_the_relational_algebra.markdown)
53+1. [TutorialD via Jupyter Notebook Walkthrough](jupyter/TutorialD%20Notebook%20Walkthrough.ipynb)
53541. [TutorialD Tutorial](docs/tutd_tutorial.markdown)
54551. [15 Minute Tutorial](docs/15_minute_tutorial.markdown)
55561. [Developer's Change Log](Changelog.markdown)
@@ -70,12 +71,16 @@ Project:M36 supports multiple frontends which target different audiences.
70711. [Serving Remote ProjectM36 Databases](docs/server_mode.markdown)
71721. [Using Notifications](docs/using_notifications.markdown)
72731. [Merge Transactions](docs/merge_transactions.markdown)
73−1. [WebSocket Server](docs/websocket_server.markdown)
74741. [Atom (Value) Functions](docs/atomfunctions.markdown)
75751. [Trans-Graph Relational Expressions](docs/transgraphrelationalexpr.markdown)
76761. [Isomorphic Schemas](docs/isomorphic_schemas.markdown)
77771. [Replication](docs/replication.markdown)
7878
79+### Integrations
80+
81+1. [WebSocket Server](docs/websocket_server.markdown)
82+1. [Jupyter Notebook Kernel](docs/jupypter_kernel.markdown)
83+
7984## Development
8085
8186Project:M36 is developed in Haskell and compiled with GHC 7.10 or GHC 8.0.2 or later.
@@ -89,6 +94,6 @@ Project:M36 is developed in Haskell and compiled with GHC 7.10 or GHC 8.0.2 or l
8994
9095## Suggested Reading
9196
92−* [Out of the Tarpit](http://shaffner.us/cs/papers/tarpit.pdf): a proposed software architecture which minimizes state and complexity. Project:M36 implements the requirements of this paper.
97+* [Out of the Tarpit](https://github.com/papers-we-love/papers-we-love/blob/2eb8d21/design/out-of-the-tar-pit.pdf): a proposed software architecture which minimizes state and complexity. Project:M36 implements the requirements of this paper.
9398* [Database Design & Relational Theory: Normal Forms and All That Jazz](http://shop.oreilly.com/product/0636920025276.do): mathematical foundations for the principles of the relational algebra
9499* [Database Explorations: Essays on the Third Manifesto and Related Topics](http://bookstore.trafford.com/Products/SKU-000177853/Database-Explorations.aspx): additional essays and debates on practical approaches to relational algebra engine design
docs/filesystem-persistence.markdownnew file+128
@@ -0,0 +1,128 @@
1+# Project-M36 Filesystem Persistence
2+
3+## Introduction
4+
5+Existing database persistent storage is not suitable for Project:M36 storage for several reasons:
6+
7+* typical SQL storage doesn't support relation-valued attributes
8+ * Project:M36 supports nested relations, relations as values- a useful feature of the relational algebra
9+* typical SQL storage relies on writing and rewriting files to save space
10+ * Project:M36 uses a write-one-read-many strategy to support database branching, concurrency, and compression
11+* typical SQL storage maps each table to one or more files
12+ * Project:M36 obviates the one-to-one mapping between relation variables and files and allows for multiple representations of a single relation variable and evaluated expressions on disk
13+* typical SQL storage maintains only the latest state of the database
14+ * Project:M36 stores a graph of transactions and supports multiple heads of contention
15+* typical key-value storage is poorly typed or even schemaless
16+ * Project:M36 is strongly-typed which enforces correctness and can benefit compression schemes and performance
17+
18+## Requirements
19+
20+* storage of an append-at-leaves (transaction) graph with highly concurrent read access
21+* database [durability](https://en.wikipedia.org/wiki/ACID#Durability)
22+* extensibility
23+ * examples include future compression types, dynamic atom (value) types, database context extensions, new relation types (such as recurrence relations), backwards-compatibility for atom values
24+* platform-agnosticism
25+ * postgresql's C-struct-to-disk design means that switching platforms requires a dump-and-restore, even if the database directory can be physically moved to the new machine- it's not always clear what constitutes a platform changes (float/int timestamps)
26+* upgradeability
27+ * postgresql software upgrades require painful database dump-and-restores or pg_upgrade, causing downtime
28+ * be able to read older formats to prevent costly data-rewrites
29+* backwards-compatibility
30+ * no one wants to rewrite their database files during upgrades, even if it is on-the-fly like with pg_upgrade
31+* multiple remote readers and writers to the storage
32+ * mutliple servers can read/write the same storage over networked filesystems as long as there are some basic locking guarantees
33+ * multiple servers can read the same storage without locking guarantees
34+ * we can protect WORM storage with fine-grained locking and still maintain high-concurrency
35+* singular format for data export/import (dump/restore)
36+ * we won't need a pseudo-cross-platform dump format (like pg_dump's output) because the file format is already cross-platform
37+ * sharing the format for internal and external usage means we can share the same tools
38+ * the dump format can be a subset of the internal format
39+* existence of a Haskell-based serialization library
40+* POSIX filesystem semantics and atomicity guarantees
41+
42+## Current, Insufficient Design
43+
44+### Filesystem Requirements
45+
46+* POSIX, journaled filesystem
47+ * ext3/4, xfs, etc.
48+ * simulate POSIX semantics on Windows possibly with locking
49+
50+The first pass of the on-disk storage for Project:M36 was implemented with a focus on correctness, but not performance or intelligent disk storage usage. As such, it is valuable as a baseline for the new implementation.
51+
52+* uses Data.Binary to serialize entire Haskell data structures to disk
53+* includes lots of redundancy
54+ * for example, each new transaction writes all current relation variables to disk- instead, we should be able to reference unchanged relvars from previous transactions (they are already safely on disk)
55+* loses the in-memory data structure sharing that was present when writing the data (one of the benefits of immutable data structures)
56+* writes a platform-specific format
57+* is not extensible, backwards-, or forwards-compatible
58+* cannot be used as a portable, exportable format
59+* compression is all-or-nothing
60+
61+### Directory Layout
62+
63+* /m36v*x* - includes a human-readable transaction graph listing. The *x* references the version of persistence format.
64+* /lockFile - an empty file used for synchronizing multiple project-m36 server contending for the transaction graph
65+* /heads - a human-readable mapping of head names to transaction ids
66+* /<transaction id> - a directory containing all data needed to reconstruct a transaction
67+ * /atomfuncs - a Binary-serialized listing of `AtomFunction`s
68+ * /dbcfuncs - a Binary-serialized listing of `DatabaseContextFunction`s
69+ * /incdeps - a Binary-serialized listing of `InclusionDependency`s
70+ * /info - a Binary-serialized listing of `TransactionInfo` (current and parent transaction ids)
71+ * /relvars - a directory containing files named after the relvars with each file's contents being a serialized `Relation`
72+ * dynamic names should not appear in the filesystem as the filesystem naming requirements will be different from Project:M36 naming requirements
73+ * /schemas - a Binary-serialized listing of alternative isomorphic schemas
74+ * /typecons - a Binary-serialized listing of `TypeConstructor`s (data types)
75+
76+## New, Shiny, Proposed Design
77+
78+### Directory Layout
79+
80+Most of the existing directory layout remains useful. The one-transaction-per-directory layout maximizes read performance using a WORM strategy. Later transactions can refer to past transactions to cut down on immediate IO. Due to the idempotence of the `DatabaseContextExpr`, we can optionally serialize the expression itself to further reduce and defer IO. Deferred updates can be resolved on-demand or in the background by replacing transaction directories, individual files, or by adding supplementary relation representations (more on these below).
81+
82+Improvements:
83+
84+* wrap up small objects together
85+ * We experienced unnecessarily high disk usage when saving atom functions in individual files. Wrap up smaller data structures (especially when less than 4KB) into one file.
86+* use [cbor](https://github.com/well-typed/cborg) binary format to allow external tools to be able to read the on-disk format
87+ * the cbor format allows for forwards and backwards compatibility for atom types and includes relevant built-in types
88+
89+### Relation representations
90+
91+One area where almost every DBMS has fallen flat is in *data independece*, the separation between the logical database state representation and the on-disk format. Project:M36 aims to remedy this with "relation representations" which acknowledges that every relation (including the results of relational expresssions) can have multiple on-disk or in-memory representations. The purpose of a query planner is to choose the optimal representation for the given request.
92+
93+Most DBMSes do make an inadvertent attempt to support multiple relation representations via indexes. An index can be viewed as an independent representation of a relation in a different format (optimized for certain queries), but these same DBMSes fail to hide this abstraction in the logical layer, instead forcing users to create and maintain indexes as a separate entity.
94+
95+Relational representations, however, are never presented to the user of the logical layer of the database. Such representations can be created automatically. For example, if a certain query result (always a relation) is requested often, the DBMS may choose to cache the result as a relation representation mapped to the query itself. Thereafter, if the query is requested again (within an unchanged database state, of course), the planner can choose to send the pre-computed result back. This is known as a "query cache", but we can extend this same model to any other possible representation.
96+
97+Another representation type includes alternative representations of a relation variable's value designed for specific projections or restrictions. For example, a relation may be stored on disk as a btree with the nodes representing a relation's key. This is obviously equivalent to an index (either populated with the remainder of the tuple attributes or not, if the query is merely checking for existence).
98+
99+Note that if two relation representations are isomorphic, as in the example of a btree containing the key and remaining tuple values, then all other representations of the same relation are redundant. Under storage pressure, we can delete all but one representation. Which one the DBMS chooses to delete is arbitrary. When we have adequate space, we can create additional representations in the background without affecting correctness.
100+
101+The on-disk representation need not correspond to the logical relation variables at all. Besides query caches, the DBMS can freely store multiple relations in the same file as long as there is a relational expression to distinguish between the logical relation variables. Conversely, a single relation variable's value could span multiple files, simulating what other DBMSes call sharding but without exposing the implementation details to the users.
102+
103+Considering that user involvement in the creation of relation representations is the norm, how can the DBMS create such representations and maintain them automatically? This will likely be an area of research, but we can start simply with a query cache and automatic cost-based indexing based on actual queries executed.
104+
105+When broken free from user control, the DBMS is free to create arbitrary representations with the only consideration being available IO and memory resources.
106+
107+### Deferred updates
108+
109+Virtually all DBMSes execute database updates against existing files which store tuples. However, because Project:M36 updates are idempotent, we can save expressions to disk for *deferred updates*. Database update expressions saved to disk can drastically increase the rate at which we can execute updates because the expression is likely far smaller than the number of tuples which the expression will change. Because Project:M36 uses per-transaction directories, once a deferred update is actually evaluated, the directory can be atomically replaced with the evaluated result. Of course, to save disk storage and IO, the DBMS can also choose to discard the evaluated result and continue to store the update. This is nothing novel- git does this by storing code diffs between commits. The only novelty is in recognizing the value of idempotent updates.
110+
111+### Linked relations
112+
113+Every database update creates a new state based on an old state and an update expression. In the previous section, we demonstrated how the expression along with the previous state is sufficient to create the new state. Alternatively, the DBMS can also store a "diff" (difference) in the form of a relational expr with a link to the previous state.
114+
115+For example, if we have a relation variable of employees' names and we insert another employee name, the new state is the union of the previous state of the relation variable and a relation containing the new employee's name. The evaluated expression which requests the new state is merely replaced with the union- similar to a macro.
116+
117+This strategy can also be viewed as a form of compression. In the general case where a relation variable has not changed from one transaction to the next transaction, we can merely store what amounts to a symbolic link to the relation variable in the previous state.
118+
119+### Risks of Redesign
120+
121+* cborg module has had only one release, is probably less than ready for production use, but the initial results are promising and is developed and backed by well-known Haskellers and a Haskell shop
122+ * are there alternatives we should consider?
123+* increased code complexity
124+ * executor needs to be separated from current relational EDSL to support scanning various types of relation representations
125+ * the new EDSL may not leak its representations to the user-level EDSL
126+ * query executor will need access to the full transaction graph to cover relation links and deferred updates; currently, all state is wrapped up into the `DatabaseContext`
127+* how and when to create new relation representations and when to defer updates will be an active area of research
128+* requires more consideration in reducing IO and storage requirements; this was not a major driver of the initial design
docs/jupyter_kernel.markdownnew file+49
@@ -0,0 +1,49 @@
1+# TutorialD Jupyter Kernel for Project:M36
2+
3+## Introduction
4+
5+[The Jupyter Notebook](https://jupyter-notebook.readthedocs.io/en/stable/notebook.html) is web-based tool for experimenting with various languages. Project:M36 offers a Jupyter kernel for the TutorialD interpreter.
6+
7+The kernel automatically starts and stops its own websocket-server-based database and offers no filesystem persistence, which is similar to other jupyter kernels.
8+
9+## Installation
10+
11+To install the `tutd` kernel:
12+
13+* install the `itutd` module into your jupyter virtual environment:
14+```
15+$ cd project-m36/jupyter/itutd
16+$ pip install .
17+```
18+
19+* install `kernel.json`:
20+
21+ ```$ mkdir ~/Library/Jupyter/kernels/itutd && cp project-m36/jupyter/kernel.json ~/Library/Jupyter/kernels/itutd``` (macOS)
22+
23+ ```$ mkdir ~/.local/share/jupyter/kernels/tutd && cp project-m36/jupyter/kernel.json ~/.local/share/jupyter/kernels/tutd``` (Linux)
24+
25+ ```$ mkdir %APPDATA%\jupyter\kernels\tutd && copy project-m36\jupyter\kernel.json %APPDATA%\jupyter\kernels\tutd``` (Windows)
26+
27+* check that the kernel is detected:
28+```
29+$ jupyter kernelspec list
30+Available kernels:
31+ python2 ...
32+ tutd ...
33+```
34+
35+* add `project-m36-server` to `PATH` environment variable
36+
37+```
38+$ export PATH=$PATH:<path/to/project-m36-websocket-server>
39+$ which project-m36-websocket-server
40+/path/to/project-m36-websocket-server
41+```
42+
43+From within this python virtual environment, you can now start jupyter with TutorialD support.
44+
45+## Usage
46+
47+With the kernel installed, run `jupyter notebook` for the web-based interface and click "New" at the top right of the page. The TutorialD kernel should be in the list.
48+
49+Once a new TutorialD notebook is opened, the interpreter is identical to the `tutd` command line interpreter and accepts the same commands. If you are new to Project:M36-flavored TutorialD, please [read the tutorial](/docs/tutd_tutorial.markdown).
docs/reaching_out_of_the_tarpit.markdown+6−6
@@ -2,11 +2,11 @@
22
33## Introduction
44
5−[Project:M36](https://github.com/agentm/project-m36) is a relational algebra database management system (RDBMS) written in the Haskell programming language whose purpose is to implement the foundation for functional relational programming as described in the excellent paper ["Out of the Tarpit"](http://shaffner.us/cs/papers/tarpit.pdf). This document describes how Project:M36 meets the design goals laid out in the paper.
5+[Project:M36](https://github.com/agentm/project-m36) is a relational algebra database management system (RDBMS) written in the Haskell programming language whose purpose is to implement the foundation for functional relational programming as described in the excellent paper ["Out of the Tarpit"](https://github.com/papers-we-love/papers-we-love/blob/2eb8d21/design/out-of-the-tar-pit.pdf). This document describes how Project:M36 meets the design goals laid out in the paper.
66
77## Why is "Out of the Tarpit" Important?
88
9−The "Out of the Tarpit" paper, written in 2006, consolidates decades of software development research and experience to explain why software written today is chock-full of software bugs. The paper categorizes sources of complexity and then proceeds to propose a software architecture which promises to reduce unnecessary software complexity so that the resultant software is maximally focused on actual business rules. It is recommended to read [the paper](http://shaffner.us/cs/papers/tarpit.pdf) before reading the remainder of this document.
9+The "Out of the Tarpit" paper, written in 2006, consolidates decades of software development research and experience to explain why software written today is chock-full of software bugs. The paper categorizes sources of complexity and then proceeds to propose a software architecture which promises to reduce unnecessary software complexity so that the resultant software is maximally focused on actual business rules. It is recommended to read [the paper](https://github.com/papers-we-love/papers-we-love/blob/2eb8d21/design/out-of-the-tar-pit.pdf) before reading the remainder of this document.
1010
1111## Laying the Foundations for Reduced Complexity
1212
@@ -21,14 +21,14 @@ The central software component of functional-relational programming is a relatio
2121* **declarative**: thus removing accidental complexity (such as accidental ordering)
2222* **restricted**: removing mistaken features; if a programming language allows a programmer to do something less-than-savory, then that misfeature will be used
2323
24−and provides ([p.37](http://shaffner.us/cs/papers/tarpit.pdf)):
24+and provides ([p.37](https://github.com/papers-we-love/papers-we-love/blob/2eb8d21/design/out-of-the-tar-pit.pdf)):
2525
2626* **structure**: a closed world system whereby all data is represented by relations
2727* **manipulation**: a means to simulate mutation of the application state over time
2828* **data integrity**: constraints which prevent illogical data constructions
2929* **data independence**: logical and physical representations of state which are clearly separated
3030
31−The relational model is a mathematical model for storing and manipulating "essential state" ([p.25](http://shaffner.us/cs/papers/tarpit.pdf)). As a mathematical model, it not beholden to any specific implementation or quirks.
31+The relational model is a mathematical model for storing and manipulating "essential state" ([p.25](https://github.com/papers-we-love/papers-we-love/blob/2eb8d21/design/out-of-the-tar-pit.pdf)). As a mathematical model, it not beholden to any specific implementation or quirks.
3232
3333Unfortunately, due to the quirks of legacy implementations, much of what developers learn about the relational model is [wrong](/docs/on_null.markdown). Project:M36 aims to rectify this situation by providing a mathematically coherent implementation of the relational model. Contrast the mathematical approach to existing databases which sacrifice or elide a mathematical model in lieu of "performance". Refer to [Knuth on premature optimization](https://en.wikiquote.org/wiki/Donald_Knuth) for additional details.
3434
@@ -70,7 +70,7 @@ TutorialD (master/main): :showexpr p
7070
7171### Infrastructure Requirements
7272
73−Project:M36 meets or exceeds the enumerated "Infrastructure" ([p.47](http://shaffner.us/cs/papers/tarpit.pdf)) requirements (except one):
73+Project:M36 meets or exceeds the enumerated "Infrastructure" ([p.47](https://github.com/papers-we-love/papers-we-love/blob/2eb8d21/design/out-of-the-tar-pit.pdf)) requirements (except one):
7474
7575* **Infrastructure for Essential State**: relation variable manipulation, storage, and data types
7676* **Infrastructure for Essential Logic**: relational expression evaluation, functions to operate on values, a language (TutorialD or Haskell), type inference, and constraints
@@ -82,7 +82,7 @@ Derived relations, as described in the paper, are not supported in Project:M36 b
8282
8383### Feeders and Observers
8484
85−The "Out of the Tarpit" paper distinguishes between pushing data to the database and pulling data out of the database ([p.46](http://shaffner.us/cs/papers/tarpit.pdf)): "feeders" continually update the state of the database to best reflect reality and "observers" request notifications on relational state changes and take action on them.
85+The "Out of the Tarpit" paper distinguishes between pushing data to the database and pulling data out of the database ([p.46](https://github.com/papers-we-love/papers-we-love/blob/2eb8d21/design/out-of-the-tar-pit.pdf)): "feeders" continually update the state of the database to best reflect reality and "observers" request notifications on relational state changes and take action on them.
8686
8787SQL developers are already intimately familiar with "feeders"- these are merely INSERT and UPDATE statements issued to the database via some middleware.
8888
jupyter/TutorialD Notebook Walkthrough.ipynbnew file+794
@@ -0,0 +1,794 @@
1+{
2+ "cells": [
3+ {
4+ "cell_type": "markdown",
5+ "metadata": {},
6+ "source": [
7+ "# Introduction to TutorialD via Jupyter Notebook\n",
8+ "\n",
9+ "Thanks for trying Project:M36 via this Jupyter notebook. TutorialD is a language for interacting with relational algebra databases, similar to SQL but without the historical cruft. This notebook is based on the [TutorialD Tutorial](https://github.com/agentm/project-m36/blob/master/docs/tutd_tutorial.markdown). \n",
10+ "\n",
11+ "[Project:M36](https://github.com/agentm/project-m36) is a feature-rich database management system (DBMS) which casts off the baggage of legacy DBMSes by adhering strictly to the mathematics of the relational algebra.\n",
12+ "\n",
13+ "Please refer to the [Project:M36 documentation](https://github.com/agentm/project-m36#documentation) for more detailed examples. Let's get started!\n",
14+ "\n",
15+ "First, let's import the Chris Date example relation variables. The relation variables provided in the example are:\n",
16+ "\n",
17+ "* `s` suppliers\n",
18+ "* `sp` supplier-parts\n",
19+ "* `p` parts"
20+ ]
21+ },
22+ {
23+ "cell_type": "code",
24+ "execution_count": 1,
25+ "metadata": {},
26+ "outputs": [
27+ {
28+ "data": {
29+ "text/html": [
30+ "ok"
31+ ],
32+ "text/plain": [
33+ "ok"
34+ ]
35+ },
36+ "metadata": {},
37+ "output_type": "display_data"
38+ }
39+ ],
40+ "source": [
41+ ":importexample date"
42+ ]
43+ },
44+ {
45+ "cell_type": "markdown",
46+ "metadata": {},
47+ "source": [
48+ "If you see an error in the previous cell regarding `project-m36-websocket-server` not being in your `PATH`, add the binary to your `PATH` and restart the jupyter server.\n",
49+ "\n",
50+ "Let's take a look at the relation variables."
51+ ]
52+ },
53+ {
54+ "cell_type": "code",
55+ "execution_count": 2,
56+ "metadata": {},
57+ "outputs": [
58+ {
59+ "data": {
60+ "text/html": [
61+ "<style>.pm36relation {empty-cells: show;} .pm36relation tbody td, .pm36relation th { border: 1px solid black;}</style><table class=\"pm36relation\"\"><tr><th>s#::Text</th><th>sname::Text</th><th>status::Integer</th><th>city::Text</th></tr><tr><td>\"S3\"</td><td>\"Blake\"</td><td>30</td><td>\"Paris\"</td></tr><tr><td>\"S5\"</td><td>\"Adams\"</td><td>30</td><td>\"Athens\"</td></tr><tr><td>\"S4\"</td><td>\"Clark\"</td><td>20</td><td>\"London\"</td></tr><tr><td>\"S1\"</td><td>\"Smith\"</td><td>20</td><td>\"London\"</td></tr><tr><td>\"S2\"</td><td>\"Jones\"</td><td>10</td><td>\"Paris\"</td></tr><tfoot><tr><td colspan=\"100%\">Finite 5 tuples</td></tr></tfoot></table>"
62+ ],
63+ "text/plain": [
64+ "┌──────────┬────────┬───────────┬───────────────┐\n",
65+ "│city::Text│s#::Text│sname::Text│status::Integer│\n",
66+ "├──────────┼────────┼───────────┼───────────────┤\n",
67+ "│\"Paris\" │\"S2\" │\"Jones\" │10 │\n",
68+ "│\"London\" │\"S1\" │\"Smith\" │20 │\n",
69+ "│\"London\" │\"S4\" │\"Clark\" │20 │\n",
70+ "│\"Athens\" │\"S5\" │\"Adams\" │30 │\n",
71+ "│\"Paris\" │\"S3\" │\"Blake\" │30 │\n",
72+ "└──────────┴────────┴───────────┴───────────────┘"
73+ ]
74+ },
75+ "metadata": {},
76+ "output_type": "display_data"
77+ }
78+ ],
79+ "source": [
80+ ":showexpr s"
81+ ]
82+ },
83+ {
84+ "cell_type": "code",
85+ "execution_count": 3,
86+ "metadata": {},
87+ "outputs": [
88+ {
89+ "data": {
90+ "text/html": [
91+ "<style>.pm36relation {empty-cells: show;} .pm36relation tbody td, .pm36relation th { border: 1px solid black;}</style><table class=\"pm36relation\"\"><tr><th>s#::Text</th><th>p#::Text</th><th>qty::Integer</th></tr><tr><td>\"S1\"</td><td>\"P2\"</td><td>200</td></tr><tr><td>\"S1\"</td><td>\"P1\"</td><td>300</td></tr><tr><td>\"S3\"</td><td>\"P2\"</td><td>200</td></tr><tr><td>\"S2\"</td><td>\"P2\"</td><td>400</td></tr><tr><td>\"S4\"</td><td>\"P5\"</td><td>400</td></tr><tr><td>\"S1\"</td><td>\"P5\"</td><td>100</td></tr><tr><td>\"S1\"</td><td>\"P6\"</td><td>100</td></tr><tr><td>\"S2\"</td><td>\"P1\"</td><td>300</td></tr><tr><td>\"S1\"</td><td>\"P4\"</td><td>200</td></tr><tr><td>\"S4\"</td><td>\"P4\"</td><td>300</td></tr><tr><td>\"S4\"</td><td>\"P2\"</td><td>200</td></tr><tr><td>\"S1\"</td><td>\"P3\"</td><td>400</td></tr><tfoot><tr><td colspan=\"100%\">Finite 12 tuples</td></tr></tfoot></table>"
92+ ],
93+ "text/plain": [
94+ "┌────────┬────────────┬────────┐\n",
95+ "│p#::Text│qty::Integer│s#::Text│\n",
96+ "├────────┼────────────┼────────┤\n",
97+ "│\"P3\" │400 │\"S1\" │\n",
98+ "│\"P2\" │200 │\"S4\" │\n",
99+ "│\"P4\" │300 │\"S4\" │\n",
100+ "│\"P4\" │200 │\"S1\" │\n",
101+ "│\"P1\" │300 │\"S2\" │\n",
102+ "│\"P6\" │100 │\"S1\" │\n",
103+ "│\"P5\" │100 │\"S1\" │\n",
104+ "│\"P5\" │400 │\"S4\" │\n",
105+ "│\"P2\" │400 │\"S2\" │\n",
106+ "│\"P2\" │200 │\"S3\" │\n",
107+ "│\"P1\" │300 │\"S1\" │\n",
108+ "│\"P2\" │200 │\"S1\" │\n",
109+ "└────────┴────────────┴────────┘"
110+ ]
111+ },
112+ "metadata": {},
113+ "output_type": "display_data"
114+ }
115+ ],
116+ "source": [
117+ ":showexpr sp"
118+ ]
119+ },
120+ {
121+ "cell_type": "code",
122+ "execution_count": 4,
123+ "metadata": {
124+ "scrolled": true
125+ },
126+ "outputs": [
127+ {
128+ "data": {
129+ "text/html": [
130+ "<style>.pm36relation {empty-cells: show;} .pm36relation tbody td, .pm36relation th { border: 1px solid black;}</style><table class=\"pm36relation\"\"><tr><th>p#::Text</th><th>pname::Text</th><th>color::Text</th><th>weight::Integer</th><th>city::Text</th></tr><tr><td>\"P3\"</td><td>\"Screw\"</td><td>\"Blue\"</td><td>17</td><td>\"Oslo\"</td></tr><tr><td>\"P2\"</td><td>\"Bolt\"</td><td>\"Green\"</td><td>17</td><td>\"Paris\"</td></tr><tr><td>\"P4\"</td><td>\"Screw\"</td><td>\"Red\"</td><td>14</td><td>\"London\"</td></tr><tr><td>\"P6\"</td><td>\"Cog\"</td><td>\"Red\"</td><td>19</td><td>\"London\"</td></tr><tr><td>\"P1\"</td><td>\"Nut\"</td><td>\"Red\"</td><td>12</td><td>\"London\"</td></tr><tr><td>\"P5\"</td><td>\"Cam\"</td><td>\"Blue\"</td><td>12</td><td>\"Paris\"</td></tr><tfoot><tr><td colspan=\"100%\">Finite 6 tuples</td></tr></tfoot></table>"
131+ ],
132+ "text/plain": [
133+ "┌──────────┬───────────┬────────┬───────────┬───────────────┐\n",
134+ "│city::Text│color::Text│p#::Text│pname::Text│weight::Integer│\n",
135+ "├──────────┼───────────┼────────┼───────────┼───────────────┤\n",
136+ "│\"Paris\" │\"Blue\" │\"P5\" │\"Cam\" │12 │\n",
137+ "│\"London\" │\"Red\" │\"P1\" │\"Nut\" │12 │\n",
138+ "│\"London\" │\"Red\" │\"P6\" │\"Cog\" │19 │\n",
139+ "│\"London\" │\"Red\" │\"P4\" │\"Screw\" │14 │\n",
140+ "│\"Paris\" │\"Green\" │\"P2\" │\"Bolt\" │17 │\n",
141+ "│\"Oslo\" │\"Blue\" │\"P3\" │\"Screw\" │17 │\n",
142+ "└──────────┴───────────┴────────┴───────────┴───────────────┘"
143+ ]
144+ },
145+ "metadata": {},
146+ "output_type": "display_data"
147+ }
148+ ],
149+ "source": [
150+ "\n",
151+ ":showexpr p"
152+ ]
153+ },
154+ {
155+ "cell_type": "markdown",
156+ "metadata": {},
157+ "source": [
158+ "## Relational Operators\n",
159+ "\n",
160+ "### Restriction\n",
161+ "\n",
162+ "We can see that there is a many-to-many relationship between `s` (suppliers) and `p` (parts) via the `sp` relation variable.\n",
163+ "\n",
164+ "Let's look at the suppliers who are in London. This is called \"restriction\".\n",
165+ "\n",
166+ "Note that the relation attributes have appropriate types which restrict the values which they can contain."
167+ ]
168+ },
169+ {
170+ "cell_type": "code",
171+ "execution_count": 5,
172+ "metadata": {},
173+ "outputs": [
174+ {
175+ "data": {
176+ "text/html": [
177+ "<style>.pm36relation {empty-cells: show;} .pm36relation tbody td, .pm36relation th { border: 1px solid black;}</style><table class=\"pm36relation\"\"><tr><th>s#::Text</th><th>sname::Text</th><th>status::Integer</th><th>city::Text</th></tr><tr><td>\"S4\"</td><td>\"Clark\"</td><td>20</td><td>\"London\"</td></tr><tr><td>\"S1\"</td><td>\"Smith\"</td><td>20</td><td>\"London\"</td></tr><tfoot><tr><td colspan=\"100%\">Finite 2 tuples</td></tr></tfoot></table>"
178+ ],
179+ "text/plain": [
180+ "┌──────────┬────────┬───────────┬───────────────┐\n",
181+ "│city::Text│s#::Text│sname::Text│status::Integer│\n",
182+ "├──────────┼────────┼───────────┼───────────────┤\n",
183+ "│\"London\" │\"S1\" │\"Smith\" │20 │\n",
184+ "│\"London\" │\"S4\" │\"Clark\" │20 │\n",
185+ "└──────────┴────────┴───────────┴───────────────┘"
186+ ]
187+ },
188+ "metadata": {},
189+ "output_type": "display_data"
190+ }
191+ ],
192+ "source": [
193+ ":showexpr s where city=\"London\""
194+ ]
195+ },
196+ {
197+ "cell_type": "markdown",
198+ "metadata": {},
199+ "source": [
200+ "### Join And Projection\n",
201+ "\n",
202+ "Let's examine which parts the London suppliers offer. In the following expression, we make use of the relational join operator (equivalent to `NATURAL JOIN` in SQL) and projection using the trailing curly brackets to only return the relation attributes which interest us."
203+ ]
204+ },
205+ {
206+ "cell_type": "code",
207+ "execution_count": 6,
208+ "metadata": {},
209+ "outputs": [
210+ {
211+ "data": {
212+ "text/html": [
213+ "<style>.pm36relation {empty-cells: show;} .pm36relation tbody td, .pm36relation th { border: 1px solid black;}</style><table class=\"pm36relation\"\"><tr><th>qty::Integer</th><th>pname::Text</th><th>p#::Text</th></tr><tr><td>100</td><td>\"Cog\"</td><td>\"P6\"</td></tr><tr><td>300</td><td>\"Nut\"</td><td>\"P1\"</td></tr><tr><td>300</td><td>\"Screw\"</td><td>\"P4\"</td></tr><tr><td>200</td><td>\"Screw\"</td><td>\"P4\"</td></tr><tfoot><tr><td colspan=\"100%\">Finite 4 tuples</td></tr></tfoot></table>"
214+ ],
215+ "text/plain": [
216+ "┌────────┬───────────┬────────────┐\n",
217+ "│p#::Text│pname::Text│qty::Integer│\n",
218+ "├────────┼───────────┼────────────┤\n",
219+ "│\"P4\" │\"Screw\" │200 │\n",
220+ "│\"P4\" │\"Screw\" │300 │\n",
221+ "│\"P1\" │\"Nut\" │300 │\n",
222+ "│\"P6\" │\"Cog\" │100 │\n",
223+ "└────────┴───────────┴────────────┘"
224+ ]
225+ },
226+ "metadata": {},
227+ "output_type": "display_data"
228+ }
229+ ],
230+ "source": [
231+ ":showexpr ((s where city=\"London\") join sp join p){p#,pname,qty}"
232+ ]
233+ },
234+ {
235+ "cell_type": "markdown",
236+ "metadata": {},
237+ "source": [
238+ "### Extend and Atom Functions\n",
239+ "\n",
240+ "We can also execute server-side functions on values and add them to our result relation."
241+ ]
242+ },
243+ {
244+ "cell_type": "code",
245+ "execution_count": 7,
246+ "metadata": {},
247+ "outputs": [
248+ {
249+ "data": {
250+ "text/html": [
251+ "<style>.pm36relation {empty-cells: show;} .pm36relation tbody td, .pm36relation th { border: 1px solid black;}</style><table class=\"pm36relation\"\"><tr><th>s#::Text</th><th>status::Integer</th><th>statusplus10::Integer</th></tr><tr><td>\"S1\"</td><td>20</td><td>30</td></tr><tr><td>\"S4\"</td><td>20</td><td>30</td></tr><tr><td>\"S5\"</td><td>30</td><td>40</td></tr><tr><td>\"S3\"</td><td>30</td><td>40</td></tr><tr><td>\"S2\"</td><td>10</td><td>20</td></tr><tfoot><tr><td colspan=\"100%\">Finite 5 tuples</td></tr></tfoot></table>"
252+ ],
253+ "text/plain": [
254+ "┌────────┬───────────────┬─────────────────────┐\n",
255+ "│s#::Text│status::Integer│statusplus10::Integer│\n",
256+ "├────────┼───────────────┼─────────────────────┤\n",
257+ "│\"S2\" │10 │20 │\n",
258+ "│\"S3\" │30 │40 │\n",
259+ "│\"S5\" │30 │40 │\n",
260+ "│\"S4\" │20 │30 │\n",
261+ "│\"S1\" │20 │30 │\n",
262+ "└────────┴───────────────┴─────────────────────┘"
263+ ]
264+ },
265+ "metadata": {},
266+ "output_type": "display_data"
267+ }
268+ ],
269+ "source": [
270+ ":showexpr (s:{statusplus10:=add(@status,10)}){s#,status,statusplus10}"
271+ ]
272+ },
273+ {
274+ "cell_type": "markdown",
275+ "metadata": {},
276+ "source": [
277+ "### Group and Ungroup\n",
278+ "\n",
279+ "Unlike most databases, Project:M36 supports relation-valued attributes. This means that database values (atoms) can also be relations. Relation-valued attributes can be represented in the notebook as nested tables of values. Note that the nested relation's type is `relation {qty::Integer,pname::Text,s#::Text}`. \n",
280+ "\n",
281+ "Relation-valued attributes reduce tuple duplication by clearly associating certain values with their related multiple tuples. The `group` operator can also be used to create the SQL equivalent of `OUTER JOIN` but without the need for representing `NULL` in any form.\n",
282+ "\n",
283+ "Let's list the available counts for each supplier for each part."
284+ ]
285+ },
286+ {
287+ "cell_type": "code",
288+ "execution_count": 8,
289+ "metadata": {},
290+ "outputs": [
291+ {
292+ "data": {
293+ "text/html": [
294+ "<style>.pm36relation {empty-cells: show;} .pm36relation tbody td, .pm36relation th { border: 1px solid black;}</style><table class=\"pm36relation\"\"><tr><th>p#::Text</th><th>availability::relation {qty::Integer,pname::Text,s#::Text}</th></tr><tr><td>\"P6\"</td><td><style>.pm36relation {empty-cells: show;} .pm36relation tbody td, .pm36relation th { border: 1px solid black;}</style><table class=\"pm36relation\"\"><tr><th>qty::Integer</th><th>pname::Text</th><th>s#::Text</th></tr><tr><td>100</td><td>\"Cog\"</td><td>\"S1\"</td></tr><tfoot><tr><td colspan=\"100%\">Finite 1 tuples</td></tr></tfoot></table></td></tr><tr><td>\"P1\"</td><td><style>.pm36relation {empty-cells: show;} .pm36relation tbody td, .pm36relation th { border: 1px solid black;}</style><table class=\"pm36relation\"\"><tr><th>qty::Integer</th><th>pname::Text</th><th>s#::Text</th></tr><tr><td>300</td><td>\"Nut\"</td><td>\"S2\"</td></tr><tr><td>300</td><td>\"Nut\"</td><td>\"S1\"</td></tr><tfoot><tr><td colspan=\"100%\">Finite 2 tuples</td></tr></tfoot></table></td></tr><tr><td>\"P2\"</td><td><style>.pm36relation {empty-cells: show;} .pm36relation tbody td, .pm36relation th { border: 1px solid black;}</style><table class=\"pm36relation\"\"><tr><th>qty::Integer</th><th>pname::Text</th><th>s#::Text</th></tr><tr><td>200</td><td>\"Bolt\"</td><td>\"S4\"</td></tr><tr><td>200</td><td>\"Bolt\"</td><td>\"S1\"</td></tr><tr><td>400</td><td>\"Bolt\"</td><td>\"S2\"</td></tr><tr><td>200</td><td>\"Bolt\"</td><td>\"S3\"</td></tr><tfoot><tr><td colspan=\"100%\">Finite 4 tuples</td></tr></tfoot></table></td></tr><tr><td>\"P4\"</td><td><style>.pm36relation {empty-cells: show;} .pm36relation tbody td, .pm36relation th { border: 1px solid black;}</style><table class=\"pm36relation\"\"><tr><th>qty::Integer</th><th>pname::Text</th><th>s#::Text</th></tr><tr><td>200</td><td>\"Screw\"</td><td>\"S1\"</td></tr><tr><td>300</td><td>\"Screw\"</td><td>\"S4\"</td></tr><tfoot><tr><td colspan=\"100%\">Finite 2 tuples</td></tr></tfoot></table></td></tr><tr><td>\"P3\"</td><td><style>.pm36relation {empty-cells: show;} .pm36relation tbody td, .pm36relation th { border: 1px solid black;}</style><table class=\"pm36relation\"\"><tr><th>qty::Integer</th><th>pname::Text</th><th>s#::Text</th></tr><tr><td>400</td><td>\"Screw\"</td><td>\"S1\"</td></tr><tfoot><tr><td colspan=\"100%\">Finite 1 tuples</td></tr></tfoot></table></td></tr><tr><td>\"P5\"</td><td><style>.pm36relation {empty-cells: show;} .pm36relation tbody td, .pm36relation th { border: 1px solid black;}</style><table class=\"pm36relation\"\"><tr><th>qty::Integer</th><th>pname::Text</th><th>s#::Text</th></tr><tr><td>400</td><td>\"Cam\"</td><td>\"S4\"</td></tr><tr><td>100</td><td>\"Cam\"</td><td>\"S1\"</td></tr><tfoot><tr><td colspan=\"100%\">Finite 2 tuples</td></tr></tfoot></table></td></tr><tfoot><tr><td colspan=\"100%\">Finite 6 tuples</td></tr></tfoot></table>"
295+ ],
296+ "text/plain": [
297+ "┌──────────────────────────────────────────────────────────┬────────┐\n",
298+ "│availability::relation {qty::Integer,pname::Text,s#::Text}│p#::Text│\n",
299+ "├──────────────────────────────────────────────────────────┼────────┤\n",
300+ "│┌───────────┬────────────┬────────┐ │\"P5\" │\n",
301+ "││pname::Text│qty::Integer│s#::Text│ │ │\n",
302+ "│├───────────┼────────────┼────────┤ │ │\n",
303+ "││\"Cam\" │100 │\"S1\" │ │ │\n",
304+ "││\"Cam\" │400 │\"S4\" │ │ │\n",
305+ "│└───────────┴────────────┴────────┘ │ │\n",
306+ "│┌───────────┬────────────┬────────┐ │\"P3\" │\n",
307+ "││pname::Text│qty::Integer│s#::Text│ │ │\n",
308+ "│├───────────┼────────────┼────────┤ │ │\n",
309+ "││\"Screw\" │400 │\"S1\" │ │ │\n",
310+ "│└───────────┴────────────┴────────┘ │ │\n",
311+ "│┌───────────┬────────────┬────────┐ │\"P4\" │\n",
312+ "││pname::Text│qty::Integer│s#::Text│ │ │\n",
313+ "│├───────────┼────────────┼────────┤ │ │\n",
314+ "││\"Screw\" │300 │\"S4\" │ │ │\n",
315+ "││\"Screw\" │200 │\"S1\" │ │ │\n",
316+ "│└───────────┴────────────┴────────┘ │ │\n",
317+ "│┌───────────┬────────────┬────────┐ │\"P2\" │\n",
318+ "││pname::Text│qty::Integer│s#::Text│ │ │\n",
319+ "│├───────────┼────────────┼────────┤ │ │\n",
320+ "││\"Bolt\" │200 │\"S3\" │ │ │\n",
321+ "││\"Bolt\" │400 │\"S2\" │ │ │\n",
322+ "││\"Bolt\" │200 │\"S1\" │ │ │\n",
323+ "││\"Bolt\" │200 │\"S4\" │ │ │\n",
324+ "│└───────────┴────────────┴────────┘ │ │\n",
325+ "│┌───────────┬────────────┬────────┐ │\"P1\" │\n",
326+ "││pname::Text│qty::Integer│s#::Text│ │ │\n",
327+ "│├───────────┼────────────┼────────┤ │ │\n",
328+ "││\"Nut\" │300 │\"S1\" │ │ │\n",
329+ "││\"Nut\" │300 │\"S2\" │ │ │\n",
330+ "│└───────────┴────────────┴────────┘ │ │\n",
331+ "│┌───────────┬────────────┬────────┐ │\"P6\" │\n",
332+ "││pname::Text│qty::Integer│s#::Text│ │ │\n",
333+ "│├───────────┼────────────┼────────┤ │ │\n",
334+ "││\"Cog\" │100 │\"S1\" │ │ │\n",
335+ "│└───────────┴────────────┴────────┘ │ │\n",
336+ "└──────────────────────────────────────────────────────────┴────────┘"
337+ ]
338+ },
339+ "metadata": {},
340+ "output_type": "display_data"
341+ }
342+ ],
343+ "source": [
344+ ":showexpr (p join sp){qty,pname,s#,p#} group ({s#,pname,qty} as availability)"
345+ ]
346+ },
347+ {
348+ "cell_type": "markdown",
349+ "metadata": {},
350+ "source": [
351+ "We can flatten the relation-valued attributes, thereby generating more top-level tuples."
352+ ]
353+ },
354+ {
355+ "cell_type": "code",
356+ "execution_count": 9,
357+ "metadata": {},
358+ "outputs": [
359+ {
360+ "data": {
361+ "text/html": [
362+ "<style>.pm36relation {empty-cells: show;} .pm36relation tbody td, .pm36relation th { border: 1px solid black;}</style><table class=\"pm36relation\"\"><tr><th>qty::Integer</th><th>pname::Text</th><th>s#::Text</th><th>p#::Text</th></tr><tr><td>400</td><td>\"Screw\"</td><td>\"S1\"</td><td>\"P3\"</td></tr><tr><td>200</td><td>\"Bolt\"</td><td>\"S4\"</td><td>\"P2\"</td></tr><tr><td>100</td><td>\"Cog\"</td><td>\"S1\"</td><td>\"P6\"</td></tr><tr><td>100</td><td>\"Cam\"</td><td>\"S1\"</td><td>\"P5\"</td></tr><tr><td>300</td><td>\"Nut\"</td><td>\"S2\"</td><td>\"P1\"</td></tr><tr><td>400</td><td>\"Cam\"</td><td>\"S4\"</td><td>\"P5\"</td></tr><tr><td>300</td><td>\"Screw\"</td><td>\"S4\"</td><td>\"P4\"</td></tr><tr><td>200</td><td>\"Bolt\"</td><td>\"S1\"</td><td>\"P2\"</td></tr><tr><td>200</td><td>\"Screw\"</td><td>\"S1\"</td><td>\"P4\"</td></tr><tr><td>200</td><td>\"Bolt\"</td><td>\"S3\"</td><td>\"P2\"</td></tr><tr><td>300</td><td>\"Nut\"</td><td>\"S1\"</td><td>\"P1\"</td></tr><tr><td>400</td><td>\"Bolt\"</td><td>\"S2\"</td><td>\"P2\"</td></tr><tfoot><tr><td colspan=\"100%\">Finite 12 tuples</td></tr></tfoot></table>"
363+ ],
364+ "text/plain": [
365+ "┌────────┬───────────┬────────────┬────────┐\n",
366+ "│p#::Text│pname::Text│qty::Integer│s#::Text│\n",
367+ "├────────┼───────────┼────────────┼────────┤\n",
368+ "│\"P2\" │\"Bolt\" │400 │\"S2\" │\n",
369+ "│\"P1\" │\"Nut\" │300 │\"S1\" │\n",
370+ "│\"P2\" │\"Bolt\" │200 │\"S3\" │\n",
371+ "│\"P4\" │\"Screw\" │200 │\"S1\" │\n",
372+ "│\"P2\" │\"Bolt\" │200 │\"S1\" │\n",
373+ "│\"P4\" │\"Screw\" │300 │\"S4\" │\n",
374+ "│\"P5\" │\"Cam\" │400 │\"S4\" │\n",
375+ "│\"P1\" │\"Nut\" │300 │\"S2\" │\n",
376+ "│\"P5\" │\"Cam\" │100 │\"S1\" │\n",
377+ "│\"P6\" │\"Cog\" │100 │\"S1\" │\n",
378+ "│\"P2\" │\"Bolt\" │200 │\"S4\" │\n",
379+ "│\"P3\" │\"Screw\" │400 │\"S1\" │\n",
380+ "└────────┴───────────┴────────────┴────────┘"
381+ ]
382+ },
383+ "metadata": {},
384+ "output_type": "display_data"
385+ }
386+ ],
387+ "source": [
388+ ":showexpr ((p join sp){qty,pname,s#,p#} group ({s#,pname,qty} as availability)) ungroup availability"
389+ ]
390+ },
391+ {
392+ "cell_type": "markdown",
393+ "metadata": {},
394+ "source": [
395+ "## Database State Operators\n",
396+ "\n",
397+ "Thus far, we have only been inspecting the database state. Next, let's cover some database update operators. These operators are all similar to SQL operators, so we don't need a lot of exposition."
398+ ]
399+ },
400+ {
401+ "cell_type": "markdown",
402+ "metadata": {},
403+ "source": [
404+ "### Insert"
405+ ]
406+ },
407+ {
408+ "cell_type": "code",
409+ "execution_count": 10,
410+ "metadata": {},
411+ "outputs": [
412+ {
413+ "data": {
414+ "text/html": [
415+ "ok"
416+ ],
417+ "text/plain": [
418+ "ok"
419+ ]
420+ },
421+ "metadata": {},
422+ "output_type": "display_data"
423+ }
424+ ],
425+ "source": [
426+ "insert s relation{tuple{s# \"S6\", sname \"Bob\", status 50, city \"Boston\"}}"
427+ ]
428+ },
429+ {
430+ "cell_type": "code",
431+ "execution_count": 11,
432+ "metadata": {},
433+ "outputs": [
434+ {
435+ "data": {
436+ "text/html": [
437+ "<style>.pm36relation {empty-cells: show;} .pm36relation tbody td, .pm36relation th { border: 1px solid black;}</style><table class=\"pm36relation\"\"><tr><th>city::Text</th><th>s#::Text</th><th>sname::Text</th><th>status::Integer</th></tr><tr><td>\"Boston\"</td><td>\"S6\"</td><td>\"Bob\"</td><td>50</td></tr><tr><td>\"Paris\"</td><td>\"S3\"</td><td>\"Blake\"</td><td>30</td></tr><tr><td>\"Athens\"</td><td>\"S5\"</td><td>\"Adams\"</td><td>30</td></tr><tr><td>\"London\"</td><td>\"S4\"</td><td>\"Clark\"</td><td>20</td></tr><tr><td>\"London\"</td><td>\"S1\"</td><td>\"Smith\"</td><td>20</td></tr><tr><td>\"Paris\"</td><td>\"S2\"</td><td>\"Jones\"</td><td>10</td></tr><tfoot><tr><td colspan=\"100%\">Finite 6 tuples</td></tr></tfoot></table>"
438+ ],
439+ "text/plain": [
440+ "┌──────────┬────────┬───────────┬───────────────┐\n",
441+ "│city::Text│s#::Text│sname::Text│status::Integer│\n",
442+ "├──────────┼────────┼───────────┼───────────────┤\n",
443+ "│\"Paris\" │\"S2\" │\"Jones\" │10 │\n",
444+ "│\"London\" │\"S1\" │\"Smith\" │20 │\n",
445+ "│\"London\" │\"S4\" │\"Clark\" │20 │\n",
446+ "│\"Athens\" │\"S5\" │\"Adams\" │30 │\n",
447+ "│\"Paris\" │\"S3\" │\"Blake\" │30 │\n",
448+ "│\"Boston\" │\"S6\" │\"Bob\" │50 │\n",
449+ "└──────────┴────────┴───────────┴───────────────┘"
450+ ]
451+ },
452+ "metadata": {},
453+ "output_type": "display_data"
454+ }
455+ ],
456+ "source": [
457+ ":showexpr s"
458+ ]
459+ },
460+ {
461+ "cell_type": "markdown",
462+ "metadata": {},
463+ "source": [
464+ "### Delete"
465+ ]
466+ },
467+ {
468+ "cell_type": "code",
469+ "execution_count": 12,
470+ "metadata": {},
471+ "outputs": [
472+ {
473+ "data": {
474+ "text/html": [
475+ "ok"
476+ ],
477+ "text/plain": [
478+ "ok"
479+ ]
480+ },
481+ "metadata": {},
482+ "output_type": "display_data"
483+ }
484+ ],
485+ "source": [
486+ "delete s where sname=\"Bob\""
487+ ]
488+ },
489+ {
490+ "cell_type": "code",
491+ "execution_count": 13,
492+ "metadata": {},
493+ "outputs": [
494+ {
495+ "data": {
496+ "text/html": [
497+ "<style>.pm36relation {empty-cells: show;} .pm36relation tbody td, .pm36relation th { border: 1px solid black;}</style><table class=\"pm36relation\"\"><tr><th>city::Text</th><th>s#::Text</th><th>sname::Text</th><th>status::Integer</th></tr><tr><td>\"Paris\"</td><td>\"S3\"</td><td>\"Blake\"</td><td>30</td></tr><tr><td>\"Athens\"</td><td>\"S5\"</td><td>\"Adams\"</td><td>30</td></tr><tr><td>\"London\"</td><td>\"S4\"</td><td>\"Clark\"</td><td>20</td></tr><tr><td>\"London\"</td><td>\"S1\"</td><td>\"Smith\"</td><td>20</td></tr><tr><td>\"Paris\"</td><td>\"S2\"</td><td>\"Jones\"</td><td>10</td></tr><tfoot><tr><td colspan=\"100%\">Finite 5 tuples</td></tr></tfoot></table>"
498+ ],
499+ "text/plain": [
500+ "┌──────────┬────────┬───────────┬───────────────┐\n",
501+ "│city::Text│s#::Text│sname::Text│status::Integer│\n",
502+ "├──────────┼────────┼───────────┼───────────────┤\n",
503+ "│\"Paris\" │\"S2\" │\"Jones\" │10 │\n",
504+ "│\"London\" │\"S1\" │\"Smith\" │20 │\n",
505+ "│\"London\" │\"S4\" │\"Clark\" │20 │\n",
506+ "│\"Athens\" │\"S5\" │\"Adams\" │30 │\n",
507+ "│\"Paris\" │\"S3\" │\"Blake\" │30 │\n",
508+ "└──────────┴────────┴───────────┴───────────────┘"
509+ ]
510+ },
511+ "metadata": {},
512+ "output_type": "display_data"
513+ }
514+ ],
515+ "source": [
516+ ":showexpr s"
517+ ]
518+ },
519+ {
520+ "cell_type": "markdown",
521+ "metadata": {},
522+ "source": [
523+ "### Update"
524+ ]
525+ },
526+ {
527+ "cell_type": "code",
528+ "execution_count": 14,
529+ "metadata": {},
530+ "outputs": [
531+ {
532+ "data": {
533+ "text/html": [
534+ "ok"
535+ ],
536+ "text/plain": [
537+ "ok"
538+ ]
539+ },
540+ "metadata": {},
541+ "output_type": "display_data"
542+ }
543+ ],
544+ "source": [
545+ "update s where city=\"London\" (status:=90)"
546+ ]
547+ },
548+ {
549+ "cell_type": "code",
550+ "execution_count": 15,
551+ "metadata": {},
552+ "outputs": [
553+ {
554+ "data": {
555+ "text/html": [
556+ "<style>.pm36relation {empty-cells: show;} .pm36relation tbody td, .pm36relation th { border: 1px solid black;}</style><table class=\"pm36relation\"\"><tr><th>city::Text</th><th>s#::Text</th><th>sname::Text</th><th>status::Integer</th></tr><tr><td>\"London\"</td><td>\"S1\"</td><td>\"Smith\"</td><td>90</td></tr><tr><td>\"Paris\"</td><td>\"S3\"</td><td>\"Blake\"</td><td>30</td></tr><tr><td>\"Athens\"</td><td>\"S5\"</td><td>\"Adams\"</td><td>30</td></tr><tr><td>\"London\"</td><td>\"S4\"</td><td>\"Clark\"</td><td>90</td></tr><tr><td>\"Paris\"</td><td>\"S2\"</td><td>\"Jones\"</td><td>10</td></tr><tfoot><tr><td colspan=\"100%\">Finite 5 tuples</td></tr></tfoot></table>"
557+ ],
558+ "text/plain": [
559+ "┌──────────┬────────┬───────────┬───────────────┐\n",
560+ "│city::Text│s#::Text│sname::Text│status::Integer│\n",
561+ "├──────────┼────────┼───────────┼───────────────┤\n",
562+ "│\"Paris\" │\"S2\" │\"Jones\" │10 │\n",
563+ "│\"London\" │\"S4\" │\"Clark\" │90 │\n",
564+ "│\"Athens\" │\"S5\" │\"Adams\" │30 │\n",
565+ "│\"Paris\" │\"S3\" │\"Blake\" │30 │\n",
566+ "│\"London\" │\"S1\" │\"Smith\" │90 │\n",
567+ "└──────────┴────────┴───────────┴───────────────┘"
568+ ]
569+ },
570+ "metadata": {},
571+ "output_type": "display_data"
572+ }
573+ ],
574+ "source": [
575+ ":showexpr s"
576+ ]
577+ },
578+ {
579+ "cell_type": "markdown",
580+ "metadata": {},
581+ "source": [
582+ "### Assign"
583+ ]
584+ },
585+ {
586+ "cell_type": "code",
587+ "execution_count": 16,
588+ "metadata": {},
589+ "outputs": [
590+ {
591+ "data": {
592+ "text/html": [
593+ "ok"
594+ ],
595+ "text/plain": [
596+ "ok"
597+ ]
598+ },
599+ "metadata": {},
600+ "output_type": "display_data"
601+ }
602+ ],
603+ "source": [
604+ "london_suppliers := s where city=\"London\""
605+ ]
606+ },
607+ {
608+ "cell_type": "code",
609+ "execution_count": 17,
610+ "metadata": {},
611+ "outputs": [
612+ {
613+ "data": {
614+ "text/html": [
615+ "<style>.pm36relation {empty-cells: show;} .pm36relation tbody td, .pm36relation th { border: 1px solid black;}</style><table class=\"pm36relation\"\"><tr><th>city::Text</th><th>s#::Text</th><th>sname::Text</th><th>status::Integer</th></tr><tr><td>\"London\"</td><td>\"S1\"</td><td>\"Smith\"</td><td>90</td></tr><tr><td>\"London\"</td><td>\"S4\"</td><td>\"Clark\"</td><td>90</td></tr><tfoot><tr><td colspan=\"100%\">Finite 2 tuples</td></tr></tfoot></table>"
616+ ],
617+ "text/plain": [
618+ "┌──────────┬────────┬───────────┬───────────────┐\n",
619+ "│city::Text│s#::Text│sname::Text│status::Integer│\n",
620+ "├──────────┼────────┼───────────┼───────────────┤\n",
621+ "│\"London\" │\"S4\" │\"Clark\" │90 │\n",
622+ "│\"London\" │\"S1\" │\"Smith\" │90 │\n",
623+ "└──────────┴────────┴───────────┴───────────────┘"
624+ ]
625+ },
626+ "metadata": {},
627+ "output_type": "display_data"
628+ }
629+ ],
630+ "source": [
631+ ":showexpr london_suppliers"
632+ ]
633+ },
634+ {
635+ "cell_type": "markdown",
636+ "metadata": {},
637+ "source": [
638+ "## Constraints\n",
639+ "\n",
640+ "Project:M36 supports too many features to cover in one notebook, but constraints are certainly essential.\n",
641+ "\n",
642+ "Let's examine the current constraints."
643+ ]
644+ },
645+ {
646+ "cell_type": "code",
647+ "execution_count": 18,
648+ "metadata": {},
649+ "outputs": [
650+ {
651+ "data": {
652+ "text/html": [
653+ "<style>.pm36relation {empty-cells: show;} .pm36relation tbody td, .pm36relation th { border: 1px solid black;}</style><table class=\"pm36relation\"\"><tr><th>name::Text</th><th>sub::Text</th><th>super::Text</th></tr><tr><td>\"p_pkey\"</td><td>\"NotEquals (Project (AttributeNames (fromList [\\\"b\\\"])) (Extend (AttributeExtendTupleExpr \\\"b\\\" (FunctionAtomExpr \\\"count\\\" [AttributeAtomExpr \\\"a\\\"] ())) (Extend (AttributeExtendTupleExpr \\\"a\\\" (RelationAtomExpr (RelationVariable \\\"p\\\" ()))) (ExistingRelation (Relation [] (RelationTupleSet {asList = [RelationTuple [] []]})))))) (Project (AttributeNames (fromList [\\\"b\\\"])) (Extend (AttributeExtendTupleExpr \\\"b\\\" (FunctionAtomExpr \\\"count\\\" [AttributeAtomExpr \\\"a\\\"] ())) (Extend (AttributeExtendTupleExpr \\\"a\\\" (RelationAtomExpr (Project (AttributeNames (fromList [\\\"p#\\\"])) (RelationVariable \\\"p\\\" ())))) (ExistingRelation (Relation [] (RelationTupleSet {asList = [RelationTuple [] []]}))))))\"</td><td>\"ExistingRelation (Relation [] (RelationTupleSet {asList = []}))\"</td></tr><tr><td>\"s_pkey\"</td><td>\"NotEquals (Project (AttributeNames (fromList [\\\"b\\\"])) (Extend (AttributeExtendTupleExpr \\\"b\\\" (FunctionAtomExpr \\\"count\\\" [AttributeAtomExpr \\\"a\\\"] ())) (Extend (AttributeExtendTupleExpr \\\"a\\\" (RelationAtomExpr (RelationVariable \\\"s\\\" ()))) (ExistingRelation (Relation [] (RelationTupleSet {asList = [RelationTuple [] []]})))))) (Project (AttributeNames (fromList [\\\"b\\\"])) (Extend (AttributeExtendTupleExpr \\\"b\\\" (FunctionAtomExpr \\\"count\\\" [AttributeAtomExpr \\\"a\\\"] ())) (Extend (AttributeExtendTupleExpr \\\"a\\\" (RelationAtomExpr (Project (AttributeNames (fromList [\\\"s#\\\"])) (RelationVariable \\\"s\\\" ())))) (ExistingRelation (Relation [] (RelationTupleSet {asList = [RelationTuple [] []]}))))))\"</td><td>\"ExistingRelation (Relation [] (RelationTupleSet {asList = []}))\"</td></tr><tr><td>\"sp_pkey\"</td><td>\"NotEquals (Project (AttributeNames (fromList [\\\"b\\\"])) (Extend (AttributeExtendTupleExpr \\\"b\\\" (FunctionAtomExpr \\\"count\\\" [AttributeAtomExpr \\\"a\\\"] ())) (Extend (AttributeExtendTupleExpr \\\"a\\\" (RelationAtomExpr (RelationVariable \\\"sp\\\" ()))) (ExistingRelation (Relation [] (RelationTupleSet {asList = [RelationTuple [] []]})))))) (Project (AttributeNames (fromList [\\\"b\\\"])) (Extend (AttributeExtendTupleExpr \\\"b\\\" (FunctionAtomExpr \\\"count\\\" [AttributeAtomExpr \\\"a\\\"] ())) (Extend (AttributeExtendTupleExpr \\\"a\\\" (RelationAtomExpr (Project (AttributeNames (fromList [\\\"p#\\\",\\\"s#\\\"])) (RelationVariable \\\"sp\\\" ())))) (ExistingRelation (Relation [] (RelationTupleSet {asList = [RelationTuple [] []]}))))))\"</td><td>\"ExistingRelation (Relation [] (RelationTupleSet {asList = []}))\"</td></tr><tfoot><tr><td colspan=\"100%\">Finite 3 tuples</td></tr></tfoot></table>"
654+ ],
655+ "text/plain": [
656+ "┌──────────┬───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┬─────────────────────────────────────────────────────────────────┐\n",
657+ "│name::Text│sub::Text │super::Text │\n",
658+ "├──────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────────────────────┤\n",
659+ "│\"sp_pkey\" │\"NotEquals (Project (AttributeNames (fromList [\\\"b\\\"])) (Extend (AttributeExtendTupleExpr \\\"b\\\" (FunctionAtomExpr \\\"count\\\" [AttributeAtomExpr \\\"a\\\"] ())) (Extend (AttributeExtendTupleExpr \\\"a\\\" (RelationAtomExpr (RelationVariable \\\"sp\\\" ()))) (ExistingRelation (Relation [] (RelationTupleSet {asList = [RelationTuple [] []]})))))) (Project (AttributeNames (fromList [\\\"b\\\"])) (Extend (AttributeExtendTupleExpr \\\"b\\\" (FunctionAtomExpr \\\"count\\\" [AttributeAtomExpr \\\"a\\\"] ())) (Extend (AttributeExtendTupleExpr \\\"a\\\" (RelationAtomExpr (Project (AttributeNames (fromList [\\\"p#\\\",\\\"s#\\\"])) (RelationVariable \\\"sp\\\" ())))) (ExistingRelation (Relation [] (RelationTupleSet {asList = [RelationTuple [] []]}))))))\"│\"ExistingRelation (Relation [] (RelationTupleSet {asList = []}))\"│\n",
660+ "│\"s_pkey\" │\"NotEquals (Project (AttributeNames (fromList [\\\"b\\\"])) (Extend (AttributeExtendTupleExpr \\\"b\\\" (FunctionAtomExpr \\\"count\\\" [AttributeAtomExpr \\\"a\\\"] ())) (Extend (AttributeExtendTupleExpr \\\"a\\\" (RelationAtomExpr (RelationVariable \\\"s\\\" ()))) (ExistingRelation (Relation [] (RelationTupleSet {asList = [RelationTuple [] []]})))))) (Project (AttributeNames (fromList [\\\"b\\\"])) (Extend (AttributeExtendTupleExpr \\\"b\\\" (FunctionAtomExpr \\\"count\\\" [AttributeAtomExpr \\\"a\\\"] ())) (Extend (AttributeExtendTupleExpr \\\"a\\\" (RelationAtomExpr (Project (AttributeNames (fromList [\\\"s#\\\"])) (RelationVariable \\\"s\\\" ())))) (ExistingRelation (Relation [] (RelationTupleSet {asList = [RelationTuple [] []]}))))))\" │\"ExistingRelation (Relation [] (RelationTupleSet {asList = []}))\"│\n",
661+ "│\"p_pkey\" │\"NotEquals (Project (AttributeNames (fromList [\\\"b\\\"])) (Extend (AttributeExtendTupleExpr \\\"b\\\" (FunctionAtomExpr \\\"count\\\" [AttributeAtomExpr \\\"a\\\"] ())) (Extend (AttributeExtendTupleExpr \\\"a\\\" (RelationAtomExpr (RelationVariable \\\"p\\\" ()))) (ExistingRelation (Relation [] (RelationTupleSet {asList = [RelationTuple [] []]})))))) (Project (AttributeNames (fromList [\\\"b\\\"])) (Extend (AttributeExtendTupleExpr \\\"b\\\" (FunctionAtomExpr \\\"count\\\" [AttributeAtomExpr \\\"a\\\"] ())) (Extend (AttributeExtendTupleExpr \\\"a\\\" (RelationAtomExpr (Project (AttributeNames (fromList [\\\"p#\\\"])) (RelationVariable \\\"p\\\" ())))) (ExistingRelation (Relation [] (RelationTupleSet {asList = [RelationTuple [] []]}))))))\" │\"ExistingRelation (Relation [] (RelationTupleSet {asList = []}))\"│\n",
662+ "└──────────┴───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┴─────────────────────────────────────────────────────────────────┘"
663+ ]
664+ },
665+ "metadata": {},
666+ "output_type": "display_data"
667+ }
668+ ],
669+ "source": [
670+ ":constraints"
671+ ]
672+ },
673+ {
674+ "cell_type": "markdown",
675+ "metadata": {},
676+ "source": [
677+ "Constraints are represented as \"inclusion dependencies\". An inclusion dependency is a generalization of all possible constraints on the database and can represent keys, uniqueness constraints, foreign key constraints, as well as span multiple relation variables.\n",
678+ "\n",
679+ "An inclusion dependency requires that the result of relational expression `sub` is a subset of relational expression `super`.\n",
680+ "\n",
681+ "Project:M36 includes some utilities to generate inclusion dependencies."
682+ ]
683+ },
684+ {
685+ "cell_type": "code",
686+ "execution_count": 19,
687+ "metadata": {},
688+ "outputs": [
689+ {
690+ "data": {
691+ "text/html": [
692+ "ok"
693+ ],
694+ "text/plain": [
695+ "ok"
696+ ]
697+ },
698+ "metadata": {},
699+ "output_type": "display_data"
700+ }
701+ ],
702+ "source": [
703+ "sandwich := relation{tuple{name \"Cheese\", price 4},tuple{name \"Tuna\", price 5}}"
704+ ]
705+ },
706+ {
707+ "cell_type": "code",
708+ "execution_count": 20,
709+ "metadata": {},
710+ "outputs": [
711+ {
712+ "data": {
713+ "text/html": [
714+ "ok"
715+ ],
716+ "text/plain": [
717+ "ok"
718+ ]
719+ },
720+ "metadata": {},
721+ "output_type": "display_data"
722+ }
723+ ],
724+ "source": [
725+ "key sandwich_name{name} sandwich"
726+ ]
727+ },
728+ {
729+ "cell_type": "code",
730+ "execution_count": 21,
731+ "metadata": {},
732+ "outputs": [
733+ {
734+ "ename": "TutorialDError",
735+ "evalue": "",
736+ "output_type": "error",
737+ "traceback": [
738+ "InclusionDependencyCheckError \"sandwich_name\""
739+ ]
740+ }
741+ ],
742+ "source": [
743+ "insert sandwich relation{tuple{name \"Tuna\", price 6}}"
744+ ]
745+ },
746+ {
747+ "cell_type": "markdown",
748+ "metadata": {},
749+ "source": [
750+ "## Jupyter Kernel Features\n",
751+ "\n",
752+ "By default, the TutorialD kernel will automatically start a `project-m36-websocket-server` when the first TutorialD command is executed. For this to work, `project-m36-websocket-server` must be in the jupyter server's `PATH` environment variable.\n",
753+ "\n",
754+ "### Magic\n",
755+ "\n",
756+ "Like the python kernel, the TutorialD kernel includes some commands to interact directly with the kernel. These commands are not passed to the TutorialD interpreter.\n",
757+ "\n",
758+ "* `%help` brings up some help which includes links to additional documentation\n",
759+ "* `%connect ws://<host>:<port> <dbname>` connects to the `project-m36-websocket-server` already running. For example:\n",
760+ " ```\n",
761+ " %connect ws://localhost:64000 employees\n",
762+ " ```\n",
763+ " connects to the websocket server running on the localhost on port 64000 and selects the database named \"employees\". Note that the database name is not part of the websocket URL.\n",
764+ " \n",
765+ "## Conclusion\n",
766+ "\n",
767+ "[Project:M36](https://github.com/agentm/project-m36) is a fully-featured relational algebra engine suitable for use a database. It sets out to prove that sticking to the mathematical underpinnings of the relational algebra is straightforward and results in improved correctness and performance.\n",
768+ "\n",
769+ "If you would like to learn more, please refer to the [documentation](https://github.com/agentm/project-m36#documentation) and join our welcoming [community](https://github.com/agentm/project-m36#community) to ask questions."
770+ ]
771+ },
772+ {
773+ "cell_type": "code",
774+ "execution_count": null,
775+ "metadata": {},
776+ "outputs": [],
777+ "source": []
778+ }
779+ ],
780+ "metadata": {
781+ "kernelspec": {
782+ "display_name": "TutorialD (Project:M36)",
783+ "language": "TutorialD",
784+ "name": "tutd"
785+ },
786+ "language_info": {
787+ "file_extension": ".tutd",
788+ "mimetype": "text/x-tutd",
789+ "name": "TutorialD"
790+ }
791+ },
792+ "nbformat": 4,
793+ "nbformat_minor": 2
794+}
jupyter/itutd/ITutorialDKernel.pynew file+179
@@ -0,0 +1,179 @@
1+from ipykernel.kernelbase import Kernel
2+import subprocess
3+from websocket import create_connection, WebSocketConnectionClosedException, WebSocketBadStatusException
4+import time
5+import select
6+import json
7+import socket
8+import errno
9+import urlparse
10+
11+class ITutorialDKernel(Kernel):
12+ implementation = 'ITutorialD'
13+ implementation_version = '0.1'
14+
15+ banner = 'TutorialD kernel using Project:M36 https://github.com/agentm/project-m36'
16+
17+ language = 'TutorialD'
18+ language_version = '0.3'
19+ language_info = {
20+ 'name': 'TutorialD',
21+ 'mimetype': 'text/x-tutd',
22+ 'file_extension': '.tutd',
23+ }
24+ dbserver = None
25+ ws = None
26+
27+# todo: implement wrong database name handling
28+ def runtutd(self, tutd):
29+ #setup server
30+ dbname = 'jupyter'
31+ port = 63000
32+ if not self.ws and not self.dbserver:
33+ try:
34+ self.dbserver = subprocess.Popen(['project-m36-websocket-server',
35+ '--database', dbname,
36+ '--port', str(port)],
37+ )
38+ except OSError as err:
39+ if err.errno == errno.ENOENT:
40+ return ('"project-m36-websocket-server" is not in the PATH environment variable and could not be started.', None)
41+ else:
42+ raise
43+ res = self.connect_ws('ws://127.0.0.1:63000', dbname, retries=10)
44+ if res:
45+ return ('Connection failed: {}'.format(res), None)
46+
47+ self.ws.send('executetutd/html+text:' + tutd)
48+ try:
49+ jsonresult = self.discard_prompts()
50+ except WebSocketConnectionClosedException:
51+ self.ws = None
52+ return ('Websocket server closed the connection.',None)
53+ res = json.loads(jsonresult)
54+ if 'displayerror' in res:
55+ return (res['displayerror']['text'], None)
56+ elif 'acknowledged' in res:
57+ return (None, {'html':'ok', 'text':'ok'})
58+ else:
59+ return (None, res['displayrelation'])
60+
61+ def connect_ws(self, url, dbname, retries=0):
62+ while 1:
63+ try:
64+ ws = create_connection(url)
65+ except socket.error as err:
66+ if err.errno in (errno.ECONNREFUSED, ):
67+ time.sleep(1.0)
68+ if retries == 0:
69+ return 'Failed to connect to websocket.'
70+ retries -= 1
71+ else:
72+ raise
73+ except WebSocketBadStatusException:
74+ return 'Failed to negotiate websocket connection.'
75+ else:
76+ break
77+ self.ws = ws
78+ self.ws.send('connectdb:{}'.format(dbname))
79+ msg = json.loads(self.ws.recv())
80+ if 'displayerror' in msg:
81+ return msg['displayerror']['tag']
82+ return None
83+
84+ def discard_prompts(self):
85+ #read from the websocket until a non-prompt message is received
86+ while 1:
87+ sel = select.select([self.ws.sock],[],[],0)
88+ if len(sel[0]) > 0:
89+ #there is something to read
90+ msg = self.ws.recv()
91+ if 'promptInfo' not in msg:
92+ return msg
93+
94+ def send_error(self, msg):
95+ stream_content = {'ename':'TutorialDError',
96+ 'traceback':[msg],
97+ 'evalue':''}
98+ self.send_response(self.iopub_socket, 'error', stream_content)
99+
100+
101+ def process_magic(self, code):
102+ #handle "magic" processing of commands which are not sent to the tutd interpreter
103+ cmds = code.split(' ')
104+ ok = {'text':'ok', 'html':'ok'}
105+
106+ if cmds[0] == '%connect' and len(cmds) == 3:
107+ #connect to alternate database
108+ url = urlparse.urlparse(cmds[1])
109+ dbname = cmds[2]
110+ if not url.scheme.startswith('ws'):
111+ return ('Rejected attempt to connect to non-websocket. Use ws:// or wss:// scheme',None)
112+ res = self.connect_ws(url.geturl(), dbname)
113+ if res: #error from connection
114+ return (res, None, None)
115+ else: #connection successful
116+ return (None, ok, None)
117+ elif cmds[0] == '%help' and len(cmds) == 1:
118+ #display some help
119+ return (None, ok, self.show_help())
120+ elif cmds[0].startswith('%'):
121+ return ('Invalid magic command.', None)
122+
123+ def do_execute(self, code, silent,
124+ store_history=True,
125+ user_expressions=False,
126+ allow_stdin=False):
127+ magic = self.process_magic(code)
128+ payload = None
129+ if magic:
130+ (err, result, payload) = magic
131+ else:
132+ (err, result) = self.runtutd(code)
133+ if not silent:
134+ if err:
135+ status = 'error'
136+ self.send_error(err)
137+ else:
138+ status = 'ok'
139+ display_content = {
140+ 'data':{'text/html':result['html'],
141+ 'text/plain':result['text']},
142+ 'metadata':{},
143+ 'transient':{}
144+ }
145+ self.send_response(self.iopub_socket, 'display_data', display_content)
146+ ret ={'status': status,
147+ 'execution_count': self.execution_count,
148+ 'payload':[payload],
149+ 'user_expressions':{}}
150+ if payload:
151+ ret['payload'] = [payload]
152+ return ret
153+
154+ def do_shutdown(self, restart):
155+ if self.dbserver:
156+ self.dbserver.terminate()
157+ self.dbserver.kill()
158+
159+ def show_help(self):
160+ return {'source': 'page',
161+ 'start':0,
162+ 'data':{'text/html':'''<h1>Project:M36 TutorialD Jupyter Kernel Help</h1>
163+<p>TutorialD is an interactive language for the relational algebra.</p>
164+
165+<p><a href="https://github.com/agentm/project-m36/blob/master/docs/15_minute_tutorial.markdown">15 Minute TutorialD Introduction</a></p>
166+<p><a href="https://github.com/agentm/project-m36/blob/master/docs/tutd_tutorial.markdown">TutorialD Documentation</a></p>
167+''',
168+ 'text/plain':
169+'''Project:M36 TutorialD Jupyter Kernel Help
170+
171+TutorialD is an interactive language for the relational algebra.
172+
173+15 Minute TutorialD Introduction: https://github.com/agentm/project-m36/blob/master/docs/15_minute_tutorial.markdown
174+TutorialD Documentation: https://github.com/agentm/project-m36/blob/master/docs/tutd_tutorial.markdown
175+'''}}
176+
177+if __name__ == '__main__':
178+ from ipykernel.kernelapp import IPKernelApp
179+ IPKernelApp.launch_instance(kernel_class=ITutorialDKernel)
jupyter/itutd/__init__.pynew file+1
@@ -0,0 +1 @@
1+
jupyter/itutd/__main__.pynew file+4
@@ -0,0 +1,4 @@
1+from ipykernel.kernelapp import IPKernelApp
2+from . import ITutorialDKernel
3+
4+IPKernelApp.launch_instance(kernel_class=ITutorialDKernel)
jupyter/kernel.jsonnew file+5
@@ -0,0 +1,5 @@
1+{
2+ "language":"TutorialD",
3+ "display_name":"TutorialD (Project:M36)",
4+ "argv":["python", "-m", "itutd.ITutorialDKernel", "-f", "{connection_file}"]
5+}
jupyter/setup.pynew file+19
@@ -0,0 +1,19 @@
1+from setuptools import setup
2+
3+setup(
4+ name='itutd',
5+ version='0.1',
6+ packages=['itutd'],
7+ description='TutorialD Jupyter kernel for Project:M36',
8+ author='AgentM',
9+ author_email='agentm@themactionfaction.com',
10+ url='https://github.com/agentm/project-m36/jupyter/',
11+ install_requires=[
12+ 'jupyter_client', 'IPython', 'ipykernel', 'websocket-client'
13+ ],
14+ classifiers=[
15+ 'Intended Audience :: Developers',
16+ 'License :: OSI Approved :: Public Domain',
17+ 'Programming Language :: Python :: 2',
18+ ],
19+)
project-m36.cabal+2−1
@@ -428,7 +428,7 @@ Test-Suite test-scripts
428428
429429Executable project-m36-websocket-server
430430 Default-Language: Haskell2010
431− Build-Depends: base, aeson, path-pieces, either, conduit, http-api-data, template-haskell, websockets, aeson, optparse-applicative, project-m36, containers, bytestring, text, vector, uuid, megaparsec, haskeline, mtl, directory, base64-bytestring, random, MonadRandom, time, network-transport-tcp, semigroups
431+ Build-Depends: base, aeson, path-pieces, either, conduit, http-api-data, template-haskell, websockets, aeson, optparse-applicative, project-m36, containers, bytestring, text, vector, uuid, megaparsec, haskeline, mtl, directory, base64-bytestring, random, MonadRandom, time, network-transport-tcp, semigroups, attoparsec
432432 Main-Is: ProjectM36/Server/WebSocket/websocket-server.hs
433433 Other-Modules: ProjectM36.Client.Json, ProjectM36.Server.RemoteCallTypes.Json, ProjectM36.Server.WebSocket, TutorialD.Interpreter, TutorialD.Interpreter.Base, TutorialD.Interpreter.DatabaseContextExpr, TutorialD.Interpreter.DatabaseContextIOOperator, TutorialD.Interpreter.Export.Base, TutorialD.Interpreter.Export.CSV, TutorialD.Interpreter.Import.Base, TutorialD.Interpreter.Import.BasicExamples, TutorialD.Interpreter.Import.CSV, TutorialD.Interpreter.Import.TutorialD, TutorialD.Interpreter.InformationOperator, TutorialD.Interpreter.RODatabaseContextOperator, TutorialD.Interpreter.RelationalExpr, TutorialD.Interpreter.TransactionGraphOperator, TutorialD.Interpreter.Types, TutorialD.Interpreter.TransGraphRelationalOperator, TutorialD.Interpreter.SchemaOperator
434434 GHC-Options: -Wall
@@ -478,6 +478,7 @@ Test-Suite test-transactiongraph-automerge
478478 type: exitcode-stdio-1.0
479479 main-is: test/TransactionGraph/Automerge.hs
480480 Build-Depends: base, HUnit, Cabal, containers, hashable, unordered-containers, mtl, vector, vector-binary-instances, time, hashable-time, bytestring, uuid, stm, deepseq, deepseq-generics, binary, parallel, cassava, attoparsec, gnuplot, directory, temporary, haskeline, megaparsec, text, base64-bytestring, data-interval, filepath, transformers, stm-containers, list-t, websockets, optparse-applicative, network, aeson, project-m36, random, MonadRandom, semigroups
481+ Other-Modules: TutorialD.Interpreter, TutorialD.Interpreter.Base, TutorialD.Interpreter.DatabaseContextExpr, TutorialD.Interpreter.DatabaseContextIOOperator, TutorialD.Interpreter.Export.Base, TutorialD.Interpreter.Export.CSV, TutorialD.Interpreter.Import.Base, TutorialD.Interpreter.Import.BasicExamples, TutorialD.Interpreter.Import.CSV, TutorialD.Interpreter.Import.TutorialD, TutorialD.Interpreter.InformationOperator, TutorialD.Interpreter.RODatabaseContextOperator, TutorialD.Interpreter.RelationalExpr, TutorialD.Interpreter.SchemaOperator, TutorialD.Interpreter.TestBase, TutorialD.Interpreter.TransGraphRelationalOperator, TutorialD.Interpreter.TransactionGraphOperator, TutorialD.Interpreter.Types
481482 Default-Extensions: OverloadedStrings
482483 GHC-Options: -Wall
483484 Hs-Source-Dirs: ./src/bin, ., test/
src/bin/ProjectM36/Server/WebSocket.hs+54−21
@@ -1,3 +1,4 @@
1+{-# LANGUAGE LambdaCase #-}
12module ProjectM36.Server.WebSocket where
23-- while the tutd client performs TutorialD parsing on the client, the websocket server will pass tutd to be parsed and executed on the server- otherwise I have to pull in ghcjs as a dependency to allow client-side parsing- that's not appealing because then the frontend is not language-agnostic, but this could change in the future, perhaps by sending different messages over the websocket
34-- ideally, the wire protocol should not be exposed to a straight string-based API ala SQL, so we could make perhaps a javascript DSL which compiles to the necessary JSON- anaylyze tradeoffs
@@ -9,11 +10,17 @@ import qualified Data.Text as T
910import qualified Network.WebSockets as WS
1011import ProjectM36.Server.RemoteCallTypes.Json ()
1112import ProjectM36.Client.Json ()
13+import ProjectM36.Relation.Show.Term
14+import ProjectM36.Relation.Show.HTML
1215import Data.Aeson
1316import TutorialD.Interpreter
1417import TutorialD.Interpreter.Base
1518import ProjectM36.Client
1619import Control.Exception
20+import Data.Attoparsec.Text
21+import Control.Applicative
22+import Text.Megaparsec.Error
23+import Data.Functor
1724
1825websocketProxyServer :: Port -> Hostname -> WS.ServerApp
1926websocketProxyServer port host pending = do
@@ -26,10 +33,9 @@ websocketProxyServer port host pending = do
2633 else do
2734 let dbname = T.unpack $ T.drop (T.length connectdbmsg) dbmsg
2835 bracket (createConnection conn dbname port host)
29− (\eDBconn -> case eDBconn of
30− Right dbconn -> close dbconn
31− Left _ -> pure ()) $ \eDBconn ->
32− case eDBconn of
36+ (\case
37+ Right dbconn -> close dbconn
38+ Left _ -> pure ()) $ \case
3339 Left err -> sendError conn err
3440 Right dbconn -> do
3541 eSessionId <- createSessionAtHead dbconn "master"
@@ -43,12 +49,11 @@ websocketProxyServer port host pending = do
4349 sendPromptInfo pInfo conn
4450 sendPromptInfo pInfo conn
4551 msg <- WS.receiveData conn :: IO T.Text
46− let tutdprefix = "executetutd:"
47− case msg of
48− _ | tutdprefix `T.isPrefixOf` msg -> do
49− let tutdString = T.drop (T.length tutdprefix) msg
52+ case parseOnly parseExecuteMessage msg of
53+ Left _ -> unexpectedMsg
54+ Right (presentation, tutdString) ->
5055 case parseTutorialD tutdString of
51− Left err -> handleOpResult conn dbconn (DisplayErrorResult ("parse error: " `T.append` T.pack (show err)))
56+ Left err -> handleOpResult conn dbconn presentation (DisplayErrorResult ("parse error: " `T.append` T.pack (parseErrorPretty err)))
5257 Right parsed -> do
5358 let timeoutFilter exc = if exc == RequestTimeoutException
5459 then Just exc
@@ -57,9 +62,8 @@ websocketProxyServer port host pending = do
5762 result <- evalTutorialD sessionId dbconn SafeEvaluation parsed
5863 pInfo' <- promptInfo sessionId dbconn
5964 sendPromptInfo pInfo' conn
60− handleOpResult conn dbconn result
61− catchJust timeoutFilter responseHandler (\_ -> handleOpResult conn dbconn (DisplayErrorResult "Request Timed Out."))
62− _ -> unexpectedMsg
65+ handleOpResult conn dbconn presentation result
66+ catchJust timeoutFilter responseHandler (\_ -> handleOpResult conn dbconn presentation (DisplayErrorResult "Request Timed Out."))
6367 pure ()
6468
6569notificationCallback :: WS.Connection -> NotificationCallback
@@ -74,15 +78,22 @@ createConnection wsconn dbname port host = connectProjectM36 (RemoteProcessConne
7478sendError :: (ToJSON a) => WS.Connection -> a -> IO ()
7579sendError conn err = WS.sendTextData conn (encode (object ["displayerror" .= err]))
7680
77−handleOpResult :: WS.Connection -> Connection -> TutorialDOperatorResult -> IO ()
78−handleOpResult conn db QuitResult = WS.sendClose conn ("close" :: T.Text) >> close db
79−handleOpResult conn _ (DisplayResult out) = WS.sendTextData conn (encode (object ["display" .= out]))
80−handleOpResult _ _ (DisplayIOResult ioout) = ioout
81−handleOpResult conn _ (DisplayErrorResult err) = WS.sendTextData conn (encode (object ["displayerror" .= err]))
82−handleOpResult conn _ (DisplayParseErrorResult _ err) = WS.sendTextData conn (encode (object ["displayparseerrorresult" .= show err]))
83−handleOpResult conn _ QuietSuccessResult = WS.sendTextData conn (encode (object ["acknowledged" .= True]))
84−handleOpResult conn _ (DisplayRelationResult rel) = WS.sendTextData conn (encode (object ["displayrelation" .= rel]))
85−
81+handleOpResult :: WS.Connection -> Connection -> Presentation -> TutorialDOperatorResult -> IO ()
82+handleOpResult conn db _ QuitResult = WS.sendClose conn ("close" :: T.Text) >> close db
83+handleOpResult conn _ _ (DisplayResult out) = WS.sendTextData conn (encode (object ["display" .= out]))
84+handleOpResult _ _ _ (DisplayIOResult ioout) = ioout
85+handleOpResult conn _ presentation (DisplayErrorResult err) = do
86+ let jsono = ["json" .= err | jsonPresentation presentation]
87+ texto = ["text" .= err | textPresentation presentation]
88+ htmlo = ["html" .= err | htmlPresentation presentation]
89+ WS.sendTextData conn (encode (object ["displayerror" .= object (jsono ++ texto ++ htmlo)]))
90+handleOpResult conn _ _ (DisplayParseErrorResult _ err) = WS.sendTextData conn (encode (object ["displayparseerrorresult" .= show err]))
91+handleOpResult conn _ _ QuietSuccessResult = WS.sendTextData conn (encode (object ["acknowledged" .= True]))
92+handleOpResult conn _ presentation (DisplayRelationResult rel) = do
93+ let jsono = ["json" .= rel | jsonPresentation presentation]
94+ texto = ["text" .= showRelation rel | textPresentation presentation]
95+ htmlo = ["html" .= relationAsHTML rel | htmlPresentation presentation]
96+ WS.sendTextData conn (encode (object ["displayrelation" .= object (jsono ++ texto ++ htmlo)]))
8697-- get current schema and head name for client
8798promptInfo :: SessionId -> Connection -> IO (HeadName, SchemaName)
8899promptInfo sessionId conn = do
@@ -92,3 +103,25 @@ promptInfo sessionId conn = do
92103
93104sendPromptInfo :: (HeadName, SchemaName) -> WS.Connection -> IO ()
94105sendPromptInfo (hName, sName) conn = WS.sendTextData conn (encode (object ["promptInfo" .= object ["headname" .= hName, "schemaname" .= sName]]))
106+
107+--a returning relation can be returned as JSON, Text (for consoles), or HTML
108+data Presentation = Presentation {
109+ jsonPresentation :: Bool,
110+ textPresentation :: Bool,
111+ htmlPresentation :: Bool }
112+
113+data PresentationFlag = JSONFlag | TextFlag | HTMLFlag
114+
115+parseExecuteMessage :: Parser (Presentation, T.Text)
116+parseExecuteMessage = do
117+ _ <- string "executetutd/"
118+ flags <- sepBy ((string "json" $> JSONFlag) <|>
119+ (string "text" $> TextFlag) <|>
120+ (string "html" $> HTMLFlag)) "+"
121+ let presentation = foldr (\flag acc -> case flag of
122+ JSONFlag -> acc {jsonPresentation = True}
123+ TextFlag -> acc {textPresentation = True}
124+ HTMLFlag -> acc {htmlPresentation = True}) (Presentation False False False) flags
125+ _ <- char ':'
126+ tutd <- T.pack <$> manyTill anyChar endOfInput
127+ pure (presentation, tutd)
src/bin/ProjectM36/Server/WebSocket/project-m36.js+3−3
@@ -82,7 +82,7 @@ ProjectM36Connection.prototype.handleResponse = function(message)
8282
8383 if(relation)
8484 {
85− this.statuscallback(new ProjectM36Status(relation, null, null));
85+ this.statuscallback(new ProjectM36Status(relation['json'], null, null));
8686 }
8787
8888 if(acknowledged)
@@ -96,7 +96,7 @@ ProjectM36Connection.prototype.handleResponse = function(message)
9696 {
9797 error=error.tag; // for error objects
9898 }
99− this.statuscallback(new ProjectM36Status(null, null, error));
99+ this.statuscallback(new ProjectM36Status(null, null, error['json']));
100100 }
101101
102102 if(prompt)
@@ -117,7 +117,7 @@ ProjectM36Connection.prototype.handleResponse = function(message)
117117*/
118118ProjectM36Connection.prototype.executeTutorialD = function(tutd)
119119{
120− this.socket.send("executetutd:" + tutd);
120+ this.socket.send("executetutd/json:" + tutd);
121121}
122122
123123/**
src/bin/ProjectM36/Server/WebSocket/websocket-client.html+1−1
@@ -108,7 +108,7 @@ margin-bottom: 5pt;
108108 <option value="ws">ws://</option>
109109 </select>
110110 <input type="text" name="host" value="localhost" placeholder="Host"/>:
111− <input id="port" type="number" placeholder="Port" name="port" min="0" max="32767" value="8888"/>/
111+ <input id="port" type="number" placeholder="Port" name="port" min="0" max="65535" value="8888"/>/
112112 <input type="text" name="path" placeholder="path" value=""/>
113113 <label>database:</label><input type="text" placeholder="Database Name" name="dbname" value="test"/>
114114 <input type="submit" name="connect" value="Connect"/>
src/bin/TutorialD/Interpreter.hs+2−2
@@ -1,4 +1,4 @@
1−{-# LANGUAGE GADTs #-}
1+{-# LANGUAGE GADTs, LambdaCase #-}
22module TutorialD.Interpreter where
33import TutorialD.Interpreter.Base
44import TutorialD.Interpreter.RODatabaseContextOperator
@@ -245,7 +245,7 @@ reprLoop config sessionId conn = do
245245 eHeadName <- C.headName sessionId conn
246246 eSchemaName <- C.currentSchemaName sessionId conn
247247 let prompt = promptText eHeadName eSchemaName
248− catchInterrupt = handleJust (\exc -> case exc of
248+ catchInterrupt = handleJust (\case
249249 UserInterrupt -> Just Nothing
250250 _ -> Nothing) (\_ -> do
251251 hPutStrLn stderr "Statement cancelled. Use \":quit\" to exit tutd."
src/bin/TutorialD/Interpreter/DatabaseContextExpr.hs+2−1
@@ -15,6 +15,7 @@ import qualified ProjectM36.RelationalExpression as RE
1515import ProjectM36.Key
1616import ProjectM36.FunctionalDependency
1717import Data.Monoid
18+import Data.Functor
1819
1920--parsers which create "database expressions" which modify the database context (such as relvar assignment)
2021databaseContextExprP :: Parser DatabaseContextExpr
@@ -101,7 +102,7 @@ addConstraintP = do
101102 reservedOp "constraint" <|> reservedOp "foreign key"
102103 constraintName <- identifier
103104 subset <- relExprP
104− op <- (reservedOp "in" *> pure SubsetOp) <|> (reservedOp "equals" *> pure EqualityOp)
105+ op <- (reservedOp "in" $> SubsetOp) <|> (reservedOp "equals" $> EqualityOp)
105106 superset <- relExprP
106107 let subsetA = incDepSet constraintName subset superset
107108 subsetB = incDepSet (constraintName <> "_eqInvert") superset subset --inverted args for equality constraint
src/bin/TutorialD/Interpreter/TransactionGraphOperator.hs+2−1
@@ -6,6 +6,7 @@ import Text.Megaparsec
66import ProjectM36.TransactionGraph hiding (autoMergeToHead)
77import ProjectM36.Client
88import ProjectM36.Base
9+import Data.Functor
910
1011data ConvenienceTransactionGraphOperator = AutoMergeToHead MergeStrategy HeadName
1112 deriving (Show)
@@ -62,7 +63,7 @@ showGraphP = do
6263 return ShowGraph
6364
6465mergeTransactionStrategyP :: Parser MergeStrategy
65−mergeTransactionStrategyP = (reserved "union" *> pure UnionMergeStrategy) <|>
66+mergeTransactionStrategyP = (reserved "union" $> UnionMergeStrategy) <|>
6667 (do
6768 reserved "selectedbranch"
6869 branch <- identifier
src/lib/ProjectM36/AtomType.hs+2−2
@@ -234,8 +234,8 @@ atomTypeVerify x@(ConstructedAtomType tConsNameA tVarMapA) (ConstructedAtomType
234234 | otherwise = Right x
235235
236236atomTypeVerify x@(RelationAtomType attrs1) y@(RelationAtomType attrs2) = do
237− _ <- mapM (\(attr1,attr2) -> let name1 = A.attributeName attr1
238− name2 = A.attributeName attr2 in
237+ mapM_ (\(attr1,attr2) -> let name1 = A.attributeName attr1
238+ name2 = A.attributeName attr2 in
239239 if notElem "_" [name1, name2] && name1 /= name2 then
240240 Left $ AtomTypeMismatchError x y
241241 else
src/lib/ProjectM36/Attribute.hs+6−3
@@ -99,9 +99,12 @@ matchingAttributeNameSet = S.intersection
9999attributeNamesNotContained :: S.Set AttributeName -> S.Set AttributeName -> S.Set AttributeName
100100attributeNamesNotContained subset superset = S.filter (`S.notMember` superset) subset
101101
102−-- this is sorted so the tuples know in which order to output- the ordering is arbitrary
103−sortedAttributeNameList :: S.Set AttributeName -> [AttributeName]
104−sortedAttributeNameList attrNameSet= L.sort $ S.toList attrNameSet
102+-- useful for display
103+orderedAttributes :: Attributes -> [Attribute]
104+orderedAttributes attrs = L.sortBy (\a b -> attributeName a `compare` attributeName b) (V.toList attrs)
105+
106+orderedAttributeNames :: Attributes -> [AttributeName]
107+orderedAttributeNames attrs = map attributeName (orderedAttributes attrs)
105108
106109-- take two attribute sets and return an attribute set with the attributes which do not match
107110attributesDifference :: Attributes -> Attributes -> Attributes
src/lib/ProjectM36/Client.hs+1−1
@@ -110,7 +110,7 @@ import qualified ProjectM36.TransactionGraph as Graph
110110import ProjectM36.TransactionGraph
111111import qualified ProjectM36.Transaction as Trans
112112import ProjectM36.TransactionGraph.Persist
113−import ProjectM36.Attribute hiding (atomTypes)
113+import ProjectM36.Attribute
114114import ProjectM36.TransGraphRelationalExpression (TransGraphRelationalExpr, evalTransGraphRelationalExpr)
115115import ProjectM36.Persist (DiskSync(..))
116116import ProjectM36.FileLock
src/lib/ProjectM36/Client/Simple.hs+1−1
@@ -84,7 +84,7 @@ withTransactionUsing (sess, conn) strat dbm = do
8484 handler :: TransactionCancelled -> IO (Either DbError a)
8585 handler (TransactionCancelled err) = pure (Left err)
8686 handle handler $ do
87− ret <- C.withTransaction sess conn (block >>= pure . Right) successFunc
87+ ret <- C.withTransaction sess conn (Right <$> block) successFunc
8888 case ret of
8989 Left err -> pure (Left (RelError err))
9090 Right val -> pure (Right val)
src/lib/ProjectM36/Error.hs+1
@@ -14,6 +14,7 @@ import Control.Exception
1414
1515data RelationalError = NoSuchAttributeNamesError (S.Set AttributeName)
1616 | TupleAttributeCountMismatchError Int --attribute name
17+ | DuplicateAttributeNamesError (S.Set AttributeName)
1718 | TupleAttributeTypeMismatchError Attributes
1819 | AttributeCountMismatchError Int
1920 | AttributeNamesMismatchError (S.Set AttributeName)
src/lib/ProjectM36/FileLock.hs+3−4
@@ -2,11 +2,9 @@
22--cross-platform file locking utilizing POSIX file locking on Unix/Linux and Windows file locking
33--hackage's System.FileLock doesn't support POSIX advisory locks nor locking file based on file descriptors, hence this needless rewrite
44module ProjectM36.FileLock where
5−import System.IO
65
76
87#if defined(mingw32_HOST_OS)
9−import ProjectM36.Win32Handle
108import System.Win32.Types
119import Foreign.Marshal.Alloc
1210import System.Win32.File
@@ -30,10 +28,10 @@ openLockFile :: FilePath -> IO LockFile
3028openLockFile path = createFile path
3129 (gENERIC_READ .|. gENERIC_WRITE)
3230 (fILE_SHARE_READ .|. fILE_SHARE_WRITE)
33− nullPtr
31+ Nothing
3432 oPEN_ALWAYS
3533 fILE_ATTRIBUTE_NORMAL
36− nullPtr
34+ Nothing
3735
3836closeLockFile :: LockFile -> IO ()
3937closeLockFile file = do
@@ -73,6 +71,7 @@ unlockFile winHandle = do
7371import qualified System.Posix.IO as P
7472import System.Posix.Types
7573import System.Posix.Files
74+import System.IO
7675
7776lockStruct :: P.LockRequest -> P.FileLock
7877lockStruct req = (req, AbsoluteSeek, 0, 0)
src/lib/ProjectM36/IsomorphicSchema.hs+8−8
@@ -1,4 +1,4 @@
1−{-# LANGUAGE DeriveGeneric, DeriveAnyClass #-}
1+{-# LANGUAGE DeriveGeneric, DeriveAnyClass, LambdaCase #-}
22module ProjectM36.IsomorphicSchema where
33import ProjectM36.Base
44import ProjectM36.Error
@@ -82,9 +82,9 @@ isomorphsOutRelVarNames morphs = S.fromList (foldr rvnames [] morphs)
8282
8383-- | Check that all mentioned relvars are actually present in the current schema.
8484validateRelationalExprInSchema :: Schema -> RelationalExpr -> Either RelationalError ()
85−validateRelationalExprInSchema schema relExprIn = relExprMogrify (\expr -> case expr of
86− RelationVariable rv () | S.notMember rv validRelVarNames -> Left (RelVarNotDefinedError rv)
87− ex -> Right ex) relExprIn >> pure ()
85+validateRelationalExprInSchema schema relExprIn = relExprMogrify (\case
86+ RelationVariable rv () | S.notMember rv validRelVarNames -> Left (RelVarNotDefinedError rv)
87+ ex -> Right ex) relExprIn >> pure ()
8888 where
8989 validRelVarNames = isomorphsInRelVarNames (isomorphs schema)
9090
@@ -133,16 +133,16 @@ applySchemaToInclusionDependencies (Schema morphs) incDeps =
133133
134134-- | Morph a relational expression in one schema to another isomorphic schema.
135135relExprMorph :: SchemaIsomorph -> (RelationalExpr -> Either RelationalError RelationalExpr)
136−relExprMorph (IsoRestrict relIn _ (relOutTrue, relOutFalse)) = \expr -> case expr of
136+relExprMorph (IsoRestrict relIn _ (relOutTrue, relOutFalse)) = \case
137137 RelationVariable rv () | rv == relIn -> Right (Union (RelationVariable relOutTrue ()) (RelationVariable relOutFalse ()))
138138 orig -> Right orig
139−relExprMorph (IsoUnion (relInT, relInF) predi relTarget) = \expr -> case expr of
139+relExprMorph (IsoUnion (relInT, relInF) predi relTarget) = \case
140140 --only the true predicate portion appears in the virtual schema
141141 RelationVariable rv () | rv == relInT -> Right (Restrict predi (RelationVariable relTarget ()))
142142
143143 RelationVariable rv () | rv == relInF -> Right (Restrict (NotPredicate predi) (RelationVariable relTarget ()))
144144 orig -> Right orig
145−relExprMorph (IsoRename relIn relOut) = \expr -> case expr of
145+relExprMorph (IsoRename relIn relOut) = \case
146146 RelationVariable rv () | rv == relIn -> Right (RelationVariable relOut ())
147147 orig -> Right orig
148148
@@ -254,7 +254,7 @@ inclusionDependencyInSchema schema (InclusionDependency rexprA rexprB) = do
254254-- also, it's inverse to IsoRestrict which adds two constraints at the base level
255255-- for IsoRestrict, consider hiding the two, generated constraints since they can never be thrown in the isomorphic schema
256256inclusionDependenciesInSchema :: Schema -> InclusionDependencies -> Either RelationalError InclusionDependencies
257−inclusionDependenciesInSchema schema incDeps = mapM (\(depName, dep) -> inclusionDependencyInSchema schema dep >>= \newDep -> pure (depName, newDep)) (M.toList incDeps) >>= pure . M.fromList
257+inclusionDependenciesInSchema schema incDeps = M.fromList <$> mapM (\(depName, dep) -> inclusionDependencyInSchema schema dep >>= \newDep -> pure (depName, newDep)) (M.toList incDeps)
258258
259259relationVariablesInSchema :: Schema -> DatabaseContext -> Either RelationalError RelationVariables
260260relationVariablesInSchema schema@(Schema morphs) context = foldM transform M.empty morphs
src/lib/ProjectM36/MiscUtils.hs+6
@@ -7,3 +7,9 @@ dupes [_] = []
77dupes [x,y] = [x | x == y]
88dupes (x:y:xs) = dupes(x:[y]) ++ dupes(y : xs)
99
10+--Data.Vector.indexed but for lists
11+indexed :: [a] -> [(Int, a)]
12+indexed = go 0
13+ where
14+ go _ [] = []
15+ go i (v:ys) = (i,v):go (i+1) ys
src/lib/ProjectM36/Relation.hs+13−6
@@ -12,6 +12,7 @@ import qualified ProjectM36.Attribute as A
1212import qualified ProjectM36.AttributeNames as AS
1313import ProjectM36.TupleSet
1414import ProjectM36.Error
15+import ProjectM36.MiscUtils
1516--import qualified Control.Parallel.Strategies as P
1617import qualified ProjectM36.TypeConstructorDef as TCD
1718import qualified ProjectM36.DataConstructorDef as DCD
@@ -19,6 +20,7 @@ import qualified Data.Text as T
1920import Data.Either (isRight)
2021import System.Random.Shuffle
2122import Control.Monad.Random
23+import Data.List (sort)
2224
2325attributes :: Relation -> Attributes
2426attributes (Relation attrs _ ) = attrs
@@ -44,12 +46,17 @@ emptyRelationWithAttrs :: Attributes -> Relation
4446emptyRelationWithAttrs attrs = Relation attrs emptyTupleSet
4547
4648mkRelation :: Attributes -> RelationTupleSet -> Either RelationalError Relation
47−mkRelation attrs tupleSet =
48− --check that all tuples have the same keys
49− --check that all tuples have keys (1-N) where N is the attribute count
50− case verifyTupleSet attrs tupleSet of
51− Left err -> Left err
52− Right verifiedTupleSet -> return $ Relation attrs verifiedTupleSet
49+mkRelation attrs tupleSet =
50+ --check that all attributes are unique- this cannot be done when creating attributes because the check can become expensive
51+ let duplicateAttrNames = dupes (sort (map A.attributeName (V.toList attrs))) in
52+ if not (null duplicateAttrNames) then
53+ Left (DuplicateAttributeNamesError (S.fromList duplicateAttrNames))
54+ else
55+ --check that all tuples have the same keys
56+ --check that all tuples have keys (1-N) where N is the attribute count
57+ case verifyTupleSet attrs tupleSet of
58+ Left err -> Left err
59+ Right verifiedTupleSet -> return $ Relation attrs verifiedTupleSet
5360
5461--less safe version of mkRelation skips verifyTupleSet
5562--useful for infinite or thunked tuple sets
src/lib/ProjectM36/Relation/Show/HTML.hs+33−10
@@ -3,22 +3,42 @@ import ProjectM36.Base
33import ProjectM36.Relation
44import ProjectM36.Tuple
55import ProjectM36.Atom
6+import ProjectM36.AtomType
67import qualified Data.List as L
7−import ProjectM36.Attribute
8−import Data.Text (append, Text, pack)
8+import Data.Text (Text, pack)
99import qualified Data.Text as T
1010import qualified Data.Text.IO as TIO
11+import Data.Monoid
12+import qualified Data.Vector as V
1113
1214attributesAsHTML :: Attributes -> Text
13−attributesAsHTML attrs = "<tr>" `append` T.concat (map oneAttrHTML attrNameList) `append` "</tr>"
15+attributesAsHTML attrs = "<tr>" <> T.concat (map oneAttrHTML (V.toList attrs)) <> "</tr>"
1416 where
15− oneAttrHTML attrName = "<th>" `append` attrName `append` "</th>"
16− attrNameList = sortedAttributeNameList (attributeNameSet attrs)
17+ oneAttrHTML attr = "<th>" <> prettyAttribute attr <> "</th>"
1718
1819relationAsHTML :: Relation -> Text
19−relationAsHTML rel@(Relation attrNameSet tupleSet) = "<table border=\"1\">" `append` attributesAsHTML attrNameSet `append` tupleSetAsHTML tupleSet `append` "<tfoot>" `append` tablefooter `append` "</tfoot></table>"
20+-- web browsers don't display tables with empty cells or empty headers, so we have to insert some placeholders- it's not technically the same, but looks as expected in the browser
21+relationAsHTML rel@(Relation attrNameSet tupleSet)
22+ | rel == relationTrue = pm36relcss <>
23+ tablestart <>
24+ "<tr><th></th></tr>" <>
25+ "<tr><td></td></tr>" <>
26+ tablefooter <> "</table>"
27+ | rel == relationFalse = pm36relcss <>
28+ tablestart <>
29+ "<tr><th></th></tr>" <>
30+ tablefooter <>
31+ "</table>"
32+ | otherwise = pm36relcss <>
33+ tablestart <>
34+ attributesAsHTML attrNameSet <>
35+ tupleSetAsHTML tupleSet <>
36+ tablefooter <>
37+ "</table>"
2038 where
21− tablefooter = "<tr><td colspan=\"100%\">" `append` pack (show (cardinality rel)) `append` " tuples</td></tr>"
39+ pm36relcss = "<style>.pm36relation {empty-cells: show;} .pm36relation tbody td, .pm36relation th { border: 1px solid black;}</style>"
40+ tablefooter = "<tfoot><tr><td colspan=\"100%\">" <> pack (show (cardinality rel)) <> " tuples</td></tr></tfoot>"
41+ tablestart = "<table class=\"pm36relation\"\">"
2242
2343writeHTML :: Text -> IO ()
2444writeHTML = TIO.writeFile "/home/agentm/rel.html"
@@ -27,11 +47,14 @@ writeRel :: Relation -> IO ()
2747writeRel = writeHTML . relationAsHTML
2848
2949tupleAsHTML :: RelationTuple -> Text
30−tupleAsHTML tuple = "<tr>" `append` T.concat (L.map tupleFrag (tupleSortedAssocs tuple)) `append` "</tr>"
50+tupleAsHTML tuple = "<tr>" <> T.concat (L.map tupleFrag (tupleAssocs tuple)) <> "</tr>"
3151 where
32− tupleFrag tup = "<td>" `append` atomToText (snd tup) `append` "</td>"
52+ tupleFrag tup = "<td>" <> atomAsHTML (snd tup) <> "</td>"
53+ atomAsHTML (RelationAtom rel) = relationAsHTML rel
54+ atomAsHTML atom = atomToText atom
3355
3456tupleSetAsHTML :: RelationTupleSet -> Text
3557tupleSetAsHTML tupSet = foldr folder "" (asList tupSet)
3658 where
37− folder tuple acc = acc `append` tupleAsHTML tuple
59+ folder tuple acc = acc <> tupleAsHTML tuple
60+
src/lib/ProjectM36/Relation/Show/Term.hs+2−9
@@ -8,7 +8,6 @@ import ProjectM36.Relation
88import ProjectM36.Attribute hiding (null)
99import qualified Data.List as L
1010import qualified Data.Text as T
11−import qualified Data.Vector as V
1211import Control.Arrow hiding (left)
1312
1413boxV :: StringType
@@ -84,8 +83,8 @@ cellSizes (header, body) = map (map maxRowWidth &&& map (length . breakLines)) a
8483relationAsTable :: Relation -> Table
8584relationAsTable rel@(Relation _ tupleSet) = (header, body)
8685 where
87− oAttrs = orderedAttributes rel
88− oAttrNames = orderedAttributeNames rel
86+ oAttrs = orderedAttributes (attributes rel)
87+ oAttrNames = orderedAttributeNames (attributes rel)
8988 header = map prettyAttribute oAttrs
9089 body :: [[Cell]]
9190 body = L.foldl' tupleFolder [] (asList tupleSet)
@@ -143,12 +142,6 @@ renderBody cellMatrix cellLocs = renderRows `T.append` renderBottomBar
143142 rowHeightMatrix = zip cellMatrix (tail rowLocations)
144143 renderBottomBar = renderHBar boxBL boxBB boxBR columnLocations
145144
146−orderedAttributes :: Relation -> [Attribute]
147−orderedAttributes rel = L.sortBy (\a b -> attributeName a `compare` attributeName b) (V.toList (attributes rel))
148−
149−orderedAttributeNames :: Relation -> [AttributeName]
150−orderedAttributeNames rel = map attributeName (orderedAttributes rel)
151−
152145repeatString :: Int -> StringType -> StringType
153146repeatString c s = T.concat (replicate c s)
154147
src/lib/ProjectM36/RelationalExpression.hs+1−1
@@ -737,7 +737,7 @@ evalAtomExpr tupIn (FunctionAtomExpr funcName arguments ()) = do
737737 if expectedArgCount /= actualArgCount then
738738 throwE (FunctionArgumentCountMismatchError expectedArgCount actualArgCount)
739739 else do
740− _ <- mapM (\(expType, actType) -> either throwE pure (atomTypeVerify expType actType)) (safeInit (zip (atomFuncType func) argTypes))
740+ mapM_ (\(expType, actType) -> either throwE pure (atomTypeVerify expType actType)) (safeInit (zip (atomFuncType func) argTypes))
741741 evaldArgs <- mapM (liftE . evalAtomExpr tupIn) arguments
742742 case evalAtomFunction func evaldArgs of
743743 Left err -> throwE (AtomFunctionUserError err)
src/lib/ProjectM36/ScriptSession.hs+1−1
@@ -22,7 +22,7 @@ import Panic
2222import Outputable --hiding ((<>))
2323import PprTyThing
2424import Unsafe.Coerce
25−import Type hiding (pprTyThing)
25+import Type
2626
2727data ScriptSession = ScriptSession {
2828 hscEnv :: HscEnv,
src/lib/ProjectM36/StaticOptimizer.hs+1−1
@@ -6,7 +6,7 @@ import ProjectM36.Error
66import qualified ProjectM36.Attribute as A
77import qualified ProjectM36.AttributeNames as AS
88import ProjectM36.TupleSet
9−import Control.Monad.State hiding (join)
9+import Control.Monad.State
1010import Data.Either (rights, lefts)
1111import Control.Monad.Trans.Reader
1212import qualified Data.Map as M
src/lib/ProjectM36/TransactionGraph.hs+1−1
@@ -164,7 +164,7 @@ validateGraph graph@(TransactionGraph _ transSet) = do
164164 --uuids = map transactionId transSet
165165 --check that all heads appear in the transSet
166166 --check that all forward and backward links are in place
167− _ <- mapM (walkParentTransactions S.empty graph) (S.toList transSet)
167+ mapM_ (walkParentTransactions S.empty graph) (S.toList transSet)
168168 mapM (walkChildTransactions S.empty graph) (S.toList transSet)
169169
170170--verify that all parent links exist and that all children exist
src/lib/ProjectM36/Tuple.hs+8−4
@@ -29,6 +29,14 @@ tupleAttributes (RelationTuple tupAttrs _) = tupAttrs
2929tupleAssocs :: RelationTuple -> [(AttributeName, Atom)]
3030tupleAssocs (RelationTuple attrVec tupVec) = V.toList $ V.map (first attributeName) (V.zip attrVec tupVec)
3131
32+orderedTupleAssocs :: RelationTuple -> [(AttributeName, Atom)]
33+orderedTupleAssocs tup@(RelationTuple attrVec _) = map (\attr -> (attributeName attr, atomForAttr (attributeName attr))) oAttrs
34+ where
35+ oAttrs = orderedAttributes attrVec
36+ atomForAttr nam = case atomForAttributeName nam tup of
37+ Left _ -> TextAtom "<?>"
38+ Right val -> val
39+
3240-- return atoms in some arbitrary but consistent key order
3341tupleAtoms :: RelationTuple -> V.Vector Atom
3442tupleAtoms (RelationTuple _ tupVec) = tupVec
@@ -146,10 +154,6 @@ tupleAtomExtend newAttrName atom tupIn = tupleExtend tupIn newTup
146154 where
147155 newTup = RelationTuple (V.singleton $ Attribute newAttrName (atomTypeForAtom atom)) (V.singleton atom)
148156
149−{- sort the associative list to match the header sorting -}
150−tupleSortedAssocs :: RelationTuple -> [(AttributeName, Atom)]
151−tupleSortedAssocs (RelationTuple tupAttrs tupVec) = V.toList $ V.map (\(index,attr) -> (attributeName attr, tupVec V.! index)) $ V.indexed tupAttrs
152−
153157--this could be cheaper- it may not be wortwhile to update all the tuples for projection, but then the attribute management must be slightly different- perhaps the attributes should be a vector of association tuples [(name, index)]
154158tupleProject :: S.Set AttributeName -> RelationTuple -> RelationTuple
155159tupleProject projectAttrs (RelationTuple attrs tupVec) = RelationTuple newAttrs newTupVec
test/Relation/Basic.hs+12−1
@@ -3,6 +3,8 @@ import ProjectM36.Base
33import ProjectM36.Relation
44import ProjectM36.Error
55import ProjectM36.DateExamples
6+import ProjectM36.TupleSet
7+import ProjectM36.Attribute hiding (null, attributeNames)
68import ProjectM36.DataTypes.Primitive
79import ProjectM36.RelationalExpression
810import ProjectM36.Tuple
@@ -22,7 +24,9 @@ testList = TestList [testRelation "relationTrue" relationTrue, testRelation "rel
2224 testRelation "suppliers" suppliersRel,
2325 testRelation "products" productsRel,
2426 testRelation "supplierProducts" supplierProductsRel,
25− testMkRelationFromExprsBadAttrs]
27+ testMkRelationFromExprsBadAttrs,
28+ testDuplicateAttributes
29+ ]
2630
2731main :: IO ()
2832main = do
@@ -93,3 +97,10 @@ testMkRelationFromExprsBadAttrs = TestCase $ do
9397 case runReader (evalRelationalExpr (MakeRelationFromExprs (Just [AttributeAndTypeNameExpr "badAttr1" (PrimitiveTypeConstructor "Int" IntAtomType) ()]) [TupleExpr (M.singleton "badAttr2" (NakedAtomExpr (IntAtom 1)))])) (mkRelationalExprState context) of
9498 Left err -> assertEqual "tuple type mismatch" (TupleAttributeTypeMismatchError (A.attributesFromList [Attribute "badAttr2" IntAtomType])) err
9599 Right _ -> assertFailure "expected tuple type mismatch"
100+
101+--creating an empty relation with duplicate attribute names should fail
102+testDuplicateAttributes :: Test
103+testDuplicateAttributes = TestCase $ do
104+ let eRel = mkRelation attrs emptyTupleSet
105+ attrs = attributesFromList [Attribute "a" IntAtomType, Attribute "a" TextAtomType]
106+ assertEqual "duplicate attribute names" (Left (DuplicateAttributeNamesError (S.singleton "a"))) eRel
test/Server/WebSocket.hs+10−7
@@ -5,6 +5,8 @@ import ProjectM36.Server.WebSocket
55import ProjectM36.Server.Config
66import ProjectM36.Server
77import ProjectM36.Client
8+import ProjectM36.Base
9+
810import Network.Socket
911import Control.Exception
1012import Control.Concurrent
@@ -12,7 +14,6 @@ import System.Exit
1214import Data.Typeable
1315import Data.Text hiding (map)
1416import Data.Aeson
15−import ProjectM36.Base
1617import qualified Data.Map as M
1718import qualified Data.ByteString.Lazy as BS
1819import ProjectM36.Relation
@@ -90,14 +91,16 @@ testTutorialD port dbname = TestCase $ basicConnectionWithDatabase port dbname t
9091
9192 testtutd conn = do
9293 discardPromptInfo conn
93− WS.sendTextData conn ("executetutd:" `append` ":showexpr true")
94+ WS.sendTextData conn ("executetutd/json:" `append` ":showexpr true")
9495 discardPromptInfo conn
9596 discardPromptInfo conn
9697
9798 --receive relation response
9899 response <- WS.receiveData conn :: IO BS.ByteString
99− let decoded = decode response :: Maybe (M.Map Text Relation)
100−
101− case decoded of
102− Just val -> assertEqual "round-trip true relation" (M.lookup "displayrelation" val) (Just relationTrue) >> WS.sendClose conn ("test close"::Text)
103− Nothing -> assertFailure ("failed to decode relation from: " ++ show response)
100+ let decoded = decode response :: Maybe (M.Map Text (M.Map Text Relation))
101+ case decoded of
102+ Nothing -> assertFailure "failed to decode"
103+ Just decoded' -> do
104+ assertEqual "round-trip true relation" ((decoded' M.! "displayrelation") M.! "json") relationTrue
105+ WS.sendClose conn ("test close" :: Text)
106+
test/TransactionGraph/Merge.hs+1−1
@@ -21,7 +21,7 @@ import Data.Maybe
2121import Data.Time.Clock
2222import Data.Time.Calendar
2323
24−import Control.Monad.State hiding (join)
24+import Control.Monad.State
2525
2626{-# ANN module ("HLint: ignore Reduce duplication" :: String) #-}
2727