on master help sign in
Merge branch 'master' into 33_autorebase

Conflicts: src/lib/ProjectM36/Client.hs src/lib/ProjectM36/TransactionGraph.hs

commit0747ada785540c830d0ac32c60d1e11c00058692 authorAgentM <agentm@themactionfaction.com> dateSun 30 Jul 2017, 09:30 (9 years ago) parent393d5fae55a674
23 files changed+254−107
README.markdown+2−1 docs/database_context_functions.markdown+4−4 docs/introduction_to_the_relational_algebra.markdown+42−11 docs/tutd_tutorial.markdown+17 project-m36.cabal+21−7 src/bin/TutorialD/Interpreter.hs+3−1 src/bin/TutorialD/Interpreter/RelationalExpr.hs+3−2 src/lib/ProjectM36/AtomFunction.hs+1 src/lib/ProjectM36/Atomable.hs+2−1 src/lib/ProjectM36/Attribute.hs+1−1 src/lib/ProjectM36/Base.hs+10−7 src/lib/ProjectM36/Client.hs+30−15 src/lib/ProjectM36/DatabaseContextFunctionUtils.hs+15 src/lib/ProjectM36/Error.hs+1−1 src/lib/ProjectM36/Key.hs+4 src/lib/ProjectM36/Relation.hs+9−9 src/lib/ProjectM36/RelationalExpression.hs+57−36 src/lib/ProjectM36/ScriptSession.hs+1 src/lib/ProjectM36/Session.hs+1 src/lib/ProjectM36/TransactionGraph.hs+5−4 src/lib/ProjectM36/Tuple.hs+3−3 test/TutorialD/Interpreter.hs+19−1 test/TutorialD/Interpreter/DatabaseContextFunctionScript.hs+3−3
README.markdown+2−1
@@ -35,6 +35,7 @@ Project:M36 supports multiple frontends which target different audiences.
3535* [Developer's Blog](https://agentm.github.io/project-m36/)
3636* [Mailing List/Discussion Group](https://groups.google.com/d/forum/project-m36)
3737* IRC Channel: chat.freenode.net #project-m36
38+* [Hackage](https://hackage.haskell.org/package/project-m36-0.1)
3839
3940## Documentation
4041
@@ -69,7 +70,7 @@ Project:M36 supports multiple frontends which target different audiences.
6970
7071## Development
7172
72−Project:M36 is developed in Haskell and compiled with GHC 7.8 or later.
73+Project:M36 is developed in Haskell and compiled with GHC 7.10 or GHC 8.0.2 or later.
7374
7475## Related Projects
7576
docs/database_context_functions.markdown+4−4
@@ -33,7 +33,7 @@ Returning a ```DatabaseContextFunctionError``` from the function ensures that an
3333Use ```adddatabasecontextfunction``` to compile and name a new database context function:
3434
3535```
36−adddatabasecontextfunction "addperson" Int -> Text -> DatabaseContext -> Either DatabaseContextFunctionError DatabaseContext """(\(age:name:_) ctx -> let newrel = MakeRelationFromExprs Nothing [TupleExpr (fromList [("name", NakedAtomExpr name),("age", NakedAtomExpr age)])] in if isRight (evalState (evalRelationalExpr (RelationVariable "person" ())) (mkRelationalExprState ctx)) then pure (execState (evalDatabaseContextExpr (Insert "person" newrel)) ctx) else pure (execState (evalDatabaseContextExpr (Assign "person" newrel)) ctx)) :: [Atom] -> DatabaseContext -> Either DatabaseContextFunctionError DatabaseContext"""
36+adddatabasecontextfunction "addperson" Int -> Text -> DatabaseContext -> Either DatabaseContextFunctionError DatabaseContext """(\(age:name:_) ctx -> let newrel = MakeRelationFromExprs Nothing [TupleExpr (fromList [("name", NakedAtomExpr name),("age", NakedAtomExpr age)])] in if isRight (executeRelationalExpr (RelationVariable "person" ()) ctx) then executeDatabaseContextExpr (Insert "person" newrel) ctx else executeDatabaseContextExpr (Assign "person" newrel) ctx) :: [Atom] -> DatabaseContext -> Either DatabaseContextFunctionError DatabaseContext"""
3737```
3838
3939This function is quite dense, so let's examine its components.
@@ -50,10 +50,10 @@ The meat of the function is obviously the Haskell, so let's lay it out:
5050
5151```haskell
5252(\(age:name:_) ctx -> let newrel = MakeRelationFromExprs Nothing [TupleExpr (fromList [("name", NakedAtomExpr name),("age", NakedAtomExpr age)])] in
53− if isRight (evalState (evalRelationalExpr (RelationVariable "person" ())) (mkRelationalExprState ctx)) then
54− pure (execState (evalDatabaseContextExpr (Insert "person" newrel)) ctx)
53+ if isRight (executeRelationalExpr (RelationVariable "person" ()) ctx) then
54+ executeDatabaseContextExpr (Insert "person" newrel) ctx
5555 else
56− pure (execState (evalDatabaseContextExpr (Assign "person" newrel)) ctx)) :: [Atom] -> DatabaseContext -> Either DatabaseContextFunctionError DatabaseContext
56+ executeDatabaseContextExpr (Assign "person" newrel) ctx)) :: [Atom] -> DatabaseContext -> Either DatabaseContextFunctionError DatabaseContext
5757```
5858
5959Line 1 sets up a new relation created from the function's arguments. In this case, we create a relation to represent a new person in our database.
docs/introduction_to_the_relational_algebra.markdown+42−11
@@ -16,7 +16,7 @@ Let us begin by starting with the smallest units in the relational model and wor
1616
1717A value is a typed, singular unit of data.
1818
19−Common examples could be a number indicating a person's age, the number of dollars in a bank account, or a street address used for delivery. In all cases, a value must be of a specific type and must support the notion of equality.
19+Common examples could be a number indicating a person's age, the number of dollars in a bank account, or a street address used for delivery. In all cases, a value must be of a specific type and must support the notion of equality.
2020
2121As an example, it doesn't make sense to ask if a person's age is equal to a street address; the relational model can enforce this- this is called "type safety" and is the motivation for using types.
2222
@@ -38,11 +38,11 @@ An attribute is a pair containing an attribute name along with an attribute type
3838
3939Examples:
4040* An "Address" attribute name is of type "Text":
41−
41+
4242(Address, Text)
4343
4444* An "Current Balance" attribute name is of type "Dollar Amount":
45−
45+
4646(Current Balance, Dollar Amount)
4747
4848#### Tuple
@@ -62,7 +62,7 @@ In the last example, the fact that the car failed the inspection is implied by t
6262
6363#### Relation
6464
65−A relation is a container of values organized by tuples of the same type. Specifically, a relation is a set of attributes (a relation header) mapped over a set of tuples (a relation body). Note that this implies that tuples cannot appear more than once since a set can only hold unique items.
65+A relation is a container of values organized by tuples of the same type. Specifically, a relation is a set of attributes (a relation header) mapped over a set of tuples (a relation body). Note that this implies that tuples cannot appear more than once since a set can only hold unique items.
6666
6767When a tuple appears in a relation, a relation's attributes and respective types must match that of all of its tuples. Thus, a relation's header summarizes the attribute names and types of all of its constituent tuples.
6868
@@ -116,7 +116,7 @@ To reiterate, the transition from T1 to T2 replaced the "Employees" relation var
116116
117117#### Relational Operators
118118
119−Since we refer to the relational algebra as a closed algebra, all relational operators necessarily operate on and generate relations. The purpose of the operators is allow data modelers to answer questions (queries) based on the current state of the database. For example, a modeler may wish to know the ages of the employees working in the accounting department. The modeler can then compose the relational operators together to answer this and virtually any other conceivable query.
119+Since we refer to the relational algebra as a closed algebra, all relational operators necessarily operate on and generate relations. The purpose of the operators is allow data modelers to answer questions (queries) based on the current state of the database. For example, a modeler may wish to know the ages of the employees working in the accounting department. The modeler can then compose the relational operators together to answer this and virtually any other conceivable query.
120120
121121##### Project
122122
@@ -186,7 +186,7 @@ Note that rename does not alter the type of the attribute in question.
186186
187187##### Join
188188
189−One of the most essential-to-understand operators is the join operator. Unlike the previous two unary (single relation argument) operators, join operates on two (or potentially more) relations. The purpose of a join is to merge multiple relations into one relation in order to determine the relationships between data across relations. For example, there are good reasons (which have not yet been explained) why a relation representing a company's products and records of when those products were shipped should be two disparate relations. The join operators merges relation A and relation B by returning a new relation with the attribute sets of both relation headers merged and with a body of the tuples merged if-and-only-if the in-common attributes' values are equal.
189+One of the most essential-to-understand operators is the join operator. Unlike the previous two unary (single relation argument) operators, join operates on two (or potentially more) relations. The purpose of a join is to merge multiple relations into one relation in order to determine the relationships between data across relations. For example, there are good reasons (which have not yet been explained) why a relation representing a company's products and records of when those products were shipped should be two disparate relations. The join operators merges relation A and relation B by returning a new relation with the attribute sets of both relation headers merged and with a body of the tuples merged if-and-only-if the in-common attributes' values are equal.
190190
191191In the case where no attributes are in common between the relations, the join operator becomes a cross-product operator; this is not a special case, rather a logical consequence of the definition.
192192
@@ -208,7 +208,7 @@ Join (Instructor relation)
208208|French 2|Bob|Room 103|
209209|English 2|Steve|Room 104|
210210
211−["Instructor Name" is the in-common attribute]
211+["Instructor Name" is the in-common attribute]
212212===>
213213
214214|(Instructor Name, Instructor Name Type)|(Office Number, Office Number Type)|(Course Name, Course Name Type)|(Course Room Number, Room Number Type)|
@@ -259,7 +259,7 @@ Examples:
259259
260260group ("Name") as "SameAgePeople"
261261
262−|(Name, Name Type)|(Age, Age Type)|
262+| (Name, Name Type)|(Age, Age Type) |
263263|-----------------|---------------|
264264|Steve|12|
265265|Cindy|12|
@@ -301,6 +301,37 @@ ungroup ("Pets")
301301
302302Note that Bob does not appear in the result relation because he has no pets.
303303
304+#### Difference
305+
306+The difference operator ("minus" in TutorialD) receives two relations of the same type (same attributes) as arguments rvA and rvB and returns a relation which contains tuples in rvA which are not found in rvB. This operator is similar to set difference.
307+
308+Example:
309+
310+* Show a relation of non-managers by starting with the relation of employees and removing the employees found in the manager relation.
311+
312+Employees
313+
314+|(Name, Name Type)|
315+|-----------------|
316+| Bob |
317+| Jim |
318+| Steve |
319+| Bart |
320+
321+Managers
322+
323+|(Name, Name Type)|
324+|-----------------|
325+| Bob |
326+| Steve |
327+
328+====>
329+
330+|(Name, Name Type)|
331+|-----------------|
332+| Jim |
333+| Bart |
334+
304335### Displaying Relations
305336
306337Note that in the above operator examples, the order of the tuples appears to shift around. Since a relation body is a set of tuples, any ordering, such as required to represent a relation in two-dimenstional space as a table, is completely arbitrary. Note that the following two relations are two representations of *same* relation:
@@ -311,7 +342,7 @@ Note that in the above operator examples, the order of the tuples appears to shi
311342|Sam|41|
312343|Cindy|42|
313344
314−and
345+and
315346
316347|(Name, Name Type)|(Age, Age Type)|
317348|-----------------|---------------|
@@ -343,7 +374,7 @@ For most companies, its data, such as customer, employee, sales, and product inf
343374
344375## What is the advantage of having relation-valued attributes?
345376
346−Since a value can store any (immutable) type, it can certainly hold a relation. At the very least, the mathematics does not exclude the possibility. This does create a recursively-defined data structure (Relation -> Tuple -> Value -> Relation), but it turns out that relations as values can be quite useful.
377+Since a value can store any (immutable) type, it can certainly hold a relation. At the very least, the mathematics does not exclude the possibility. This does create a recursively-defined data structure (Relation -> Tuple -> Value -> Relation), but it turns out that relations as values can be quite useful.
347378
348379For example, in SQL, if one wants to collect some aggregate data over some tuples while still returning the individual tuples, in the best case, the aggregate information must be repeated for every individual tuple. In the worst case, two separate queries are required since nested tables are not supported.
349380
@@ -372,7 +403,7 @@ Example:
372403
373404## Suggested Reading
374405
375−Chris Date has been researching the relational model and its consequences for over forty years. Date has published his findings in various papers but also summarizes his work in a series of introductory and advanced books. Date freely admits that his work on the relational algebra has been corrected over time, so it is best to read his latest works.
406+Chris Date has been researching the relational model and its consequences for over forty years. Date has published his findings in various papers but also summarizes his work in a series of introductory and advanced books. Date freely admits that his work on the relational algebra has been corrected over time, so it is best to read his latest works.
376407
377408Despite that fact that modern SQL databases do not adhere to the principles of relational algebra doesn't mean that the principles cannot be applied. Date often discusses and compares SQL constructions with his own "TutorialD" language. Project:M36 also supports TutorialD.
378409
docs/tutd_tutorial.markdown+17
@@ -320,6 +320,23 @@ TutorialD (master/main): :showexpr (s group ({sname,status,s#} as subrel)) ungro
320320└──────┴──┴─────┴──────┘
321321```
322322
323+#### Minus
324+
325+The binary minus operator takes two relation variables of the same type and returns the tuples which appear in the first relation variable but not in the second relation.
326+
327+```
328+TutorialD (master/main): x:=relation{tuple{name "Steve"},tuple{name "Bob"},tuple{name "Jim"},tuple{name "Bart"}}
329+TutorialD (master/main): y:=relation{tuple{name "Jim"},tuple{name "Bart"}}
330+TutorialD (master/main): :showexpr x minus y
331+┌──────────┐
332+│name::Text│
333+├──────────┤
334+│"Steve" │
335+│"Bob" │
336+└──────────┘
337+```
338+
339+
323340## State Operators
324341
325342While relational operators compose to relational expressions representing queries of the database, state operators change the state of the database.
project-m36.cabal+21−7
@@ -7,13 +7,24 @@ Bug-Reports: https://github.com/agentm/project-m36/issues
77Author: AgentM
88Stability: experimental
99Category: Relational Algebra
10+Maintainer: agentm@themactionfaction.com
1011Cabal-Version: >= 1.10
1112Synopsis: Relational Algebra Engine
1213Description: A relational algebra engine which can be used to persist and query Haskell data types.
14+Extra-Source-Files: Changelog.markdown README.markdown
15+
16+Source-Repository head
17+ Type: git
18+ location: https://github.com/agentm/project-m36
19+
20+Flag profiler
21+ Description: Enable Haskell-specific profiling support
22+ Default: False
23+ Manual: True
1324
1425Library
15− Build-Depends: base>=4.8 && < 5.0
16− ,ghc >= 7.8 && <= 8.1
26+ Build-Depends: base>=4.8 && < 6.0
27+ ,ghc >= 7.8 && < 8.3
1728 ,ghc-paths
1829 ,mtl
1930 ,containers
@@ -62,8 +73,8 @@ Library
6273 ,distributed-process-async >= 0.2.4
6374 ,network-transport-tcp
6475 ,network-transport
65− ,stm-containers
6676 ,list-t
77+ ,stm-containers >= 0.2.15
6778 ,optparse-applicative
6879 ,Glob
6980--used for hashing the transaction graph file to check for differences
@@ -86,6 +97,7 @@ Library
8697 ProjectM36.ScriptSession,
8798 ProjectM36.DatabaseContextFunction,
8899 ProjectM36.DatabaseContextFunctionError,
100+ ProjectM36.DatabaseContextFunctionUtils,
89101 ProjectM36.Key,
90102 ProjectM36.FunctionalDependency,
91103 ProjectM36.DatabaseContext,
@@ -146,7 +158,7 @@ Library
146158
147159Executable tutd
148160 Build-Depends: base >=4.8 && <5.0,
149− ghc >= 7.8 && <= 8.1,
161+ ghc >= 7.8 && < 8.3,
150162 ghc-paths,
151163 project-m36 == 0.1,
152164 containers,
@@ -219,7 +231,7 @@ Executable tutd
219231
220232Executable project-m36-server
221233 Build-Depends: base,
222− ghc >= 7.8 && <= 8.1,
234+ ghc >= 7.8 && < 8.3,
223235 ghc-paths,
224236 project-m36 == 0.1,
225237 binary,
@@ -251,7 +263,8 @@ Executable project-m36-server
251263 Main-Is: ./src/bin/ProjectM36/Server/project-m36-server.hs
252264 CPP-Options: -DPROJECTM36_VERSION="v0.1"
253265 GHC-Options: -Wall -threaded -rtsopts
254− GHC-Prof-Options: -fprof-auto -rtsopts -threaded -Wall
266+ if flag(profiler)
267+ GHC-Prof-Options: -fprof-auto -rtsopts -threaded -Wall
255268 Default-Language: Haskell2010
256269 Default-Extensions: OverloadedStrings
257270
@@ -267,7 +280,8 @@ Executable bigrel
267280 GHC-Options: -Wall -threaded -rtsopts
268281 CPP-Options: -DPROJECTM36_VERSION="v0.1"
269282 HS-Source-Dirs: ./src/bin
270− GHC-Prof-Options: -fprof-auto -rtsopts -threaded -Wall
283+ if flag(profiler)
284+ GHC-Prof-Options: -fprof-auto -rtsopts -threaded -Wall
271285
272286Test-Suite test-tutoriald
273287 Default-Language: Haskell2010
src/bin/TutorialD/Interpreter.hs+3−1
@@ -143,7 +143,7 @@ evalTutorialD sessionId conn safe expr = case expr of
143143 unsafeError
144144 else
145145 case evalInformationOperator execOp of
146− Left err -> barf err
146+ Left err -> pure (DisplayErrorResult err)
147147 Right info -> pure (DisplayResult info)
148148
149149 (RelVarExportOp execOp@(RelVarDataExportOperator relExpr _ _)) -> do
@@ -167,6 +167,8 @@ evalTutorialD sessionId conn safe expr = case expr of
167167 where
168168 needsSafe = safe == SafeEvaluation
169169 unsafeError = pure $ DisplayErrorResult "File I/O operation prohibited."
170+ barf :: RelationalError -> IO TutorialDOperatorResult
171+ barf (ScriptError (OtherScriptCompilationError errStr)) = pure (DisplayErrorResult (T.pack errStr))
170172 barf err = return $ DisplayErrorResult (T.pack (show err))
171173 eHandler io = do
172174 eErr <- io
src/bin/TutorialD/Interpreter/RelationalExpr.hs+3−2
@@ -148,7 +148,7 @@ restrictionPredicateP = makeExprParser predicateTerm predicateOperators
148148 [InfixL (reservedOp "and" >> return AndPredicate)],
149149 [InfixL (reservedOp "or" >> return OrPredicate)]
150150 ]
151− predicateTerm = parens restrictionPredicateP
151+ predicateTerm = try (parens restrictionPredicateP)
152152 <|> try restrictionAtomExprP
153153 <|> try restrictionAttributeEqualityP
154154 <|> try relationalBooleanExprP
@@ -156,7 +156,8 @@ restrictionPredicateP = makeExprParser predicateTerm predicateOperators
156156
157157relationalBooleanExprP :: RelationalMarkerExpr a => Parser (RestrictionPredicateExprBase a)
158158relationalBooleanExprP = do
159− relexpr <- relExprP
159+ relexpr <- parens relExprP <|> relTerm
160+ --we can't actually detect if the type is relational boolean, so we just pass it to the next phase
160161 return $ RelationalExprPredicate relexpr
161162
162163restrictionAttributeEqualityP :: RelationalMarkerExpr a => Parser (RestrictionPredicateExprBase a)
src/lib/ProjectM36/AtomFunction.hs+1
@@ -57,6 +57,7 @@ atomFunctionScript :: AtomFunction -> Maybe AtomFunctionBodyScript
5757atomFunctionScript func = case atomFuncBody func of
5858 AtomFunctionBody script _ -> script
5959
60+-- | Create a 'DatabaseContextIOExpr' which can be used to load a new atom function written in Haskell and loaded at runtime.
6061createScriptedAtomFunction :: AtomFunctionName -> [TypeConstructor] -> TypeConstructor -> AtomFunctionBodyScript -> DatabaseContextIOExpr
6162createScriptedAtomFunction funcName argsType retType script = AddAtomFunction funcName (
6263 argsType ++ [ADTypeConstructor "Either" [
src/lib/ProjectM36/Atomable.hs+2−1
@@ -29,6 +29,7 @@ data Test2T = Test2C Int Int
2929data Test3T = Test3Ca | Test3Cb
3030 deriving (Generic, Show, Eq, Binary, NFData, Atomable)
3131-}
32+-- | All database values ("atoms") adhere to the 'Atomable' typeclass. This class is derivable allowing new datatypes to be easily marshaling between Haskell values and database values.
3233class (Eq a, NFData a, Binary a, Show a) => Atomable a where
3334 toAtom :: a -> Atom
3435 default toAtom :: (Generic a, AtomableG (Rep a)) => a -> Atom
@@ -47,7 +48,7 @@ class (Eq a, NFData a, Binary a, Show a) => Atomable a where
4748 default toAtomType :: (Generic a, AtomableG (Rep a)) => a -> AtomType
4849 toAtomType v = toAtomTypeG (from v)
4950
50− --creates DatabaseContextExpr necessary to load the type constructor and data constructor into the database
51+ -- | Creates DatabaseContextExpr necessary to load the type constructor and data constructor into the database.
5152 toDatabaseContextExpr :: a -> DatabaseContextExpr
5253 default toDatabaseContextExpr :: (Generic a, AtomableG (Rep a)) => a -> DatabaseContextExpr
5354 toDatabaseContextExpr v = toDatabaseContextExprG (from v) (toAtomType v)
src/lib/ProjectM36/Attribute.hs+1−1
@@ -71,7 +71,7 @@ atomTypeForAttributeName attrName attrs = do
7171
7272attributeForName :: AttributeName -> Attributes -> Either RelationalError Attribute
7373attributeForName attrName attrs = case V.find (\attr -> attributeName attr == attrName) attrs of
74− Nothing -> Left $ NoSuchAttributeNamesError (S.singleton attrName)
74+ Nothing -> Left (NoSuchAttributeNamesError (S.singleton attrName))
7575 Just attr -> Right attr
7676
7777attributesForNames :: S.Set AttributeName -> Attributes -> Attributes
src/lib/ProjectM36/Base.hs+10−7
@@ -63,7 +63,7 @@ instance Binary Day where
6363 return (fromGregorian y m d)
6464
6565-- I suspect the definition of ConstructedAtomType with its name alone is insufficient to disambiguate the cases; for example, one could create a type named X, remove a type named X, and re-add it using different constructors. However, as long as requests are served from only one DatabaseContext at-a-time, the type name is unambiguous. This will become a problem for time-travel, however.
66−-- | The AtomType must uniquely identify the type of a atom.
66+-- | The AtomType uniquely identifies the type of a atom.
6767data AtomType = IntAtomType |
6868 DoubleAtomType |
6969 TextAtomType |
@@ -167,6 +167,7 @@ instance Hashable Relation where
167167
168168instance Binary Relation
169169
170+-- | Used to represent the number of tuples in a relation.
170171data RelationCardinality = Countable | Finite Int deriving (Eq, Show, Generic, Ord)
171172
172173-- | Relation variables are identified by their names.
@@ -224,7 +225,7 @@ data Notification = Notification {
224225
225226type TypeVarName = StringType
226227
227−-- | "data Either a b"
228+-- | Metadata definition for type constructors such as @data Either a b@.
228229data TypeConstructorDef = ADTypeConstructorDef TypeConstructorName [TypeVarName] |
229230 PrimitiveTypeConstructorDef TypeConstructorName AtomType
230231 deriving (Show, Generic, Binary, Eq, NFData)
@@ -242,7 +243,7 @@ type TypeConstructorArgName = StringType
242243type DataConstructorName = StringType
243244type AtomTypeName = StringType
244245
245−-- | Used to define a data constructor in a type constructor context. (Left a | Right b)
246+-- | Used to define a data constructor in a type constructor context such as @Left a | Right b@
246247data DataConstructorDef = DataConstructorDef DataConstructorName [DataConstructorDefArg] deriving (Eq, Show, Binary, Generic, NFData)
247248
248249type DataConstructorDefs = [DataConstructorDef]
@@ -285,7 +286,7 @@ data DatabaseContext = DatabaseContext {
285286
286287type IncDepName = StringType
287288
288−-- | Inclusion dependencies represent every possible database constraints. Constraints enforce specific, arbitrarily-complex rules to which the database context must adhere.
289+-- | Inclusion dependencies represent every possible database constraint. Constraints enforce specific, arbitrarily-complex rules to which the database context's relation variables must adhere unconditionally.
289290data InclusionDependency = InclusionDependency RelationalExpr RelationalExpr deriving (Show, Eq, Generic, NFData)
290291
291292instance Binary InclusionDependency
@@ -295,7 +296,7 @@ type AttributeNameAtomExprMap = M.Map AttributeName AtomExpr
295296--used for returning information about individual expressions
296297type DatabaseContextExprName = StringType
297298
298−-- | Database context expressions modify the database context. The expression name is useful
299+-- | Database context expressions modify the database context.
299300data DatabaseContextExpr =
300301 NoOperation |
301302 Define RelVarName [AttributeExpr] |
@@ -329,10 +330,10 @@ data DatabaseContextIOExpr = AddAtomFunction AtomFunctionName [TypeConstructor]
329330 AddDatabaseContextFunction DatabaseContextFunctionName [TypeConstructor] DatabaseContextFunctionBodyScript
330331 deriving (Show, Eq, Generic, Binary)
331332
332−-- | Restriction predicate are boolean algebra components which, when composed, indicate whether or not a tuple should be retained during a restriction (filtering) operation.
333333
334334type RestrictionPredicateExpr = RestrictionPredicateExprBase ()
335335
336+-- | Restriction predicates are boolean algebra components which, when composed, indicate whether or not a tuple should be retained during a restriction (filtering) operation.
336337data RestrictionPredicateExprBase a =
337338 TruePredicate |
338339 AndPredicate (RestrictionPredicateExprBase a) (RestrictionPredicateExprBase a) |
@@ -392,9 +393,9 @@ instance Eq Transaction where
392393instance Ord Transaction where
393394 compare (Transaction uuidA _ _) (Transaction uuidB _ _) = compare uuidA uuidB
394395
395−-- | An atom expression represents an action to take when extending a relation or when statically defining a relation or a new tuple.
396396type AtomExpr = AtomExprBase ()
397397
398+-- | An atom expression represents an action to take when extending a relation or when statically defining a relation or a new tuple.
398399data AtomExprBase a = AttributeAtomExpr AttributeName |
399400 NakedAtomExpr Atom |
400401 FunctionAtomExpr AtomFunctionName [AtomExprBase a] a |
@@ -465,10 +466,12 @@ data PersistenceStrategy = NoPersistence | -- ^ no filesystem persistence/memory
465466
466467type AttributeExpr = AttributeExprBase ()
467468
469+-- | Create attributes dynamically.
468470data AttributeExprBase a = AttributeAndTypeNameExpr AttributeName TypeConstructor a |
469471 NakedAttributeExpr Attribute
470472 deriving (Eq, Show, Generic, Binary, NFData)
471473
474+-- | Dynamically create a tuple from attribute names and 'AtomExpr's.
472475data TupleExprBase a = TupleExpr (M.Map AttributeName (AtomExprBase a))
473476 deriving (Eq, Show, Generic, NFData)
474477
src/lib/ProjectM36/Client.hs+30−15
@@ -1,11 +1,16 @@
11{-# LANGUAGE DeriveAnyClass, DeriveGeneric, ScopedTypeVariables, BangPatterns #-}
2+{-|
3+Module: ProjectM36.Client
4+
5+Client interface to local and remote Project:M36 databases. To get started, connect with 'connectProjectM36', then run some database changes with 'executeDatabaseContextExpr', and issue queries using 'executeRelationalExpr'.
6+-}
27module ProjectM36.Client
38 (ConnectionInfo(..),
49 Connection(..),
510 Port,
611 Hostname,
712 DatabaseName,
8− ConnectionError,
13+ ConnectionError(..),
914 connectProjectM36,
1015 close,
1116 closeRemote_,
@@ -20,7 +25,6 @@ module ProjectM36.Client
2025 typeForRelationalExpr,
2126 inclusionDependencies,
2227 planForDatabaseContextExpr,
23− processTransactionGraphPersistence,
2428 currentSchemaName,
2529 SchemaName,
2630 HeadName,
@@ -166,7 +170,7 @@ data RequestTimeoutException = RequestTimeoutException
166170
167171instance Exception RequestTimeoutException
168172
169−-- | Construct a 'ConnectionInfo' to describe how to make the 'Connection'.
173+-- | Construct a 'ConnectionInfo' to describe how to make the 'Connection'. The database can be run within the current process or running remotely via distributed-process.
170174data ConnectionInfo = InProcessConnectionInfo PersistenceStrategy NotificationCallback [GhcPkgPath]|
171175 RemoteProcessConnectionInfo DatabaseName NodeId NotificationCallback
172176
@@ -183,18 +187,24 @@ data EvaluatedNotification = EvaluatedNotification {
183187 }
184188 deriving(Binary, Eq, Show, Generic)
185189
190+
191+-- | Create a 'NodeId' for use in connecting to a remote server using distributed-process.
186192createNodeId :: Hostname -> Port -> NodeId
187193createNodeId host port = NodeId $ encodeEndPointAddress host (show port) 1
188194
195+-- | Use this for connecting to remote servers on the default port.
189196defaultServerPort :: Port
190197defaultServerPort = 6543
191198
199+-- | Use this for connecting to remote servers with the default database name.
192200defaultDatabaseName :: DatabaseName
193201defaultDatabaseName = "base"
194202
203+-- | Use this for connecting to remote servers with the default head name.
195204defaultHeadName :: HeadName
196205defaultHeadName = "master"
197206
207+-- | Create a connection configuration which connects to the localhost on the default server port and default server database name. The configured notification callback is set to ignore all events.
198208defaultRemoteConnectionInfo :: ConnectionInfo
199209defaultRemoteConnectionInfo = RemoteProcessConnectionInfo defaultDatabaseName (createNodeId "127.0.0.1" defaultServerPort) emptyNotificationCallback
200210
@@ -268,7 +278,7 @@ createScriptSession ghcPkgPaths = do
268278 Left err -> hPutStrLn stderr ("Failed to load scripting engine- scripting disabled: " ++ (show err)) >> pure Nothing --not a fatal error, but the scripting feature must be disabled
269279 Right s -> pure (Just s)
270280
271−-- | To create a 'Connection' to a remote or local database, create a connectionInfo and call 'connectProjectM36'.
281+-- | To create a 'Connection' to a remote or local database, create a 'ConnectionInfo' and call 'connectProjectM36'.
272282connectProjectM36 :: ConnectionInfo -> IO (Either ConnectionError Connection)
273283--create a new in-memory database/transaction graph
274284connectProjectM36 (InProcessConnectionInfo strat notificationCallback ghcPkgPaths) = do
@@ -403,7 +413,7 @@ closeSession :: SessionId -> Connection -> IO ()
403413closeSession sessionId (InProcessConnection conf) = do
404414 atomically $ STMMap.delete sessionId (ipSessions conf)
405415closeSession sessionId conn@(RemoteProcessConnection _) = remoteCall conn (CloseSession sessionId)
406−-- | 'close' cleans up the database access connection. Note that sessions persist even after the connection is closed.
416+-- | 'close' cleans up the database access connection and closes any relevant sockets.
407417close :: Connection -> IO ()
408418close (InProcessConnection conf) = do
409419 atomically $ do
@@ -415,8 +425,6 @@ close (InProcessConnection conf) = do
415425
416426close conn@(RemoteProcessConnection conf) = do
417427 _ <- (remoteCall conn Logout) :: IO Bool
418− --putStrLn ("closing " ++ show (localNodeId (rLocalNode conf)))
419−
420428 closeLocalNode (rLocalNode conf)
421429 closeTransport (rTransport conf)
422430
@@ -490,6 +498,7 @@ sessionAndSchema sessionId sessions = do
490498 Left err -> pure (Left err)
491499 Right schema -> pure (Right (session, schema))
492500
501+-- | Returns the name of the currently selected isomorphic schema.
493502currentSchemaName :: SessionId -> Connection -> IO (Either RelationalError SchemaName)
494503currentSchemaName sessionId (InProcessConnection conf) = atomically $ do
495504 let sessions = ipSessions conf
@@ -499,6 +508,7 @@ currentSchemaName sessionId (InProcessConnection conf) = atomically $ do
499508 Right session -> pure (Right (Sess.schemaName session))
500509currentSchemaName sessionId conn@(RemoteProcessConnection _) = remoteCall conn (RetrieveCurrentSchemaName sessionId)
501510
511+-- | Switch to the named isomorphic schema.
502512setCurrentSchemaName :: SessionId -> Connection -> SchemaName -> IO (Either RelationalError ())
503513setCurrentSchemaName sessionId (InProcessConnection conf) sname = atomically $ do
504514 let sessions = ipSessions conf
@@ -571,7 +581,7 @@ autoMergeToHead sessionId (InProcessConnection conf) strat headName' = do
571581 Right (discon', graph') -> pure (Right (discon', graph', True))
572582autoMergeToHead sessionId conn@(RemoteProcessConnection _) strat headName' = remoteCall conn (ExecuteAutoMergeToHead sessionId strat headName')
573583
574−-- | Execute a database context IO-monad-based expression for the given session and connection. `DatabaseContextIOExpr` modify the DatabaseContext but cannot be purely implemented.
584+-- | Execute a database context IO-monad-based expression for the given session and connection. `DatabaseContextIOExpr`s modify the DatabaseContext but cannot be purely implemented.
575585--this is almost completely identical to executeDatabaseContextExpr above
576586executeDatabaseContextIOExpr :: SessionId -> Connection -> DatabaseContextIOExpr -> IO (Either RelationalError ())
577587executeDatabaseContextIOExpr sessionId (InProcessConnection conf) expr = excEither $ do
@@ -617,6 +627,7 @@ executeCommitExprSTM_ oldContext newContext nodes = do
617627 pure (evaldNots, nodes)
618628
619629-- | Execute a transaction graph expression in the context of the session and connection. Transaction graph operators modify the transaction graph state.
630+
620631-- OPTIMIZATION OPPORTUNITY: no locks are required to write new transaction data, only to update the transaction graph id file
621632-- if writing data is re-entrant, we may be able to use unsafeIOtoSTM
622633-- perhaps keep hash of data file instead of checking if our head was updated on every write
@@ -718,6 +729,7 @@ executeTransGraphRelationalExpr _ (InProcessConnection conf) tgraphExpr = excEit
718729 Right rel -> pure (force (Right rel))
719730executeTransGraphRelationalExpr sessionId conn@(RemoteProcessConnection _) tgraphExpr = remoteCall conn (ExecuteTransGraphRelationalExpr sessionId tgraphExpr)
720731
732+-- | Schema expressions manipulate the isomorphic schemas for the current 'DatabaseContext'.
721733executeSchemaExpr :: SessionId -> Connection -> Schema.SchemaExpr -> IO (Either RelationalError ())
722734executeSchemaExpr sessionId (InProcessConnection conf) schemaExpr = atomically $ do
723735 let sessions = ipSessions conf
@@ -737,7 +749,7 @@ executeSchemaExpr sessionId (InProcessConnection conf) schemaExpr = atomically $
737749 pure (Right ())
738750executeSchemaExpr sessionId conn@(RemoteProcessConnection _) schemaExpr = remoteCall conn (ExecuteSchemaExpr sessionId schemaExpr)
739751
740−-- | After modifying a session, 'commit' the transaction to the transaction graph at the head which the session is referencing. This will also trigger checks for any notifications which need to be propagated.
752+-- | After modifying a 'DatabaseContext', 'commit' the transaction to the transaction graph at the head which the session is referencing. This will also trigger checks for any notifications which need to be propagated.
741753commit :: SessionId -> Connection -> IO (Either RelationalError ())
742754commit sessionId conn@(InProcessConnection _) = executeGraphExpr sessionId conn Commit
743755commit sessionId conn@(RemoteProcessConnection _) = remoteCall conn (ExecuteGraphExpr sessionId Commit)
@@ -748,11 +760,12 @@ sendNotifications pids localNode nots = mapM_ sendNots pids
748760 sendNots remoteClientPid = do
749761 when (not (M.null nots)) $ runProcess localNode $ send remoteClientPid (NotificationMessage nots)
750762
751−-- | Discard any changes made in the current session. This resets the disconnected transaction to reference the original database context of the parent transaction and is a very cheap operation.
763+-- | Discard any changes made in the current 'Session' and 'DatabaseContext'. This resets the disconnected transaction to reference the original database context of the parent transaction and is a very cheap operation.
752764rollback :: SessionId -> Connection -> IO (Either RelationalError ())
753765rollback sessionId conn@(InProcessConnection _) = executeGraphExpr sessionId conn Rollback
754766rollback sessionId conn@(RemoteProcessConnection _) = remoteCall conn (ExecuteGraphExpr sessionId Rollback)
755767
768+-- | Write the transaction graph to disk. This function can be used to incrementally write new transactions to disk.
756769processTransactionGraphPersistence :: PersistenceStrategy -> TransactionGraph -> IO ()
757770processTransactionGraphPersistence NoPersistence _ = pure ()
758771processTransactionGraphPersistence (MinimalPersistence dbdir) graph = transactionGraphPersist NoDiskSync dbdir graph >> pure ()
@@ -818,10 +831,10 @@ planForDatabaseContextExpr sessionId (InProcessConnection conf) dbExpr = do
818831planForDatabaseContextExpr sessionId conn@(RemoteProcessConnection _) dbExpr = remoteCall conn (RetrievePlanForDatabaseContextExpr sessionId dbExpr)
819832
820833-- | Return a relation which represents the current state of the global transaction graph. The attributes are
821−-- * current- boolean attribute representing whether or not the current session references this transaction
822−-- * head- text attribute which is a non-empty 'HeadName' iff the transaction references a head.
823−-- * id- id attribute of the transaction
824−-- * parents- a relation-valued attribute which contains a relation of transaction ids which are parent transaction to the transaction
834+-- * current- boolean attribute representing whether or not the current session references this transaction
835+-- * head- text attribute which is a non-empty 'HeadName' iff the transaction references a head.
836+-- * id- id attribute of the transaction
837+-- * parents- a relation-valued attribute which contains a relation of transaction ids which are parent transaction to the transaction
825838transactionGraphAsRelation :: SessionId -> Connection -> IO (Either RelationalError Relation)
826839transactionGraphAsRelation sessionId (InProcessConnection conf) = do
827840 let sessions = ipSessions conf
@@ -836,6 +849,7 @@ transactionGraphAsRelation sessionId (InProcessConnection conf) = do
836849
837850transactionGraphAsRelation sessionId conn@(RemoteProcessConnection _) = remoteCall conn (RetrieveTransactionGraph sessionId)
838851
852+-- | Returns the names and types of the relation variables in the current 'Session'.
839853relationVariablesAsRelation :: SessionId -> Connection -> IO (Either RelationalError Relation)
840854relationVariablesAsRelation sessionId (InProcessConnection conf) = do
841855 let sessions = ipSessions conf
@@ -885,6 +899,7 @@ headName sessionId (InProcessConnection conf) = do
885899 atomically (headNameSTM_ sessionId sessions graphTvar)
886900headName sessionId conn@(RemoteProcessConnection _) = remoteCall conn (ExecuteHeadName sessionId)
887901
902+-- | Returns a listing of all available atom types.
888903atomTypesAsRelation :: SessionId -> Connection -> IO (Either RelationalError Relation)
889904atomTypesAsRelation sessionId (InProcessConnection conf) = do
890905 let sessions = ipSessions conf
@@ -1008,4 +1023,4 @@ writeDisconAndGraph_ graphTvar sessionId session sessions discon graph = do
10081023 writeTVar graphTvar graph
10091024 let newSession = Session discon (Sess.schemaName session)
10101025 STMMap.insert newSession sessionId sessions
1011−-}
1026+-}
src/lib/ProjectM36/DatabaseContextFunctionUtils.hsnew file+15
@@ -0,0 +1,15 @@
1+module ProjectM36.DatabaseContextFunctionUtils where
2+import ProjectM36.RelationalExpression
3+import ProjectM36.Base
4+import ProjectM36.DatabaseContextFunctionError
5+import ProjectM36.Error
6+import Control.Monad.State
7+import Control.Monad.Trans.Reader
8+
9+executeDatabaseContextExpr :: DatabaseContextExpr -> DatabaseContext -> Either DatabaseContextFunctionError DatabaseContext
10+executeDatabaseContextExpr expr context = case runState (evalDatabaseContextExpr expr) (freshDatabaseState context) of
11+ (Nothing, (context', _, _)) -> pure context'
12+ (Just err, _) -> error (show err)
13+
14+executeRelationalExpr :: RelationalExpr -> DatabaseContext -> Either RelationalError Relation
15+executeRelationalExpr expr context = runReader (evalRelationalExpr expr) (mkRelationalExprState context)
src/lib/ProjectM36/Error.hs+1−1
@@ -123,7 +123,7 @@ data ScriptCompilationError = TypeCheckCompilationError String String | --expect
123123 SyntaxErrorCompilationError String |
124124 ScriptCompilationDisabledError |
125125 OtherScriptCompilationError String
126− deriving (Show,Eq, Generic, Binary, Typeable, NFData)
126+ deriving (Show, Eq, Generic, Binary, Typeable, NFData)
127127
128128instance Exception ScriptCompilationError
129129
src/lib/ProjectM36/Key.hs+4
@@ -21,6 +21,8 @@ example:
2121│5│
2222└─┘
2323-}
24+
25+-- | Create a uniqueness constraint for the attribute names and relational expression. Note that constraint can span multiple relation variables.
2426inclusionDependencyForKey :: AttributeNames -> RelationalExpr -> InclusionDependency
2527inclusionDependencyForKey attrNames relExpr = --InclusionDependency name (exprCount relExpr) (exprCount (projectedOnKeys relExpr))
2628 InclusionDependency equalityExpr (ExistingRelation relationFalse)
@@ -31,9 +33,11 @@ inclusionDependencyForKey attrNames relExpr = --InclusionDependency name (exprCo
3133 projectionForCount expr = Project (AttributeNames $ S.fromList ["b"]) expr
3234 equalityExpr = NotEquals (exprCount relExpr) (exprCount (projectedOnKeys relExpr))
3335
36+-- | Create a 'DatabaseContextExpr' which can be used to add a uniqueness constraint to attributes on one relation variable.
3437databaseContextExprForUniqueKey :: RelVarName -> [AttributeName] -> DatabaseContextExpr
3538databaseContextExprForUniqueKey rvName attrNames = AddInclusionDependency (rvName <> "_key") $ inclusionDependencyForKey (AttributeNames (S.fromList attrNames)) (RelationVariable rvName ())
3639
40+-- | Create a foreign key constraint from the first relation variable and attributes to the second.
3741databaseContextExprForForeignKey :: IncDepName -> (RelVarName, [AttributeName]) -> (RelVarName, [AttributeName]) -> DatabaseContextExpr
3842databaseContextExprForForeignKey fkName (rvA, attrsA) (rvB, attrsB) = AddInclusionDependency fkName $ InclusionDependency (Project (attrsL attrsA) (RelationVariable rvA ())) (Project (attrsL attrsB) (RelationVariable rvB ()))
3943 where
src/lib/ProjectM36/Relation.hs+9−9
@@ -12,7 +12,7 @@ import qualified ProjectM36.Attribute as A
1212import qualified ProjectM36.AttributeNames as AS
1313import ProjectM36.TupleSet
1414import ProjectM36.Error
15−import qualified Control.Parallel.Strategies as P
15+--import qualified Control.Parallel.Strategies as P
1616import qualified ProjectM36.TypeConstructorDef as TCD
1717import qualified ProjectM36.DataConstructorDef as DCD
1818import qualified Data.Text as T
@@ -165,8 +165,8 @@ group groupAttrNames newAttrName rel = do
165165restrictEq :: RelationTuple -> Relation -> Either RelationalError Relation
166166restrictEq tuple rel = restrict rfilter rel
167167 where
168− rfilter :: RelationTuple -> Bool
169− rfilter tupleIn = tupleIntersection tuple tupleIn == tuple
168+ rfilter :: RelationTuple -> Either RelationalError Bool
169+ rfilter tupleIn = pure (tupleIntersection tuple tupleIn == tuple)
170170
171171-- unwrap relation-valued attribute
172172-- return error if relval attrs and nongroup attrs overlap
@@ -203,11 +203,11 @@ attributesForRelval relvalAttrName (Relation attrs _) = do
203203 (RelationAtomType relAttrs) -> Right relAttrs
204204 _ -> Left $ AttributeIsNotRelationValuedError relvalAttrName
205205
206−restrict :: (RelationTuple -> Bool) -> Relation -> Either RelationalError Relation
207−--restrict rfilter (Relation attrs tupset) = Right $ Relation attrs $ HS.filter rfilter tupset
208−restrict rfilter (Relation attrs tupset) = Right $ Relation attrs processedTupSet
209− where
210− processedTupSet = RelationTupleSet ((filter rfilter (asList tupset)) `P.using` (P.parListChunk 1000 P.rdeepseq))
206+type RestrictionFilter = RelationTuple -> Either RelationalError Bool
207+restrict :: RestrictionFilter -> Relation -> Either RelationalError Relation
208+restrict rfilter (Relation attrs tupset) = do
209+ tuples <- filterM rfilter (asList tupset)
210+ Right $ Relation attrs (RelationTupleSet tuples)
211211
212212--joins on columns with the same name- use rename to avoid this- base case: cartesian product
213213--after changing from string atoms, there needs to be a type-checking step!
@@ -233,7 +233,7 @@ difference relA relB =
233233 where
234234 attrsA = attributes relA
235235 attrsB = attributes relB
236− rfilter tupInA = relFold (\tupInB acc -> if acc == False then False else if tupInB == tupInA then False else True) True relB
236+ rfilter tupInA = relFold (\tupInB acc -> if acc == Right False then pure False else pure (tupInB /= tupInA)) (Right True) relB
237237
238238--a map should NOT change the structure of a relation, so attributes should be constant
239239relMap :: (RelationTuple -> Either RelationalError RelationTuple) -> Relation -> Either RelationalError Relation
src/lib/ProjectM36/RelationalExpression.hs+57−36
@@ -195,7 +195,8 @@ evalRelationalExpr (Restrict predicateExpr relExpr) = do
195195 eFilterFunc <- predicateRestrictionFilter (attributes rel) predicateExpr
196196 case eFilterFunc of
197197 Left err -> return $ Left err
198− Right filterfunc -> return $ restrict filterfunc rel
198+ Right filterfunc -> do
199+ pure (restrict filterfunc rel)
199200
200201evalRelationalExpr (Equals relExprA relExprB) = do
201202 evaldA <- evalRelationalExpr relExprA
@@ -334,7 +335,7 @@ evalDatabaseContextExpr (Update relVarName atomExprMap restrictionPredicateExpr)
334335 if cardinality restrictedPortion == Finite 0 then
335336 pure Nothing
336337 else do
337− unrestrictedPortion <- restrict (not . predicateFunc) rel
338+ unrestrictedPortion <- restrict (\tup -> predicateFunc tup >>= pure . not) rel
338339 updatedPortion <- relMap (updateTupleWithAtomExprs atomExprMap context) restrictedPortion
339340 updatedRel <- union updatedPortion unrestrictedPortion
340341 pure (Just updatedRel)
@@ -614,52 +615,73 @@ liftE v = do
614615 Right val -> pure val
615616
616617{- used for restrictions- take the restrictionpredicate and return the corresponding filter function -}
617−predicateRestrictionFilter :: Attributes -> RestrictionPredicateExpr -> RelationalExprState (Either RelationalError (RelationTuple -> Bool))
618+predicateRestrictionFilter :: Attributes -> RestrictionPredicateExpr -> RelationalExprState (Either RelationalError RestrictionFilter)
618619predicateRestrictionFilter attrs (AndPredicate expr1 expr2) = do
619620 runExceptT $ do
620621 expr1v <- liftE (predicateRestrictionFilter attrs expr1)
621622 expr2v <- liftE (predicateRestrictionFilter attrs expr2)
622− pure (\x -> expr1v x && expr2v x)
623+ pure (\x -> do
624+ ev1 <- expr1v x
625+ ev2 <- expr2v x
626+ pure (ev1 && ev2))
623627
624628predicateRestrictionFilter attrs (OrPredicate expr1 expr2) = do
625629 runExceptT $ do
626630 expr1v <- liftE (predicateRestrictionFilter attrs expr1)
627631 expr2v <- liftE (predicateRestrictionFilter attrs expr2)
628− pure (\x -> expr1v x || expr2v x)
632+ pure (\x -> do
633+ ev1 <- expr1v x
634+ ev2 <- expr2v x
635+ pure (ev1 || ev2))
629636
630−predicateRestrictionFilter _ TruePredicate = pure (Right (\_ -> True))
637+predicateRestrictionFilter _ TruePredicate = pure (Right (\_ -> pure True))
631638
632639predicateRestrictionFilter attrs (NotPredicate expr) = do
633640 runExceptT $ do
634641 exprv <- liftE (predicateRestrictionFilter attrs expr)
635− pure (\x -> not (exprv x))
636−
637−predicateRestrictionFilter _ (RelationalExprPredicate relExpr) = runExceptT $ do
638− --merge attrs into to state attributes
639− rel <- liftE (evalRelationalExpr relExpr)
640− if rel == relationTrue then
641− pure (\_ -> True)
642− else if rel == relationFalse then
643− pure (\_ -> False)
644− else
645− throwE (PredicateExpressionError "Relational restriction filter must evaluate to 'true' or 'false'")
642+ pure (\x -> do
643+ ev <- exprv x
644+ pure (not ev))
645+
646+--optimization opportunity: if the subexpression does not reference attributes in the top-level expression, then it need only be evaluated once, statically, outside the tuple filter- see historical implementation here
647+predicateRestrictionFilter _ (RelationalExprPredicate relExpr) = do
648+ rstate <- ask
649+ pure (Right (\tup -> case runReader (evalRelationalExpr relExpr) (mergeTuplesIntoRelationalExprState tup rstate) of
650+ Left err -> Left err
651+ Right rel -> if arity rel /= 0 then
652+ Left (PredicateExpressionError "Relational restriction filter must evaluate to 'true' or 'false'")
653+ else
654+ pure (rel == relationTrue)))
646655
647656predicateRestrictionFilter attrs (AttributeEqualityPredicate attrName atomExpr) = do
648− --merge attrs into the state attributes
649657 rstate <- ask
658+ let (attrs', ctxtup') = case rstate of
659+ RelationalExprStateElems _ -> (attrs, emptyTuple)
660+ RelationalExprStateAttrsElems _ _ -> (attrs, emptyTuple)
661+ RelationalExprStateTupleElems _ ctxtup -> (A.union attrs (tupleAttributes ctxtup), ctxtup)
650662 runExceptT $ do
651− atomExprType <- liftE (typeFromAtomExpr attrs atomExpr)
652− attr <- either throwE pure (A.attributeForName attrName attrs)
663+ atomExprType <- liftE (typeFromAtomExpr attrs' atomExpr)
664+ attr <- either throwE pure $ case A.attributeForName attrName attrs of
665+ Right attr -> Right attr
666+ Left (NoSuchAttributeNamesError _) -> case A.attributeForName attrName (tupleAttributes ctxtup') of
667+ Right ctxattr -> Right ctxattr
668+ Left err2@(NoSuchAttributeNamesError _) -> Left err2
669+ Left err -> Left err
670+ Left err -> Left err
653671 if atomExprType /= A.atomType attr then
654672 throwE (TupleAttributeTypeMismatchError (A.attributesFromList [attr]))
655673 else
656− pure $ \tupleIn -> case atomForAttributeName attrName tupleIn of
657− Left _ -> False
658− Right atomIn ->
659− let atomEvald = runReader (evalAtomExpr tupleIn atomExpr) rstate in
660− case atomEvald of
661− Right atomCmp -> atomCmp == atomIn
662− Left _ -> False
674+ pure $ \tupleIn -> let evalAndCmp atomIn = case atomEvald of
675+ Right atomCmp -> atomCmp == atomIn
676+ Left _ -> False
677+ atomEvald = runReader (evalAtomExpr tupleIn atomExpr) rstate
678+ in
679+ pure $ case atomForAttributeName attrName tupleIn of
680+ Left (NoSuchAttributeNamesError _) -> case atomForAttributeName attrName ctxtup' of
681+ Left _ -> False
682+ Right ctxatom -> evalAndCmp ctxatom
683+ Left _ -> False
684+ Right atomIn -> evalAndCmp atomIn
663685-- in the future, it would be useful to do typechecking on the attribute and atom expr filters in advance
664686predicateRestrictionFilter attrs (AtomExprPredicate atomExpr) = do
665687 --merge attrs into the state attributes
@@ -670,13 +692,13 @@ predicateRestrictionFilter attrs (AtomExprPredicate atomExpr) = do
670692 throwE (AtomTypeMismatchError aType BoolAtomType)
671693 else
672694 pure (\tupleIn -> do
673− case runReader (evalAtomExpr tupleIn atomExpr) rstate of
695+ pure $ case runReader (evalAtomExpr tupleIn atomExpr) rstate of
674696 Left _ -> False
675697 Right boolAtomValue -> boolAtomValue == BoolAtom True)
676698
677699tupleExprCheckNewAttrName :: AttributeName -> Relation -> Either RelationalError Relation
678−tupleExprCheckNewAttrName attrName rel = if isRight $ attributeForName attrName rel then
679− Left $ AttributeNameInUseError attrName
700+tupleExprCheckNewAttrName attrName rel = if isRight (attributeForName attrName rel) then
701+ Left (error "SPAMMIT" $ AttributeNameInUseError attrName)
680702 else
681703 Right rel
682704
@@ -742,19 +764,18 @@ evalAtomExpr tupIn cons@(ConstructedAtomExpr dConsName dConsArgs ()) = runExcept
742764
743765typeFromAtomExpr :: Attributes -> AtomExpr -> RelationalExprState (Either RelationalError AtomType)
744766typeFromAtomExpr attrs (AttributeAtomExpr attrName) = do
745− --first, check if the attribute is in the immediate attributes
746767 rstate <- ask
747768 case A.atomTypeForAttributeName attrName attrs of
748769 Right aType -> pure (Right aType)
749770 Left err@(NoSuchAttributeNamesError _) -> case rstate of
750771 RelationalExprStateAttrsElems _ attrs' -> case A.attributeForName attrName attrs' of
751− Left err' -> pure (Left err')
772+ Left err' -> pure (error "SPAMMO2" $ Left err')
752773 Right attr -> pure (Right (A.atomType attr))
753− RelationalExprStateElems _ -> pure (Left err)
774+ RelationalExprStateElems _ -> pure (error (show attrs) $ Left err)
754775 RelationalExprStateTupleElems _ tup -> case atomForAttributeName attrName tup of
755− Left err' -> pure (Left err')
776+ Left err' -> pure (error "STAMP" $ Left err')
756777 Right atom -> pure (Right (atomTypeForAtom atom))
757− Left err -> pure (Left err)
778+ Left err -> pure (error "GONK" $ Left err)
758779typeFromAtomExpr _ (NakedAtomExpr atom) = pure (Right (atomTypeForAtom atom))
759780typeFromAtomExpr attrs (FunctionAtomExpr funcName atomArgs _) = do
760781 context <- liftM stateElemsContext ask
@@ -795,7 +816,7 @@ verifyAtomExprTypes relIn (AttributeAtomExpr attrName) expectedType = runExceptT
795816 RelationalExprStateTupleElems _ _ -> throwE err
796817 RelationalExprStateElems _ -> throwE err
797818 RelationalExprStateAttrsElems _ attrs -> case A.attributeForName attrName attrs of
798− Left err' -> throwE err'
819+ Left err' -> throwE (error "GONK" err')
799820 Right attrType -> either throwE pure (atomTypeVerify expectedType (A.atomType attrType))
800821 Left err -> throwE err
801822verifyAtomExprTypes _ (NakedAtomExpr atom) expectedType = pure (atomTypeVerify expectedType (atomTypeForAtom atom))
src/lib/ProjectM36/ScriptSession.hs+1
@@ -114,6 +114,7 @@ initScriptSession ghcPkgPaths = do
114114 "ProjectM36.Relation",
115115 "ProjectM36.AtomFunctionError",
116116 "ProjectM36.DatabaseContextFunctionError",
117+ "ProjectM36.DatabaseContextFunctionUtils",
117118 "ProjectM36.RelationalExpression"]
118119 qualifiedModules = map (\(modn, qualNam) -> IIDecl $ safeImportDecl (mkModuleName modn) (Just (mkModuleName qualNam))) [
119120 ("Data.Text", "T")
src/lib/ProjectM36/Session.hs+1
@@ -8,6 +8,7 @@ type SessionId = UUID
88
99--the persistence of a session is as long as the life of the database (not serialized to disk)
1010-- sessions are not associated with connections and have separate lifetimes
11+-- | Represents a pointer into the database's transaction graph which the 'DatabaseContextExpr's can then modify subsequently be committed to extend the transaction graph. The session contains staged (uncommitted) database changes as well as the means to switch between isomorphic schemas.
1112data Session = Session DisconnectedTransaction SchemaName
1213
1314defaultSchemaName :: SchemaName
src/lib/ProjectM36/TransactionGraph.hs+5−4
@@ -24,12 +24,13 @@ data TransactionIdLookup = TransactionIdLookup TransactionId |
2424 TransactionIdHeadNameLookup HeadName [TransactionIdHeadBacktrack]
2525 deriving (Show, Eq, Binary, Generic)
2626
27−-- topic~3^2
28−data TransactionIdHeadBacktrack = TransactionIdHeadParentBacktrack Int | -- git ~: walk back n parents, arbitrarily choosing a parent when a choice must be made
29− TransactionIdHeadBranchBacktrack Int -- git ^: walk back one parent level to the nth arbitrarily-chosen parent
27+-- | Used for git-style head backtracking such as topic~3^2.
28+data TransactionIdHeadBacktrack = TransactionIdHeadParentBacktrack Int | -- ^ git equivalent of ~: walk back n parents, arbitrarily choosing a parent when a choice must be made
29+ TransactionIdHeadBranchBacktrack Int -- ^ git equivalent of ^: walk back one parent level to the nth arbitrarily-chosen parent
3030 deriving (Show, Eq, Binary, Generic)
3131
32−--operators which manipulate a transaction graph
32+
33+-- | Operators which manipulate a transaction graph and which transaction the current 'Session' is based upon.
3334data TransactionGraphOperator = JumpToHead HeadName |
3435 JumpToTransaction TransactionId |
3536 Branch HeadName |
src/lib/ProjectM36/Tuple.hs+3−3
@@ -33,9 +33,9 @@ tupleAtoms (RelationTuple _ tupVec) = tupVec
3333
3434atomForAttributeName :: AttributeName -> RelationTuple -> Either RelationalError Atom
3535atomForAttributeName attrName (RelationTuple tupAttrs tupVec) = case V.findIndex (\attr -> attributeName attr == attrName) tupAttrs of
36− Nothing -> Left $ NoSuchAttributeNamesError (S.singleton attrName)
36+ Nothing -> Left (NoSuchAttributeNamesError (S.singleton attrName))
3737 Just index -> case tupVec V.!? index of
38− Nothing -> Left $ NoSuchAttributeNamesError (S.singleton attrName)
38+ Nothing -> Left (NoSuchAttributeNamesError (S.singleton attrName))
3939 Just atom -> Right atom
4040
4141{- -- resolve naming clash with Attribute and Relation later
@@ -173,7 +173,7 @@ tupleIntersection tuple1 tuple2 = RelationTuple newAttrs newTupVec
173173 newAttrs = indexFilter attrs1
174174 newTupVec = indexFilter (tupleAtoms tuple1)
175175
176− -- | An optimized form of tuple update which updates vectors efficiently.
176+-- | An optimized form of tuple update which updates vectors efficiently.
177177updateTupleWithAtoms :: (M.Map AttributeName Atom) -> RelationTuple -> RelationTuple
178178updateTupleWithAtoms updateMap (RelationTuple attrs tupVec) = RelationTuple attrs newVec
179179 where
test/TutorialD/Interpreter.hs+19−1
@@ -15,6 +15,7 @@ import ProjectM36.Base hiding (Finite)
1515import ProjectM36.TransactionGraph
1616import ProjectM36.Client
1717import qualified ProjectM36.DisconnectedTransaction as Discon
18+import qualified ProjectM36.AttributeNames as AN
1819import qualified ProjectM36.Session as Sess
1920import qualified ProjectM36.Attribute as A
2021import qualified Data.Map as M
@@ -89,7 +90,7 @@ main = do
8990 Right $ updateTupleWithAtoms (M.singleton "status" (IntAtom ((castInt statusAtom) + 10))) tuple
9091 else Right tuple) suppliersRel
9192 dateExampleRelTests = [("x:=s where true", Right suppliersRel),
92− ("x:=s where city = \"London\"", restrict (\tuple -> atomForAttributeName "city" tuple == (Right $ TextAtom "London")) suppliersRel),
93+ ("x:=s where city = \"London\"", restrict (\tuple -> pure $ atomForAttributeName "city" tuple == (Right $ TextAtom "London")) suppliersRel),
9394 ("x:=s where false", Right $ Relation (attributes suppliersRel) emptyTupleSet),
9495 ("x:=p where color=\"Blue\" and city=\"Paris\"", mkRelationFromList (attributes productsRel) [[TextAtom "P5", TextAtom "Cam", TextAtom "Blue", IntAtom 12, TextAtom "Paris"]]),
9596 ("a:=s; update a (status:=50); x:=a{status}", mkRelation (A.attributesFromList [Attribute "status" IntAtomType]) (RelationTupleSet [mkRelationTuple (A.attributesFromList [Attribute "status" IntAtomType]) (V.fromList [IntAtom 50])])),
@@ -375,6 +376,23 @@ testRelationalExprStateTupleElems = TestCase $ do
375376 [TextAtom "Athens", IntAtom 0]]
376377 assertEqual "validate parts count" expectedRel eRv
377378
379+ executeTutorialD sessionId dbconn "rv1:=relation{tuple{test 1}}"
380+ executeTutorialD sessionId dbconn "rv2:=relation{tuple{val 1},tuple{val 2}}"
381+ --check subexpression evaluation in restriction predicate
382+ -- "rv1 where ((rv2 where val=@test) {})"
383+ let correctSubexpr = Restrict (AttributeEqualityPredicate "val" (AttributeAtomExpr "test")) (RelationVariable "rv2" ())
384+ mainExpr subexpr = (Restrict
385+ (RelationalExprPredicate
386+ (Project AN.empty subexpr)) (RelationVariable "rv1" ()))
387+ eRv2 <- executeRelationalExpr sessionId dbconn (mainExpr correctSubexpr)
388+ let expectedRel2 = mkRelationFromList (attributesFromList [Attribute "test" IntAtomType]) [[IntAtom 1]]
389+ assertEqual "validate sub-expression attribute" expectedRel2 eRv2
390+
391+ --check error in subexpression
392+ let wrongSubexpr = Restrict (AttributeEqualityPredicate "nosuchattr" (AttributeAtomExpr "test")) (RelationVariable "rv2" ())
393+ eRv3 <- executeRelationalExpr sessionId dbconn (mainExpr wrongSubexpr)
394+ assertEqual "validate missing attribute in subexpression" (Left (NoSuchAttributeNamesError (S.singleton "nosuchattr"))) eRv3
395+
378396-- | Add a functional dependency on sname -> status and insert one tuple which is valid and another tuple which is invalid.
379397testFunctionalDependencies :: Test
380398testFunctionalDependencies = TestCase $ do
test/TutorialD/Interpreter/DatabaseContextFunctionScript.hs+3−3
@@ -8,7 +8,7 @@ import qualified Data.Text as T
88main :: IO ()
99main = do
1010 tcounts <- runTestTT (TestList [testBasicDBCFunction,
11− testErrorDBCFunction,
11+ testErrorDBCFunction,
1212 testExceptionDBCFunction,
1313 testDBCFunctionWithAtomArguments
1414 ])
@@ -20,7 +20,7 @@ adddatabasecontextfunction "addtruerv" DatabaseContext -> DatabaseContext """(\[
2020testBasicDBCFunction :: Test
2121testBasicDBCFunction = TestCase $ do
2222 (sess, conn) <- dateExamplesConnection emptyNotificationCallback
23− let addfunc = "adddatabasecontextfunction \"addTrue2\" DatabaseContext -> Either DatabaseContextFunctionError DatabaseContext \"\"\"(\\[] ctx -> pure $ execState (evalDatabaseContextExpr (Assign \"true2\" (ExistingRelation relationTrue))) ctx) :: [Atom] -> DatabaseContext -> Either DatabaseContextFunctionError DatabaseContext\"\"\""
23+ let addfunc = "adddatabasecontextfunction \"addTrue2\" DatabaseContext -> Either DatabaseContextFunctionError DatabaseContext \"\"\"(\\[] ctx -> executeDatabaseContextExpr (Assign \"true2\" (ExistingRelation relationTrue)) ctx) :: [Atom] -> DatabaseContext -> Either DatabaseContextFunctionError DatabaseContext\"\"\""
2424 executeTutorialD sess conn addfunc
2525 executeTutorialD sess conn "execute addTrue2()"
2626 let true2Expr = RelationVariable "true2" ()
@@ -50,7 +50,7 @@ testDBCFunctionWithAtomArguments :: Test
5050testDBCFunctionWithAtomArguments = TestCase $ do
5151 --test function with creation of a relvar with some arguments
5252 (sess, conn) <- dateExamplesConnection emptyNotificationCallback
53− let addfunc = "adddatabasecontextfunction \"multiArgFunc\" Int -> Text -> DatabaseContext -> Either DatabaseContextFunctionError DatabaseContext \"\"\"(\\(age:name:_) ctx -> pure $ execState (evalDatabaseContextExpr (Assign \"person\" (MakeRelationFromExprs Nothing [TupleExpr (fromList [(\"name\", NakedAtomExpr name), (\"age\", NakedAtomExpr age)])]))) ctx) :: [Atom] -> DatabaseContext -> Either DatabaseContextFunctionError DatabaseContext\"\"\""
53+ let addfunc = "adddatabasecontextfunction \"multiArgFunc\" Int -> Text -> DatabaseContext -> Either DatabaseContextFunctionError DatabaseContext \"\"\"(\\(age:name:_) ctx -> executeDatabaseContextExpr (Assign \"person\" (MakeRelationFromExprs Nothing [TupleExpr (fromList [(\"name\", NakedAtomExpr name), (\"age\", NakedAtomExpr age)])])) ctx) :: [Atom] -> DatabaseContext -> Either DatabaseContextFunctionError DatabaseContext\"\"\""
5454 executeTutorialD sess conn addfunc
5555 executeTutorialD sess conn "execute multiArgFunc(30,\"Steve\")"
5656 result <- executeRelationalExpr sess conn (RelationVariable "person" ())