on master help sign in
add Merkle hash validation at database startup

add Merkle hash validation at arbitrary times with command add Merkle hash documentation

commit467de7e349818e3f39dc4ab07395e58e991d7174 authorAgentM <agentm@themactionfaction.com> dateSun 31 May 2020, 22:28 (6 years ago) parent634dd65
16 files changed+210−32
README.markdown+1 docs/merkle_transaction_graph.markdown+33 docs/transaction_graph_operators.markdown+6−4 project-m36.cabal+1 src/bin/ProjectM36/Client/Json.hs+4−1 src/bin/ProjectM36/Server/RemoteCallTypes/Json.hs+10 src/bin/TutorialD/Interpreter/TransactionGraphOperator.hs+12−3 src/lib/ProjectM36/Base.hs+2−1 src/lib/ProjectM36/Client.hs+28−10 src/lib/ProjectM36/Error.hs+3 src/lib/ProjectM36/MerkleHash.hs+10 src/lib/ProjectM36/Server.hs+1 src/lib/ProjectM36/Server/EntryPoints.hs+4 src/lib/ProjectM36/Server/RemoteCallTypes.hs+3 src/lib/ProjectM36/TransactionGraph.hs+28−6 test/TransactionGraph/Persist.hs+64−7
README.markdown+1
@@ -78,6 +78,7 @@ Project:M36 supports multiple frontends which target different audiences.
78781. [Isomorphic Schemas](docs/isomorphic_schemas.markdown)
79791. [Replication](docs/replication.markdown)
80801. [Basic Operator Benchmarks](https://rawgit.com/agentm/project-m36/master/docs/basic_benchmarks.html)
81+1. [Merkle Transaction Hashes](docs/merkle_transaction_graph.markdown)
8182
8283### Integrations
8384
docs/merkle_transaction_graph.markdownnew file+33
@@ -0,0 +1,33 @@
1+# Project:M36: Merkle Hashing for Transaction Graph
2+
3+## Introduction
4+
5+Project:M36 supports an append-only transaction graph model (similar to git) whereby new transactions can be added to the graph but not removed. As with any graph, each node can contain a Merkle hash which ensures not only that it's state since being added to the graph has not changed but also that none of its parent nodes all the way to the root node (initial state) have not been modified. This validation has a number of use cases.
6+
7+## Usage
8+
9+Project:M36 automatically adds Merkle hashes to every transaction. All data which represents the database state at that transaction is included in the hash including (but not limited to) relation variable definitions, atom and database context functions, constraints, and types.
10+
11+Project:M36 validates all merkle hashes at database startup. For users who need validation after the database is started, the user can issue `:validatemerklehashes` at the `tutd` command line.
12+
13+### Audit Logging
14+
15+Project:M36 supports audit logging by allowing users to travel to transactions in the past and recreate the database state at that past point-in-time. To ensure that those past states cannot be modified either by malicious on-disk or in-memory changes, each transaction is protected by its own Merkle hash. If the calculated hash does not match the stored hash, Project:M36 can immediately report which transaction triggered the tamper-proof log.
16+
17+### Reproducibility
18+
19+Since a Merkle tree is effectively immutable except to add more nodes, it can be used to save and restore a whole database with very high confidence in the results. This feature could also be used to restore a database partially; for example, to duplicate just the latest database state on a second machine for OLAP analysis.
20+
21+### High-Availability
22+
23+Merkle tree hashes can be used to assert database integrity across high-availibility or database replication. Since the hash covers all database state, it can be used to validate log-shipping-based replication. Corrupted cache tuple data can be easily detected and flushed.
24+
25+## Merkle Hashes in Other DBMSes
26+
27+While Merkle hashes have become prevalent in blockchain-style databases, they have not made many appearances in typical SQL databases. This is likely due to the fact that such databases consider any additional operations against the large tuple stores as prohibitively expensive. Project:M36 is able to skirt this cost by only hashing database update expressions rather than the result of these executed expressions. This also means that the Merkle hash can be computed against transactions which have been created but never executed.
28+
29+## Conclusion
30+
31+Merkle hashes are an obvious improvement to any graph-based transaction log and are a low-cost method of ensuring data integrity.
32+
33+A further improvement could allow users to specify an initial key provided by an admin user to salt the hash used to create the database. Additionally, a multi-user system could allow individual users to sign their committed transactions. The signature would then be included in the Merkle hash.
docs/transaction_graph_operators.markdown+6−4
@@ -43,15 +43,17 @@ TutorialD (master/main): :rollback
4343In order to navigate the transaction graph, one must specify transaction UUIDs. Viewing the entirety of the transaction graph as a relation is a single command.
4444
4545```
46−TutorialD (master/main): :showgraph
47−┌─────────────┬──────────┬──────────────────────────────────────┬────────────────────────────┬──────────────────────────────┐
48−│current::Bool│head::Text│id::Text │parents::relation {id::Text}│stamp::DateTime │
49−├─────────────┼──────────┼──────────────────────────────────────┼────────────────────────────┼──────────────────────────────┤
46+┌─────────────┬────────────────────────────────────────────┬──────────┬──────────────────────────────────────┬────────────────────────────────────────┬─────────────────────────────────┐
47+│current::Bool│hash::ByteString │head::Text│id::Text │parents::relation {id::Text} │stamp::DateTime │
48+├─────────────┼────────────────────────────────────────────┼──────────┼──────────────────────────────────────┼────────────────────────────────────────┼─────────────────────────────────┤
49+
5050
5151<big result elided>
5252```
5353The "current" attribute is a boolean value. If true, the transaction is the parent of the current database context. Commits will be children of this transaction.
5454
55+The "hash" attribute contains the merkle tree hash for the given transaction. This can be used to ensure that the transaction graph has been used in an append-only fashion, ensuring historical state and audit log integrity.
56+
5557The "head" attribute is a boolean value indicating whether or not the transaction is a "head" node of its graph. Head nodes are nodes which have no children.
5658
5759The "id" attribute is the UUID needed in order to navigate the graph.
project-m36.cabal+1
@@ -79,6 +79,7 @@ Library
7979 ProjectM36.Relation.Parse.CSV,
8080 ProjectM36.IsomorphicSchema,
8181 ProjectM36.InclusionDependency,
82+ ProjectM36.MerkleHash,
8283 ProjectM36.Client,
8384 ProjectM36.Client.Simple,
8485 ProjectM36.Persist,
src/bin/ProjectM36/Client/Json.hs+4−1
@@ -3,6 +3,7 @@ module ProjectM36.Client.Json where
33import Data.Aeson
44import ProjectM36.Server.RemoteCallTypes.Json ()
55import ProjectM36.Client
6+import ProjectM36.TransactionGraph
67import Control.Exception (IOException)
78
89instance ToJSON EvaluatedNotification
@@ -11,4 +12,6 @@ instance FromJSON EvaluatedNotification
1112instance ToJSON ConnectionError
1213
1314instance ToJSON IOException where
14− toJSON err = object ["IOException" .= show err]
15+ toJSON err = object ["IOException" .= show err]
16+
17+instance ToJSON MerkleValidationError
src/bin/ProjectM36/Server/RemoteCallTypes/Json.hs+10
@@ -10,6 +10,7 @@ import ProjectM36.DataTypes.Primitive
1010import ProjectM36.Error
1111import ProjectM36.IsomorphicSchema
1212import ProjectM36.Server.RemoteCallTypes
13+import ProjectM36.MerkleHash
1314
1415import Data.Aeson
1516import Data.ByteString.Base64 as B64
@@ -132,6 +133,15 @@ instance FromJSON Notification
132133instance ToJSON ScriptCompilationError
133134instance FromJSON ScriptCompilationError
134135
136+instance ToJSON MerkleHash where
137+ toJSON h = object [ "merklehash" .= decodeUtf8 (B64.encode (_unMerkleHash h))]
138+instance FromJSON MerkleHash where
139+ parseJSON = withObject "merklehash" $ \o -> do
140+ b64bs <- encodeUtf8 <$> o .: "merklehash"
141+ case B64.decode b64bs of
142+ Left err -> fail ("Failed to parse merkle hash: " ++ err)
143+ Right bs -> pure (MerkleHash bs)
144+
135145instance ToJSON RelationalError
136146instance FromJSON RelationalError
137147
src/bin/TutorialD/Interpreter/TransactionGraphOperator.hs+12−3
@@ -2,8 +2,9 @@
22module TutorialD.Interpreter.TransactionGraphOperator where
33import TutorialD.Interpreter.Base
44import ProjectM36.TransactionGraph hiding (autoMergeToHead)
5−import ProjectM36.Client
5+import ProjectM36.Client as C
66import ProjectM36.Base
7+import ProjectM36.Relation (relationTrue)
78import Data.Functor
89
910data ConvenienceTransactionGraphOperator = AutoMergeToHead MergeStrategy HeadName
@@ -71,6 +72,9 @@ mergeTransactionsP = do
7172 reservedOp ":mergetrans"
7273 MergeTransactions <$> mergeTransactionStrategyP <*> identifier <*> identifier
7374
75+validateMerkleHashesP :: Parser ROTransactionGraphOperator
76+validateMerkleHashesP = reservedOp ":validatemerklehashes" $> ValidateMerkleHashes
77+
7478transactionGraphOpP :: Parser TransactionGraphOperator
7579transactionGraphOpP =
7680 jumpToHeadP
@@ -83,7 +87,7 @@ transactionGraphOpP =
8387 <|> mergeTransactionsP
8488
8589roTransactionGraphOpP :: Parser ROTransactionGraphOperator
86−roTransactionGraphOpP = showGraphP
90+roTransactionGraphOpP = showGraphP <|> validateMerkleHashesP
8791
8892
8993
@@ -101,6 +105,11 @@ interpretOps newUUID trans@(DisconnectedTransaction _ context) transGraph instri
101105
102106evalROGraphOp :: SessionId -> Connection -> ROTransactionGraphOperator -> IO (Either RelationalError Relation)
103107evalROGraphOp sessionId conn ShowGraph = transactionGraphAsRelation sessionId conn
108+evalROGraphOp sessionId conn ValidateMerkleHashes = do
109+ eVal <- C.validateMerkleHashes sessionId conn
110+ case eVal of
111+ Left err -> pure (Left err)
112+ Right _ -> pure (Right relationTrue)
104113
105114evalConvenienceGraphOp :: SessionId -> Connection -> ConvenienceTransactionGraphOperator -> IO (Either RelationalError ())
106−evalConvenienceGraphOp sessionId conn (AutoMergeToHead strat head') = autoMergeToHead sessionId conn strat head'
115+evalConvenienceGraphOp sessionId conn (AutoMergeToHead strat head') = autoMergeToHead sessionId conn strat head'
src/lib/ProjectM36/Base.hs+2−1
@@ -4,6 +4,7 @@
44module ProjectM36.Base where
55import ProjectM36.DatabaseContextFunctionError
66import ProjectM36.AtomFunctionError
7+import ProjectM36.MerkleHash
78
89import qualified Data.Map as M
910import qualified Data.HashSet as HS
@@ -414,7 +415,7 @@ transactionsForGraph (TransactionGraph _ ts) = ts
414415data TransactionInfo = TransactionInfo {
415416 parents :: TransactionParents,
416417 stamp :: UTCTime,
417− merkleHash :: ByteString
418+ merkleHash :: MerkleHash
418419 } deriving (Show, Generic)
419420
420421type TransactionParents = NE.NonEmpty TransactionId
src/lib/ProjectM36/Client.hs+28−10
@@ -80,6 +80,7 @@ module ProjectM36.Client
8080 databaseContextExprForUniqueKey,
8181 databaseContextExprForForeignKey,
8282 createScriptedAtomFunction,
83+ ProjectM36.Client.validateMerkleHashes,
8384 AttributeExprBase(..),
8485 TypeConstructorBase(..),
8586 TypeConstructorDef(..),
@@ -118,7 +119,7 @@ import Control.Monad.State
118119import qualified ProjectM36.RelationalExpression as RE
119120import ProjectM36.DatabaseContext (basicDatabaseContext)
120121import qualified ProjectM36.TransactionGraph as Graph
121−import ProjectM36.TransactionGraph
122+import ProjectM36.TransactionGraph as TG
122123import qualified ProjectM36.Transaction as Trans
123124import ProjectM36.TransactionGraph.Persist
124125import ProjectM36.Attribute
@@ -265,6 +266,7 @@ data Connection = InProcessConnection InProcessConnectionConf |
265266data ConnectionError = SetupDatabaseDirectoryError PersistenceError |
266267 IOExceptionError IOException |
267268 NoSuchDatabaseByNameError DatabaseName |
269+ DatabaseValidationError [MerkleValidationError] |
268270 LoginError
269271 deriving (Show, Eq, Generic)
270272
@@ -379,12 +381,15 @@ connectPersistentProjectM36 strat sync dbdir freshGraph notificationCallback ghc
379381 case graph of
380382 Left err' -> return $ Left (SetupDatabaseDirectoryError err')
381383 Right graph' -> do
382− tvarGraph <- newTVarIO graph'
383− sessions <- StmMap.newIO
384− clientNodes <- StmSet.newIO
385− (localNode, transport) <- createLocalNode
386− lockMVar <- newMVar digest
387− let conn = InProcessConnection InProcessConnectionConf {
384+ case TG.validateMerkleHashes graph' of
385+ Left merkleErrs -> pure (Left (DatabaseValidationError merkleErrs))
386+ Right _ -> do
387+ tvarGraph <- newTVarIO graph'
388+ sessions <- StmMap.newIO
389+ clientNodes <- StmSet.newIO
390+ (localNode, transport) <- createLocalNode
391+ lockMVar <- newMVar digest
392+ let conn = InProcessConnection InProcessConnectionConf {
388393 ipPersistenceStrategy = strat,
389394 ipClientNodes = clientNodes,
390395 ipSessions = sessions,
@@ -395,9 +400,9 @@ connectPersistentProjectM36 strat sync dbdir freshGraph notificationCallback ghc
395400 ipLocks = Just (lockFileH, lockMVar)
396401 }
397402
398− notificationPid <- startNotificationListener localNode notificationCallback
399− addClientNode conn notificationPid
400− pure (Right conn)
403+ notificationPid <- startNotificationListener localNode notificationCallback
404+ addClientNode conn notificationPid
405+ pure (Right conn)
401406
402407-- | Create a new session at the transaction id and return the session's Id.
403408createSessionAtCommit :: Connection -> TransactionId -> IO (Either RelationalError SessionId)
@@ -1205,3 +1210,16 @@ executeDataFrameExpr sessionId conn@(InProcessConnection _) dfExpr = do
12051210 pure (Right dFrame'')
12061211executeDataFrameExpr sessionId conn@(RemoteProcessConnection _) dfExpr = remoteCall conn (ExecuteDataFrameExpr sessionId dfExpr)
12071212
1213+validateMerkleHashes :: SessionId -> Connection -> IO (Either RelationalError ())
1214+validateMerkleHashes sessionId (InProcessConnection conf) = do
1215+ let sessions = ipSessions conf
1216+ atomically $ do
1217+ eSession <- sessionForSessionId sessionId sessions
1218+ case eSession of
1219+ Left err -> pure (Left err)
1220+ Right _ -> do
1221+ graph <- readTVar (ipTransactionGraph conf)
1222+ case Graph.validateMerkleHashes graph of
1223+ Left merkleErrs -> pure $ Left $ someErrors (map (\(MerkleValidationError tid expected actual) -> MerkleHashValidationError tid expected actual) merkleErrs)
1224+ Right () -> pure (Right ())
1225+validateMerkleHashes sessionId conn@RemoteProcessConnection{} = remoteCall conn (ExecuteValidateMerkleHashes sessionId)
src/lib/ProjectM36/Error.hs+3
@@ -1,6 +1,7 @@
11{-# LANGUAGE DeriveGeneric, DeriveAnyClass #-}
22module ProjectM36.Error where
33import ProjectM36.Base
4+import ProjectM36.MerkleHash
45import ProjectM36.DatabaseContextFunctionError
56import ProjectM36.AtomFunctionError
67import qualified Data.Set as S
@@ -101,6 +102,8 @@ data RelationalError = NoSuchAttributeNamesError (S.Set AttributeName)
101102
102103 | NoUncommittedContextInEvalError
103104 | TupleExprsReferenceMultipleMarkersError
105+
106+ | MerkleHashValidationError TransactionId MerkleHash MerkleHash
104107
105108 | MultipleErrors [RelationalError]
106109 deriving (Show,Eq,Generic,Binary,Typeable, NFData)
src/lib/ProjectM36/MerkleHash.hsnew file+10
@@ -0,0 +1,10 @@
1+{-# LANGUAGE GeneralizedNewtypeDeriving, DeriveGeneric #-}
2+module ProjectM36.MerkleHash where
3+import Data.ByteString (ByteString)
4+import GHC.Generics
5+import Data.Binary (Binary)
6+import Control.DeepSeq (NFData)
7+
8+newtype MerkleHash = MerkleHash { _unMerkleHash :: ByteString }
9+ deriving (Show, Eq, Binary, Generic, Monoid, Semigroup, NFData)
10+
src/lib/ProjectM36/Server.hs+1
@@ -53,6 +53,7 @@ serverDefinition testBool ti = defaultProcess {
5353 handleCall (\conn (RetrieveSessionIsDirty sessionId) -> handleRetrieveSessionIsDirty ti sessionId conn),
5454 handleCall (\conn (ExecuteAutoMergeToHead sessionId strat headName') -> handleExecuteAutoMergeToHead ti sessionId conn strat headName'),
5555 handleCall (\conn (RetrieveTypeConstructorMapping sessionId) -> handleRetrieveTypeConstructorMapping ti sessionId conn),
56+ handleCall (\conn (ExecuteValidateMerkleHashes sessionId) -> handleValidateMerkleHashes ti sessionId conn),
5657 handleCall (\conn Logout -> handleLogout ti conn)
5758 ] ++ testModeHandlers,
5859 unhandledMessagePolicy = Terminate
src/lib/ProjectM36/Server/EntryPoints.hs+4
@@ -172,3 +172,7 @@ handleRetrieveTypeConstructorMapping ti sessionId conn = do
172172 ret <- timeoutOrDie ti (C.typeConstructorMapping sessionId conn)
173173 reply ret conn
174174
175+handleValidateMerkleHashes :: Timeout -> SessionId -> Connection -> Reply (Either RelationalError ())
176+handleValidateMerkleHashes ti sessionId conn = do
177+ ret <- timeoutOrDie ti (C.validateMerkleHashes sessionId conn)
178+ reply ret conn
src/lib/ProjectM36/Server/RemoteCallTypes.hs+3
@@ -69,3 +69,6 @@ data ExecuteAutoMergeToHead = ExecuteAutoMergeToHead SessionId MergeStrategy Hea
6969 deriving (Binary, Generic)
7070data RetrieveTypeConstructorMapping = RetrieveTypeConstructorMapping SessionId
7171 deriving (Binary, Generic)
72+
73+data ExecuteValidateMerkleHashes = ExecuteValidateMerkleHashes SessionId
74+ deriving (Binary, Generic)
src/lib/ProjectM36/TransactionGraph.hs+28−6
@@ -9,6 +9,7 @@ import ProjectM36.TupleSet
99import ProjectM36.Tuple
1010import ProjectM36.RelationalExpression
1111import ProjectM36.TransactionGraph.Merge
12+import ProjectM36.MerkleHash
1213import qualified ProjectM36.DisconnectedTransaction as Discon
1314import qualified ProjectM36.Attribute as A
1415
@@ -30,7 +31,6 @@ import Data.Monoid
3031import Control.Arrow
3132import Data.Maybe
3233import Data.UUID.V4
33−import qualified Data.ByteString as BS
3434import qualified Data.ByteString.Lazy as BL
3535import ProjectM36.DatabaseContext as DBC
3636import Crypto.Hash.SHA256
@@ -62,7 +62,7 @@ isCommit :: TransactionGraphOperator -> Bool
6262isCommit Commit = True
6363isCommit _ = False
6464
65−data ROTransactionGraphOperator = ShowGraph
65+data ROTransactionGraphOperator = ShowGraph | ValidateMerkleHashes
6666 deriving Show
6767
6868bootstrapTransactionGraph :: UTCTime -> TransactionId -> DatabaseContext -> TransactionGraph
@@ -313,7 +313,7 @@ graphAsRelation (DisconnectedTransaction parentId _ _) graph@(TransactionGraph _
313313 tupleGenerator transaction = case transactionParentsRelation transaction graph of
314314 Left err -> Left err
315315 Right parentTransRel -> Right [TextAtom $ T.pack $ show (transactionId transaction),
316− ByteStringAtom $ merkleHash (transactionInfo transaction),
316+ ByteStringAtom $ _unMerkleHash (merkleHash (transactionInfo transaction)),
317317 DateTimeAtom (timestamp transaction),
318318 RelationAtom parentTransRel,
319319 BoolAtom $ parentId == transactionId transaction,
@@ -592,13 +592,13 @@ addMerkleHash graph trans = Transaction (transactionId trans) newInfo (schemas t
592592 newInfo = (transactionInfo trans) { merkleHash = calculateMerkleHash trans graph }
593593
594594-- the new hash includes the parents' ids, the current id, and the hash of the context, and the merkle hashes of the parent transactions
595−calculateMerkleHash :: Transaction -> TransactionGraph -> BS.ByteString
596−calculateMerkleHash trans graph = hashlazy (mconcat [transIds,
595+calculateMerkleHash :: Transaction -> TransactionGraph -> MerkleHash
596+calculateMerkleHash trans graph = MerkleHash $ hashlazy (mconcat [transIds,
597597 dbcBytes,
598598 schemasBytes,
599599 parentMerkleHashes])
600600 where
601− parentMerkleHashes = BL.fromChunks $ map getMerkleHash parentTranses
601+ parentMerkleHashes = BL.fromChunks $ map (_unMerkleHash . getMerkleHash) parentTranses
602602 parentTranses =
603603 case transactionsForIds (parentIds trans) graph of
604604 Left RootTransactionTraversalError -> []
@@ -609,3 +609,25 @@ calculateMerkleHash trans graph = hashlazy (mconcat [transIds,
609609 dbcBytes = DBC.hashBytes (concreteDatabaseContext trans)
610610 schemasBytes = B.encode (subschemas trans)
611611
612+validateMerkleHash :: Transaction -> TransactionGraph -> Either MerkleValidationError ()
613+validateMerkleHash trans graph =
614+ if expectedHash /= actualHash then
615+ Left (MerkleValidationError (transactionId trans) expectedHash actualHash)
616+ else
617+ pure ()
618+ where
619+ expectedHash = merkleHash (transactionInfo trans)
620+ actualHash = calculateMerkleHash trans graph
621+
622+data MerkleValidationError = MerkleValidationError TransactionId MerkleHash MerkleHash
623+ deriving (Show, Eq, Generic)
624+
625+validateMerkleHashes :: TransactionGraph -> Either [MerkleValidationError] ()
626+validateMerkleHashes graph =
627+ if null errs then pure () else Left errs
628+ where
629+ errs = S.foldr validateTrans [] (transactionsForGraph graph)
630+ validateTrans trans acc =
631+ case validateMerkleHash trans graph of
632+ Left err -> err : acc
633+ _ -> acc
test/TransactionGraph/Persist.hs+64−7
@@ -3,11 +3,12 @@ import Test.HUnit
33import ProjectM36.Base
44import ProjectM36.Persist (DiskSync(NoDiskSync))
55import ProjectM36.TransactionGraph.Persist
6−import ProjectM36.TransactionGraph
6+import ProjectM36.TransactionGraph as TG
77import ProjectM36.Transaction
88import ProjectM36.DateExamples
99import System.IO.Temp
1010import System.Exit
11+import System.Directory
1112import Data.Either
1213import Data.UUID.V4 (nextRandom)
1314import System.FilePath
@@ -16,10 +17,9 @@ import qualified Data.Map as M
1617import qualified Data.Set as S
1718import Data.Time.Clock
1819import Data.Time.Calendar
19−#ifdef PM36_HASKELL_SCRIPTING
20−import ProjectM36.Client
20+import ProjectM36.Client as C
2121import ProjectM36.Relation
22−#endif
22+import ProjectM36.Transaction.Persist
2323
2424main :: IO ()
2525main = do
@@ -29,7 +29,8 @@ main = do
2929testList :: Test
3030testList = TestList [testBootstrapDB,
3131 testDBSimplePersistence,
32− testFunctionPersistence]
32+ testFunctionPersistence,
33+ testMerkleHashValidation]
3334
3435
3536stamp' :: UTCTime
@@ -74,7 +75,62 @@ testDBSimplePersistence = TestCase $ withSystemTempDirectory "m36testdb" $ \temp
7475 case graphErr of
7576 Left err -> assertFailure (show err)
7677 Right graph'' -> assertBool "graph equality" (mapEq graph'' == mapEq graph')
77−
78+
79+testMerkleHashValidation :: Test
80+testMerkleHashValidation = TestCase $
81+ -- add a commit and validate the hashes successfully
82+ withSystemTempDirectory "m36testdb" $ \tempdir -> do
83+ let dbdir = tempdir </> "dbdir"
84+ connInfo = InProcessConnectionInfo (MinimalPersistence dbdir) emptyNotificationCallback []
85+ conn <- assertIOEither $ connectProjectM36 connInfo
86+ sess <- assertIOEither $ createSessionAtHead conn "master"
87+ Right _ <- executeDatabaseContextExpr sess conn (Assign "x" (ExistingRelation relationTrue))
88+ Right _ <- commit sess conn
89+ val <- C.validateMerkleHashes sess conn
90+ assertEqual "merkle success" (Right ()) val
91+
92+ --read graph from disk
93+ conn' <- assertIOEither $ connectProjectM36 connInfo
94+ sess' <- assertIOEither $ createSessionAtHead conn' "master"
95+
96+ val' <- C.validateMerkleHashes sess' conn'
97+ assertEqual "merkle read again success" (Right ()) val'
98+
99+ --alter the on-disk representation and check that the hash validation fails
100+ eGraph <- transactionGraphLoad dbdir emptyTransactionGraph Nothing
101+ case eGraph of
102+ Left err -> assertFailure "failed to load graph"
103+ Right graph -> do
104+ let trans = transactionHeadsForGraph graph M.! "master"
105+ updatedTrans = Transaction (transactionId trans) (transactionInfo trans) updatedSchemas
106+ transactionDir = dbdir </> show (transactionId trans)
107+ updatedSchemas =
108+ case schemas trans of
109+ Schemas ctx sschemas ->
110+ let updatedContext = ctx {
111+ relationVariables = M.insert "malicious" (ExistingRelation relationFalse) (relationVariables ctx)
112+ } in
113+ Schemas updatedContext sschemas
114+ maliciousGraph = TransactionGraph malHeads malTransSet
115+ malHeads = M.insert "master" updatedTrans (transactionHeadsForGraph graph)
116+ malTransSet = S.insert updatedTrans (transactionsForGraph graph)
117+ malMerkleHash = calculateMerkleHash updatedTrans maliciousGraph
118+ regMerkleHash = merkleHash (transactionInfo trans)
119+ --validate and fail
120+ let val'' = TG.validateMerkleHashes maliciousGraph
121+ assertEqual "loaded graph merkle hashes" (Left [MerkleValidationError (transactionId trans) regMerkleHash malMerkleHash]) val''
122+ --delete existing transaction directory
123+ removeDirectoryRecursive transactionDir
124+ writeTransaction NoDiskSync dbdir updatedTrans
125+
126+ --read graph from disk
127+ eConnFail <- connectProjectM36 connInfo
128+ case eConnFail of
129+ Left err ->
130+ assertEqual "open connection merkle validation" (DatabaseValidationError [MerkleValidationError (transactionId trans) regMerkleHash malMerkleHash]) err
131+ Right _ -> assertFailure "open connection validation"
132+
133+
78134
79135--only Haskell-scripted dbc and atom functions can be serialized
80136testFunctionPersistence :: Test
@@ -103,10 +159,11 @@ testFunctionPersistence = TestCase $
103159 let expectedRel = mkRelationFromList (attributesFromList [Attribute "a" IntAtomType]) [[IntAtom 3]]
104160 assertEqual "testdisk dbc function run" expectedRel res
105161
162+#endif
163+
106164assertIOEither :: (Show a) => IO (Either a b) -> IO b
107165assertIOEither x = do
108166 ret <- x
109167 case ret of
110168 Left err -> assertFailure (show err) >> undefined
111169 Right val -> pure val
112−#endif