git.iko.soy/project-m36
on master help sign in
add merge notes document
8448917AgentM
docs/merge_notes.markdownnew file+182
compare with master
unifiedsplit
@@ -0,0 +1,182 @@
1+# Automatic Merge Notes
2+
3+These notes present cases where two diverging transaction branches can be automatically merged. Later, we cover transaction branches which cannot be automatically merged without user intervention (either in advance or in real-time).
4+
5+In the following diagrams, "A" is the divergence point (parent) for two branches "B1" and "B2". "B1" and "B2" are subsequently merged to "C".
6+
7+## 1. No-op
8+
9+![graph](http://g.gravizo.com/g?
10+ digraph G {
11+ node [shape=rect];
12+ A -> B1 [label="no change"];
13+ A -> B2 [label="no change"];
14+ B1 -> C [label="merge"];
15+ B2 -> C [label="merge"];
16+ }
17+)
18+
19+In this base case, we see that the merge is always successful if the two branches since the parent transaction A is equivalent to B1, B2, and C.
20+
21+## 2. One transaction changes
22+
23+![graph](http://g.gravizo.com/g?
24+digraph G {
25+node [shape=record];
26+A [label="A|x|a Int|[]"];
27+A -> B1 [label="insert x [4]"];
28+A -> B2 [label="no change"];
29+B1 -> C [label="merge"];
30+B2 -> C [label="merge"];
31+C [label="C|x|a Int|[4]"];
32+}
33+)
34+
35+The resultant merge is equivalent to A->B1->C ignoring B2. Since B2 adds nothing to the transaction stream, an equivalent graph would be the flat A->B1->B2->C. Such an application is similar to how git "merges" uses the fast-forward technique.
36+
37+## 3. Both transactions make equivalent changes
38+
39+![graph](http://g.gravizo.com/g?
40+digraph G {
41+node [shape=record];
42+A [label="A|x|a Int|[]"];
43+A -> B1 [label="insert x relation[tuple[a 4]]"];
44+A -> B2 [label="insert x relation[tuple[a 4]]"];
45+B1 -> C [label="merge"];
46+B2 -> C [label="merge"];
47+C [label="C|x|a Int|[[4]]"];
48+}
49+)
50+
51+Both branches add the "4" tuple, thus the result must also contain "4". There is no conflict here because, unlike in SQL, relations may not contain duplicate tuples, so the merge is unambiguous. Because both branches agree on the "truth", the merge succeeds regardless of the actual changes (insert, delete, update). Thus, the result C is equivalent to executing just one branch of changes.
52+
53+## 4. Two transactions add/remove different tuples
54+
55+![graph](http://g.gravizo.com/g?
56+ digraph G {
57+ node [shape=record];
58+ A [label="A|x|a Int|[]"];
59+ A -> B1 [label="insert x relation[tuple[a 4]]"];
60+ A -> B2 [label="insert x relation[tuple[a 5]]"];
61+ B1 -> C [label="merge"];
62+ B2 -> C [label="merge"];
63+ C [label="C|x|a Int|[[4],[5]]"];
64+ }
65+)
66+
67+Two relations add two unrelated tuples to the relation. We know they are unrelated because the merge does not trigger any constraint violations. A constraint violation during the merge would necessitate conflict resolution. In this case, the result is equivalent to the application of the branches in either order: A->B1->B2->C or A->B2->B1->C.
68+
69+## 5. Two transactions update a tuple with the same key
70+
71+![graph](http://g.gravizo.com/g?
72+digraph G {
73+node [shape=record];
74+A [label="A|employee|id Int,name Text|[[303,\"Dr. Garbez\"]]"];
75+A -> B1 [label="update employee where id=303 [name:=\"Jim Garbez\"]"];
76+A -> B2 [label="update employee where id=303 [name:=\"James Garbez\"]"];
77+B1 -> C [label="merge"];
78+B2 -> C [label="merge"];
79+B1 [label="B1|employee|id Int,name Text|[[303,\"Jim Garbez\"]]"];
80+B2 [label="B2|employee|id Int,name Text|[[303,\"James Garbez\"]]"];
81+C [label="C|employee|id Int,name Text|[[303,???]]"];
82+}
83+)
84+
85+This type of merge will always result in a key constraint violation. Logically, the branches are referring to the same tuple, so the key constraint is logically violated as if the merge engine tried to insert both changed tuples into the result.
86+
87+There are multiple possible conflict resolutions:
88+
89+### 5a. Prefer a branch name
90+
91+If a specifically-named branch is "preferred", the merge can proceed automatically with the other branch being completely discarded. This would be used, for example, when it is known that data flowing into a specific branch is authoritative. Note that this strategy (called "ours/theirs" merging in git) should discard all of a branches changes and not just the conflicted tuple changes.
92+
93+### 5b. Perform some logic to determine how to merge conflicting tuple atoms
94+
95+A stored procedure could determine which branch to choose or even how to merge values at the tuple or atom level. For example, the stored procedure could determine the "name" atom in the merged tuple result to be "Jim or James Garbez" (product) or "J. Garbez" (commonality) or even just "Garbez" (commonality) or the procedure could claim that the longer name is more likely to be correct.
96+
97+In general, the stored procedure could inspect (reading only) the two branches in the tree and make any deterministic decision it wishes to make based solely on those branches (maintaining the purity of the function).
98+
99+### 5c. Use the earlier/later changed
100+
101+The chosen branch is determined by its commit timestamp. The other branch's changes are discarded entirely.
102+
103+### 5d. Revert to shared ancestor
104+
105+Due to ambiguity, the rule could be to revert to the parent ancestor state which is the first knwon consistent state prior to the branching.
106+
107+### 6. User intervention
108+
109+If none of the above strategies result in a non-constraint-violating merge transaction, then the merge must fail, perhaps marking some state for a human to intervene and take action (which may include no action).
110+
111+# Schema changes
112+
113+## 1. Adding/removing a relation
114+
115+![graph](http://g.gravizo.com/g?
116+digraph G {
117+node [shape=record];
118+A;
119+A -> B1 [label="x:=true"];
120+A -> B2 [label="y:=true"];
121+B1 -> C [label="merge"];
122+B2 -> C [label="merge"];
123+C [label="C|x:=true and y:=true"];
124+}
125+)
126+
127+If a relation has no cross-constraints with another relation, then adding either at any point in the timeline cannot interfere with the other.
128+
129+## 2. Changing constraints
130+
131+![graph](http://g.gravizo.com/g?
132+digraph G {
133+node [shape=record];
134+A [label="A|employee[id Int, name Text]|[[3, \"Jim Garbez\"]]"];
135+A -> B1 [label="+ key employee[id]"];
136+A -> B2 [label="+ key employee[name]"];
137+B1 -> C [label="merge"];
138+B2 -> C [label="merge"];
139+C [label="C"];
140+}
141+)
142+
143+From all the examples, this is likely the most interesting since detecting when constraints conflict is not immediately obvious. Consider, for example, if A->B2 added a constraint "+ key employee[id,name]". This would not create a conflict under the conditions that both constraints can be added to the merge without causing a constraint violation, but is that the genuine intention of the user?
144+
145+Consider this example:
146+
147+![graph](http://g.gravizo.com/g?
148+digraph G {
149+node [shape=record];
150+A [label="A|integers[i Int]|[]"];
151+A -> B1 [label="+ constraint i<0"];
152+A -> B2 [label="+ constraint i>0"];
153+B1 -> C [label="merge"];
154+B2 -> C [label="merge"];
155+C [label="C"];
156+}
157+)
158+
159+The end result is a relation "i" which cannot contain any tuples. Clearly, neither branch intended this to be the case, so does it make sense to merely concatenate the constraints?
160+
161+## In general
162+
163+The function to perform the merge should be of type:
164+
165+```haskell
166+mergeBranches :: TransactionGraph -> BranchName -> BranchName -> TransactionGraph
167+mergeBranches subGraph branchNameA branchNameB = ...
168+```
169+
170+The subGraph argument is a subGraph of the entire database transaction graph starting at the common ancestor and including all transactions from both branches until the potential merge point. This is a sub graph because I have not been able to think of a circumstance where more context would be necessary. For convenience, the merge system should also include a transaction diff function of type:
171+
172+```haskell
173+diffTransactions :: Transaction -> Transaction -> DatabaseContextExpr
174+```
175+
176+This function should make a best effort to compress the actual differences into as few actions as possible, though further research will likely discover a canonical representation. Specifically, when diff'ing two transaction in sequence, this function should hopefully return the minimal statements needed to get from one transaction state to the next. This diff function's result can then be analyzed by the merge function in its decision-making.
177+
178+Each merge strategy can be implemented using separate functions of the above type, but deciding which to apply or which strategy should win automatically is likely an open research problem. Likely, this decision-making must be left up to the database developer who has the full context needed in order to make the decision.
179+
180+In addition, it would seem valuable to the algorithms to determine which constraints would be violated without executing the action. Even if this were determinable some of the time, it would appear to be a valuable optimization.
181+
182+The parallels between transaction merging and parallel lock management should not be underestimated. However, instead of critical sections, constraints appear to be able to represent additional complexity not usually attributed to locking.