mutationstore: insert mutation info in a stable order

Summary:
When inserting multiple rows using `INSERT IGNORE`, it's possible to hit
deadlocks if two competing transactions insert the same rows in a different
order.

This can be triggered in the mutation store if two clients are pushing
overlapping sets of commits, and the arbitrary changeset order from the
`HashSet` gives the commits in a different order.  For example the first
transaction might be inserting "A, B, C", be holding the lock on row A, and be
waiting for the lock on row C, meanwhile another transaction is inserting "C,
D, A", holding the lock on row C, and waiting for the lock on row A.

To avoid these deadlocks, always insert the mutation changesets in a stable
order (sorted by changeset hash or successor hash).

Reviewed By: krallin

Differential Revision: D22067338

fbshipit-source-id: aa54c8a9c0cac4e4ae35855b44e759f8b6cb4f59

eden/mononoke/mercurial/mutation/src/store.rs

@@ -131,6 +131,16 @@ impl SqlHgMutationStore {
             }
         }
 
+        // Sort the entries so that they are inserted in a stable
+        // order.  This prevents deadlocks where one transaction is
+        // adding commits "A, B, C" and gains the lock on A while
+        // another transaction is adding "C, D, A" and gains the lock
+        // on C.
+        db_csets.sort_unstable();
+        db_entries.sort_unstable();
+        db_preds.sort_unstable();
+        db_splits.sort_unstable();
+
         // Convert the numbers to references.
         let ref_db_entries: Vec<_> = db_entries
             .iter()