From 297ab79a2af959356583e5f7466cf2f2f000ad1e Mon Sep 17 00:00:00 2001
From: Liubov Dmitrieva <liubovd@fb.com>
Date: Tue, 11 May 2021 13:59:06 -0700
Subject: [PATCH] split full sync into 3 passes

Summary:
split full sync into 3 steps

Commit cloud by default pulls only 30 days of commits.

Users often see some of their commits are missing in their smartlog.

I discovered that most of the users know the '--full' option (`hg cloud sync --full`) but not the 'max_sync_age' config option.

So, they try --full option but it could fail due to very very old commits we haven't migrated to Mononoke.

Users often don't really need those commits but it's not nice that the whole sync run failed.

We know that at least latest 2 years of commits are present in Mononoke.

So if we split a bit how we sync with --full option works, it would at least result in partially successfully sync for the latest 2/3 years of commits.

Reviewed By: mitrandir77

Differential Revision: D28352355

fbshipit-source-id: b5bacd7d5256191528613e3c0bcbb21b0104ac3c
---
 eden/scm/edenscm/hgext/commitcloud/sync.py    | 26 +++++++++++++++++++
 .../tests/test-commitcloud-sync-omission.t    | 10 +++++++
 2 files changed, 36 insertions(+)

diff --git a/eden/scm/edenscm/hgext/commitcloud/sync.py b/eden/scm/edenscm/hgext/commitcloud/sync.py
index 9f7f2e930e..cf55711cb6 100644
--- a/eden/scm/edenscm/hgext/commitcloud/sync.py
+++ b/eden/scm/edenscm/hgext/commitcloud/sync.py
@@ -137,6 +137,32 @@ def _sync(
     startnode = repo["."].node()
 
     if full:
+        # split the full sync into few passes:
+        #   sync the first 2 years of commits (commits should be present on the server)
+        #   sync the first 3 years of commits (commits should be probably present on the server for most of repos)
+        #   sync the rest of commits (some commits could be missing, haven't been migrated from the old backend)
+        for years in [2, 3]:
+            with ui.configoverride({("commitcloud", "max_sync_age"): years * 365}):
+                ui.status(
+                    _(
+                        "latest %d years of commits will be attempted to synchronize first\n"
+                    )
+                    % years,
+                    component="commitcloud",
+                )
+                _sync(
+                    repo,
+                    cloudrefs=cloudrefs,
+                    full=False,
+                    cloudversion=cloudversion,
+                    cloudworkspace=cloudworkspace,
+                    connect_opts=connect_opts,
+                    dest=dest,
+                )
+                ui.status(
+                    _("latest %d years of commits synchronized\n") % years,
+                    component="commitcloud",
+                )
         maxage = None
     else:
         maxage = ui.configint("commitcloud", "max_sync_age", None)
diff --git a/eden/scm/tests/test-commitcloud-sync-omission.t b/eden/scm/tests/test-commitcloud-sync-omission.t
index ec28bf39ef..70df57ce9f 100644
--- a/eden/scm/tests/test-commitcloud-sync-omission.t
+++ b/eden/scm/tests/test-commitcloud-sync-omission.t
@@ -552,6 +552,7 @@ Move the bookmark locally - this still gets synced ok.
 A full sync pulls the old commits in
   $ cd ../client3
   $ hgfakedate 1990-03-05T12:01Z cloud sync --full
+  commitcloud: latest 2 years of commits will be attempted to synchronize first
   commitcloud: synchronizing 'server' with 'user/test/default'
   pulling d133b886da68 from ssh://user@dummy/server
   searching for changes
@@ -561,6 +562,15 @@ A full sync pulls the old commits in
   added 2 changesets with 2 changes to 2 files
   commitcloud: commits synchronized
   finished in * (glob)
+  commitcloud: latest 2 years of commits synchronized
+  commitcloud: latest 3 years of commits will be attempted to synchronize first
+  commitcloud: synchronizing 'server' with 'user/test/default'
+  commitcloud: commits synchronized
+  finished in * (glob)
+  commitcloud: latest 3 years of commits synchronized
+  commitcloud: synchronizing 'server' with 'user/test/default'
+  commitcloud: commits synchronized
+  finished in * (glob)
 
   $ tglogp
   o  d133b886da68 draft 'midstack-feb9'