Commit Graph

31 Commits

Author SHA1 Message Date
Jason Fried
8d130ad96d pep-479 codemod
Summary:
https://www.python.org/dev/peps/pep-0479/

If you are in a generator (a function that uses "yield") you are never allowed to raise StopIteration instead you should just `return` or pass out of scope (implicit return None)

Reviewed By: thatch

Differential Revision: D17749640

fbshipit-source-id: 9f1be673cf877ff193a0379a0208d037dd2d7bae
2019-10-04 15:24:23 -07:00
Aleksei Kulikov
1c2460479e snapshot: clean the internal files given the --clean flag
Summary: `merge/*` and `rebasestate` files.

Reviewed By: mitrandir77

Differential Revision: D17719134

fbshipit-source-id: a7ed2258396e8b758debdb9fa28c953b0a22e749
2019-10-04 10:07:55 -07:00
Aleksei Kulikov
b802a42448 snapshot: store metadata content
Summary:
Make it possible to store small files directly inside the metadata blob:
```
{
  "files" : {
    "unknown": {
      "some/big/untracked/file": {"oid": "oid0"},
      "some/small/untracked/file": {"content": <binary file contents>},
    },
    . . .
  },
  "version" : "version of metadata format"
}
```

Reviewed By: mitrandir77

Differential Revision: D17689055

fbshipit-source-id: 7ab18d8f012a20be04d65fcbe2365ff0157386f1
2019-10-04 10:07:54 -07:00
Aleksei Kulikov
a45cf73590 snapshot: fix smartlog predicate for snapshots
Summary: This is a better way to wrap smartlog functionality.

Reviewed By: mitrandir77

Differential Revision: D17686738

fbshipit-source-id: 770b32c19c89a59aee0f8b25251faaae7685a4dd
2019-10-04 10:07:54 -07:00
Aleksei Kulikov
41c7d139af snapshot: fix graphnode and phabstatus templates for snapshots
Summary: The graphnode templatekw must be wrapped in the keywords dict, and the phabstatus template needs to work if the requested revision is hidden.

Reviewed By: mitrandir77

Differential Revision: D17685575

fbshipit-source-id: 5e4a8b45dfe6564af3f72de3511d834aa498d154
2019-10-04 10:07:53 -07:00
Aleksei Kulikov
e635171709 snapshot: fix rebasestate show snapshot
Reviewed By: markbt

Differential Revision: D17666034

fbshipit-source-id: 67005c07c1c583b709310861c3ece38c4be825a3
2019-10-02 11:54:25 -07:00
Aleksei Kulikov
1a6762b57d snapshot: match files correctly during checkout, update tests
Summary: The previous matcher constructed wrong (with the extra `cwd` part) paths.

Reviewed By: markbt

Differential Revision: D17659134

fbshipit-source-id: 8556b8acda515ac68c2a72f2ede6408172d42575
2019-10-02 11:54:25 -07:00
Aleksei Kulikov
21837851bd commitcloud: sync snapshots on the client side
Summary:
Essentially, snapshots are being synced exactly like heads are.
A dedicated table in CC DB etc.

Reviewed By: markbt

Differential Revision: D17345491

fbshipit-source-id: a1e65c2a4815f437a11da68f9d46c27f046453a9
2019-10-02 11:54:24 -07:00
Aleksei Kulikov
041fe70d99 snapshot: update the docs
Summary: Now help text for the snapshot extension doesn't have any TODOs.

Reviewed By: markbt

Differential Revision: D17627630

fbshipit-source-id: 3e02f6540ab84010712a146a069dbb4a3968182f
2019-10-01 06:48:43 -07:00
Aleksei Kulikov
8a79cfa7e8 snapshot: add the unionstore to handle the rebundling of snapshot bundles
Summary:
If some bundle consisted of several snapshots, and the client requests a part of that data, the server can trigger the rebundling process.
This diff teaches snapshots to the rebundling process.

The `bundlerepo` object should store data in memory, so a `unionstore` class is made up.
It allows to "pair" in-memory store with a regular local store and query them via the same API.

Reviewed By: markbt

Differential Revision: D17684945

fbshipit-source-id: 0bdb1f70b777fb2d8193b3bfa46b5915c2019663
2019-10-01 06:46:43 -07:00
Aleksei Kulikov
65c288f459 snapshot: hide and unhide snapshots
Summary:
`hg hide <snapshot id>` should remove it from the list of snapshots (=> no sync and not shown in smartlog).
`hg unhide <snapshot id>` should restore the snapshot in the list, but do not unhide the underlying commit.

Reviewed By: markbt

Differential Revision: D17570798

fbshipit-source-id: 895f7b53464860b80f4f2498ea97f8db270b4d67
2019-09-26 14:20:23 -07:00
Aleksei Kulikov
aaca5459b6 snapshot: add snapshot show command to view individual snapshots
Summary:
Need a cmd which is equivalent to `hg show <commit id>` for snapshots.
```
$ hg snapshot show <snapshot id>
<the regular patch message for commits>
  ? untrackedfile
  @@ -0,0 +1,1 @@  # diff-like content
  +fizz
  ! missingfile
  @@ -1,1 +0,0 @@  # diff-like content
  -foo
  ? .hg/merge/fc4ffdcb8ed23cecd44a0e11d23af83b445179b4
  @@ -0,0 +1,1 @@
  +b

  The snapshot is in an unfinished *merge* state.
```

Reviewed By: mitrandir77

Differential Revision: D17529050

fbshipit-source-id: 331c68f64a5ad71d6d77363a9dd1aed6e569efa5
2019-09-26 14:20:22 -07:00
Aleksei Kulikov
a829fee85c snapshot: show snapshots in smartlog
Summary:
Snapshots can be marked with letter `s`:
```
  s    changeset:   8:510948b1f172
  |\   parent:      7:fdf2c0326bba
  | |  parent:      6:55f800d93e34
  | |  user:        test
  | |  date:        Thu Jan 01 00:00:00 1970 +0000
  | |  summary:     snapshot
  | |
  | @  changeset:   7:fdf2c0326bba
  | |  parent:      2:fa948fa73a59
  | |  user:        test
  | |  date:        Thu Jan 01 00:00:00 1970 +0000
  | |  summary:     merge #2
  | |
  @ |  changeset:   6:55f800d93e34
  | |  user:        test
  | |  date:        Thu Jan 01 00:00:00 1970 +0000
  | |  summary:     merge #1
  | |
  o |  changeset:   5:afed36f05817
  | |  user:        test
  | |  date:        Thu Jan 01 00:00:00 1970 +0000
  | |  summary:     draft2
```

Reviewed By: mitrandir77

Differential Revision: D17533027

fbshipit-source-id: da22fef6c23933e2c91d6f5b4ff6d4adb2714e31
2019-09-26 14:20:22 -07:00
Aleksei Kulikov
08cdcd0ea2 snapshot: store a list of snapshots instead of set
Summary:
Storing nodes in a `set` does not make much sense because of possible aliases etc.
It is possible to remove duplicates via `util.removeduplicates` function.
List, on the other hand, preserves order and there will be less conversions during sync (later in the stack).

Differential Revision: D17397594

fbshipit-source-id: ba76e0a1f53516544f055355356d0beaf1c8bfa8
2019-09-26 04:50:18 -07:00
Aleksei Kulikov
83708239f0 snapshot: store hex nodes of the snapshots
Differential Revision: D17397473

fbshipit-source-id: 36e956fa3031bbc2eb92a75a0c3a18d312df5ef6
2019-09-26 04:50:17 -07:00
Aleksei Kulikov
14d3872615 snapshot: make the snapshots invisible after pull
Summary:
Right now `hg pull -r REV` marks all pulled revs as visible.
This makes sense for regular commits (if they exist in the cloud, then they are not hidden).
For snapshots, it is wrong, thus we need not to mark snapshots as visible when pulling.

Differential Revision: D17289438

fbshipit-source-id: d16237b9e471f49bb44317104bc445472bedc16c
2019-09-26 04:50:17 -07:00
Aleksei Kulikov
fe325e6d93 snapshot: make snapshotlist cacheable in localrepo
Summary: It will be needed to sync snapshots while bypassing the `sync` transaction.

Reviewed By: markbt

Differential Revision: D17571650

fbshipit-source-id: ff6c0382a1b1e763c3e37610d188f5bbfb382510
2019-09-25 08:29:22 -07:00
Aleksei Kulikov
012583d390 snapshot: backup snapshot commits without --hidden flag
Summary:
Snapshots are essentially hidden commits that can be backed up/synced via infinitepush.
When we back up the repo state, we want it to access snapshots without explicitly specifying that they are hidden.

Differential Revision: D17260966

fbshipit-source-id: aa7f24043dddb052cab89c1ba939440887e99339
2019-09-23 04:58:11 -07:00
Aleksei Kulikov
1ff54e0b70 snapshot: encode snapshot bundlepart with CBOR
Summary:
We decided not to re-serialize the existing metadata files.
Even though CBOR is good, multiple cycles of serialization can lead to ambiguous encoding.
Thus, the bundle part would look like that:
```
{
    "metadatafiles": {
        <metadata oid>: {
            "content": <binary CBOR-serialized metadata content>,
        },
        . . .
    },
    "auxfiles": {
        <file oid>: {
            "content": <binary file content>,
        },
        . . .
    }
}
```

Reviewed By: quark-zju

Differential Revision: D17466032

fbshipit-source-id: 6d81aacbe76e2037b4b9acae760b759f1c31ecac
2019-09-21 14:17:06 -07:00
Aleksei Kulikov
ff51ebebf6 snapshot: use cbor to store metadata locally
Summary: This is better that `json` because it will allow to store any binary data inside the metadata (e.g. file content).

Reviewed By: quark-zju

Differential Revision: D17464502

fbshipit-source-id: 3d559f390cc7827300fc51c23f004f1d833f0ab4
2019-09-21 14:17:06 -07:00
Aleksei Kulikov
3c5be085f0 snapshot: remove the debug commands
Summary: They became redundant and impose restrictions on the code.

Reviewed By: quark-zju

Differential Revision: D17463968

fbshipit-source-id: 83cb8d9dca037747e62f9e8c41a4081259c92e91
2019-09-21 14:17:05 -07:00
Aleksei Kulikov
23d65a01d5 snapshot: refactor the metadata class
Summary: Do not keep a reference to the repo and provide a better API.

Reviewed By: quark-zju

Differential Revision: D17463871

fbshipit-source-id: d54256208b5f571ca8a4f4e2d07353edd9e825b2
2019-09-21 14:17:05 -07:00
Aleksei Kulikov
b94ce91b33 snapshot: fix coding utf-8 comments
Reviewed By: markbt

Differential Revision: D17228034

fbshipit-source-id: b3fb280bcaede2c42029b38c8e145e12695741c8
2019-09-11 06:38:41 -07:00
Aleksei Kulikov
a0e092de37 snapshot: add a command to list the local snapshots
Reviewed By: markbt

Differential Revision: D17227370

fbshipit-source-id: 363f45431824b574cc298eb5c0f40c334f1d916b
2019-09-11 06:38:41 -07:00
Aleksei Kulikov
b7ee506bfa snapshot: keep a list of snapshots in .hg/store/snapshotlist
Reviewed By: markbt

Differential Revision: D17226709

fbshipit-source-id: fefa31227f77295793ccaa3448f4488274f70848
2019-09-09 10:38:22 -07:00
Aleksei Kulikov
377e1200a7 snapshot: add -m option to set up a custom snapshot message
Reviewed By: markbt

Differential Revision: D17206496

fbshipit-source-id: a6e190ca87aa5309e359a5cecd6c6bc40634ca8d
2019-09-06 08:18:36 -07:00
Aleksei Kulikov
ccc05d0d65 snapshot: remove lfs dependency completely
Summary: Also remove debug commands that are strictly related to remote lfs.

Reviewed By: markbt

Differential Revision: D17184902

fbshipit-source-id: da38a2150212500bab62191ddcfab0990276605e
2019-09-06 08:18:35 -07:00
Aleksei Kulikov
22dce8230d snapshot: bundle metadata and the related files
Summary:
Instead of the lfs remote storage it was chosen to send the snapshot metadata via bundles.
Snapshot metadata consists of: the actual metadata blob + several other blobs (untracked files etc).
If we have several snapshot revisions in a single bundle, the blobs could repeat.
Then we should store each blob as a separate entry in a binary stream, keeping its id and contents.

Here we introduce a new bundle part type `"b2x:snapshotmetadataparttype"`.
```
1 byte of version info
[ # a list of binary entries, each corresponds to a separate file
  # (either a metadata file itself or a related -- externally stored -- file)
  <oid><length><data>
  :oid: is a 64char string with the hash of the file
  :length: is an unsigned int with length of the data
  :data: is binary data of length <length>, the actual file contents
]
```

So far there is an ongoing discussion on the exact format of serialization.
Actual state is at [the quip doc](https://fb.quip.com/R5OVAzabX8oo).

Reviewed By: markbt

Differential Revision: D17184222

fbshipit-source-id: 90f833ec71556e90d513e3be3f3efa7f870b037d
2019-09-06 08:18:35 -07:00
Aleksei Kulikov
bcd08fcda3 snapshot: store metadata version as int
Summary: That way it'll be easier to pack it into a blob.

Reviewed By: markbt

Differential Revision: D17183018

fbshipit-source-id: 44e21103f201aafb6f417a5b5a7b3d4735f32039
2019-09-06 08:18:35 -07:00
Aleksei Kulikov
732a5baa9a snapshot: use local snapshot storage instead of lfs
Summary: In the next diff I will replace remote lfs with bundle2

Reviewed By: markbt

Differential Revision: D17132405

fbshipit-source-id: a0dfff3ebad067abb0231cf31de08ae62affe7ce
2019-09-06 08:18:35 -07:00
Aleksei Kulikov
1cc68aac23 snapshot: split the extension code into files
Summary: Preparation for lfs deprecation in snapshot extension.

Reviewed By: quark-zju

Differential Revision: D17112040

fbshipit-source-id: aa55a631d1f03180ffea741c71bcf801fb4fc147
2019-09-04 11:09:36 -07:00