Summary:
An alternative to D29363808 (e396cab669). The benefit is that parents_and_head is used by
both the client and the server. So we don't need to duplicate D29363808 (e396cab669) in
Mononoke code.
Reviewed By: andll
Differential Revision: D29365079
fbshipit-source-id: bca60ba2b3df477929d8e72b2363e5a0f744b35d
Summary:
Pull fast path uses `reload` which drops pending changes.
To avoid misuse, raise an error if pending changes are present.
Reviewed By: andll
Differential Revision: D29363799
fbshipit-source-id: 8f520d2c5553432abc452bc7b2b59d7af80e0a99
Summary:
The import pull data logic used low-level locking, persisting APIs, it does not
write cached idmap to disk. So we need to manually insert the idmap remote
lookup result to the actual local idmap explicitly.
This addressed an issue that verify_missing fails in the pull fast path.
Reviewed By: andll
Differential Revision: D29363813
fbshipit-source-id: 2749855a6c8c924bd1b772173de066d400f73764
Summary:
For a NameDag, `IdConvert` on `self.map` cannot resolve names remotely, but
`IdConvert` on `self` can. Use the latter. This is similar to D27547584 (af3c3b3fd0) where
some `self.map` are updated to `self`.
This addressed an issue found in the pull fast path test. Note there is another
issues to solve.
Reviewed By: andll
Differential Revision: D29363810
fbshipit-source-id: 28ba583ed14bbc5d52af81d4128d965f24eef011
Summary: The test pulls when the client has a lazy graph, and the server has a few merges.
Reviewed By: andll
Differential Revision: D29363806
fbshipit-source-id: 09bc3c4c3d21924f500ca86e8d86f58a15159169
Summary:
`fmt::Debug` for a NameDag is too verbose. Separate part of it so we can debug
print segments for a given (group, level). This will be used by upcoming
changes.
Reviewed By: andll
Differential Revision: D29363805
fbshipit-source-id: e1c6713be10b8b64fc7a42178117e724e0d691d0
Summary:
The client TestDag might have outdated server Dag as the remote protocol,
because it is a static "snapshot". Ensure the remote Dag is updated when
using the pull API.
This is an issue solved by tracking down issues in tests added in upcoming
diffs.
Reviewed By: andll
Differential Revision: D29363807
fbshipit-source-id: a560b2e91999873338604907a6d83cc7d2ff5c58
Summary: It will be used by the next change.
Reviewed By: andll
Differential Revision: D29363802
fbshipit-source-id: 842735ac05ea5fea4ea0c3625a68d06d27bc37d5
Summary:
It is useful when drawdag itself triggers remote fetches.
This was used but is not used after some refactoring. I think it might be useful
in the future so kept it.
Reviewed By: andll
Differential Revision: D29363803
fbshipit-source-id: fa178ac9783d1dc1b73525eeb8cd3d766cf46a0f
Summary: The test will be used to verify upcoming changes.
Reviewed By: andll
Differential Revision: D29363809
fbshipit-source-id: d34d13123914cfabb5c82dee3873b6e0c4979ee2
Summary: Make it easier to write more tests around pull.
Reviewed By: andll
Differential Revision: D29363804
fbshipit-source-id: 5b2cf8675343898fabc1d8845228e240e463edf8
Summary:
The roots data will be useful for the client to check if the pulled commits are
going to overlap with its existed DAG.
Reviewed By: andll
Differential Revision: D29363808
fbshipit-source-id: e09d924d65537f59fd4ea209b568265d07a80e46
Summary: Minor change to make the code a little bit more straightforward.
Reviewed By: andll
Differential Revision: D29363801
fbshipit-source-id: 2c4bd6ece07282f044622227a3c077cb31db6d17
Summary: Make the docstring a bit more consistent.
Reviewed By: andll
Differential Revision: D29363798
fbshipit-source-id: 1b4e2a7a1af4c4cffe3693e437a831bab1b43fd7
Summary: This is required to make sure segmented changelog has all the data needed
Reviewed By: quark-zju
Differential Revision: D29347285
fbshipit-source-id: 82ee1ffca178492b7ad363c53cee7ec57058733f
Summary:
Add git LFS support to gitimport and grepo branch_forest.
I did not want to add the parsing of .gitattributes and .lfsconfig to the gitimport library. This needs to be done by the users of gitimport before the import is started, And the GitImportLfs object needs to be configured accordingly. Currently we are extrating this data from the manifest files for the "g"repo imports.
I am not sure the simple git-lfs download client works with other git-lfs server back ends then Dewey. But it is a fairly simple implementation and it should be easy to extend to be more generic.
Reviewed By: farnz
Differential Revision: D29082867
fbshipit-source-id: a7b0272147b3d44a0b6b9782d2a1b8ec94653b8f
Summary:
Sometimes we need repo internal states for deeper investigation. Right now
the process is quite adhoc - telling people to figure out where the "backing"
repo is, excluding unwanted files like 00changelog.d, etc. Let's just make
it easier so we have a dedicated command to tar up selected internal states.
Reviewed By: andll
Differential Revision: D29242716
fbshipit-source-id: 85604d0df5b3a12f1aed29e4f31fcd5c166f9066
Summary: It's useful to be able to copy multiple dirs at once
Reviewed By: markbt
Differential Revision: D29358375
fbshipit-source-id: f1cc351195cc2c19de36a1b6936b598e314848c3
Summary:
Previously only conversion between bonsai and hg was supported. Let's add git
as well.
Obviously you can use `scsc lookup`, but mononoke_admin can be useful for repos
that are not on scs yet.
Reviewed By: farnz
Differential Revision: D29360793
fbshipit-source-id: eb2b71eab192b3456ba3d580f7eb8c4a85b2fd1d
Summary: Very simple refactor. This logic was already used twice and I will use it another time in following diffs.
Reviewed By: markbt
Differential Revision: D29033594
fbshipit-source-id: 96040a2eee2b58f6851646e51b67c46c6bf334fe
Summary:
Implement get and put for the ephemeral blobstore. This allows blobs to
be stored and retrieved in bubbles.
Ephemeral bubbles always have a repo associated with them when they are opened,
to simplify blob prefixing. It is valid for a bubble id to have multiple repos
associated with it, but they must be accessed separately, and in practice this
won't be used.
Reviewed By: StanislavGlebik
Differential Revision: D29067722
fbshipit-source-id: d870f695fc1d0c825fdaec9337c82a13209165ce
Summary:
Extend metaconfig to include configuration for the ephemeral blobstore.
An ephemeral blobstore is optional: repos without an ephemeral blobstore cannot
store ephemeral commits or snapshots.
Reviewed By: StanislavGlebik
Differential Revision: D29067719
fbshipit-source-id: fe7d42173d5c34a937c99c72f4b2bd08af503889
Summary:
Packblob currently expects key prefixes of the form `repoNNNN.` to be stripped , but also allows keys without this prefix. For the ephemeral blobstore we want to allow prefixes of the form `ephXXX.repoNNNN.` as well.
Generalise packblob so that we can have multiple key prefixes.
Packblob will enforce that none of the blobs in the packblob have a prefix that matches any of the patterns - this will prevent us from accidentally storing `repoNNNN.`-prefixed blobs in an ephemeral blobstore that requires `ephXXX.repoNNNN.` prefixes, for example.
Reviewed By: liubov-dmitrieva
Differential Revision: D29067720
fbshipit-source-id: 953909d47c9c4af91b529bcc684340d26411463d
Summary: Make it clearer which of the TailParams are only required when chunking, removing parallel Option<> so that all items that should be set together are inside one optional item.
Reviewed By: farnz
Differential Revision: D29264647
fbshipit-source-id: d64cddf94b35e62d6e50cd8afe906eef2444c730
Summary: Makes defer_visit return result, so we can detect if it is called when not chunking.
Reviewed By: farnz
Differential Revision: D29268346
fbshipit-source-id: b8ea503c2848adb5d7ca3fb0e61399be2930c3de
Summary:
It looks like 20s might not be sufficient anymore as the test is failing too
frequently with `wait` completing without the process having completed:
```
eden/fs/service/test/StartupLoggerTest.cpp:352: Failure
Expected equality of these values:
"exited with status 0"
returnCode.str()
Which is: "running"
eden/fs/service/test/StartupLoggerTest.cpp:363: Failure
Value of: isWritablePipeBroken(fd)
Actual: false
Expected: true
```
Reviewed By: kmancini
Differential Revision: D29342002
fbshipit-source-id: d457b632a2666356e9053d2da66a3227e6524b68
Summary: Verify we actually have pending keys to fetch before attempting a remote request in scmstore TreeStore.
Reviewed By: kulshrax
Differential Revision: D29345214
fbshipit-source-id: 328bdcbc41429e59de6ceb488533bafa97518fcc
Summary: Previously, it was not possible to interrupt `hg` during EdenAPI fetch operations. This made it impossible to interrupt long-running fetches, which is very frustating to users. This can be simply fixed by using `block_unless_interrupted` in place of `block_on`.
Reviewed By: quark-zju
Differential Revision: D29344670
fbshipit-source-id: 3b0d36dda28f5f7cc812a07981f295f8d0fbdd8a
Summary:
This is simple command mostly to be used by testing before we fully integrate with hg pull
This command does not perform discovery and requires from/to revision to be passed in cmd line
Reviewed By: quark-zju
Differential Revision: D29315647
fbshipit-source-id: 26d67031e566b7c99af1e2a5ab287f02b52f7db0
Summary: This is breaking the Windows release, reverting.
Reviewed By: fanzeyi
Differential Revision: D29339787
fbshipit-source-id: 22d8ff5db5619194e4597754dc37343cf0bc3286
Summary: `|&` apparently fails on mactest, so I've replaced it with `2>/dev/stdout | ` which works on my devserver and macbook.
Reviewed By: andll
Differential Revision: D29337621
fbshipit-source-id: eaac2592f4c7bfda6696c2500f3b08441b596c39
Summary:
Introduce basic contentstore fallback tracking to help monitor the scmstore shim rollout.
This will be expanded to a general fetch metrics system for scmstore in a future change.
Reviewed By: kulshrax
Differential Revision: D29305839
fbshipit-source-id: c6cc3ea15a3bb7b90f4ec298febc911ec4e2af91
Summary: This is rougly similar to algorithm in NameDag
Reviewed By: quark-zju
Differential Revision: D29318721
fbshipit-source-id: 51a9123daa2b4cf0fbe2346a8a0c7e75172d9afb
Summary: The naming is used in other parts of dag crate - this introduce mononoke side binding for corresponding functions on dag side
Reviewed By: quark-zju
Differential Revision: D29318722
fbshipit-source-id: e9eea5536b041b6ab2ce578914817bca43a10d48
Summary:
In D29079762 (ea2e2f8bbd), globbing was fixed to not match the recursive glob (**) against
the entire path, as this would lead some paths to be matched while they
shouldn't. It however introduced another bug: in some cases, recursive globs
would no longer match paths that should be matched.
To fix both, a partial revert of the original diff is done with a small tweak:
the path that is matched against no longer starts at the root of the
repository. This will prevent `a/b/**/b/c.txt` to match `a/b/c.txt` as
`**/b/c.txt` would only be matched against `c.txt`, and not `a/b/c.txt` like it
was previously.
Reviewed By: fanzeyi
Differential Revision: D29175333
fbshipit-source-id: 1a4137d6f64f6cb77c4be09bd143f72630aa58d5
Summary:
Bring Watchman closer to the EdenFS file name convention by moving
source files and includes into watchman/.
Reviewed By: fanzeyi
Differential Revision: D29242789
fbshipit-source-id: 6e29a4a50e7202dbf6b603ccc7e4c8184afeb115
Summary:
When buck kill fails, eden rm will also fail. This has caused some checkouts
to not be removed when they could be. Stopping aux processes is a nice thing
to do before we unmount. It ensures these processes close file handles in the
mount, but we force unmount anyways so open file handles should not be able to
block the umount call.
Reviewed By: xavierd
Differential Revision: D29205962
fbshipit-source-id: a899940efa5cc1d960cd14a775b7053c34f5d6f2
