Summary: There's no need to create an empty commit.
Reviewed By: krallin
Differential Revision: D24537264
fbshipit-source-id: 41a80fcbf5cab28b731dae53c58653ec4b434716
Summary: To make it clearer what's going on
Reviewed By: krallin
Differential Revision: D24534961
fbshipit-source-id: 049edbf1b6b648670a6c6ef68b14698a27ea1a0e
Summary:
There are two new options (and one option was renamed):
1) First option (--file-size-limit) adds a way to limit total size of moved files
2) Second option (--lfs-threshold) adds a way to exclude size of the files that
are above the threshold from --file-size-limit calculations. This option can be
useful if we want to limit the total size of the bundle that will be created in
mononoke hg sync job - files that are above or equal to lfs threshold are not included
in the bundle, so no need to count their size.
Reviewed By: krallin
Differential Revision: D24534959
fbshipit-source-id: 6323e4d52802c00dc67270b08f671b0789ab262f
Summary:
a simple command that can be used to pushrebase a commit. Note that this
command is very low level so it skips a lot of checks that we do during the
normal push process (e.g. it doesn't run hooks).
Reviewed By: krallin
Differential Revision: D24534960
fbshipit-source-id: 0f1e27005fa450c86c310d66e3215747ca6a49e2
Summary: This change adds wire types for the history API in the most straightforward way possible. In a future change, I'll move the `WireHistoryEntry` / `HistoryResponseChunk` conversion logic into the `ToApi` implementation. This implementation also doesn't add per-item errors, or standardize `history` to match the protocol evolution standards used by `trees`.
Reviewed By: kulshrax
Differential Revision: D24342046
fbshipit-source-id: d46403a823f2a1e89ad9d6d2074241d8bfe4810e
Summary: This change introduces custo `Serialize` and `Deserialize` implementations for wire types containing a fixed-length byte array. This change is more verbose than should be necessary. Because `TryFrom` is implemented to convert `&[T]` to `[T; N]` and `AsRef` to convert `[T; N]` to `&[T]`, we should be able to use a generic serde `with = ` attribute on the byte array fields of the appropriate structs. Unfortunately, I'm getting a lifetime issue that I haven't been able to resolve, and const generics aren't available on stable to implement the trait bounds without an explicit lifetime.
Reviewed By: kulshrax
Differential Revision: D24460527
fbshipit-source-id: d3a4179b833a523ce164d3df934e4cbdc2202546
Summary:
Removes top-level metadata from `TreeEntry` and add a new, specialized type for carrying child entry metadata, `TreeChildEntry`. This change also fully removes the `revisionstore_types::Metadata` from EdenAPI trees, which is only used on files.
In a follow up change I'll optimize handlings of paths / path segments in `TreeChildEntry` keys. Right now they need to be joined manually.
Reviewed By: kulshrax
Differential Revision: D24434716
fbshipit-source-id: d0739471b1f6cef58b435e10b5fb774bfb08f7f6
Summary: Rather than silently dropping entries which cannot be fetched, this change has the `WireTreeEntry` type carry optional error information, allowing it to be (de)serialized to / from `Result<TreeEntry, EdenApiServerError>` instead of a bare `TreeEntry`. Currently, handling of these failures is up to the individual application code, but it might be useful to introduce utility functions to drop failed entries and log errors.
Reviewed By: kulshrax
Differential Revision: D24315399
fbshipit-source-id: 94e4593b77cf2dc12d0dcc93d174c8a4eda95344
Summary:
`message` is not longer an attribute of an exception object in Python 3. Avoid
using it. This avoids turning rendering `{mirrornode}` template into a crash like:
Traceback (most recent call last):
File "edenscm/hgext/fbscmquery.py", line 78, in mirrornode
return client.getmirroredrev(reponame, "hg", torepo, totype, node)
File "edenscm/hgext/extlib/phabricator/graphql.py", line 399, in getmirroredrev
self._raise_errors(ret)
File "edenscm/hgext/extlib/phabricator/graphql.py", line 504, in _raise_errors
raise ClientError(None, errormsg)
edenscm.hgext.extlib.phabricator.graphql.ClientError: The provided crypto auth token(s) are expired. Check to make sure you have specified a long enough expiration time for your crypto auth token(s).
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "edenscm/mercurial/commands/__init__.py", line 4225, in log
displayer.show(
....
File "edenscm/mercurial/templatefilters.py", line 422, in byteify
return b"".join([byteify(t) for t in thing if t is not None])
....
File "edenscm/mercurial/templater.py", line 391, in evalstring
return stringify(evalrawexp(context, mapping, arg))
File "edenscm/mercurial/templater.py", line 350, in evalrawexp
return func(context, mapping, data)
File "edenscm/hgext/fbscmquery.py", line 83, in mirrornode
mapping["repo"].ui.warn(_x(str(e.message) + "\n"))
AttributeError: 'ClientError' object has no attribute 'message'
Reported by: arnabde03
Reviewed By: mofarrell
Differential Revision: D24528362
fbshipit-source-id: 57d093811986f3a65d1201de1f9aed3770db2ad1
Summary: In D24447404, I provided some utility functions that allowed me to avoid constructing and/or passing around a ConfigStore. Remove those functions and fix up the code to run.
Reviewed By: krallin
Differential Revision: D24502692
fbshipit-source-id: 742dbc54fbcf735895d6829745b9317af14dfa0b
Summary: It's designed as a singleton store for normal use - rather than have lots of ways to get different config stores, let's use one global store
Reviewed By: krallin
Differential Revision: D24447404
fbshipit-source-id: 6ecc46351b14794471f654cec98527a11a93d3ef
Summary:
Sometimes you only want to trace writes into the EdenFS checkout, so
add the ability to run `eden strace` with `--reads`, `--writes`, or
both in order to see only those events.
Reviewed By: wez
Differential Revision: D24468539
fbshipit-source-id: a1b3c730987cf86ce3d39952c6a5e9c5edaddfc2
Summary:
Before replacing the existing debugInodeStatus changes, make some
small refactorings to help me understand it.
Reviewed By: xavierd
Differential Revision: D24299970
fbshipit-source-id: 3b60aa8365bb9a45e12357374282ba837ccc36eb
Summary:
Add a function that makes it easy to traverse EdenFS's view of the
traversed-so-far filesystem.
Reviewed By: xavierd
Differential Revision: D24299962
fbshipit-source-id: 275c849846bf45a2e80780411d60266b961d825b
Summary:
Adding number of changed files to mononoke_commit and mononoke_commit_draft scribe categories.
I need to create scuba table for mononoke_commit scribe category.
Reviewed By: krallin
Differential Revision: D24390738
fbshipit-source-id: 80b9917751a42377b49190cbb2f650a21b086e74
Summary:
- convert ManifestOps to new style futures
- at this point `//eden/manifest:manifest` crate is completely free from old style futures
Reviewed By: krallin
Differential Revision: D24502214
fbshipit-source-id: f1cdb11bd8234f22af5c905243f71e1e9fca11f1
Summary: `derive_manifest_*` functions are now based on new futures and use new bounded_traversal
Reviewed By: krallin
Differential Revision: D24448824
fbshipit-source-id: 5af06851f9c48a73e53daafe1f4972c186a5ca80
Summary: Somewhat more convenient to work with, no need to look inside.
Reviewed By: StanislavGlebik
Differential Revision: D24474898
fbshipit-source-id: 7ee0920e7d0d5a2102c68695a5cc0d9e237d958d
Summary:
Users get confused and try to pull from the wrong repo - e.g. `hg checkout D23665606` in `fbsource` - and find the resulting output about missing hashes scary.
Give them a nice clear error instead.
Reviewed By: wez
Differential Revision: D24485097
fbshipit-source-id: e9aeb7ed6aaf494f828d736ca5203100807fb321
Summary:
indexedlogdatastore is supposed to use remotefilelog.cachelimit to set
the max size, but instead it was setting the max per-log size, which means the
max size was N times bigger. Let's fix that.
Reviewed By: xavierd
Differential Revision: D24483181
fbshipit-source-id: f33cedbfdbb318e9d5eb9fda497645050b93e9fe
Summary:
This updates Thrift codegen for Rust to call hooks on a context stack in a
similar fashion as the C++ codegen.
There is still a bit of room for improvement here. Notably, we don't know how
much data we actually read or sent to the client, so for now those are
hard-coded to zero in the codegen. That's better than not calling those hooks
at all (which is what happens right now), but it could stand to be improved.
Reviewed By: jsgf
Differential Revision: D24445298
fbshipit-source-id: 470daf03057424dc300b6a193668be835ae28452
Summary:
In the next diff in this stack, I'm changing the Rust thrift library as well as
the codegen, and this is causing quite a bit of pain with regard to this
codegen that is checked in to the hg build:
- Regenerating the codegen isn't super obvious.
- The Sandcastle build fails because it uses the current codegen with the old
library.
According to lukaspiatkowski, this has also been a problem in the past when Eden got
migrated to Tokio 0.2, so I'd like to save myself and others some pain by just
not generating the server side codegen and not checking it in, since it's
unused. This reduces the surface of stuff that might go out of sync.
#forcetdhashing
Reviewed By: markbt
Differential Revision: D23649604
fbshipit-source-id: d684bec427431a366de42c88e53072caa98d5b2f
Summary:
Now that Linux/macOS startup no longer uses fork it becomes trivial to share
the same code with Windows. This improves Windows startup in several different ways:
- `edenfsctl start` now displays the status of the start process in the console
- `edenfsctl start` no longer has an arbitrary timeout after which it reports
a timeout even though EdenFS is still starting up.
This also kills a bunch of Windows specific code that is no longer needed.
Reviewed By: fanzeyi
Differential Revision: D24393690
fbshipit-source-id: 28100aec96da81c92d5b592353edceed332e2364
Summary:
Previously, when that code was ported on Windows, paths separator were
converted from '\' to '/' when a wide string was provided, all the other paths
were treated as is.
The main issue with this strategy is that not all paths can be converted, the
non-stored ones for instance are immutable, which leads to some subtle bugs
down the line. For instance, the paths: "Z:/foo/bar/baz" and "Z:\foo/bar\baz"
would not be equal as the path separator isn't the same, but both of these are
actually the same path underneath.
To solve this, this diff first introduce a Windows path separator, and then
modifies the path comparison functions to ignore the path separator and only
compare the components.
I'm definitively not a fan of the pattern I use for searching for both / and \
in paths, suggestions are welcome for how to improve that.
Reviewed By: chadaustin
Differential Revision: D24376980
fbshipit-source-id: 0702bf775c7c3937b2138abd5a63d339ac80aaed
Summary:
The path iterator functions are skipping sanity checking of paths as they
assume that the path has already been validated. Unfortunately, that isn't the
case as the only sanity checking we are doing is on the beginning and end of
the string that is passed in. Notably, `RelativePathPiece{"./foo"}` would not
fail the sanity checking even though it it's not made of valid components.
Reviewed By: chadaustin
Differential Revision: D24439804
fbshipit-source-id: 852b3a1180b185cb0bfb96bf5bcdc98b231f32c5
Summary:
As the string is created with MAX_PATH wide characters, a lot of them are going
to stay nul. A future diff adds a sanity check to the PathComponent that
validates that it doesn't contain any nul bytes, which chokes on these paths.
Reviewed By: chadaustin
Differential Revision: D24479715
fbshipit-source-id: e51c4b1c53a3f375664c0c0a1e325ebe302b0cd3
Summary:
When the new rust store is enabled, it can throw exceptions if the rust
code tries to fetch multiple trees when the server doesn't support bfs fetching.
The python code path ignores these requests, letting the eventual single tree
lookup path (i.e. not the prefetch path) perform the download. Let's let the
rust code do the same for rust stores.
Eventually every repo will support bfs fetching and we can delete this code path
entirely.
Reviewed By: quark-zju
Differential Revision: D24459106
fbshipit-source-id: d9a931364bd3235113b0dbcf09cba35a16e16e8e
Summary: Comment out the tests that do not compile on Windows.
Reviewed By: fanzeyi
Differential Revision: D24458438
fbshipit-source-id: 14bd4803ccd68ea6e02f95f868b8a801161518ea
Summary: let's make sure we don't create too big commits by setting the limit
Reviewed By: ikostia
Differential Revision: D24451569
fbshipit-source-id: 7b4f653509fce47d921e55dddad2de49ac5672a3
Summary:
This is a command that creates a commit that copies one directory into another.
Later you can take a commit and push it.
At the moment the command is very simple and also not very useful. At the very
least we need a way to limit the size of the commit, and this option is coming
in the next diff.
Note that the command doesn't overwrite the files if they exist in the target
directory - we may add an option to do so in future. Also if a file exists in
target directory but doesn't exist in source directory then it's left
untouched.
Reviewed By: ikostia
Differential Revision: D24451568
fbshipit-source-id: 2c7b945aadc99a1ca960349992c3684ef02ba35e
Summary: This merely filters out the files that don't compile on mode/win.
Reviewed By: chadaustin
Differential Revision: D24456917
fbshipit-source-id: d314ca2936e064133f2fc6a6199b38d17f2d9ef7
Summary:
FUSE_ACCESS was incorrectly categorized as an 'other' FUSE request
type. Categorize it as 'read'.
Reviewed By: wez
Differential Revision: D24437458
fbshipit-source-id: 5ea0a7d5a0fd57b9230f37674fd2568a4917bab5
Summary:
I noticed while playing with `eden strace` that, on my Linux machine,
running `ls -lA` would cause a pile of FUSE_GETXATTR requests for
`system.posix_acl_access` and `system.posix_acl_default`.
FUSE does not cache getxattr yet, but the kernel can cache ACLs if we
pretend to support them. Thus, advertise FUSE_POSIX_ACL support, but
continue returning ENOSYS for all setxattr calls.
Reviewed By: wez
Differential Revision: D24039130
fbshipit-source-id: a2e69c43b728fb51fb1d1b41ca9d31eba8efaa19
Summary:
Add an `eden strace` command that subscribes to all FUSE events for
the specified mount (or current directory), and streams them to stdout
with response latency measurements and the requesting process.
If there are any pending requests at the time `eden strace` is run,
they are printed first.
Reviewed By: wez
Differential Revision: D24038978
fbshipit-source-id: 59a2112dbdb1708571d12b04bdccaf9eca725cf7
Summary:
This is to be able to run the hook tailer on fbsource commits, which had been
push-redirected from ovrsource.
Reviewed By: StanislavGlebik
Differential Revision: D24447814
fbshipit-source-id: bfb164ec45dd2d4d5c3c83f133b48000a71a50a3
Summary: As well as the unixname of the pusher, log all identities provided by the pusher.
Reviewed By: krallin
Differential Revision: D24446453
fbshipit-source-id: 0854b3a10221da16d87b53fd849f033f7bb2a27e
Summary:
Use the new API `smartset.prefetch` for the prefetching logic for `log`
commands.
Prefetching is only enabled if the template contains related symbols.
Reviewed By: DurhamG
Differential Revision: D24289052
fbshipit-source-id: 810da3b249d45a7516aaf39a0b74f6965b87f96f
Summary: This makes `log` command more efficient using the `hybrid` backend.
Reviewed By: DurhamG
Differential Revision: D24293498
fbshipit-source-id: db3bac28a0b04ba6af449357bc08b285cd62780d
Summary:
Add prefetch APIs on smartset to make it easier to specify what need to be
prefetched.
This is intended to be used by `{phabstatus}`, log with lazy commit text backend,
and revsets like `author(x)`, etc.
Reviewed By: sfilipco
Differential Revision: D24321292
fbshipit-source-id: 7e645ccbe39feeae1e2da7bfb115623945bf9cef
Summary:
This makes it possible to "fork" sets and set different prefetch properties on
them.
Reviewed By: sfilipco
Differential Revision: D24365329
fbshipit-source-id: c4bdd6ab21e6a891a02d450f9e6edab3a3781eca
Summary:
The generatorset has a pure Python implementation for rewindable generator.
However it is not thread-safe, and we want thread-safety since the set
iteration will be driven by async Rust in multiple threads.
One of the issues is, during thread 1 `next(gen)`, thread 2 might call
`next(gen)`, and that's not allowed by Python.
Fix them by switching to the Rust RGenerator.
Reviewed By: DurhamG
Differential Revision: D24365328
fbshipit-source-id: 2785e80c7c460a7f754ed23e3af99f4a5c9fbcdf
Summary:
Buck-out can contain some files that we wont have the ability to read on mac os
(like .Spotlight-V100 and .Trashes). This means `eden du` will error out on
reading them.
We handle FileNotFound but not permission errors, lets treat them the same.
Reviewed By: genevievehelsel
Differential Revision: D24396252
fbshipit-source-id: 31e2ecfeff1783ae9c3479f59335d95a0d9b6de7
Summary:
Similarly to the indexedlogdatastore, the LFS stores can become corrupted on
power loss. This is happening fairly frequently on Windows Sandcastle due to
the OS being virtualized and power being cut abruptly.
For now this only attempts to repair the shared stores, in theory we could also
try to repair the local stores but haven't looked into it.
Reviewed By: DurhamG
Differential Revision: D24449202
fbshipit-source-id: 605a7943a0850b625bf00c514879b3da1ab2b406
Summary:
The ContentStore/Metadatastore are made of several different stores, attempting
to expose all of them to Python to drive the repair logic from there would leak
implementation detail of how the stores are implemented.
Instead, let's simply expose a single `repair` function out of the
pyrevisionstore crate that takes care of repairing all of the underlying
stores. For now, this is just moving code around, but a future diff will
integrate the LFS stores.
Reviewed By: DurhamG
Differential Revision: D24449203
fbshipit-source-id: 1631ced9068716453cb404bf7e65cefbf2db5247
