Summary:
Migrating buck integration to use configurations, which is more performant and is the primary supported way to run Pyre.
Mypy is no longer supported at Facebook, so mypy buck integration is covered in this migration as well.
Configuration target automatically expanded to include all subtargets, expanding type coverage while introducing no more than 2 fixmes per file.
Reviewed By: dkgi
Differential Revision: D21474723
fbshipit-source-id: 2501af8250b4998fe331350b80c9dc8b4ba2dd1a
Summary: If no LFS blobs needs uploading, then don't try to connect to the LFS server in the first place.
Reviewed By: DurhamG
Differential Revision: D21478243
fbshipit-source-id: 81fa960d899b14f47aadf2fc90485747889041e1
Summary:
Sometimes, in shell one-liners, it's convenient to insert the current
eden pid. We print the pid as part of `eden status` (like nginx and
many other daemonizing programs), but that is not useful for shell
substitution. So a pid subcommand that succeeds and prints the pid to
stdout if running and fails if edenfs is not running.
An example use would be `strace -fyp $(eden pid)`
Reviewed By: genevievehelsel
Differential Revision: D21397118
fbshipit-source-id: 06010150557ec882b452180665b84452fa06ebbd
Summary:
Add a simple `/repos` endpoint that returns the list of repos available in a JSON response.
While the handler itself is quite simple, this diff establishes the general pattern by which new handlers will be added to the server.
Reviewed By: krallin
Differential Revision: D21330778
fbshipit-source-id: 77f57c969c34c8c1f7c94979fac383ec442a1e14
Summary:
The information available in `eden top` is growing and as we add more and
more to eden top, we want to make sure that eden top users can understand
what `eden top` is telling them. Particularly if we want eden end users to be
able to debug themselves, providing them enough context on the
metrics eden top provides is important.
The help page explains what data is included, why this data is helpful and
when this data indicates a concern.
Reviewed By: wez
Differential Revision: D20974262
fbshipit-source-id: caaef70e76d283968bee8533c7c34d63a95c5490
Summary:
this is preperation for adding a help page to eden top (D20974262)
This allows adding a scrollable section to `eden top`. This is particuarly needed
so that the help page is readable when it does not fit all in one screen. This
also allows scrolling the process table on the main page to see all the
processes listed (this is added here).
Reviewed By: chadaustin
Differential Revision: D20974245
fbshipit-source-id: 402409efdc451db32e7a267b0750204f2322dc4f
Summary:
this is preparation for adding a help page to eden top (D20974262)
This moves all code for writing to the screen to its own class so that
it is not scattered through out the rest of the eden top logic. This is
primarlity so that it is easier to make changes the screen writing logic to
allow scrolling, which is needed so that the full help screen can be read
when it takes up more space than is avaiable.
Reviewed By: chadaustin
Differential Revision: D20965954
fbshipit-source-id: c2690a293579706605d26be9d3bbe491104e6847
Summary:
When I refactored MPath to limit path length to 255 throughout, I had to change
the logic in this hook because it couldn't represent problematic paths anymore.
Unfortunately, I didn't realize that this would break in cases where the file
fits in 254 or 255 characters with one of the less compact encodings (but that
doesn't fit in 255 if you add `.i`), but also fits in 255 characters once you
add `.i` using one of the more compact encodings.
This results in the hook rejecting things that could have been represented in a
more compact encoding un-necessarily.
This fixes that, but to do so it also requires no requiring MPath in fsencode
(and instead allowing any slice of bytes) which is basically the bulk of
changes here.
Reviewed By: StanislavGlebik
Differential Revision: D21462205
fbshipit-source-id: d4fe6129b379675e842bff5b20bd776cb39157b2
Summary:
This diff logs the delay in deriving data. In particular it logs how much time
has left since an underived commit was created.
Note that this code makes an assumption about monotonic dates - for repos with pushrebase
repos that should be the case.
Reviewed By: krallin
Differential Revision: D21427265
fbshipit-source-id: bfddf594467dfd2424f711f895275fb54a4e1c60
Summary:
Two things will be simplified:
1) Do not pass sqlbookmarks, we can always get them from blobrep
2) Instead of passing repo per derived data type let's just always pass
unredacted repo
Add a very simple unittest
Differential Revision: D21426885
fbshipit-source-id: 712ef23340466438bf34a086517f7ba33d4eabed
Summary: Small refactoring that will make the next diffs easier
Differential Revision: D21426166
fbshipit-source-id: f3c3ae00794046828eaf3c0912dbabc233c97e77
Summary:
The transformation is pretty direct. I didn't add additional functionality
to the IdMap and I did not update the construction algorithm yet. The querying
method on IdMap were updated to async and then there are the SQL interaction
details.
In follow up changes I want to update the construction algorithm and add support
for multiple repositories.
I am not happy with the names of the columns or naming in general in this code.
Open to suggestions. One idea could be matching the client nomenclature as much
as possible.
Reviewed By: StanislavGlebik
Differential Revision: D20929576
fbshipit-source-id: 12104892faa69f37c141e8baf54d5fb24fc5df6b
Summary: This also unblocks the MacOS Mononoke builds, so enabling them back
Reviewed By: farnz
Differential Revision: D21455422
fbshipit-source-id: 4eae10785db5b93b1167f580a1c887ee4c8a96a2
Summary:
Our use of the term "client" to refer to a checkout is
deprecated. Rename some internal functions that use the term client.
Reviewed By: simpkins
Differential Revision: D21395159
fbshipit-source-id: fa96ba593f53b493e5ae816fa686f333a132c232
Summary:
When LFS is enabled, only bundle3 is supported, so we have to hack the exchange
code a bit in this case to always chose bundle3.
This is copied verbatim from the lfs extension.
Reviewed By: DurhamG
Differential Revision: D21459734
fbshipit-source-id: 41c867cec09e2485ec1e9d91545b61da568f4766
Summary:
Add new fields to EdenConfig to control log rotation settings, and update
edenfs_monitor to set up the log rotation strategy using these settings.
Reviewed By: chadaustin
Differential Revision: D20427271
fbshipit-source-id: 9960bdb6f4d077a4e21fb4a6209aa02ab21ad653
Summary:
Update the LogFile class in edenfs_monitor to support log rotation when the
file size exceeds a set limit.
There is a generic API to abstract out the log rotation details. For now
I have only implemented a relatively straightforward implementation that
appends the current timestamp to the log path, and prunes old log files.
Reviewed By: chadaustin
Differential Revision: D20427310
fbshipit-source-id: f6aefc8f2fe604fdd1d6a72c3d30666612912f41
Summary:
See the comment in FuseChannel.cpp, but it's currently not easy to
avoid the "security.capability" getxattr request for every
write. Since we can't avoid the request, the fastest thing we can do
is branch, strcmp, and fast-path a result on the same thread.
This appears to save three or four microseconds in the 4k random write
benchmark.
Reviewed By: wez
Differential Revision: D21341973
fbshipit-source-id: a23620767f4bdec4daf02ecfe3acb924dd57857a
Summary:
On Windows, we've had a couple of cases where the overlay is corrupted which
prevent remounting EdenFS. In this case, one entry of a directory was an empty
path. Let's try to catch the error when it happens, instead of after the fact.
Reviewed By: wez
Differential Revision: D21443002
fbshipit-source-id: 47e59aee803cc0b4befd6e0585836d5c68e905b1
Summary: All of the tests in hg/status_test.py pass except for these 3.
Reviewed By: genevievehelsel
Differential Revision: D21410570
fbshipit-source-id: cf9a87ce0f025860c1e95927a780757fda6801ad
Summary:
- If we fail to create symlinks ignore this error on Windows
- Using a file as a directory produces an ENOENT error instead of ENOTDIR
on Windows.
- Deal with some issues around CR -> CRLF translation when writing files in
non-binary mode on Windows.
- Move several POSIX-specific tests out into a separate test class.
Reviewed By: wez
Differential Revision: D21214598
fbshipit-source-id: 587a755bd06990bf88d26017254a69ff8869edbc
Summary:
On Windows we can only run EdenFS if ProjectedFSLib.dll is available. It
turns out that some Windows systems have ProjectedFSLib.lib available, so we
can build `edenfs.exe`, but are missing ProjectedFSLib.dll, so we cannot
actually run it.
Reviewed By: wez
Differential Revision: D21443255
fbshipit-source-id: 6396fce8bd71adc0695468a2df3fd5f584909138
Summary:
The disabled tests are due to the precision of st_mtime on Windows only being
a second, but the tests are changing the config faster than that.
Reviewed By: wez
Differential Revision: D21319023
fbshipit-source-id: bcdce24f70ce99984cabb290338ac94a2459e9de
Summary: This is according to the suggestion in the discussion referenced in the task. Per quark-zju we do need to change `rage` to use `hg sparse` rather than `hg sparse show`.
Reviewed By: quark-zju
Differential Revision: D21422005
fbshipit-source-id: 6dd0e20125635c7fb9b6ea6c9e2b35c8fb517d5d
Summary: Added the `status` field to json in order to provide that information to the automated client, as well as match similar output of `hg status`.
Reviewed By: quark-zju
Differential Revision: D21421494
fbshipit-source-id: 2a8b80068f2068b09930b90c43252003421b324e
Summary: Fixed that `hg sparse show` failed on missing profiles. Added them to be shown in the output with "!" symbol and in cyan color - which matches output of deleted files in `hg status`.
Reviewed By: quark-zju
Differential Revision: D21419278
fbshipit-source-id: 5581e67774686a5240dceb9aac428fac3b1b73c2
Summary:
Remove HgIdDataStore::get_delta and all implementations. Remove HgIdDataStore::get_delta_chain from trait, remove all unnecessary implentations, remove all implementations from public Rust API. Leave Python API and introduce "delta-wrapping".
MutableDataPack::get_delta_chain must remain in some form, as it necessary to implement get using a sequence of Deltas. It has been moved to a private inherent impl.
DataPack::get_delta_chain must remain in some form for the same reasons, and in fact both implenetations can probably be merged, but it is also used in repack.rs for the free function repack_datapack. There are a few ways to address this without making DataPack::get_delta_chain part of the public API. I've currently chosen to make the method pub(crate), ie visible only within the revisionstore crate. Alternatively, we could move the repack_datapack function to a method on DataPack, or use a trait in a private module, or some other technique to restrict visibility to only where necessary.
UnionDataStore::get has been modified to call get on it's sub-stores and return the first which matches the given key.
MultiplexDeltaStore has been modified to implement get similarly to UnionDataStore.
Reviewed By: xavierd
Differential Revision: D21356420
fbshipit-source-id: d04e18a0781374a138395d1c21c3687897223d15
Summary:
These 3 tests compile without issues on Windows. The RocksDB one is weird,
while it compiles with no hickups, I simply cannot run the resulting test
binary, and I'm not sure how to debug this. The local store one fails in folly.
Reviewed By: chadaustin
Differential Revision: D21393724
fbshipit-source-id: db90bf20a9d116bc8aa493703997c5e8da76eb1f
Summary: All the tests are passing.
Reviewed By: chadaustin
Differential Revision: D21341730
fbshipit-source-id: 90a3872b190879ec163935ff53703157028f87bc
Summary:
The modeFromEntryType and treeEntryTypeFromMode tests for symlinks and
executable had to be disabled as these function explicitely do not support
these. Since mode bits are a bit meaningless on Windows, this is probably OK.
Reviewed By: chadaustin
Differential Revision: D21341728
fbshipit-source-id: 86acf24d9ab67a02ecab33b7ebe82a456295fc3c
Summary: All of these tests are passing on Windows with no changes.
Reviewed By: chadaustin
Differential Revision: D21341729
fbshipit-source-id: 2b4d52751e74fa953bfe5143dc0c5735de2d34cf
Summary:
All of these were simply NOT_IMPLEMENTED on Windows, but the code compiles
and doesn't break any existing tests. The underlying called functions might
have been implemented already, or are NOT_IMPLEMENTED, either way, this reduces
the amount of `#ifdef _WIN32`.
Reviewed By: chadaustin
Differential Revision: D21405622
fbshipit-source-id: bdc2de41d6a57e1c0b532e76eeb2c0c86180d558
Summary: What it says in the title. I'd like to set up alarms on this.
Reviewed By: farnz
Differential Revision: D21450584
fbshipit-source-id: 539299407cea84c67ff14b30184e8df4282415f8
Summary:
Update `contrib/check-code.py` to Python 3.
Mostly it was already compatible, however stricter regular expression parsing
revealed a case where one of our tests wasn't working, and as a result lots of
instances of `open(file).read()` existed that this test should have caught.
I have fixed up most of the instances in the code, although there are many
in the test suite that I have ignored for now.
Reviewed By: quark-zju
Differential Revision: D21427212
fbshipit-source-id: 7461a7c391e0ade947f779a2b476ca937fd24a8d
Summary: Reformat using a newer version of Black.
Reviewed By: quark-zju
Differential Revision: D21426337
fbshipit-source-id: 1ac7f6e85a06feec0d41e9509eca09194f421a1d
Summary:
The latest version of Black removes unneccessary parenthesis. Mercurial's
test-check-code currently uses extra parentheses to signal untranslated strings, so
Black's reformatting breaks this test.
Capitulate to Black by adding a new `_x` translation marker that means "untranslated".
Reviewed By: quark-zju
Differential Revision: D21426335
fbshipit-source-id: a6c26d7c6365c49530a7dee3a5f9ed71ff166835
Summary:
If a bundle comes from the commit cloud forward filler, we need to ignore
and not record it.
To do so, we need to start paying attention to stream-level params for the
first time.
Reviewed By: krallin
Differential Revision: D21427620
fbshipit-source-id: 9ee417cd2a5f2f5bb6ec342cd63071c6ca822475
Summary:
We want to be able to record all the bundles Mononoke processes to be later
replayed by Mercurail.
Reviewed By: krallin
Differential Revision: D21427622
fbshipit-source-id: b88e10e03d07dae35369286fe31022f36a1ee5cf
Summary: To make it easier to navigate the codebase the oss-only code will be from now on stored in a separate module, similarly to how the fbcode-only code is stored.
Reviewed By: markbt
Differential Revision: D21429060
fbshipit-source-id: aa7e80961de2897dae31bd0ec83488c683633b7a
Summary:
Something was up yesterday with the warm bookmarks cache. It started failing on
some hosts, and went out of sync for > 1 hour on some hosts. The logs reported
a lot of failures, but without any context they weren't super useful:
P130438422.
This adds a bit more logging. If this happens again, we'll be able to better
understand what happened.
Reviewed By: StanislavGlebik
Differential Revision: D21447043
fbshipit-source-id: 67a3924c4486991df5e4d38a995ff8054c145cf9
Summary: This is an sqlite equivalent of what exists in xdb now.
Reviewed By: krallin
Differential Revision: D21427621
fbshipit-source-id: 7024fbf7a8773c4465d2e6ee327aadeaf87cb213
Summary:
Implement autopull so non-automation `hg pull -r Dxxxx`, `hg up Dxxxx` will
pull `Dxxxx` automatically.
Since we now autopull the commits, error messages are removed. The old code
actually causes issues because it will raise at `"D1234" in repo`, which is
a surprise to many code paths, including the revset autopull logic, which
uses `x in repo` to decide whether `x` is unknown.
Note `hg pull -r Dxxxx` is not using the revset layer and needs to be handled
separately. It works for hg servers right now because the server can translate
`Dxxx` to a commit hash. It probably does not work for a Mononoke server.
Reviewed By: sfilipco
Differential Revision: D21320626
fbshipit-source-id: 939abe12e3a9a8ed5ca7ed29bb4f90fb39e7674a
Summary:
Change the interface to return infallible `Optional[node]` instead of fallible
`List[rev]`. In the next diff, we're going to use the 'node' information to
implement autopull.
Reviewed By: sfilipco
Differential Revision: D21320628
fbshipit-source-id: 6f7c070faba667cc85313cc78d6149c787ca8593
Summary:
Currently, phrevset picks the "successor" that has the maximum revision
number. That depends on the assumption that larger revision numbers
are modified last. Howevver, that's not always true with commit cloud sync. For example, in my repo I have D21179514 modified from 63768bf43
to 684612d5d. Phabricator has 63768bf43. Local successor is 684612d5d,
but the revision number of 684612d5d is smaller than 63768bf43:
quark@devvm1939 ~/hg/edenscm/hgext % hg log -r 'D21179514'
changeset: 63768bf436d01982a8d42ce97160ac6d9ae2cdad D21179514
user: Jun Wu <quark@fb.com>
date: Wed, 22 Apr 2020 09:45:50 -0700
summary: [hg] commitcloud: log metalog root during update references
quark@devvm1939 ~/hg/edenscm/hgext % lhg log -r 'D21179514'
changeset: 684612d5d606b01c224889f2b3f87aff7b93db49 D21179514 (@)
user: Jun Wu <quark@fb.com>
date: Wed, 22 Apr 2020 10:10:37 -0700
summary: [hg] commitcloud: log metalog root during update references
quark@devvm1939 ~/hg/edenscm/hgext % lhg log -r 'successors(63768bf436d0198
2a8d42ce97160ac6d9ae2cdad)' -T '{node} {rev}\n'
684612d5d606b01c224889f2b3f87aff7b93db49 76718
63768bf436d01982a8d42ce97160ac6d9ae2cdad 95363
Improve it by actually prefer selecting a non-obsoleted successor.
Reviewed By: sfilipco
Differential Revision: D21267552
fbshipit-source-id: d43d72a7c273c55af70bb41ad967fff0c78a452a
Summary: This is more efficient (if pullattempts have high chance to succeed).
Reviewed By: sfilipco
Differential Revision: D21320627
fbshipit-source-id: a2166f5a59f98c7d705c806b9d152ceb9981f3be
Summary:
Make it so that the autopull functions just describe what to do (using the
`pullattempt` struct) instead of having the side effect directly. This will be
useful in multiple cases:
- The actual autopull logic becomes easier to implement.
- The revset autopull layer can merge `pullattempt`s to fetch things in one go.
- The `pull -r` logic can reuse the `pullattempt`s to translate what to pull
(ex. from `D1234` to a commit hash).
This has subtle changes that multiple "remote"s are no longer properly
supported. That is probably fine in our production use-cases but we might
want to revisit if we want to support remotes "properly". Currently, this
greatly simplifies things due to the fact that "infinitepush" or
"infinitepushbookmark" have to be used in certain cases.
Reviewed By: sfilipco
Differential Revision: D21320633
fbshipit-source-id: e38b68abf69a34a97431685aa7ab0d2fe022fda8
Summary: Add a way for extensions to register how to auto pull unknown names.
Reviewed By: sfilipco
Differential Revision: D21320630
fbshipit-source-id: d20e11cff83b8a7e15a5085b0508a3e5bef305c3
Summary:
We're going to make autopull more complicated. Move it to a separate module as
it is not directly related to revset.
Reviewed By: sfilipco
Differential Revision: D21320631
fbshipit-source-id: fe60bc53ebf1c75f8bf66156805cbe2801fe6532
