Summary:
With narrow-heads before the new dag structure, reading phases is expensive.
Avoid it.
This is part of a stack of optimization that makes a 20s rebase to 5s (0.6s per
commit to 0.07s per commit).
Reviewed By: xavierd
Differential Revision: D18838468
fbshipit-source-id: 0847d2508114203f1d16a7240a43375d325103a5
Summary:
With narrow-heads, rebase operation does not need to read existing phases to
decide new phases. Therefore remove it.
This is part of a stack of optimization that makes a 20s rebase to 5s (0.6s per
commit to 0.07s per commit).
Reviewed By: xavierd
Differential Revision: D18838469
fbshipit-source-id: 1298d4f4b0d72a764a286478e76c90b42b3fc569
Summary:
In the future we might have revlog or more complicated storage for maintaining
the IdMap. Define a trait for it.
Reviewed By: sfilipco
Differential Revision: D18664208
fbshipit-source-id: 242611d78852e8d82a52ab52192db87c13b848e7
Summary: This is the reverse of first_ancestor_nth. It will be used later.
Reviewed By: sfilipco
Differential Revision: D18664209
fbshipit-source-id: 8d3706cdc080086ec60da60870fde76c8933f6a8
Summary: This will be used in an upcoming change.
Reviewed By: sfilipco
Differential Revision: D18664207
fbshipit-source-id: 3030bb02e172b95c6dbabda74c318e7f16212f59
Summary:
With the ONLY_HEAD flag, ancestors can be sped up sometimes. Since ancestors
is used widely in other DAG calculations, perf improvements are visible across
the benchmark. The trade-off is to spend a bit more time on building segments
to figure out ONLY_HEAD.
After this diff:
building segments 576.625 ms
ancestors 47.119 ms
children 436.967 ms
common_ancestors (spans) 257.096 ms
descendants (small subset) 367.316 ms
gca_one (2 ids) 146.315 ms
gca_one (spans) 253.059 ms
gca_all (2 ids) 236.075 ms
gca_all (spans) 266.695 ms
heads 220.766 ms
heads_ancestors 36.258 ms
is_ancestor 98.967 ms
parents 219.173 ms
parent_ids 9.282 ms
range (2 ids) 99.859 ms
range (spans) 205.154 ms
roots 429.047 ms
Before this and previous diff:
building segments 542.564 ms
ancestors 316.075 ms
children 428.612 ms
common_ancestors (spans) 452.850 ms
descendants (small subset) 537.483 ms
gca_one (2 ids) 526.112 ms
gca_one (spans) 446.726 ms
gca_all (2 ids) 780.590 ms
gca_all (spans) 490.906 ms
heads 214.005 ms
heads_ancestors 217.558 ms
is_ancestor 256.769 ms
parents 213.170 ms
parent_ids 9.313 ms
range (2 ids) 259.435 ms
range (spans) 358.934 ms
roots 423.005 ms
Reviewed By: sfilipco
Differential Revision: D18642797
fbshipit-source-id: 27f1ee5797a92142b2c19195eb26baef58769553
Summary: The pre-calculated flag can be used for some fast paths.
Reviewed By: sfilipco
Differential Revision: D18642796
fbshipit-source-id: bc16b3517cc29e23478f2027e83e2a2282be0d0a
Summary:
The bindings APIs are out of date.
The latest idea is to write non-master commits to disk, in a different group.
There are no "segments in memory". Therefore:
- Rename `builddisk` to `build`.
- Make sure `build` updates internal states by calling `reload`.
- Drop `buildmem`.
- In case `builddisk` gets called very frequently, reduce its overhead by
doing nothing if all commits are built already.
The constructor is revised to match the latest interface (no
"max segment level" and "set_new_segment_size" needs to be called).
Reviewed By: sfilipco
Differential Revision: D18640897
fbshipit-source-id: 76a5d0db363713dc5b763503c238c1bfa61ea02e
Summary: Make it possible to build non-master segments.
Reviewed By: sfilipco
Differential Revision: D18641507
fbshipit-source-id: 4d0c46cdea240849af37d28219b6023578514323
Summary:
`build_segments_volatile(id)` should build flat segments so it includes `id`.
Right now it can fail that in a corner case. Fix it.
Reviewed By: sfilipco
Differential Revision: D18641509
fbshipit-source-id: ad1de694e4d31c51f29e9bc5800291066bf16dbb
Summary:
The IdMap has assertions about 1:1 mapping between Id and Slice.
Practically, a non-master commit can be merged into a master commit.
That means a commit hash might map to 2 Ids: [master-id, non-master-id].
Allow that to happen (temporarily).
Note: The non 1:1 mapping causes more headache than benefit and
D18838995 restores 1:1 mapping. But it still requires temporary
non-1:1 mapping to work.
Reviewed By: sfilipco
Differential Revision: D18641508
fbshipit-source-id: bc4bc831d5ab5ebfc2588bafd48114ae49d31e0f
Summary: fix the `build-eden-thrift-deps.py` to generate a new zip file, upload it to lfs, and point setup.py to use this new generated thrift code
Reviewed By: quark-zju
Differential Revision: D18836545
fbshipit-source-id: 52c58c50319c878f2f6f708b952f0aa158d2ef4e
Summary: panic on drop is a terrible idea. It can cause SIGILL kill without meaningful messages or meaningful backtrace. Remove it.
Reviewed By: xavierd
Differential Revision: D18640894
fbshipit-source-id: 3ab52ded9875e17e891acd8f9b45d82964615f01
Summary: This makes the code harder to crash in abnormal cases.
Reviewed By: sfilipco
Differential Revision: D18640895
fbshipit-source-id: ee81dd7739739d3d15052bdb7bb2ba80ed48cd0a
Summary:
This allows us to allocate Ids for non-master commits in a separate range
so they wouldn't cause fragmentation for commits in the master group.
Reviewed By: sfilipco
Differential Revision: D18640896
fbshipit-source-id: c19c53342583ded8795849584cf5f31bc86b2818
Summary: Migrate the codebase to use the new Id type instead of an alias of u64.
Reviewed By: sfilipco
Differential Revision: D18640898
fbshipit-source-id: 415db9afeda1f8960e18c69e70f53a9eeb63cbbe
Summary:
Background:
- Tests and Nuclide haven't removed usage of revision numbers.
- If we build segments for non-master commits on demand, there are performance
concerns about very long release branches.
- It's very desirable to keep `ancestors(master)` continuous. In other words,
`ancestors(master)` is a single integer span. This affects performance for
DAG algorithms a lot.
To solve the problems, I'm going to use separate ranges for master and
non-master commit Ids (so assigning Ids to non-master commits wouldn't affect
what numbers can be used for the master group). To make the code easier to
read, I'm changing `type Id = u64` to `struct Id` where the struct is
group-aware.
This diff introduces related types and basic operations.
Reviewed By: sfilipco
Differential Revision: D18640899
fbshipit-source-id: 8f7ecbee6e0d1703d1225f18e92742981dbf3757
Summary:
sysplatform is defined as a byte string, and thus a byte string needs to be
provided.
Reviewed By: ikostia
Differential Revision: D18816931
fbshipit-source-id: c653f1319f160bc00d7a1c90b06ff3b4f4d035ad
Summary:
In Python3, unicode doesn't exist, instead of re-inventing the wheel, just use
unicode from the pycompat module.
Reviewed By: ikostia
Differential Revision: D18816929
fbshipit-source-id: d3ec9ae4f2190db64866568b1945b26e05b9c2a5
Summary:
Somehow, pyre isn't able to inline the value of ispy3 and propagate it properly
and thus complains about lots of code paths. Let's inline it everywhere.
Reviewed By: ikostia
Differential Revision: D18816333
fbshipit-source-id: 80f81cb91bef4c6831952f9469401d87040335e2
Summary:
The None value has an Optional type, but was overwritten by a string in the
subclass, let's just change the base value to an empty string.
Reviewed By: ikostia
Differential Revision: D18805103
fbshipit-source-id: 30113d5b4394163554b340917f3cf32178e4fe9e
Summary:
The copy method is inherited from list and pyre requires proper type
annotation.
Reviewed By: ikostia
Differential Revision: D18805104
fbshipit-source-id: efe828c6033af48754c0ef28ee52340616582e23
Summary:
The run method is inherited from Thread, and pyre requires proper type
annotation.
Reviewed By: ikostia
Differential Revision: D18805026
fbshipit-source-id: 4f9addc5d1d9655f3796342d99366df9f0cfe102
Summary:
The branchmap type inherit from dict and override the copy method, pyre
requires types for it.
Reviewed By: ikostia
Differential Revision: D18805028
fbshipit-source-id: 43de36df6f40e5a3485e83e8056b2a70cdebcfac
Summary:
These methods are inherited from Thread, and pyre wants proper type annotation
in this case.
Reviewed By: ikostia
Differential Revision: D18805027
fbshipit-source-id: 1d479c7af4da74220be3e175c8d1c0803da6d9e4
Summary: This updates the hg and telemetry wrapper callsites of getScmStatus to first try running getScmStatusV2() with fallback option. This does not retry `hg status` while a checkout is in progress.
Reviewed By: simpkins
Differential Revision: D18209899
fbshipit-source-id: e7a77b902f5a0ee624e4ea3185a1901bdac090e6
Summary:
When changing a file to a folder in a commit this function would try to query
the storage using the path and the id of the file. Storage would not have
such an entry and would cause the operation to fail.
Fixing the issue by filtering out ids of files in the section for known
manifests. Filtering already occurs in the section for transmitted manifest.
Reviewed By: quark-zju
Differential Revision: D18797368
fbshipit-source-id: 2ed727b5eaaab2f17cd988d6c8a3a1055ac8bd2a
Summary:
On the LFS Server, we've noted that LFS batch requests from Mercurial have
about ~40ms latency, but that the same request sent through curl doesn't.
Those requests, when sent through curl, complete in < 1ms, so thats a lot of overhead.
After adding more logging in the LFS server (D18636703), it turned out that that is
because the body of the request shows up 40ms after the request itself.
After capturing the traffic on the backend (D18689109), it turned out that
Mercurial is sending the headers and the response body in 2 separate TCP
packets.
The reason for that is delayed TCP acks. What happens is that when we call
`h.endheaders()`, that calls `send(2)`, which in turns send the packet with the
headers. Later, when we call `h.send(data)`, we make another `send(2)` syscall,
but that doesn't go over the wire immediately: because of Nagle's algorithm, we
wait for an ACK for the first packet we send before sending a new one, and
because of delayed ACKs, the ACK takes 40ms for this ACK to show up (which is
over 40 times the roundtrip latency!).
This diff passes the data to `h.endheaders()`, which in Python's httplib will
concatenate the headers and the data into a single `send(2)` syscall if the
data isn't a file-like object (which it isn't for batch requests), and
otherwise fall back to doing exactly what we're doing right now.
This will result in everything in a single packet if we're fetching a single
file (and AFAICT, without further delay if we're fetching enough files that the
request doesn't fit in a single packet).
An alternative approach would be to set `TCP_NODELAY` on our socket here, but
for now this seems less intrusive and just as effective, so I've opted for that
approach.
Reviewed By: farnz
Differential Revision: D18811419
fbshipit-source-id: 4cf5719e5eed90e5dd994e6c8861aceb69373d89
Summary:
The Mercurial codebase contains over 500 errors, let's ignore them for now, we
can go back to them later to fix them.
Besides the manual change to .pyre_configuration.local, the changes were
generated with:
pyre --output=json check | pyre-upgrade fixme
Reviewed By: singhsrb
Differential Revision: D18803908
fbshipit-source-id: 724db7bd864c0de47a97ef2092bdee9f2cda531f
Summary:
The line causes pyre to enter an infinite loop, for now, let's simply ignore it
when type checking is enabled.
Reviewed By: singhsrb
Differential Revision: D18803909
fbshipit-source-id: d89b4cd0311a4a5416dd31197a8c69f4a6b65944
Summary:
When user types Dxxx as a revset locally they usually mean the latest version
of the commit - not neccesarily the one in phabricator. This usecase was
usually handled by doing local lookup which can be very slow in case of slow
commits: see for example those user complaints:
https://fb.workplace.com/groups/scm/permalink/2487795837936688
Reviewed By: farnz
Differential Revision: D18809252
fbshipit-source-id: b3442d6fa2ef9c9c0dff4909c874689810fbfa88
Summary: Log files that required manual merge during a rebase to dev command timers so we have an idea of how often they happen and on which files
Reviewed By: simpkins
Differential Revision: D18648175
fbshipit-source-id: 83ffe6d9177ca00b8fd1095745c585186bc2c8e9
Summary:
These are not currently used by mercurial and are superseded
by the newer more general `watchman_client` crate on crates.io:
https://docs.rs/watchman_client/0.2.0/watchman_client/
Some of the implementation of that crate was based on the work I'm
removing here.
I was going to update hg_watchman_client to be implemented in terms
of the new crate, but since it isn't used and it would only be a thin
wrapper, I figure that it is best to delete this code instead so that
we can integrate it in terms of the new API when we're ready.
Reviewed By: singhsrb
Differential Revision: D18777161
fbshipit-source-id: efacb749e6d35ff4cc1fdc7d99cdca9ed42c9cfe
Summary:
Since it's not calling conduit anymore we can remove all conduit related
functions.
Reviewed By: quark-zju
Differential Revision: D18733121
fbshipit-source-id: 20a6022d9ac7ae6e9afea2791a8daac57cefb22c
Summary:
conduit is going away, phabricator team wants us to stop using
it in hg.
Reviewed By: quark-zju
Differential Revision: D18732257
fbshipit-source-id: c4faf83e36af80fc616e91adede141ba12de5446
Summary: conduit is going away, phabricator team wants us to stop using it in hg
Reviewed By: quark-zju
Differential Revision: D18727399
fbshipit-source-id: c208f5fec5fefd83aa37629b56d2bbaa2532d30e
Summary:
It would be nice if the insert method would give back the path that
it failed to insert.
Reviewed By: dtolnay
Differential Revision: D18739978
fbshipit-source-id: 4c16d09750ade2f01397161129c31bcf0059a957
Summary:
The Mercurial code has the bad habbit of inserting files in what can
appear impossible locations. Sometimes files with directory names and
sometimes directories in file locations. This happens because the initial
code would do additions before deletions on Manifest implementations that
were rudimentary. As soon as we introduce validation various code paths
surface.
I tried to fix the codepaths that modify manifests but it's a losing game.
I fixed the issue that appeared in tests and a couple of issues people
reported but more situations crop up.
This is giving up on the python code.
Reviewed By: quark-zju
Differential Revision: D18737678
fbshipit-source-id: 0c97128ff67e5ba2334942b6afc404aa64a2e5f4
Summary:
In NFS v4.x, flock ends up inheriting some semantics of fcntl,
including that write locks can only be created against file handles
opened for O_WRONLY or O_RDWR.
This diff changes the mode of 'lockpath' to O_RDWR to solve this.
Reviewed By: simpkins
Differential Revision: D18714734
fbshipit-source-id: 84ba4a6a5de034a4942b1ca3aa8b0c92f882ce38
Summary:
`path = ...` is enough. This should resolve the following build issues:
error: failed to select a version for the requirement `serde_bser = "^0.1.0"`
candidate versions found which didn't match: 0.2.0
location searched: ...
required by package `watchman_client v0.1.0 (...)
... which is depended on by `hg_watchman_client v0.1.0 (...)`
Reviewed By: lukaspiatkowski
Differential Revision: D18738176
fbshipit-source-id: 725840895a8e988b35000a48cf92018b14cb4dee
Summary:
Since bookmarks are stored in svfs, track it in metalog for better transaction
support.
This affects some transaction / rollback behaviors. Since our plan is to remove
`hg rollback` eventually, the compatibility is intentionally broken.
Reviewed By: xavierd
Differential Revision: D18538147
fbshipit-source-id: 6070772a4bce88d3808547207f5df194696b1b52
Summary:
Existing tools (ex. shell complete script) still read from
`.hg/store/{remotenames,bookmarks}`. Be compatible with them by double writing
content to both metalog and filesystem. Metalog is the source of truth for hg.
Reviewed By: xavierd
Differential Revision: D18524106
fbshipit-source-id: c37bc86b7443bba1b65735f29243767650b22ecd
Summary:
Similar to D17199834, by moving 'bookmarks' from localvfs and sharedvfs to
storevfs, we can make it part of metalog and get better transaction support.
Reviewed By: xavierd
Differential Revision: D18524104
fbshipit-source-id: ae148c1f02fc83b5c2d73102ecab39ff223ea5df
Summary:
I'm going to move `bookmarks` to "metalog". So operating on the `.hg/bookmarks`
file no longer works.
Reviewed By: xavierd
Differential Revision: D18524105
fbshipit-source-id: dc31b13e1acc171d2e8b32cdfea7028faf6dc4d3
Summary:
Right now `bookmarks` is stored at both "local" vfs and the "shared" vfs.
In an upcoming change I'd like to move bookmarks to the "metalog" to get
better transaction. Therefore make it always shared.
The shared bookmark feature is turned on by default. Assuming nobody manually
turns it off, production hosts all have shared bookmark turned on.
Reviewed By: simpkins
Differential Revision: D18524103
fbshipit-source-id: 3160146f357dd01f2d78f5d2f4a14838f8f12a94
Summary:
We have seen cases on Windows where the hg process gets stuck in
CreateToolhelp32Snapshot. Let's be defensive and always exit the loop.
Reviewed By: singhsrb
Differential Revision: D18729720
fbshipit-source-id: fb8602ce231eec01b6b42c6759849d56e5db2030
Summary:
The only valid place that metalog can be changed is inside a transaction, since
it's transaction.{close,abort} that writes or discards metalog changes.
In other words, metalog should not be changed during A, B, or C:
|<- A ->|<----------- repo lock --------->|
|<- B ->|<- transaction ->|<- C ->|
Add detection for them.
Reviewed By: xavierd
Differential Revision: D18538143
fbshipit-source-id: 036286ed32a897fe3ce0a91c1e3c848cc6167b1d
Summary:
This helps test if there are uncommitted changes. It can help detecting
programming errors. For example, is metalog changed before starting a
transaction?
Reviewed By: xavierd
Differential Revision: D18538148
fbshipit-source-id: 9c82838d8174e6a93fde7108503a025660191cbf
Summary:
Before this diff, metalog has a same lifetime as repo.svfs and never gets
reloaded or dropped. That is problematic in case external processes also
make changes, ex. running `hg amend` in `histedit exec`.
pid 1> histedit action foo (a single transaction, metalog loaded here)
pid 1> histedit action exec ... (a single transaction)
pid 2> hg amend (changes metalog)
pid 1> histedit action bar (cannot perform this action because metalog has
conflicts!)
This is why test-mutation.t didn't work with metalog.
Fix it by discarding metalog state at transaction/lock boundary (enter/exit).
The next two diffs add checks so we wouldn't discard uncommitted data silently.
Reviewed By: xavierd
Differential Revision: D18519995
fbshipit-source-id: ceb030362d66ad4be142e81accb82a4afa67f305
Summary: This resolves ProgrammingErrors detected by a later change.
Reviewed By: xavierd
Differential Revision: D18538145
fbshipit-source-id: 650e956707af6024457cdc7dabf516d087ad03cd
Summary:
The current `commit` code path errors out without telling much about what
conflicts are. Improve it by showing what are actually conflicting, and allows
users to replace the resolver function to do customized merges.
Reviewed By: xavierd
Differential Revision: D18519996
fbshipit-source-id: 78f42ada1df45659783ce1eba30a19f6fc6b3b62
Summary:
Those tests will crash `run-tests.py` because `svn` is unknown to `hghave`.
Alternatively we can revert `hghave` changes in D18088850. But it seems there
is little interest in converting svn repos using the convert extension.
Reviewed By: singhsrb
Differential Revision: D18713922
fbshipit-source-id: 1a70b3d5d0fe8d02cc0549eabeb609ab1d9c12e6
Summary:
"outstanding merge conflicts" is not a helpful message to inexperienced users.
Print out commands to get more context.
Reviewed By: simpkins
Differential Revision: D18535010
fbshipit-source-id: 035ec9f3d79bf04a997ee907469f2e3d749a1d0e
Summary: Some commits in AOSP have empty fields for author/committer email addresses.
Reviewed By: tchebb
Differential Revision: D18661778
fbshipit-source-id: e975392da677879d598eb9fc77558251a55c2f23
Summary:
Remove some unnecessary forward declarations, and use full paths in includes
so autodeps can recognize which rule these headers come from.
Reviewed By: fanzeyi
Differential Revision: D18652106
fbshipit-source-id: e2c84e1aaf846c91c18242050c4015eca8c501d3
Summary:
Now that the fb-mercurial sources are available in the Eden repository, update
our CMake build files to always build them. This moves the build logic from
the centralized `FBMercurialFeatures.cmake` file into `CMakeLists.txt` files
in the appropriate subdirectories.
Reviewed By: chadaustin
Differential Revision: D18588011
fbshipit-source-id: ded9decde5c2ec766aae0bb0f4f5b021d1044a98
Summary:
Remove the standalone fb-mercurial-rust target that was an internal-only
dependency for the Eden build. This build step is now done entirely in the
Eden build.
Reviewed By: fanzeyi
Differential Revision: D18623943
fbshipit-source-id: c62a1155ddd1c0a6b2270c472176ba25194c6145
Summary:
D8997967 changed `hg rage` to pull from a path that uses the machine's unix user. But The write in the background backup for commit cloud used `util.shortuser(ui.username())`. This means that in On Demand, it would write to `$LOGDIR/samfu/$REPO$DATE` but `hg rage` would look in `$LOGDIR/svcscm/$REPO$DATE`. Changing this to the same `util.getuser()` should fix this.
Caveat: I didn't actually test this. I just know that my `hg rage` says "log directory does not exist: /tmp/svcscm" (P122851697)
Reviewed By: markbt
Differential Revision: D18654245
fbshipit-source-id: ccb78eba778ddedb23c7947a3eeb16e9f8fd2021
Summary:
This diff replaces eden's dependencies on failure::Error with anyhow::Error.
Failure's error type requires all errors to have an implementation of failure's own failure::Fail trait in order for cause chains and backtraces to work. The necessary methods for this functionality have made their way into the standard library error trait, so modern error libraries build directly on std::error::Error rather than something like failure::Fail. Once we are no longer tied to failure 0.1's Fail trait, different parts of the codebase will be free to use any std::error::Error-based libraries they like while still working nicely together.
Reviewed By: xavierd
Differential Revision: D18576093
fbshipit-source-id: e2d862b659450f2969520d9b74877913fabb2e5d
Summary:
[svcscm@hg022.prn2 /data/scm/www]$ hg log -r master
transaction abort!
rollback completed
abort: File name too long: '/data/scm/www-merge-generaldelta/.hg/store/data/flib/intern/____generated____/_graph_q_l_meerkat_step/flib/intern/entschema/generated/entity/bizapps/dcp__insight/application__config/_ent_d_c_p_insights_application_config_action.php/._g_q_l_g___intern_____set_permission_manager_domain_dcp_insights_application_config_data_____ent_d_c_p_insights_application_config_action____gen_perform_graph_q_l_set_permission_manager_domain_dcp_insights_application_config_mutation_coercer.php.i-U0sMqX~'
On hg022.prn2 www currently failes under certain circumstances because filenames longer than
255 characters are currently created. I traced this back to mktempcopy and made sure that
we truncate filenames when they are created too long.
Reviewed By: quark-zju
Differential Revision: D18639433
fbshipit-source-id: 20e4b086ca5c775b6054b642ecedfb05efd00fb9
Summary:
Add Event::Tags so we can tag commands with one or more names.
Add index for the tags so we can search them more efficiently.
As we're here, update hgcommands to read EDENSCM_BLACKBOX_TAGS and write
Event::Tags automatically just after Event::Start.
Differential Revision: D18610733
fbshipit-source-id: c69b6e3d34b317e92a1c4d76f29e832b6b01c4fa
Summary:
Previously, the blackbox index was hardcoded on the timestamp, and is not
Event-aware. This diff changes the index to parse Event and build up indexes
for various common events (ex. Start, Finish).
For now, the index just includes start, finish timestamps and duration.
Differential Revision: D18610734
fbshipit-source-id: bf572f3138c8c18d3aab06eadcc83dd5b0b9cf02
Summary:
Previously, `match_pattern` only returns a `bool`. This diff implements an
enhanced version `capture_pattern` to be able to capture matched objects.
This will be used to match query patterns and apply optimization.
Differential Revision: D18610732
fbshipit-source-id: 39e989e7633b834c04997bf8665494556f3b62b2
Summary: Migrate callsites to use the new pattern matching filtering APIs.
Differential Revision: D18610738
fbshipit-source-id: 242182f4e6a58bbff5dddbb177287921c85519bc
Summary:
See the previous diff. As part of migrating away from IndexFilter. There needs
to be a way to filter commands by absolute clock time using pattern matching
against Events. To achieve that, add the timestamp field to Start and Finish
events.
Differential Revision: D18610735
fbshipit-source-id: 9e1898a87bc93e47d0f0c77342428b5c42ef6f24
Summary:
The existing query API `filter` has 2 issues:
- It does querying and filtering together, making it hard to express certain
queries. For example: "for commands taking more than 10 seconds, get all
network requests used by those commands".
- The `IndexFilter` abstraction overlaps with `pattern`s. `IndexFilter` is
built upon the `Entry` structure, and does not work for `Event`s. In the
future, we want a `Event::Tags` to tag commands, and build indexes on it.
That's not very compatible with `IndexFilter`.
To address the issues, I'm going to:
- Decouple querying and filtering by splitting it into 2 methods glued by
`SessionId`s.
- Deprecate `IndexFilter`. Make `patterns` the only way to filter entries.
- Add dedicated indexes for certain events.
This diff adds the related APIs towards that direction.
Differential Revision: D18610737
fbshipit-source-id: c6ab92fd6fa7bb2ee7e14f567d72ac98b287c143
Summary:
The type `Entry<T>` provides flexibility on serializing different types.
Practically, only the `Event` type is used. Let's remove `T` so `blackbox` can
assume it's the `Event` type.
Differential Revision: D18610736
fbshipit-source-id: 102a5f61c758fa83cd1994c7f813576dcf5b59d5
Summary:
Similar to D2756217. But skip sync directly if the repo has been synced
recently.
This will help reducing CPU load on the DB tier.
Reviewed By: ikostia
Differential Revision: D18621013
fbshipit-source-id: 83370669ea61eb693d06495aaa13664648b97aca
Summary:
Add tracing spans so we can get some insights about statistics from the pure
Rust land.
Reviewed By: xavierd
Differential Revision: D18625646
fbshipit-source-id: 15951fe2f8270a70f647a0f4e09819bbc33919c5
Summary:
It seems an emtpy GraphQL query always times out. So let's avoid such queries
from the client side.
Reviewed By: xavierd
Differential Revision: D18626286
fbshipit-source-id: 70f0061f7f7ad8c7d8be4e15fc419039dc07d61e
Summary:
Update `RustStaticLibrary.cmake` to let CMake know that running `cargo` will
produce the Rust static library output. Without this CMake does not know how
to correctly wire up dependencies for libraries. Previously the code used the
`ALL` target to ensure that the Rust static library was built, but this did
not ensure that it was built before executables that require it.
Reviewed By: pkaush
Differential Revision: D18588010
fbshipit-source-id: 1a1880993bdfa3fcb876a0f6507e5cddfea9b738
Summary:
Update the `RustStaticLibrary.cmake` code to explicitly find and invoke the
Python interpreter when running the Python script to download vendored crates.
This is required to work on Windows.
Reviewed By: pkaush
Differential Revision: D18588012
fbshipit-source-id: 4c2dfbc786adf0a7aa6b43483060800627a8aca1
Summary:
Update `RustStaticLibrary.cmake` to automatically decide to try and download
vendored Rust crates or not, based on the presence of the script it uses to
download them.
Reviewed By: fanzeyi
Differential Revision: D18588007
fbshipit-source-id: 995365f0cc2368897ddf94b5617bdb9bc0682edc
Summary:
This updates the `RustStaticLibrary.cmake` file to download vendored
`crates.io` crates only once, rather than once per library. It also now
downloads them into the build directory, rather than into the source
directory.
This can probaly still be improved more in the future: the vendored crates
should probably just be treated as a top-level getdeps manifest entry.
Currently if multiple projects build rust static libraries we will still end
up downloading the vendored crates once per project.
Reviewed By: fanzeyi
Differential Revision: D18548094
fbshipit-source-id: 83060e679455267bdf197e5ff2d33b1b7f0204fc
Summary:
Many people hit TApplicationException recently. Most of them can be fixed
by `eden doctor`. Instead of crashing and showing an unhelpful backtrace, show
a shorter error message and prompt the user to run `eden doctor`.
Reviewed By: fanzeyi
Differential Revision: D18609445
fbshipit-source-id: 5492fa92008cb33e1122b72b48d0858db0d5f994
Summary: This diff adds Rust structs to represent the data EdenFS wants from a manifest so we can pass these to EdenFS.
Reviewed By: wez
Differential Revision: D18365440
fbshipit-source-id: b474ceb698d9230e1b02c2cce1bec2a32bd3f94f
Summary:
Some clean up.
`CByte` should clean up `vec` in `Drop` since we are going to have `CBytes` in other struct's field and this will allow Rust automatically manage this memory.
Reviewed By: chadaustin
Differential Revision: D18365441
fbshipit-source-id: 7038bc6ff457415ee8234cc4c6b7df0f70005cfe
Summary: `manifest::Tree` wants to have a store that implements `TreeStore`. However at current we don't want to introduce dependency between `manifest` and `contentstore`. So we create this wrapper struct to provide the trait implementation. This will allow us to use `manifest` to parse manifest data we get from hg store.
Reviewed By: chadaustin, xavierd
Differential Revision: D18365354
fbshipit-source-id: 3687032c7f51570ef51ccc4004efc87f64ef2188
Summary: This diff adds `Option<HgId>` to `FsNode::Directory` because EdenFS needs the tree hash for the directories in the tree.
Reviewed By: sfilipco
Differential Revision: D18307205
fbshipit-source-id: 93db944ff1686a1c5927aa9a876a0fd008edbf9a
Summary:
The test needs to be updated since we now print commit hashes instead of
revision numbers in `phase` output.
Reviewed By: akushner
Differential Revision: D18596300
fbshipit-source-id: b42bb6ed68dfc33977f0c9df5018a5bf365c2909
Summary:
The restackonce function warns about not using it and using `rebase.rebase`
directly, so let's just do that.
Reviewed By: quark-zju
Differential Revision: D18574767
fbshipit-source-id: 56829bccbba7f727867ba2b9bd0a81530ba29bf3
Summary: I got unused warning every time I build Mononoke.
Reviewed By: singhsrb
Differential Revision: D18573764
fbshipit-source-id: 23921f581bdc74655041a2413f80cb159b4ba010
Summary: The test was about `simplemerge`, which was removed. Therefore, remove the test.
Reviewed By: singhsrb
Differential Revision: D18587461
fbshipit-source-id: daf12f0c38ec08a0f73d834b8f32dc1a65620a8f
Summary:
Edenapi uses a group of impls like the following as the canonical way to construct ApiError.
```
impl From<Context<ApiErrorKind>> for ApiError {
```
Downstream code would write:
```
use failure::ResultExt;
let stream_trees = config
.get_or_default("edenapi", "streamtrees")
.context(ApiErrorKind::BadConfig("edenapi.streamtrees"))?;
```
This relies on the way that ResultExt::context returns a unique type that is different from failure::Error.
This diff introduces a dedicated extension trait to allow the same code in the caller to continue to work even without a failure::Context<T> type, which will be required for dropping our dependency on failure::Fail and failure.
```
pub trait ApiErrorContext<T> {
fn context(self, kind: ApiErrorKind) -> ApiResult<T>;
}
```
We also introduce a public constructor for ApiError and replace indirect construction that used to look like this:
```
error.context(ApiErrorKind::Curl).into()
```
with this instead:
```
ApiError::new(ApiErrorKind::Curl, error)
```
which is the same number of characters but clearer. The argument order matches [std::io::Error::new](https://doc.rust-lang.org/std/io/struct.Error.html#method.new).
Reviewed By: kulshrax
Differential Revision: D18574668
fbshipit-source-id: 0a56297bb942a26d75a62ca39fc16abeb4486345
Summary:
On Windows, the paths passed to the `match` functions are not encoded in utf-8
yet. Do not crash if received them.
Reviewed By: sfilipco
Differential Revision: D18574951
fbshipit-source-id: c22ce95cb6538db0c15d60789286e2ff7d2148bc
Summary: The CommitCloudHttpError seems to have been working around a limitation of failure's derive that no longer applies to thiserror.
Reviewed By: xavierd
Differential Revision: D18523907
fbshipit-source-id: 54efaedc405911a412ce842dff6285fb80d875b8
Summary:
I have seen errors like:
Traceback (most recent call last):
File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner
self.run()
File "/usr/lib64/python2.7/threading.py", line 765, in run
self.__target(*self.__args, **self.__kwargs)
File "./run-tests.py", line 2636, in job
del runningtests[test.name]
File "/usr/lib64/python2.7/collections.py", line 68, in __delitem__
dict_delitem(self, key)
KeyError: 'test-empty-t.py'
It's not fatal and is caused by race condition. So let's just ignore it.
Reviewed By: ikostia
Differential Revision: D18538388
fbshipit-source-id: 85e38578bea4c9b27439e6f10abb7619a8bb9238
Summary:
Before this patch, if `test-foo-t.py.out` exists, the output of `test-foo-t.py`
will be diffed against it, something like:
% ./run-tests.py test-empty-t.py
--- test-empty-t.py.out
+++ test-empty-t.py.err
@@ -1,4 +1,4 @@
Traceback (most recent call last):
File "tests/test-empty-t.py", line 77, in <module>
- raise RuntimeError('a')
-RuntimeError: a
+ raise RuntimeError('ab')
+RuntimeError: ab
However, the dott Python tests are not using "reference output" at all. If they
output something, it's mostly likely some crash or back traces that we want to
see the full content of it.
This patch makes it so. The new output looks like:
% ./run-tests.py test-empty-t.py
Traceback (most recent call last):
File "tests/test-empty-t.py", line 77, in <module>
raise RuntimeError('ab')
RuntimeError: ab
ERROR: test-empty-t.py output changed
As we're here, also improve the detection of dott Python test so they don't
have to have the `-t.py` suffix in file name.
Reviewed By: ikostia
Differential Revision: D18534830
fbshipit-source-id: 826328be5de5841081e8f093b07295bc2fadc6b1
Summary: Eden sets the "portablefilenames" to ignore and the current Mercurial code doesn't consider the setting on Windows, which triggers iterate on the Eden clone. Iterate is an unsupported function on Eden clone.
Reviewed By: quark-zju
Differential Revision: D18195607
fbshipit-source-id: a21e1fcec3ee74398f22eaf56c70116468a45b45
Summary:
Now that fmt is available in Folly builds (D14813810), use it to reduce binary code size in Folly Logger. This is done by moving most of the formatting logic behind the type-erased `vformat` API. Previously it was instantiated for all combinations of formatting argument types used in calls to `FB_LOGF` and `XLOGF` in a program.
The effect of this change can be illustrated by looking at symbol sizes as given by `nm -S -td` for the following test function:
```
void test_log() {
FB_LOGF(logger, WARN, "num events: {:06d}, duration: {:6.3f}", 1234, 5.6789);
}
```
compiled in `opt` mode.
`nm` before:
```
0000000004236736 0000000000000231 T test_log()
0000000004236992 0000000000001002 W std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > folly::LogStreamProcessor::formatLogString<int, double>(folly::Range<char const*>, int const&, double const&)
```
`nm` after:
```
0000000004237536 0000000000000231 T test_log()
0000000004237792 0000000000000251 W std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > folly::LogStreamProcessor::formatLogString<int, double>(folly::Range<char const*>, int const&, double const&)
0000000004238048 0000000000000740 W folly::LogStreamProcessor::vformatLogString[abi:cxx11](folly::Range<char const*>, fmt::v5::format_args, bool&)
```
Before we had one 1002 byte instantiation of `formatLogString<int, double>`. With this change it was reduced 4x to 251 bytes and non-template function `vformatLogString` was added which is shared among all logging calls. The size of `test_log` remained unchanged. There are even bigger savings from Folly Formatter instantiations which are no longer needed, e.g.
```
0000000004238032 0000000000001363 W _ZNK5folly13BaseFormatterINS_9FormatterILb0EJRKiRKdEEELb0EJS3_S5_EEclIZNKS7_8appendToINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEEENSt9enable_ifIXsr12IsSomeStringIT_EE5valueEvE4typeERSH_EUlNS_5RangeIPKcEEE_EEvSK_
```
So in total this change results in ~5x per-call/instantiation binary size. It is possible to reduce binary size even further but it is not done in the current diff to keep it manageable.
In addition to binary size improvements, switching to fmt will potentially
* allow catching errors in format strings at compile time,
* simplify future migration to C++20 [`std::format`](http://eel.is/c++draft/format).
Reviewed By: simpkins
Differential Revision: D15485589
fbshipit-source-id: 06db4436839f11c2c3dbed7b36658e2193343411
Summary:
The caller already pass in a node, so there is no need to convert the rev
number to a node. This broke `hg next -B`.
Reviewed By: quark-zju
Differential Revision: D18539038
fbshipit-source-id: 44afdd184bc7c949f8efb863702af8a301bbcfb7
Summary:
Remove files that are ancient, not used, or not referred.
The file list was initially generated via a Ruby script:
(Dir['{contrib,tests}/**']-Dir['tests/test-*']).select{|name| `rg #{File.basename(name).split('.')[0]}`.empty?}
Plus some manual selection.
Some notes about deleted files:
- Revset benchmarks: Benchmarks on the Rust side is a cleaner choice.
- editmerge and hg-new-workdir: Newer versions live in fb/.
Reviewed By: xavierd
Differential Revision: D18541783
fbshipit-source-id: f08933d5c1a9c46d25322adbc2cc1e8a1b505d70
Summary: The commit timestamps in Git have second granularity. As a result, sometimes multiple succeeding commits have numerically equivalent timestamps. This means that sorting by timestamp may not result in the correct ordering of commits. This is particularly true because the sort we use is a stable sort, which means that commits with the same timestamp must go into the sort in ascending commit history order to have the correct order coming out of the sort. This commit updates the log that generates the list to use ascending commit order rather than descending.
Reviewed By: tchebb
Differential Revision: D18486839
fbshipit-source-id: 7a0ff099375a250d92b0f1a846e1105bf7bb9c86
Summary: The existing logic for parsing rev-list data relies very heavily on split, which creates many intermediate strings that take up a lot of memory. This commit updates the logic to use indices into the original output to extract individual strings. It also includes a new unit test for the list parsing. Big thanks to Tom Hebb for identifying the excessive memory usage.
Reviewed By: tchebb
Differential Revision: D18513949
fbshipit-source-id: c3b4e420ae2635904b42b84f2973e83c81c21dd7
Summary:
Revision numbers are deprecated, let's only show the short hash of the commit
instead.
Reviewed By: quark-zju
Differential Revision: D18519655
fbshipit-source-id: df277cc7e99ad747899d1fae2d92cd88eebea0f1
Summary:
This diff replaces code of the form:
```
use failure::Fail;
#[derive(Fail, Debug)]
pub enum ErrorKind {
#[fail(display = "something failed {} times", _0)]
Failed(usize),
}
```
with:
```
use thiserror::Error;
#[derive(Error, Debug)]
pub enum ErrorKind {
#[error("something failed {0} times")]
Failed(usize),
}
```
The former emits an implementation of failure 0.1's `Fail` trait while the latter emits an impl of `std::error::Error`. Failure provides a blanket impl of `Fail` for any type that implements `Error`, so these `Error` impls are strictly more general. Each of these error types will continue to have exactly the same `Fail` impl that it did before this change, but now also has the appropriate `std::error::Error` impl which sets us up for dropping our various dependencies on `Fail` throughout the codebase.
Reviewed By: Imxset21
Differential Revision: D18523700
fbshipit-source-id: 0e43b10d5dfa79820663212391ecbf4aeaac2d41
Summary:
The finddate function is used when using `hg update --date`, and since it
returned a revision number, the user would see the revision number deprecation
warning. Using nodes solves this.
Reviewed By: quark-zju
Differential Revision: D18486760
fbshipit-source-id: e1a51f624d8e7133fbf334f9b8b4b4c3d5aff2a0
Summary:
`globset` supports multiple matching strategies, including literal prefix
(backed by AhoCorasick), or regexp, etc.
In theory patterns like `foo/**` (where `*` cannot match `/`) can use `foo`
prefix strategy. However, the implementation detail of `globset` wouldn't
accept it as a prefix. But `foo/*` (where `*` can match `/`) can be treated as
a prefix. Transform the former pattern to the latter to hint `globset` to use
the optimal strategies.
Reviewed By: sfilipco
Differential Revision: D18500298
fbshipit-source-id: 39e604d6157a919b75c392488b6d42375e518c16
Summary:
This diff is preparation for migrating off of failure::Fail / failure::Error for errors in favor of errors that implement std::error::Error. The Fallible terminology is unique to failure and in non-failure code we should be using Result<T>. To minimize the size of the eventual diff that removes failure, this codemod replaces all use of Fallible with Result by:
- In modules that do not use Result<T, E>, we import `failure::Fallible as Result`;
- In modules that use a mix of Result<T, E> and Fallible<T> (only 5) we define `type Result<T, E = failure::Error> = std::result::Result<T, E>` to allow both Result<T> and Result<T, E> to work simultaneously.
Reviewed By: Imxset21
Differential Revision: D18499758
fbshipit-source-id: 9f5a54c47f81fdeedbc6003cef42a1194eee55bf
Summary:
simpkins encountered a case where the treematcher does not work with many
patterns. It turns out `globset` has a hard-coded regex size limit (10MB).
Implement proper error handling so we can detect such issues and fallback to
slower paths.
Reviewed By: sfilipco
Differential Revision: D18500299
fbshipit-source-id: 0122ba9b0246c1536b2069a40e13261ee47f8bba
Summary: Add `hg debuglfsgc` command to reclaim disk space. Make it part of `hg gc`.
Reviewed By: xavierd
Differential Revision: D18417035
fbshipit-source-id: 7c0a445a0d4405df5cff960c0e28c4fc5a1f2c31
Summary:
Even keep was being passed in, rebase still added mutation markers to the old
commits, which would then make them hidden in the smartlog output. Not adding
the mutation markers in this case solves the issue.
Reviewed By: quark-zju
Differential Revision: D18496840
fbshipit-source-id: f5d0f8920bd00041598841a65356b88df00708d7
Summary:
Merge the fb-mercurial code into the Eden repository, under the
`eden/scm` subdirectory.
Reviewed By: quark-zju
Differential Revision: D18445774
fbshipit-source-id: fc3307f9937e0c7e1c8f7d03c5102c4fe5dedb10
Summary:
In preparation for merging fb-mercurial sources to the Eden repository,
move everything from the top-level directory into an `eden/scm`
subdirectory.
