Summary:
See also D5271419. Decorators in `bundle2` and `exchange` are not side-effect
free. Move them to `uisetup` for better chg compatibility.
Reviewed By: StanislavGlebik
Differential Revision: D18926783
fbshipit-source-id: 9855b8debe3f809139a9fc3c6eea2775a1562cfd
Summary:
See also D5271419. Decorators in `bundle2` and `exchange` are not side-effect
free. Move them to `uisetup` for better chg compatibility.
Reviewed By: StanislavGlebik
Differential Revision: D18926787
fbshipit-source-id: cfbf8de6d8c841478d7aa3de180a0816349fa4e4
Summary: This looks like an omission from D18912817
Reviewed By: mitrandir77, ikostia
Differential Revision: D18935510
fbshipit-source-id: f72d95155c54d7eb36ff7c868f5e6b0ff6d6e39c
Summary:
The `reset` extension does its own obsmarker creation, rather than going
through `scmutil.cleanupnodes`. This means it doesn't support new-style
visibility tracking.
Fix this by making it use `scmutil.cleanupnodes`. This isn't completely
straightforward:
* The revset it uses to work out what to prune might accidentally include
public commits (e.g. when you reset to a different public branch). Make sure
these are filtered.
* The tests originally had strip and obsmarker based tests. The strip tests,
when converted to obsmarkers and using `scmutil.cleanupnodes` stop working
because they strip and revive the same commit over and over, which is an edge
case that obsmarkers can't handle well. Fix this by restoring the strip tests
as strip tests. A separate test handles the new-style visibility.
* Reset's behaviour is still a bit wonky. If an ancestor of the source commit
has other (non-bookmarked) descendants, then reset will try to prune those
commits. New-style visibility will ignore this, and this is tested in the
new test.
Reviewed By: farnz
Differential Revision: D18912817
fbshipit-source-id: cc115333407cf67d339c24fcd0807ddedce2660d
Summary:
D18820280 increased the log level for `globFiles()` calls from DBG3 to DBG2.
However, buck and other tools often end up huge numbers of glob calls, and
this ends up causing log spam.
This change reduces the log level of `globFiles()` back to DBG3. In a
subsequent diff I will update the code to increase the log level for slow
calls. This does mean that we will only get a log message for slow glob calls
after the call completes, but this is better than not getting a message at
all.
Reviewed By: pkaush
Differential Revision: D18924780
fbshipit-source-id: 86e83614f51424aca078e4de8a56c45018a6d286
Summary: was looking in this area and saw this TODO. moves the `setParentCommits()` call into `CheckoutConflict::finish()` so it is called while we are still holding the parents lock.
Reviewed By: chadaustin
Differential Revision: D18668329
fbshipit-source-id: 8415c792193e5b89737f15141f33c5a9799f527b
Summary:
This solve the following issue when running on devserver with demandimport enabled:
In [1]: ui.config?
abort: No module named windll!
Reviewed By: xavierd
Differential Revision: D18898009
fbshipit-source-id: bf4f3b12ac6f0a1628ed8ef805f6843868363542
Summary:
A spike in automatic GCs usually implies something has gone wrong. Log
an event for each one, recording the cache size prior to the GC and
the cache size after.
Reviewed By: simpkins
Differential Revision: D18902580
fbshipit-source-id: 158b2635733a415a9fcc7c412b2c0f44ed04aa01
Summary:
On Windows the error message is different. In this test we only need to verify
a command fails. Therefore remove the output and only keep the exit code.
Reviewed By: xavierd
Differential Revision: D18915580
fbshipit-source-id: bd1031170335918d507e6225b02541ec52bc62fa
Summary: Enabling getScmStatusBetweenRevisions to work on Windows and its tests.
Reviewed By: simpkins
Differential Revision: D18431271
fbshipit-source-id: eee82538e2fc3d7e371c96fc271cd9662ea6d737
Summary: This diff contains the tests for hg status. This is not comprehensive list of tests but the tests that we have working at this time. More tests will be added as we test and fix issues.
Reviewed By: simpkins
Differential Revision: D18454851
fbshipit-source-id: 80e3ff4d24873016dc420f202fbfe53fcffc24f2
Summary: The current implementation of EdenDispatcher is projectedfs specific and was linking against its dll. This was adding the need to add Prjfs dll for running our tests. Moving it to prjfs channel should fix that.
Reviewed By: simpkins
Differential Revision: D18462171
fbshipit-source-id: ff7bfcdd14d19552d49b61ff03cdd770badfcd68
Summary: This class provides an interface to create an EdenMount for testing and use it through the APIs. It's similar to TestMount for Eden on Posix.
Reviewed By: simpkins
Differential Revision: D18454852
fbshipit-source-id: 91d64b2c535060979ef634fe4e7779f50f169fc5
Summary: WinStore interface was taking wstring for path and we were converting wchar pointers to wstring which were later converted to RelativePath. In this diff we pass the path as is and save one copy per operation.
Reviewed By: simpkins
Differential Revision: D18444773
fbshipit-source-id: fa6a3efb6a7e91cf9b0dca919e80870e1bcf0804
Summary: Moved the current FsChannel implementation in to PrjfsChannel and converted the FsChannel.h into an abstract class. Created a TestFsChannel class for testing.
Reviewed By: simpkins
Differential Revision: D18440401
fbshipit-source-id: 30d6742cd18b4b76b664e3af7ed1a528cf876046
Summary:
A successful run of hg commit ends in call to resetParents() to reset the current parent commit.
We also need resetParent to run the Status tests.
Reviewed By: genevievehelsel
Differential Revision: D18439954
fbshipit-source-id: 44f7a009e050bd5e2130d8c4fc3a85a8ec3bfad1
Summary:
GenerateStatus will parse the CurrentState and compute the status. The algorithm here is little different than of Eden with FUSE, Eden with Prjfs will loop over the CurrentState and only compare the entries present in it. CurrentState has the list of all the entries modified and a complete status could be generated from it.
This doesn't solve all the scenarios. I have few unit tests running at the top of this stack and those cases are working. I need to test the other cases and make them work. One case that I already know doesn't work is when a user resets the parent and doesn't make any change. I will work on fixing the other cases and put up diffs for that.
Reviewed By: simpkins
Differential Revision: D18195573
fbshipit-source-id: 3650f26387f1a5f8d467fd81148c266fa1307a3c
Summary: This diff makes EdenFS to be able to use EdenApi to import blobs and trees when requested objects are not present in local hgcache.
Reviewed By: chadaustin
Differential Revision: D18605547
fbshipit-source-id: 4acd2e07cfd9de6b6775ded30ea22a4478b9f1e4
Summary: This will allow us to use EdenApi in EdenFS to fetch trees and blobs.
Reviewed By: xavierd
Differential Revision: D18622844
fbshipit-source-id: 59a9091e9f2fdbcae078da2fb24ee9c0dd18505b
Summary:
Add an option `experimental:use-edenapi` to `EdenConfig`.
See the next diff for usage.
Reviewed By: chadaustin
Differential Revision: D18605549
fbshipit-source-id: 2786c21bb38a76229078662cc5c1ddf906d1be4a
Summary:
I think we need to make `EdenConfig` available for testing as more things are using `EdenConfig` to configure behaviors.
Right now it does not do much things other than just construct an `EdenConfig` instance without manually providing these parameters.
Reviewed By: chadaustin
Differential Revision: D18798484
fbshipit-source-id: b4a85d966a9b4f425c547bef9eb6e7570d7e2170
Summary:
Two bugs conspired to cause edenfs after a graceful restart to think
the kernel supported FUSE_NO_OPENDIR_SUPPORT when it didn't: the
connection info struct wasn't zeroed, and FUSE connection capabilities
weren't properly mirrored into the Dispatcher upon graceful
restart. Fix both and add an integration test.
Reviewed By: simpkins
Differential Revision: D18903761
fbshipit-source-id: 23f4db3e240ee7d035f707820072c606a45f1138
Summary: Ideas for improvements. Put them in the code for future consideration.
Reviewed By: quark-zju
Differential Revision: D18870140
fbshipit-source-id: a2a84d94d72303adc64614b06e36588ff7778aab
Summary:
The list functionality is required by EdenFs. We want this functionality
to be well supported by the Manifest.
Reviewed By: quark-zju
Differential Revision: D18870143
fbshipit-source-id: 1ebaa713ff521226e6ace22cbd35cc841d967298
Summary:
Consistency in naming. The general idea is to have Metadata types that don't
contain paths. Then we will have File, Directory and eventually FsNode that
will contain paths.
Reviewed By: quark-zju
Differential Revision: D18870141
fbshipit-source-id: a1f09add7f1c3dd4fa0348693cd3ce2fd5767fa7
Summary:
This rename is going to make it easier to import and use outside of
manifest specific crates.
Reviewed By: quark-zju
Differential Revision: D18870142
fbshipit-source-id: 2f3ea460170308162ee834efc038b2dcedd9e233
Summary:
Since we got `repair` APIs in the Python land, use them to attempt to repair
things.
This would hopefully be able to fix some user-reported issues, usually
caused by hard reboots.
Reviewed By: xavierd
Differential Revision: D18737907
fbshipit-source-id: ef3deccbf4a1c959cff7e3eb78771acd9ae9bd01
Summary:
Register the logging and logging_full option handlers in the open
source and macOS builds of Eden.
Reviewed By: simpkins
Differential Revision: D18870545
fbshipit-source-id: 140debe77be071a9fa2d628ad305517273e095e3
Summary:
From Dulwich changelog:
0.19.0 2018-03-10
API CHANGES
GitClient.send_pack now accepts a generate_pack_data rather than a
generate_pack_contents function for performance reasons.
(Note: this ignores all push blocking failures!)
Reviewed By: quark-zju
Differential Revision: D18901269
fbshipit-source-id: 9510b3a516cb2fcbeae9f1f2072d8445e442a9ed
Summary: Adds a function which takes both the manifestID and the commitID to get a Tree. This will be used in `checkOutRevision()` and this allows us to skip looking up the manifestID since the caller can just pass it in themselves.
Reviewed By: wez
Differential Revision: D18719405
fbshipit-source-id: 919f0a7c84bff4a2f0bc20110c45bd272f9e9107
Summary:
It is very painful for Eden's automatic GC to delete trees from the
local store. An unfortunate number of programs scan the entire repo,
causing every tree to be loaded. If Eden's automatic GC kicks in, it
causes all trees to be downloaded again, blocking both FUSE and Thrift
requests.
Temporarily mark trees as persistent.
Reviewed By: fanzeyi
Differential Revision: D18842066
fbshipit-source-id: 4a22defcb1151912337d4167ac49852519c7789d
Summary:
`buck run :run_tests -- ...` was added to be a buck version of `run-tests.py`.
However it does not work because the watchman path provided was invalided.
Reviewed By: xavierd
Differential Revision: D18892361
fbshipit-source-id: 8745a33e0d19b14f042ef1cd255d10c5737dffb1
Summary:
Add `#chg-compatible` to 572 tests that seem to pass with chg enabled.
This should make them run faster.
Reviewed By: xavierd
Differential Revision: D18870507
fbshipit-source-id: fe895e733efffc9286cd3d17c7a156c803124395
Summary:
Previously, it's fine to migrate up to metalog, and fine to migrate down once
since we double write data. However, re-enabling metalog can be problematic
since there is no code path to do "migrate up" again.
This diff fixes the issue by tracking what files (or "keys") are using metalog
as the source of truth in metalog. So we can accurately figure out whether to
migrate svfs files to metalog on demand.
Reviewed By: xavierd
Differential Revision: D18864424
fbshipit-source-id: e61e1790c231f9c88de869f413f27bb954a29920
Summary: Expose the repair API to the Python world.
Reviewed By: xavierd
Differential Revision: D18737913
fbshipit-source-id: c31085727589b6938c2fafb28897925aea617bc4
Summary:
Provide a way to repair the MetaLog. It's not just repairing 2 indexedlog
structures, but also checking the relationships between them.
Reviewed By: xavierd
Differential Revision: D18737904
fbshipit-source-id: fcf8ae56a1fdbb0561765701d962dfad4a8b5bd4
Summary:
Detect `#chg-compatible` in test header and force enable chg. This would allow
us to mark tests as chg-compatible to get some speed ups.
Reviewed By: xavierd
Differential Revision: D18870508
fbshipit-source-id: 923d6cb10b0c621d9995809b60ae535126f6f2a5
Summary:
The chg logic is inlined in the "hg" binary so the old logic running "chg" as
an external binary needs to be changed.
The version check part can be annoying - it requires more frequent `make local`
to run the tests while `make local` is actually not necessary. So let's just
skip the version check if running in tests.
Reviewed By: ahornby
Differential Revision: D18870510
fbshipit-source-id: 2343549c4080a0425098698b5423c6cfc1f60c77
Summary:
We only support the "C" policy, therefore let's inline it and simplify the code
significantly.
Reviewed By: quark-zju
Differential Revision: D18819680
fbshipit-source-id: 0bc0a12b85ddc42af007d3a20c0f4e8497bedb21
Summary:
This is not available on Python3, instead stat.ST_MTIME and friends should be
used to get an integer time. This is achieved by wrapping the output of the
stat function and overriding the st_[acm]time properties.
Reviewed By: quark-zju
Differential Revision: D18819679
fbshipit-source-id: 2911f3d47506456e927872d57f69253d903618e2
Summary:
The environment variable `EDENSCM_TRACE_OUTPUT` specifies where to write the
tracing output for the current command. Environment variables are inherited
by subprocesses by default. That is undesirable because another hg command
(triggered by hook, or background maintaince) will rewrite the trace output.
Avoid it by unsetting EDENSCM_TRACE_OUTPUT for subprocesses.
Reviewed By: xavierd
Differential Revision: D18892973
fbshipit-source-id: 575b6c0df2e7a0775172893e4aa72ca33fa4658c
Summary:
Some users were reporting hg gc failures because it couldn't access a
hidden commit. It looks like there's a `repo.unfiltered().revs("draft()")`
line to get the list of draft commits, so we need to also use an unfiltered repo
when extracting all the pointers.
Reviewed By: quark-zju
Differential Revision: D18863471
fbshipit-source-id: 61220c1ec3dcbf55879211bb271b4d9b686d66a1
Summary:
D18732257 changed our fastlog integration from using conduit to using
scmquery. This changed the result type from a hash to a dictionary of {"hash":
"xxx"}, but the consumer code wasn't updated appropriately. This fixes it.
Reviewed By: mitrandir77
Differential Revision: D18833566
fbshipit-source-id: 2573cbf4f93c5e4aeb2515d42696526ceb51428e
Summary: Saves loadFileFromPath in the DiffContext. This allows the source control differ ( at `eden/fs/store/Diff.cpp`) to call this function while it is trying to load gitIgnores and resolve symlinks.
Reviewed By: simpkins
Differential Revision: D18647088
fbshipit-source-id: bddff0158130f0c96ac263bcce81dc51fc017800
Summary: Adds a helper function that can take a path to an Inode and load a file by simply getting the Inode then calling the existing `loadFile()` function
Reviewed By: simpkins
Differential Revision: D18647090
fbshipit-source-id: 89c25bef10a7a0d0ffd660293f85479ac0e9f9a9
Summary: Move the bulk of the gitignore loading to EdenMount, this is needed later in the stack to pass this function as a callback through the DiffContext to enable gitignore loading from the source control tree differ.
Reviewed By: simpkins
Differential Revision: D18565265
fbshipit-source-id: 062349d81bbfb78235ccb34d6ec72ad445a6e2b0
Summary:
The previously behavior silently uses the empty Root upon "id not found" cases,
which hides data corruption issues captured by the next changes.
Reviewed By: xavierd
Differential Revision: D18737906
fbshipit-source-id: 0f2cfe1b05f11e9d2584d96a14afcb1aed80da4d
Summary:
Recently many tests become unreliable due to MySQL errors like:
OperationalError: 2055: Lost connection to MySQL server at '<host:port>', system error: 32 Broken pipe
That makes the testing infra think our tests are broken and cause:
# Test Failure Bot
Test is now disabled because it has been consistently broken
Avoid that by detecting MySQL errors and mark tests as skipped.
Reviewed By: simpkins
Differential Revision: D18668303
fbshipit-source-id: 02cb09f31c712fed41903f1e8c4ad45a1d277c30
Summary:
It was used to highlight "more interesting" (aka. extension) code. Nowadays the
non-extension part of code is also interesting. Therefore drop the extension.
Reviewed By: simpkins
Differential Revision: D18666053
fbshipit-source-id: 027190f742015bdb9e7dba109e7a28987af3cf68
Summary: Otherwise the repair will fail with "unable to lock directory" errors.
Reviewed By: xavierd
Differential Revision: D18737911
fbshipit-source-id: 2c47b5b9a071e0db2d20fc385ef466aef1e732f6
Summary: This makes it possible for other crates to implement the `Repair` trait.
Reviewed By: xavierd
Differential Revision: D18737909
fbshipit-source-id: 3d098e583b45e84eb4bd01ef45569e9694b63ed6
Summary:
This simplifies the logic a bit by getting a free `repair` method from
indexedlog.
Reviewed By: xavierd
Differential Revision: D18737908
fbshipit-source-id: 4988c1a83b7709b751cd1899c5663acc0c42e313
Summary:
The goal is to make it easier for different structures to implement `repair` by
just providing `OpenOptions`.
Reviewed By: xavierd
Differential Revision: D18737903
fbshipit-source-id: af3c18038c5bb2f91f53ab53926ec097e1d87e6b
Summary: Uses `folly::IOBuf`'s `TakeOwnership` instead of `CopyBuffer` to achieve zero-copy of file content we get from Rust.
Reviewed By: chadaustin
Differential Revision: D18264900
fbshipit-source-id: 45baa70cb771a2bff0b31feaa1b85569625218eb
Summary:
This isolates the core types in manifest so that it is harder to
create unsound dependencies with specific implementations.
Reviewed By: quark-zju
Differential Revision: D18843133
fbshipit-source-id: 4b866ad84d2e7d0ff2dc4ec6bd65f66548c3fe4a
Summary: Signal more clearly that the iteration in the cursor is depth first.
Reviewed By: quark-zju
Differential Revision: D18843135
fbshipit-source-id: 8ee2c612963eaf6fec3ae48f75ebf16bd177fed4
Summary: Items feels a bit generic as a name.
Reviewed By: quark-zju
Differential Revision: D18843136
fbshipit-source-id: 29a904f3a8a1996760e71bd78610bdbfef4ad65b
Summary: No longer strictly related to files.
Reviewed By: quark-zju
Differential Revision: D18843128
fbshipit-source-id: c7853e08aa5d0256616e777c5bb6dd477b9ebc97
Summary:
`hgid` can be easily derived from the link member variable. Using link is
preferable because link is the owner of the data. It makes the logic more
specific, less bug prone to use the true source of the data.
Reviewed By: quark-zju
Differential Revision: D18843137
fbshipit-source-id: 443f1e5eb4498fa4f2ff9904dba7cc3dc4305add
Summary:
This structure is not public. It is private to the manifest crate so there is
no point to have those modifiers.
Reviewed By: quark-zju
Differential Revision: D18843134
fbshipit-source-id: 0794bbbf578863cabbd85ada87669b791ebcc72d
Summary:
The purpose of this change is to disambiguate from the other Directory struct
that we have in the Manifest. Right now that other is in manifest/lib.rs
Reviewed By: quark-zju
Differential Revision: D18843130
fbshipit-source-id: f25a60f3040cc19abcab99ac5f2cdbff83ccf7ee
Summary:
Directory<'a> which wraps a link is closely linked with the Link abstraction.
It can be seem as a convenience wrapper around Link.
Reviewed By: quark-zju
Differential Revision: D18843129
fbshipit-source-id: 2b677d71101bff3708a65e03bbcddf5074d51387
Summary:
Diff is a core operation for manifests. The Manifest trait should require that
it is implemented.
Reviewed By: quark-zju
Differential Revision: D18843132
fbshipit-source-id: 5e3febab669470e266878d934561dfc323b71c7f
Summary:
A previous change to the BFS iterator fixed it to return the root of the repository.
This influenced the `debugmanifestdirs` command which was manually
handling the root of the repository. This diff updates the command
to match the old behavior. Basically after the fix to BFS we are
not printing any characters for the path of the root. Forcing the debug command
to print of a '/'.
(Note: this ignores all push blocking failures!)
Reviewed By: quark-zju
Differential Revision: D18862167
fbshipit-source-id: 9ca6da707ae8ab037e88a2bbce3b3993b5d219e3
Summary:
In most cases nowadays people run into worse experiences when their operations
run through the fsmonitor path. Watchman is reliable enough that we can rely
on it to work better than the default crawling algorithms.
Reviewed By: quark-zju
Differential Revision: D18835322
fbshipit-source-id: 30873c7d4b1ec306e59b6152c20833cc0027cad5
Summary:
This is a test that aims to validate that commit manifest match between the C++
and Rust implementations.
Reviewed By: quark-zju
Differential Revision: D18521823
fbshipit-source-id: 678b82239e9be7ad078f1e04a6467e87adaac966
Summary: The conversion from tuple is not used much. I don't like them that much either.
Reviewed By: quark-zju
Differential Revision: D18843131
fbshipit-source-id: fb06b72a279b59ee8af5e43d83a10e39a576730b
Summary:
Querying all the files in a Manifest is core functionality that every Manifest
implementation should implement. The main question is how do we describe that.
We want to return an interator over the files of the manifest. The problem is
that the manifest is a trait and functions on traits can't return traits.
The solution is to Box the results. Not pretty but allows us to move the code
forward.
Reviewed By: quark-zju
Differential Revision: D18822854
fbshipit-source-id: ca44b24816203049f747ad67af48e33370861024
Summary:
The purpose of the rename is to signal that this function may potentially do
a lot of work, even calling and waiting on network requests.
Reviewed By: quark-zju
Differential Revision: D18822855
fbshipit-source-id: 98471944c8e6c7cffa7a2a11f2f733274802b376
Summary:
Removing the contructor for File that takes dependency on
tree implementation details.
Reviewed By: kulshrax
Differential Revision: D18822858
fbshipit-source-id: aed33b146dfdbdb23721a0f6e96977025d71a627
Summary: The types in file.rs being in a separate file confuse people.
Reviewed By: quark-zju
Differential Revision: D18822856
fbshipit-source-id: 29aa10c76bb5ef74f1fc0bf726e6b3cfaf19b428
Summary:
For now pushrebase updates working parent, bookmarks. This diff makes it also
update remote bookmarks.
This resolves a race condtion in remotenames where the pulled remote bookmarks
have unknown nodes and cause update to silently fail.
Reviewed By: sfilipco
Differential Revision: D18601035
fbshipit-source-id: 6f2c318cbf6b29a6427eeac6d374c1fb13e6155c
Summary:
Enable visibility and mutation to get modern features. Remove legacy configs.
As we're here, add more `log -G` outputs to make the test easier to reason about.
Reviewed By: xavierd
Differential Revision: D18601036
fbshipit-source-id: f0d656c696708c045c63f5b8e59b9a40f2dbbf20
Summary:
Move it to configuration.
This gives better control over how grep is executed. Previously the assumption
was that xargs was present in PATH. This updates removes that assumption.
Reviewed By: quark-zju
Differential Revision: D18715471
fbshipit-source-id: 09c3098984588cd7478172db0bcfb8d5cb8243fd
Summary:
`reachableroots(roots, heads, includepath=False)` returns a subset of `roots`.
I implemented `reachableroots` using the documented revset using segmented
changelog APIs and noticed smartlog graph is not correct. This diff fixes
the documentation.
This diff was also sent upstream as https://phab.mercurial-scm.org/D7518.
Reviewed By: ikostia
Differential Revision: D18723363
fbshipit-source-id: ed56ae85345dcf102159c4df0eb19c02cef2bf27
Summary:
The `update` hook gets called even for in-memory updates that does not affect
real filesystem (or edenfs). That hook is set to `edenfsctl redirect fixup`,
which adds 0.35s overhead per commit when doing in-memory rebase.
Define a edenfs-only update hook to remove the overhead.
This is part of a stack of optimization that makes a 20s rebase to 5s (0.6s per
commit to 0.07s per commit).
Reviewed By: xavierd
Differential Revision: D18838470
fbshipit-source-id: cbe81f9dd89bff6d7c4bec3063d6014211f9aac6
Summary:
With narrow-heads before the new dag structure, reading phases is expensive.
Avoid it.
This is part of a stack of optimization that makes a 20s rebase to 5s (0.6s per
commit to 0.07s per commit).
Reviewed By: xavierd
Differential Revision: D18838468
fbshipit-source-id: 0847d2508114203f1d16a7240a43375d325103a5
Summary:
With narrow-heads, rebase operation does not need to read existing phases to
decide new phases. Therefore remove it.
This is part of a stack of optimization that makes a 20s rebase to 5s (0.6s per
commit to 0.07s per commit).
Reviewed By: xavierd
Differential Revision: D18838469
fbshipit-source-id: 1298d4f4b0d72a764a286478e76c90b42b3fc569
Summary:
Use all available lines, now that we ignore the spurious errors that
can be thrown when calling `addnstr()` on the last line.
Reviewed By: wez
Differential Revision: D18826918
fbshipit-source-id: 7e5739207ba1a5bfa0d290da1345c9d9606b89fb
Summary:
Add a wrapper function around `addnstr()` that silently ignores any errors.
When writing to the last cell of a terminal curses will successfully display
the data, but report an error since the logical cursor cannot be moved to the
next cell. This error is not important and can be ignored.
The errors reported by the `curses` module unfortunately are not very useful
(they normally just report `addnwstr() returned ERR`). For now we silently
ignore all errors, although this diff does contain some stub code that can be
enabled to examine them during development.
Reviewed By: wez
Differential Revision: D18826917
fbshipit-source-id: a4c5394b49aea1334def1d047eec7b5e743fb9f2
Summary:
In the future we might have revlog or more complicated storage for maintaining
the IdMap. Define a trait for it.
Reviewed By: sfilipco
Differential Revision: D18664208
fbshipit-source-id: 242611d78852e8d82a52ab52192db87c13b848e7
Summary: This is the reverse of first_ancestor_nth. It will be used later.
Reviewed By: sfilipco
Differential Revision: D18664209
fbshipit-source-id: 8d3706cdc080086ec60da60870fde76c8933f6a8
Summary: This will be used in an upcoming change.
Reviewed By: sfilipco
Differential Revision: D18664207
fbshipit-source-id: 3030bb02e172b95c6dbabda74c318e7f16212f59
Summary:
With the ONLY_HEAD flag, ancestors can be sped up sometimes. Since ancestors
is used widely in other DAG calculations, perf improvements are visible across
the benchmark. The trade-off is to spend a bit more time on building segments
to figure out ONLY_HEAD.
After this diff:
building segments 576.625 ms
ancestors 47.119 ms
children 436.967 ms
common_ancestors (spans) 257.096 ms
descendants (small subset) 367.316 ms
gca_one (2 ids) 146.315 ms
gca_one (spans) 253.059 ms
gca_all (2 ids) 236.075 ms
gca_all (spans) 266.695 ms
heads 220.766 ms
heads_ancestors 36.258 ms
is_ancestor 98.967 ms
parents 219.173 ms
parent_ids 9.282 ms
range (2 ids) 99.859 ms
range (spans) 205.154 ms
roots 429.047 ms
Before this and previous diff:
building segments 542.564 ms
ancestors 316.075 ms
children 428.612 ms
common_ancestors (spans) 452.850 ms
descendants (small subset) 537.483 ms
gca_one (2 ids) 526.112 ms
gca_one (spans) 446.726 ms
gca_all (2 ids) 780.590 ms
gca_all (spans) 490.906 ms
heads 214.005 ms
heads_ancestors 217.558 ms
is_ancestor 256.769 ms
parents 213.170 ms
parent_ids 9.313 ms
range (2 ids) 259.435 ms
range (spans) 358.934 ms
roots 423.005 ms
Reviewed By: sfilipco
Differential Revision: D18642797
fbshipit-source-id: 27f1ee5797a92142b2c19195eb26baef58769553
Summary: The pre-calculated flag can be used for some fast paths.
Reviewed By: sfilipco
Differential Revision: D18642796
fbshipit-source-id: bc16b3517cc29e23478f2027e83e2a2282be0d0a
Summary:
The bindings APIs are out of date.
The latest idea is to write non-master commits to disk, in a different group.
There are no "segments in memory". Therefore:
- Rename `builddisk` to `build`.
- Make sure `build` updates internal states by calling `reload`.
- Drop `buildmem`.
- In case `builddisk` gets called very frequently, reduce its overhead by
doing nothing if all commits are built already.
The constructor is revised to match the latest interface (no
"max segment level" and "set_new_segment_size" needs to be called).
Reviewed By: sfilipco
Differential Revision: D18640897
fbshipit-source-id: 76a5d0db363713dc5b763503c238c1bfa61ea02e
Summary: Make it possible to build non-master segments.
Reviewed By: sfilipco
Differential Revision: D18641507
fbshipit-source-id: 4d0c46cdea240849af37d28219b6023578514323
Summary:
`build_segments_volatile(id)` should build flat segments so it includes `id`.
Right now it can fail that in a corner case. Fix it.
Reviewed By: sfilipco
Differential Revision: D18641509
fbshipit-source-id: ad1de694e4d31c51f29e9bc5800291066bf16dbb
Summary:
The IdMap has assertions about 1:1 mapping between Id and Slice.
Practically, a non-master commit can be merged into a master commit.
That means a commit hash might map to 2 Ids: [master-id, non-master-id].
Allow that to happen (temporarily).
Note: The non 1:1 mapping causes more headache than benefit and
D18838995 restores 1:1 mapping. But it still requires temporary
non-1:1 mapping to work.
Reviewed By: sfilipco
Differential Revision: D18641508
fbshipit-source-id: bc4bc831d5ab5ebfc2588bafd48114ae49d31e0f
Summary: fix the `build-eden-thrift-deps.py` to generate a new zip file, upload it to lfs, and point setup.py to use this new generated thrift code
Reviewed By: quark-zju
Differential Revision: D18836545
fbshipit-source-id: 52c58c50319c878f2f6f708b952f0aa158d2ef4e
Summary: panic on drop is a terrible idea. It can cause SIGILL kill without meaningful messages or meaningful backtrace. Remove it.
Reviewed By: xavierd
Differential Revision: D18640894
fbshipit-source-id: 3ab52ded9875e17e891acd8f9b45d82964615f01
Summary: This makes the code harder to crash in abnormal cases.
Reviewed By: sfilipco
Differential Revision: D18640895
fbshipit-source-id: ee81dd7739739d3d15052bdb7bb2ba80ed48cd0a
Summary:
This allows us to allocate Ids for non-master commits in a separate range
so they wouldn't cause fragmentation for commits in the master group.
Reviewed By: sfilipco
Differential Revision: D18640896
fbshipit-source-id: c19c53342583ded8795849584cf5f31bc86b2818
Summary: Migrate the codebase to use the new Id type instead of an alias of u64.
Reviewed By: sfilipco
Differential Revision: D18640898
fbshipit-source-id: 415db9afeda1f8960e18c69e70f53a9eeb63cbbe
Summary:
Background:
- Tests and Nuclide haven't removed usage of revision numbers.
- If we build segments for non-master commits on demand, there are performance
concerns about very long release branches.
- It's very desirable to keep `ancestors(master)` continuous. In other words,
`ancestors(master)` is a single integer span. This affects performance for
DAG algorithms a lot.
To solve the problems, I'm going to use separate ranges for master and
non-master commit Ids (so assigning Ids to non-master commits wouldn't affect
what numbers can be used for the master group). To make the code easier to
read, I'm changing `type Id = u64` to `struct Id` where the struct is
group-aware.
This diff introduces related types and basic operations.
Reviewed By: sfilipco
Differential Revision: D18640899
fbshipit-source-id: 8f7ecbee6e0d1703d1225f18e92742981dbf3757
Summary:
sysplatform is defined as a byte string, and thus a byte string needs to be
provided.
Reviewed By: ikostia
Differential Revision: D18816931
fbshipit-source-id: c653f1319f160bc00d7a1c90b06ff3b4f4d035ad
Summary:
In Python3, unicode doesn't exist, instead of re-inventing the wheel, just use
unicode from the pycompat module.
Reviewed By: ikostia
Differential Revision: D18816929
fbshipit-source-id: d3ec9ae4f2190db64866568b1945b26e05b9c2a5
Summary:
Somehow, pyre isn't able to inline the value of ispy3 and propagate it properly
and thus complains about lots of code paths. Let's inline it everywhere.
Reviewed By: ikostia
Differential Revision: D18816333
fbshipit-source-id: 80f81cb91bef4c6831952f9469401d87040335e2
Summary:
The None value has an Optional type, but was overwritten by a string in the
subclass, let's just change the base value to an empty string.
Reviewed By: ikostia
Differential Revision: D18805103
fbshipit-source-id: 30113d5b4394163554b340917f3cf32178e4fe9e
Summary:
The copy method is inherited from list and pyre requires proper type
annotation.
Reviewed By: ikostia
Differential Revision: D18805104
fbshipit-source-id: efe828c6033af48754c0ef28ee52340616582e23
Summary:
The run method is inherited from Thread, and pyre requires proper type
annotation.
Reviewed By: ikostia
Differential Revision: D18805026
fbshipit-source-id: 4f9addc5d1d9655f3796342d99366df9f0cfe102
Summary:
The branchmap type inherit from dict and override the copy method, pyre
requires types for it.
Reviewed By: ikostia
Differential Revision: D18805028
fbshipit-source-id: 43de36df6f40e5a3485e83e8056b2a70cdebcfac
Summary:
These methods are inherited from Thread, and pyre wants proper type annotation
in this case.
Reviewed By: ikostia
Differential Revision: D18805027
fbshipit-source-id: 1d479c7af4da74220be3e175c8d1c0803da6d9e4
Summary:
Increase the log level for messages about `globFiles()` thrift calls so that
they are enabled by default in our normal production configuration.
`globFiles()` calls can potentially cause EdenFS to fetch a lot of data, so it
is useful to be able to see them in the logs when investigating performance
issues.
Reviewed By: genevievehelsel
Differential Revision: D18820280
fbshipit-source-id: e351a9541e420f5eb47ca14d6620366437773ce4
Summary:
This message is pretty noisy when running EdenFS with an increased debug
logging level. Bump the debug level down so it doesn't spam the logs when
running at a slightly elevated debug level. Also fix a minor typo.
Reviewed By: fanzeyi
Differential Revision: D18820281
fbshipit-source-id: 668367337f4a8cf2a38003c8e0034969d335c2c0
Summary: There are a couple of functions in `store/Diff.h` which are not used elsewhere except for testing. These functions are tested by `diffCommitsForStatus()` anyways so these standalone functions are not needed.
Reviewed By: chadaustin
Differential Revision: D18690006
fbshipit-source-id: f2b24575c17403d7241896f35f4e0e16bb03b7ce
Summary: This updates the hg and telemetry wrapper callsites of getScmStatus to first try running getScmStatusV2() with fallback option. This does not retry `hg status` while a checkout is in progress.
Reviewed By: simpkins
Differential Revision: D18209899
fbshipit-source-id: e7a77b902f5a0ee624e4ea3185a1901bdac090e6
Summary:
When changing a file to a folder in a commit this function would try to query
the storage using the path and the id of the file. Storage would not have
such an entry and would cause the operation to fail.
Fixing the issue by filtering out ids of files in the section for known
manifests. Filtering already occurs in the section for transmitted manifest.
Reviewed By: quark-zju
Differential Revision: D18797368
fbshipit-source-id: 2ed727b5eaaab2f17cd988d6c8a3a1055ac8bd2a
Summary:
Each entry in InodeMap::unloadedInodes_ stored the InodeNumber
twice. Remove one of them.
Reviewed By: wez
Differential Revision: D18651007
fbshipit-source-id: be85c34cb2b38fc0b2875d0874cecd1ef274aca4
Summary: Small readability improvements I noticed while reading the diff algorithm.
Reviewed By: genevievehelsel
Differential Revision: D18650987
fbshipit-source-id: 19346362711455ae87d6770812e647544a4576bf
Summary:
On the LFS Server, we've noted that LFS batch requests from Mercurial have
about ~40ms latency, but that the same request sent through curl doesn't.
Those requests, when sent through curl, complete in < 1ms, so thats a lot of overhead.
After adding more logging in the LFS server (D18636703), it turned out that that is
because the body of the request shows up 40ms after the request itself.
After capturing the traffic on the backend (D18689109), it turned out that
Mercurial is sending the headers and the response body in 2 separate TCP
packets.
The reason for that is delayed TCP acks. What happens is that when we call
`h.endheaders()`, that calls `send(2)`, which in turns send the packet with the
headers. Later, when we call `h.send(data)`, we make another `send(2)` syscall,
but that doesn't go over the wire immediately: because of Nagle's algorithm, we
wait for an ACK for the first packet we send before sending a new one, and
because of delayed ACKs, the ACK takes 40ms for this ACK to show up (which is
over 40 times the roundtrip latency!).
This diff passes the data to `h.endheaders()`, which in Python's httplib will
concatenate the headers and the data into a single `send(2)` syscall if the
data isn't a file-like object (which it isn't for batch requests), and
otherwise fall back to doing exactly what we're doing right now.
This will result in everything in a single packet if we're fetching a single
file (and AFAICT, without further delay if we're fetching enough files that the
request doesn't fit in a single packet).
An alternative approach would be to set `TCP_NODELAY` on our socket here, but
for now this seems less intrusive and just as effective, so I've opted for that
approach.
Reviewed By: farnz
Differential Revision: D18811419
fbshipit-source-id: 4cf5719e5eed90e5dd994e6c8861aceb69373d89
Summary:
The Mercurial codebase contains over 500 errors, let's ignore them for now, we
can go back to them later to fix them.
Besides the manual change to .pyre_configuration.local, the changes were
generated with:
pyre --output=json check | pyre-upgrade fixme
Reviewed By: singhsrb
Differential Revision: D18803908
fbshipit-source-id: 724db7bd864c0de47a97ef2092bdee9f2cda531f
Summary:
The line causes pyre to enter an infinite loop, for now, let's simply ignore it
when type checking is enabled.
Reviewed By: singhsrb
Differential Revision: D18803909
fbshipit-source-id: d89b4cd0311a4a5416dd31197a8c69f4a6b65944
Summary:
Now that we've transitioned to the newer redirections
configuration we can remove the older bind mounts configuration
parsing, and that is what this diff does.
This is helpful because in situations where the user has run
`hg co null` as part of some troubleshooting advice, the legacy
bind mounts would remain mounted and obstruct some of the
steps that would have fixed up the users repo.
Reviewed By: pkaush
Differential Revision: D18337246
fbshipit-source-id: 23f27787d609e1c38a9c98b8b6596bb40743b9ca
Summary:
When user types Dxxx as a revset locally they usually mean the latest version
of the commit - not neccesarily the one in phabricator. This usecase was
usually handled by doing local lookup which can be very slow in case of slow
commits: see for example those user complaints:
https://fb.workplace.com/groups/scm/permalink/2487795837936688
Reviewed By: farnz
Differential Revision: D18809252
fbshipit-source-id: b3442d6fa2ef9c9c0dff4909c874689810fbfa88
Summary:
The intent is to allow macOS to make a best effort attempt
to detach any disk images associated with redirections.
D18795800 taught redirect to detach rather than a simple unmount,
and we need to give it an opportunity to detach when we unmount
gracefully so that all resources are released on shutdown.
The implementation of this diff ties into the same mechanism that
we use to shutdown the buck daemon when we unmount or restart which
has the nice side effect of showing the output from the underlying
`eden redirect unmount` command.
In an earlier iteration of this diff I tried making the server code
run `eden redirect unmount` but it happened too late in the process
to be effective: the thrift server had already been torn down in the
shutdown case and we need privs to perform the unmount.
Reviewed By: pkaush
Differential Revision: D18804080
fbshipit-source-id: 0b409130752121c56a46c9b2e46b50e5abee8200
Summary:
The purpose of this command is to unmount/unlink any configured
redirections without removing their configuration.
The intent is to call this for a mount when we are unmounting; I'll do
that bit in a follow on diff.
Reviewed By: pkaush
Differential Revision: D18801872
fbshipit-source-id: 096d9595091da72aa85f4259cbab022a1fe0c01f
Summary:
unmount will unmount from the VFS but still leave the image
associated with the disk image machinery and visible to `hdiutil info`.
This leaves resources associated with the mount even after it has
logically gone away, and that is undesirable.
This diff switches to using `detach` instead which unmounts and removes
the image from the disk image state.
The effect of this is that running `hg co null ; eden redirect fixup`
will completely unmount the redirections.
However, it doesn't cause them to be unmounted when the eden daemon is
stopped: that is something to address in a separate diff.
Also worth noting: only HFS+ mounts get detached successfully using this method. APFS mounts will remain regardless, so its another reason for switching to HFS+ in D18795799.
Reviewed By: pkaush
Differential Revision: D18795800
fbshipit-source-id: dfc86d86016a0c78e67f6ae2651db681669a5b14
Summary:
Update Eden's CMake build to use the new `install_fb_python_executable()`
function to install `edenfsctl`
Reviewed By: wez
Differential Revision: D18774538
fbshipit-source-id: 462c0127d79edcb6235a629cb97ea481493e1906
Summary: Log files that required manual merge during a rebase to dev command timers so we have an idea of how often they happen and on which files
Reviewed By: simpkins
Differential Revision: D18648175
fbshipit-source-id: 83ffe6d9177ca00b8fd1095745c585186bc2c8e9
Summary:
We've seen occasional weirdness with "resource temporarily unavailable" when working with disk images.
The internet, in the form of this stackoverflow post: https://stackoverflow.com/q/48368389/149111
suggests that this is related to APFS.
This commit switches over to HFS+ in the hope that we won't see it any more.
Reviewed By: chadaustin
Differential Revision: D18795799
fbshipit-source-id: 68e08e852770e311bcf04a8d12cb20670babf889
Summary:
These are not currently used by mercurial and are superseded
by the newer more general `watchman_client` crate on crates.io:
https://docs.rs/watchman_client/0.2.0/watchman_client/
Some of the implementation of that crate was based on the work I'm
removing here.
I was going to update hg_watchman_client to be implemented in terms
of the new crate, but since it isn't used and it would only be a thin
wrapper, I figure that it is best to delete this code instead so that
we can integrate it in terms of the new API when we're ready.
Reviewed By: singhsrb
Differential Revision: D18777161
fbshipit-source-id: efacb749e6d35ff4cc1fdc7d99cdca9ed42c9cfe
Summary:
Since it's not calling conduit anymore we can remove all conduit related
functions.
Reviewed By: quark-zju
Differential Revision: D18733121
fbshipit-source-id: 20a6022d9ac7ae6e9afea2791a8daac57cefb22c
Summary:
conduit is going away, phabricator team wants us to stop using
it in hg.
Reviewed By: quark-zju
Differential Revision: D18732257
fbshipit-source-id: c4faf83e36af80fc616e91adede141ba12de5446
Summary: conduit is going away, phabricator team wants us to stop using it in hg
Reviewed By: quark-zju
Differential Revision: D18727399
fbshipit-source-id: c208f5fec5fefd83aa37629b56d2bbaa2532d30e
Summary:
It would be nice if the insert method would give back the path that
it failed to insert.
Reviewed By: dtolnay
Differential Revision: D18739978
fbshipit-source-id: 4c16d09750ade2f01397161129c31bcf0059a957
Summary:
The Mercurial code has the bad habbit of inserting files in what can
appear impossible locations. Sometimes files with directory names and
sometimes directories in file locations. This happens because the initial
code would do additions before deletions on Manifest implementations that
were rudimentary. As soon as we introduce validation various code paths
surface.
I tried to fix the codepaths that modify manifests but it's a losing game.
I fixed the issue that appeared in tests and a couple of issues people
reported but more situations crop up.
This is giving up on the python code.
Reviewed By: quark-zju
Differential Revision: D18737678
fbshipit-source-id: 0c97128ff67e5ba2334942b6afc404aa64a2e5f4
Summary:
In NFS v4.x, flock ends up inheriting some semantics of fcntl,
including that write locks can only be created against file handles
opened for O_WRONLY or O_RDWR.
This diff changes the mode of 'lockpath' to O_RDWR to solve this.
Reviewed By: simpkins
Differential Revision: D18714734
fbshipit-source-id: 84ba4a6a5de034a4942b1ca3aa8b0c92f882ce38
Summary:
`path = ...` is enough. This should resolve the following build issues:
error: failed to select a version for the requirement `serde_bser = "^0.1.0"`
candidate versions found which didn't match: 0.2.0
location searched: ...
required by package `watchman_client v0.1.0 (...)
... which is depended on by `hg_watchman_client v0.1.0 (...)`
Reviewed By: lukaspiatkowski
Differential Revision: D18738176
fbshipit-source-id: 725840895a8e988b35000a48cf92018b14cb4dee
Summary:
Since bookmarks are stored in svfs, track it in metalog for better transaction
support.
This affects some transaction / rollback behaviors. Since our plan is to remove
`hg rollback` eventually, the compatibility is intentionally broken.
Reviewed By: xavierd
Differential Revision: D18538147
fbshipit-source-id: 6070772a4bce88d3808547207f5df194696b1b52
Summary:
Existing tools (ex. shell complete script) still read from
`.hg/store/{remotenames,bookmarks}`. Be compatible with them by double writing
content to both metalog and filesystem. Metalog is the source of truth for hg.
Reviewed By: xavierd
Differential Revision: D18524106
fbshipit-source-id: c37bc86b7443bba1b65735f29243767650b22ecd
Summary:
Similar to D17199834, by moving 'bookmarks' from localvfs and sharedvfs to
storevfs, we can make it part of metalog and get better transaction support.
Reviewed By: xavierd
Differential Revision: D18524104
fbshipit-source-id: ae148c1f02fc83b5c2d73102ecab39ff223ea5df
Summary:
I'm going to move `bookmarks` to "metalog". So operating on the `.hg/bookmarks`
file no longer works.
Reviewed By: xavierd
Differential Revision: D18524105
fbshipit-source-id: dc31b13e1acc171d2e8b32cdfea7028faf6dc4d3
Summary:
Right now `bookmarks` is stored at both "local" vfs and the "shared" vfs.
In an upcoming change I'd like to move bookmarks to the "metalog" to get
better transaction. Therefore make it always shared.
The shared bookmark feature is turned on by default. Assuming nobody manually
turns it off, production hosts all have shared bookmark turned on.
Reviewed By: simpkins
Differential Revision: D18524103
fbshipit-source-id: 3160146f357dd01f2d78f5d2f4a14838f8f12a94
Summary:
We have seen cases on Windows where the hg process gets stuck in
CreateToolhelp32Snapshot. Let's be defensive and always exit the loop.
Reviewed By: singhsrb
Differential Revision: D18729720
fbshipit-source-id: fb8602ce231eec01b6b42c6759849d56e5db2030
Summary:
The only valid place that metalog can be changed is inside a transaction, since
it's transaction.{close,abort} that writes or discards metalog changes.
In other words, metalog should not be changed during A, B, or C:
|<- A ->|<----------- repo lock --------->|
|<- B ->|<- transaction ->|<- C ->|
Add detection for them.
Reviewed By: xavierd
Differential Revision: D18538143
fbshipit-source-id: 036286ed32a897fe3ce0a91c1e3c848cc6167b1d
