Summary:
The linknode is part of the history, not the data, no need to force prefetch
the blobs.
Reviewed By: quark-zju
Differential Revision: D19394351
fbshipit-source-id: 2e63d82928ebd5cf8e3a0d1b87d115b45d1428a7
Summary:
Mercurial assumes some specific ordering, let's comment about this to make sure
we don't re-order and introduce subtle bugs.
Reviewed By: quark-zju
Differential Revision: D19394352
fbshipit-source-id: 0f9e02d2c6addf040311a54b8161b06bbeaa6be9
Summary:
The error message and the exception type are slightly different between the
Rust and Python ContentStore. For now, let's just fix this up manually.
Reviewed By: quark-zju
Differential Revision: D19394350
fbshipit-source-id: e432094a9dfcf605568a1890c0303b733e98d203
Summary: The rust telemetry wrapper and the python code were not in sync in how they presented errors to the user on failure. This makes them identical in message and error code (however the "abort" from python is red in color).
Reviewed By: quark-zju
Differential Revision: D19313848
fbshipit-source-id: d5b95e8d5130f20fc85c6664933ee18702f7cc5e
Summary: This eliminates the broken pipes that have been more prevalent recently.
Reviewed By: DurhamG
Differential Revision: D19412153
fbshipit-source-id: 0062eb57ec07dfcbfee3ed9b77cf82cdcda09fb3
Summary:
Previously, NameDag only supports writing directly to disk. That is not easy
for hg to use, since hg knows commits before bookmark names during pull, while
the API requires both commits and bookmarks (to decide which commits belong to
the "master" group) at the same time.
The old API also does not match other storage layers like indexedlog and
metalog, where writes are buffered until explicitly flushed.
This diff adds the "write-in-memory" (named "add_heads"), and "flush" APIs to
make NameDag easier to use. Under the hood, it just copies the "added" portion
of the DAG, and re-use the old API ("build", renamed to "add_heads_and_flush")
to do the job.
Note: The buffered changes pattern cannot be used in IdDag, since Ids might
have to be re-assigned on flush. But NameDag does not expose the raw Ids in its
"normal" interface, so it's fine to reassign Ids on flush.
Reviewed By: markbt
Differential Revision: D19405474
fbshipit-source-id: 75e5e5815c78c3577a0138f48185f6c4b5a80891
Summary:
This is more consistent with the name "IdDag", because both "Name" and "Id" are
noun.
Reviewed By: singhsrb
Differential Revision: D19405473
fbshipit-source-id: a3b7546dd0ddcae5d9ed562838bd6be10a47063c
Summary: This is more consistent with the name "NamedDag".
Reviewed By: singhsrb
Differential Revision: D19405472
fbshipit-source-id: f7023307acaf96bf77c9fa9704dcaf6fc59b56f2
Summary:
This is part of proof-of-concept to test Watchman over TCP, a means
to accelerate hg workflows for the ASIC teams within Facebook.
For more context, refer to master task T55191832
Reviewed By: xavierd
Differential Revision: D18482107
fbshipit-source-id: 08ba3641854d81fe2a1cd8739bb4e9abcbc700ed
Summary: Copied changes from watchman dev area to hg
Reviewed By: wez
Differential Revision: D18574426
fbshipit-source-id: 6c360e42621cef373c5c1d4af34011d80eafc7bf
Summary: `hg gc` can now clean things up there, so suggest that.
Reviewed By: quark-zju, fugalh
Differential Revision: D19413816
fbshipit-source-id: 1c6c08ed9fb4757390883d908531d9b3a7da302d
Summary:
The git repo has a lot of merges. So it's interesting as a testing graph.
The file was generated by:
git clone https://github.com/git/git --bare
hg --config extensions.hggit= clone git.git git-hg
hg --cwd git-hg debugbindag -r 'all()' -o git.bindag
Reviewed By: DurhamG
Differential Revision: D19411825
fbshipit-source-id: 0fec8a16786dc8e6790986bca5c62a76276aa942
Summary:
This changes the pattern commonly seen in smartlog:
╷ o e7f5f529 ...
╭─╯
│ o a1b2773d ...
╭─╯
o ecbb4eaa
╷
to:
╷ o e7f5f529 ...
├─╯
│ o a1b2773da ...
├─╯
o ecbb4eaa
╷
The change only applies to commits with a single parent. Vertical lines from
merge commits stay disconnected intentionally. For example:
│ │ o I
│ │ │
│ o │ H
╭─┼─┬─┬─╮
│ │ │ │ o G
│ │ │ │ │
This makes it more obvious that `H` has 5 parents while `I` only has 1 parent -
`I` does not "borrow" `H`'s parents. This problem does not exist if `H` only
has 1 parent.
Reviewed By: quark-zju
Differential Revision: D19407687
fbshipit-source-id: 1046c8e2309f50e3f1620ed21f1b10573759a5f8
Summary:
Without this change, the next diff will fail tests with metalog programming
errors (write outside transaction).
Some test changes are caused by the being deprecated `rollback` command.
Reviewed By: DurhamG
Differential Revision: D19380939
fbshipit-source-id: 7c893d3025bb697102835670b8a38f8fb9a624c8
Summary:
Reverts https://www.mercurial-scm.org/repo/hg/rev/eb586ed5d8ce.
The colon syntax (x:y) is deprecated and is unsupported by segmented changelog.
Reviewed By: DurhamG
Differential Revision: D19394101
fbshipit-source-id: 8c66756f1035ab7660180716a2afa052879f384e
Summary:
As part of requiring transactions for the dirstate, let's wrap its use
in gpg.
Reviewed By: quark-zju
Differential Revision: D18213029
fbshipit-source-id: b05afa6d465e8d2a7b1637fa1c1903e1625a7104
Summary:
As part of making the dirstate require transactions, let's wrap
histedits dirstate changes in transactions.
Reviewed By: quark-zju
Differential Revision: D18213045
fbshipit-source-id: 49baf87e8f8abfe500409f0f79a52eeae09f8ada
Summary:
As part of requiring transactions for the dirstate, let's wrap eol's
manipulation of it.
Reviewed By: quark-zju
Differential Revision: D18213039
fbshipit-source-id: 28b52057e61fe9d88ce9f26be2cbf7482837ba84
Summary:
As part of moving the dirstate to require a transaction, let's wrap the
update in commitcloud.
Reviewed By: quark-zju
Differential Revision: D18213046
fbshipit-source-id: e59c880ea5e8d8808499f94c983c26e7466204e2
Summary:
As part of requiring the dirstate to be part of a transaction, let's
wrap its usage in automv.
Reviewed By: quark-zju
Differential Revision: D18213034
fbshipit-source-id: f8557453351e665ac59da9614c84ab6312cff4ef
Summary:
As part of moving the dirstate to always be part of a transaction,
let's wrap its usage in our amend extension.
Reviewed By: quark-zju
Differential Revision: D18213028
fbshipit-source-id: 3562bb589265dceb3d89c997710738f44eb49f33
Summary:
As part of making the dirstate always participate in the transaction,
let's wrap it's use in perf.py.
Reviewed By: quark-zju
Differential Revision: D18213036
fbshipit-source-id: cd4a7d942556f032d32cbe0630d16414b819ea34
Summary:
A future diff will move all dirstate writes to be in a transaction.
Let's start by moving the transaction write logic to it's own function. A future
diff will delete the non-transaction "write" function.
Reviewed By: markbt
Differential Revision: D18213554
fbshipit-source-id: 7fa414314583a64ccada0f8be287d6064024e020
Summary:
Now that the post dirstate handlers are largely unused, let's move the
fsmonitor state warning into the fsmonitor filesystem. This also allows us to
remove the temporary _newid variable that was used to allow post-transaction
detection of if the transaction happened.
Reviewed By: quark-zju
Differential Revision: D18259479
fbshipit-source-id: da88638d34a80480bbeba8b1531d523785d72c30
Summary:
Previously the localrepo object contained a list of handlers to invoke
after status finished executing. This was used for 1) updating the fsmonitor
state with the new clock, ignore hash, and list of notable files, and 2)
updating the list of ignored files in treestate.
As part of moving all fsmonitor subtlities behind the filesystem abstraction,
this diff moves the fsmonitorstate file updating to be in the filesystem layer.
This also moves it inside the same transaction as the lookup file cleanup, which
removes the need for the hacky self._newid logic to detect if the dirstate was
changed by an external process.
(self._newid was previously added so that our second dirstate transaction, which
updates the fsmonitorstate, could determine if the dirstate had either been
changed by this process earlier in the lookup cleanup transaction, or if it had
been changed by an external process).
Reviewed By: quark-zju
Differential Revision: D18076655
fbshipit-source-id: c5175f61432f8e121b6baacd9587fcd288f67098
Summary:
A future diff is going to move the fsmonitor post-status fixups into
the same transaction as the lookup-marking logic. To prepare for this, let's
refactor the lookup marking logic to separate out the transaciton management
from the actual marking logic.
Reviewed By: quark-zju
Differential Revision: D18076656
fbshipit-source-id: 7711d5b086be7baaff0d9736cf51c6b0a46f237e
Summary:
In an earlier diff we moved the lookup-state resolution from the repo
object to the dirstate object. Now that a filesystem layer exists, let's move
lookup resolution into the filesystem layer so that filesystem.pendingchanges
can now return perfect information.
Reviewed By: quark-zju
Differential Revision: D17749373
fbshipit-source-id: d03177e5e5374bd38c6c9adaa0fe8959052de19e
Summary:
The Python one only does it when needed, let's fix the 2 tests that were
relying on this.
Reviewed By: quark-zju
Differential Revision: D19393501
fbshipit-source-id: 94e004999045264adce2ffa17b4697583ee0fcbd
Summary:
The phases of a commit is a more precise metric for whether a change is public
or not, let's use this instead of testing for data to be present in the local
contentstore.
Reviewed By: quark-zju
Differential Revision: D19393502
fbshipit-source-id: b1c735c8400e8d5a1a7f7013d9a4a1ad423c5023
Summary:
We started to see a lot of tests failing with mononoke timing out at startup.
Increasing the timeout to mitigate the issue.
Reviewed By: krallin
Differential Revision: D19397440
fbshipit-source-id: a8037102b3c8389b98a7c49824fa013fd3e6a080
Summary:
Moving `store_element` and `get_hgid` from testutil.rs to lib.rs in order to
get rid of some buck build warnings. They are only used in lib.rs.
Reviewed By: markbt
Differential Revision: D19350343
fbshipit-source-id: 081783d753425daaae6fadc38589da5e4e8b745d
Summary:
It makes sense for manifest-tree to expose a testutil version for building a
TreeManifest. No reason for the testutil function to be limited to the crate.
The only issue is that `make_tree` is to generic so we rename the function to
`make_tree_manifest`.
Reviewed By: markbt
Differential Revision: D19350351
fbshipit-source-id: 06fe04d99bf820d3e81417f5a5514aa4b0d282e6
Summary:
Types that are defined by the manifest core crate should have test utilities
defined in the same crate.
This is motivated by various warning in the buck build.
Reviewed By: markbt
Differential Revision: D19350350
fbshipit-source-id: a6a7c3fb54b465aa09a28ff8b70b49a355b328fc
Summary: Without this, `hg status --print0` can output nothing and break tests.
Reviewed By: xavierd
Differential Revision: D19387317
fbshipit-source-id: 4feb82889df9bd1705526027f491db90a2a5f946
Summary:
There is a bug in Projected FS on Windows 1903 and 1909, which BSODs the system on SetInformation call. Projected FS seems to be dereferencing a nullptr while attempting to check if the file is in the backing store. Providing a separate callback to check if the file exists in the backing store will avoid the buggy code (suggested by Microsoft).
This will enable Eden FS on Windows 1903 and 1909.
Reviewed By: simpkins
Differential Revision: D19377522
fbshipit-source-id: ab5e9bc8e2472f3e689d71d05a560bac86211295
Summary: This is a followup on D18666585.
Reviewed By: chadaustin
Differential Revision: D19377523
fbshipit-source-id: f9ab4011d0978a6934de77d1837a37082603cf52
Summary: We missed updating this in D18669664.
Reviewed By: simpkins
Differential Revision: D19377524
fbshipit-source-id: e23fb039047855454b4f12353bc375b1a2c8d962
Summary: This is needed in Python3, export it.
Reviewed By: quark-zju
Differential Revision: D19379402
fbshipit-source-id: e5d095005f20c8a940b31e0296293aa8a85c4e98
Summary:
Previously, if the rust walker encountered any issues with individual
files it would stop and throw an exception or in some cases panic. To replace
the python walker we need to handle errors more gracefully, so let's add logic
for reporting which files had issues and what those issues were.
This also contains a minor one-line fix to prevent the walker from traversing
directories that contain a '.hg' directory.
Reviewed By: quark-zju
Differential Revision: D19376540
fbshipit-source-id: ede40f2b31aa1e856c564dd9956b9f593f21cf27
Summary:
The command shows how visibility, bookmarks, remotenames change in the repo
over time, using metalog data.
Also add this command to hg rage report.
Reviewed By: markbt
Differential Revision: D19358668
fbshipit-source-id: 9bddc039143b45c29c0c937ee60906ab93a2defe
Summary: The colors are from the our smartlog template. They're used in the next change.
Reviewed By: markbt
Differential Revision: D19358670
fbshipit-source-id: 7476f95767c5fcf1cccdce2119a2ab7ce45d270e
Summary:
This simplifies the error handling and makes it more compatible with things that
might capture the output, including the Python testing framework.
Reviewed By: DurhamG
Differential Revision: D19325642
fbshipit-source-id: 53de8b9a8294219e2b8e62831dce236841bd4cbb
Summary: Make `hg status` use the EdenFS Thrift path, similar to the telemetry wrapper.
Reviewed By: DurhamG
Differential Revision: D19325641
fbshipit-source-id: 14777a252d7cb433316511a2a1f1a6649e9cb020
Summary:
Make it possible to define reusable flags like walkopts, templateopts that can
be shared among commands.
Reviewed By: DurhamG
Differential Revision: D19325640
fbshipit-source-id: 89a9ad0e59672b1f8cfa22daaacd3c75d8cff61a
Summary:
Add a crate aiming for talking to EdenFS via Thrift.
This is mostly ported from telemetry. There are lots of things that can be
cleaned up like moving the command portion to hgcommands, or using a better
abstracted rendering layer for outputting, etc. But for now, I just ported
the code as-is.
Reviewed By: DurhamG
Differential Revision: D19325639
fbshipit-source-id: ac70315ed166024857a90aab070e4d521f3e3b58
Summary:
The main motivation is to replace (super slow) Python Thrift serialization.
It also makes it possible to port the native "status" command from telemetry
wrapper to the main executable.
The crate is made to be built with either buck or cargo in fbcode environment:
- Cargo build uses `build.rs` to re-generate (checked-in) Thrift code.
- Buck build uses TARGETS to build Thrift code on the fly.
Reviewed By: DurhamG
Differential Revision: D19325638
fbshipit-source-id: 1d3287d8d6c674065500929827fa13ba9d448708
Summary:
We don't want automigration to block normal execution of the command,
so let's take the lock with a short timeout, then skip automigration if we don't
get it.
This should solve an issue where debugedenimporthelpers were hanging because
they tried to take a lock that a hg process that was accessing eden held.
Reviewed By: quark-zju
Differential Revision: D19375544
fbshipit-source-id: 6c107aac87b5b839cc2d347a3b40ce67da627b71
Summary:
On Windows, select cannot be used for pipes, and thus the doublepipe class
would end up busy spinning trying to read from it. Since stderr is almost
always empty let's move it to a background thread so the main code path
doesn't have to deal with it.
This proves to be a significant win for `hg pull`. For an empty manifest
cache, `hg pull` used to take ~5 minutes, it's now around 1 minute, with
Mononoke taking 40 seconds out of that.
Reviewed By: DurhamG
Differential Revision: D19352144
fbshipit-source-id: 77112c1c18d7dab15f6946a645f73e89908802f2
Summary:
Enhance the tests to test whether the width of the graph matches the width that
was given by the `width` method before `next_row` was called.
The width we are interested in is the number of cells the string would occupy,
not the string length, so use the `unicode-width` crate to determine this.
Reviewed By: quark-zju
Differential Revision: D19282571
fbshipit-source-id: d9852c4c9e0f76c78db047f0da5dd34723a62a2a
Summary: Switch from `:` to `.` for the ancestor lines. I think they look better.
Reviewed By: quark-zju
Differential Revision: D19371721
fbshipit-source-id: e18f1a62e23620a82007e2c377607a0a61623830
Summary:
Use `encoding.unitolocal` to convert the Rust-generated unicode strings to the
local encoding.
Reviewed By: quark-zju
Differential Revision: D19371723
fbshipit-source-id: 6b37ef51f37f2212ca2d9ab20f9ed4da29b3d63c
Summary:
Box-drawing characters with curves aren't reliably renderable on Windows. Add
a "square" glyph set that uses right-angle characters. These characters are
available in cp437 and cp850, so should be available on most Windows systems.
For completeness, add a "dec" glyph set that uses DEC line drawing characters,
for use in old terminals like xterm.
Reviewed By: quark-zju
Differential Revision: D19371722
fbshipit-source-id: 35887243cceab66c702e2b5278b572f77946805f
Summary:
This applies the new feature to all tests.
The tests do not use revlog fallback but we plan to use revlog fallback
in production for faster migration.
Reviewed By: DurhamG
Differential Revision: D18737259
fbshipit-source-id: 69633c7b2026fb3d19ca16d40368bded972efd86
Summary: Add hg.pdb and pure Python source code on Windows for easier debugging.
Reviewed By: markbt
Differential Revision: D19351559
fbshipit-source-id: 8515a84dc58df7c44fd324cc69b7b142fd063b8f
Summary:
D19340538 deleted `eden/scm/edenscm/mercurial/rust/__init__.py`, but
chg's `_preimportmodules` still tries to import it, which breaks Mononoke
```
$ hg init repo-hg
+ Traceback (most recent call last):
+ File "/data/users/torozco/fbsource/fbcode/buck-out/dev/gen/eden/scm/__hg__/runtime_resources/__default__/eden/scm/hglib/edenscm/__init__.py", line 61, in run
+ dispatch.runchgserver()
+ File "/data/users/torozco/fbsource/fbcode/buck-out/dev/gen/eden/scm/__hg__/runtime_resources/__default__/eden/scm/hglib/edenscm/mercurial/dispatch.py", line 336, in runchgserver
+ _preimportmodules()
+ File "/data/users/torozco/fbsource/fbcode/buck-out/dev/gen/eden/scm/__hg__/runtime_resources/__default__/eden/scm/hglib/edenscm/mercurial/dispatch.py", line 319, in _preimportmodules
+ __import__(name)
+ ImportError: No module named rust
+ [1]
```
NOTE: I had a `__init__.pyc` left in the directory removed in D19340538, which
I'm guessing is why was still working on a `make local` build. After removing
that directory, a `make local`-built hg crashes consistently with the same
error for me.
Reviewed By: farnz
Differential Revision: D19371748
fbshipit-source-id: 4f57b5e1f75b2bd6e857303111fc01446124417b
Summary:
The expected error type is RevlogError. Previously that RevlogError
was converted from RuntimeError. However, D19169527 stack changed
RuntimeError to RustError, causing clindex partialmatch to raise
RustError instead, which is not handled by the Python layer.
Convert RustError to RevlogError to solve the issue.
Context:
https://fb.workplace.com/groups/scm/permalink/2591821277534143/
Reviewed By: DurhamG
Differential Revision: D19359074
fbshipit-source-id: 7b7b500cdf01272791a73ea0d94397e1397ff80b
Summary:
This makes it possible to use zstore instead of 00changelog.d to store commit
data. Right now it double writes to zstore and 00changelog.d if turned on.
In the future we can switch to only writing to zstore if 00changelog.i
alternative is available.
Some related configs were added for fallback strategy: revlog or server.
The revlog fallback allows us to do fast migration. The server fallback
(not implemented in this diff) allows us to stop depending on revlog.d.
Reviewed By: DurhamG
Differential Revision: D18737260
fbshipit-source-id: 3c9605f0babd8a215ee74bdf1275cc4e9dbd766c
Summary:
The `00changelog.d` file contains commit data (user, date, message, but no
ordered parents).
By setting the `zstore` attribute on `changelog`, reads from `00changelog.d`
(aka. the `revision` method) are going to use `zstore` instead. Writes will
go to both `00changelog.d` and `zstore`.
Reviewed By: DurhamG, xavierd
Differential Revision: D18737258
fbshipit-source-id: fd1cf52a77fe396df66b77c4df77089326f5d4c2
Summary:
We have seen issues parsing '0' as date somehow passes on OSX, and parsing
'Apr 2018' fails on Windows with GitBash. Let's workaround the tests while
I'm going to find better fixes.
Reviewed By: xavierd
Differential Revision: D19353054
fbshipit-source-id: c0302071964191b97eea6572ffd985f831db6791
Summary:
Using RepoPath::arbitrary() has a high chance or producing a lot of different
top level directories. The helper method added by this change,
`generate_repo_paths` simulates directories having various files and several
directories.
Reviewed By: quark-zju
Differential Revision: D19321366
fbshipit-source-id: 5c1aec78b6157f3cbea3d0673b29b3a676de88c0
Summary:
The TestStore can be leveraged by methods outside the manifest-tree crate.
Anything that wants to instantiate a manifest for test purposes could
leverage it.
Reviewed By: quark-zju
Differential Revision: D19204894
fbshipit-source-id: 4ac42d09855c70f829feefc6c71dcdbf7211cae3
Summary:
For some reason, our linker flag `-uinitthreading` no longer works in buck opt
build. This affects the `indexes` and `threading` Python extensions. Move them
to `bindings` to solve the problem.
Reviewed By: fanzeyi
Differential Revision: D19340538
fbshipit-source-id: 0c82b472f2916229d3c5e481954b69415b5b0dc2
Summary: I would like to link the Phabricator Diff Number to specific mercurial revisions. After talking to Durham, the least invasive and best method for accomplishing this is to log the Phabricator Diff Number at runtime. Hence, whenever someone modifies the current commit, log the Phabricator Diff Number (when possible). The resulting table will theoretically have a map from "revision" -> "Phabricator diff number".
Reviewed By: DurhamG
Differential Revision: D19334379
fbshipit-source-id: eb04a8af4dbeb9a7e130a268fce8a6c5d26ebfa9
Summary:
In the refactor to the new tracing framework, the threshold comparison
got changed and became a seconds-vs-milliseconds check, which therefore almost
never passes. Let's change it back to seconds-vs-seconds.
Reviewed By: quark-zju
Differential Revision: D19322203
fbshipit-source-id: d9cfb3854d3ee6b8c7ab5309544abd64af51027f
Summary:
D19114546 introduced a new rollback mechanism where it updates to the
original commit if the user quits. Unfortunatley it only did the update if the
user had made a commit, so if they exitted immediately before making a commit,
it left them on the wrong commit and all their changes are pending.
Reviewed By: quark-zju
Differential Revision: D19334201
fbshipit-source-id: c959096adfa3563894578436e8252c037b50e32c
Summary: `folly::cold_detail::ColdClass` was marking things (like `folly::Unexpected`) cold, but at the cost of inhibiting the inliner from doing its job. This is leading to bad codegen, which offsets any small wins we mind get for the `cold` attribute.
Reviewed By: yfeldblum
Differential Revision: D19324159
fbshipit-source-id: 7ed431b6c9d6e963c3bf438c707fa6cf6a38bf9d
Summary:
The Rust bindings now provide a subset of time parsing features. Replace the
old Python implementation. This has multiple benefits:
- Strong gurarnatee that Rust and Python behave the same.
- Parse relative time (ex. `5 minutes ago`)
- Parse date beyound i32 range (ex. >= year 2038)
Reviewed By: DurhamG
Differential Revision: D18946332
fbshipit-source-id: 721f47bc5b2835d7ca0a05ab34ea4faa1a411a4e
Summary:
If the user specifies a timezone-less date, ex. `2000-7-1`, it's incorrect
to use the same offset of `Local::now()`, because of daylight savings.
Fix it by going through proper type conversion.
Reviewed By: DurhamG
Differential Revision: D19179800
fbshipit-source-id: 6fd433757e7f6e4baf4625944d4061548a436f11
Summary:
This is causing a deadlock when an hg process takes the lock, accesses
an edenfs file, then eden launches debugedenimporthelper which also tries to
take the lock. Let's back it out for now.
Reviewed By: krallin
Differential Revision: D19331038
fbshipit-source-id: b8480c2e07079c3babeae8de48c02affa43188fc
Summary:
Previously, between 100 to 500 runs, the test is likely to fail:
---- log::tests::test_repair_and_delete_content stdout ----
thread 'log::tests::test_repair_and_delete_content' panicked at 'assertion failed: `(left == right)`
left: `"Verified first 141 entries, 999 of 1400012 bytes in log\nReset log size to 999\nIndex \"c\" is incompatible with (truncated) log\nRebuilt index \"c\""`,
right: `"Rebuilt metadata\nVerified first 141 entries, 999 of 1400012 bytes in log\nReset log size to 999\nRebuilt index \"c\""`', src/log.rs:3414:9
The "Rebuilt metadata" part is missing.
This is because after D17742003, the end of `meta` is the `epoch`, which is a
random number. The test is rewriting the end of `meta` to corrupt it. That
has a chance to not corrupt the file. Change the test to corrupt the fixed
header of meta so it can always corrupt the file and therefore stabilize
the test.
Reviewed By: xavierd
Differential Revision: D19325087
fbshipit-source-id: 53d46c8a6cb9771582f8f37e27f185d303fde0ce
Summary:
The Rust Contentstore code assumes that the prefetch request doesn't contain duplicated
filenodes so let's do it on the Python side.
Reviewed By: quark-zju
Differential Revision: D19318847
fbshipit-source-id: 73375de60f4fb81021eba16506037984eebdf048
Summary:
The Rust ContentStore will not print these, and in order to better understand
what tests are broken with it enabled, let's make them optional.
Reviewed By: quark-zju
Differential Revision: D19318848
fbshipit-source-id: 87a1da510b51adc40a0d08157925a990456d3e9c
Summary:
Per discussion, we decided to use "VertexName" as the struct name for things
like commit hashes, or the string names in tests (or the Mozilla DAG in tests).
Therefore, introduce a dedicated VertexName type and repalce all callsites to
use it.
`bytes::Bytes` is used so copying the `VertexName` is somewhat considered cheap.
This adds some overhead copying slices (and `Bytes` has some overhead). It
regresses the "building segments" benchmark from 673ms to 773ms, which seems
okay given the cleaner interface.
Reviewed By: markbt
Differential Revision: D19154905
fbshipit-source-id: 4c6d4eca67c11c10ed5f21999ccdc3f1b01695e8
Summary:
Previously, each test uses a separate chg server, because there are different
states in different tests. So chg servers cannot be reused across tests.
Nowdays chg servers are more stateless, and can be shared for all tests.
Do that to reduce chg server startup overhead.
With this change, `./run-tests.py --time $(rg -l chg-compatible)` went down
from 4:57 to 4:04 on my devserver.
A configuration `--chg-sock-path` is added for advanced use-cases.
Reviewed By: DurhamG
Differential Revision: D18924911
fbshipit-source-id: c2f7d84ee4cbfb5d599cfdc0ab47f922fae4a67e
Summary:
Redefine Id::{MIN, MAX} using Group so Id::MAX.group() is a valid group.
Previously Id::MAX has an invalid group.
Reviewed By: sfilipco
Differential Revision: D19154908
fbshipit-source-id: 6108f02da786eb228a7d1b772ca5134c1ebb6f6b
Summary:
Address review feedback on D18670160.
This also fixes the definition of "universal" a bit - the new code no longer
assumes there are only one head in the master group.
Reviewed By: sfilipco
Differential Revision: D19154906
fbshipit-source-id: 4fbf080354318300ce09dc7fba2dcc5f942b6308
Summary:
The check of "is this high level segment necessary or not" should be done
before dropping the last segment. This fixes an issue discovered in D18838992.
Reviewed By: sfilipco
Differential Revision: D19154904
fbshipit-source-id: fb4c83c39d66215bae168ad98e5cf78de91cc5a3
Summary:
NamedDag is the high-level interface that is more interesting to test.
High-level segments are changed subtly because NamedDag syncs them to disk
first.
Reviewed By: sfilipco
Differential Revision: D18838992
fbshipit-source-id: c6a557e0a44a1d24320ea4a9e4283262f6f30f67
Summary: Those warnings are not seen on POSIX build.
Reviewed By: sfilipco
Differential Revision: D19320441
fbshipit-source-id: feddf728eb9627834559b87d83e20f0afd9080c8
Summary:
We had eden-backing repos that were using formats from years ago
because they never ran pull to trigger the automigrate. Let's automigrate as
part of debugedenimporthelper to keep these repos up-to-date.
Reviewed By: quark-zju
Differential Revision: D19320510
fbshipit-source-id: 475a6acf8748677de3183dfb5d4c6df7caca606d
Summary: The case collision test broke when I disabled tags.
Reviewed By: singhsrb
Differential Revision: D19321272
fbshipit-source-id: d59a0ad38b80f35931bbb4fb52e400d714ebbf37
Summary:
D19135344 broke EdenFS's cmake builds, and our macOS releases haven't
been going out. Rather than pass the same streaming options to the
Thrift compiler in the CMake build, remove deprecated_server_streams
from both builds.
Reviewed By: simpkins
Differential Revision: D19204397
fbshipit-source-id: c22e634ded82a683daff5c92182c8a1eb1ed4d21
Summary:
If users or automation invoke a bunch of really long commands, it can
occupy a lot of space in our logs. The full command for really long commands is
unlikely to be useful, so let's truncate them to 256 bytes.
Reviewed By: ikostia
Differential Revision: D19317932
fbshipit-source-id: 428479a18dd7e5a7ae4d8f862cf26c02c15b1fcb
Summary:
Python errors wrapped in a Rust Error would be incorrectly changed into a
RuntimeError instead of the original one. The map_pyerr method does the right
thing, and is also shorter, let's use it.
Reviewed By: quark-zju
Differential Revision: D19313883
fbshipit-source-id: 1ecb41a8eef9d41618905d6d00f199252d373f96
Summary:
It turns out some old repos are still using the legacy treedirstate
format, which broke when I made this a hard requirement. Let's roll this back
for now.
Reviewed By: singhsrb
Differential Revision: D19313773
fbshipit-source-id: ded78be033cb04323a2e1d237059947dee509df9
Summary:
Interactive smartlog history relies on the order from the server, which has
changed, and so is now incorrect. Sort the versions from the server so
we can be sure they're always in the right order.
Interactive smartlog history also ignores the workspace parameters to specify
the workspace. Fix that.
Reviewed By: farnz
Differential Revision: D19309253
fbshipit-source-id: 9fc1b443e72557331e22d0987c73286f5c64e16d
Summary:
We no longer user tags in any of our repositories, so let's delete all
the code and remove some tech debt.
Previously drawdag was our last remaining use of tags, but in a previous diff I
switched that to use bookmarks.
Reviewed By: quark-zju
Differential Revision: D19042226
fbshipit-source-id: d7871c71c1026599aa70fdd806418347ea214a8a
Summary:
We want to delete all the non-treestate dirstate implementations. Let's
start throwing an exception if treestate is not enabled. We temporarily have a
bypass in case we break an important usecase in the process.
This also sets the standard new repo to be created in treestate mode, but adding
treestate to newreporequirements.
Reviewed By: quark-zju
Differential Revision: D19204621
fbshipit-source-id: 61c0c69ce6e559615ef570d2343bea78c1b5d5ee
Summary:
Tweakdefaults adds the ability for `hg rebase -d` to simply fastforward
the user if the destination is a descendant of the current working copy parent.
If the working copy is dirty, it merged the files, potentially leaving the
working copy with merge conflicts that are hard to recover from.
A normal hg update disallows merges that would result in potential conflicts.
Let's update the rebase code to use the same merge logic, so fast forward
rebases simply abort if there are potential conflicts.
Reviewed By: quark-zju
Differential Revision: D19204198
fbshipit-source-id: 1b0c2c0fdab909eebe74d1a211dbfb2d780058cd
Summary:
Allow rendering of cloud smartlogs using renderdag. Use the ability of renderdag
to reserve columns to provide the smartlog-like indentation of draft stacks.
Reviewed By: quark-zju
Differential Revision: D19272577
fbshipit-source-id: 3311d83511b0d5347b45f33c282ce26b8133d047
Summary:
Allow rendering of graphs using renderdag by setting `experimental.graph.renderer` to `ascii`, `ascii-large` or `lines`.
The minimum height of each row can be set using `experimental.graph.min-row-height`.
Reviewed By: quark-zju
Differential Revision: D19272580
fbshipit-source-id: ac5eff8446bd3c09d07624a193195b4224055ffc
Summary:
Move the test fixtures into a common module, so that they don't need to be
repeated in each test. Since each fixtures is now a struct, this also makes it
clearer what each of the items are.
Reviewed By: quark-zju
Differential Revision: D19288290
fbshipit-source-id: 394805c652592177f11ccb096b8e5e95361456e4
Summary:
Generalize construction of output renderers (renderers that render to `String`)
to avoid duplication of options. At the moment there is only one, but later we
may add new options.
This also allows us to construct output renderers from any renderer that
renders to `GraphRow`, which means we can create adapters that modify or
re-order the rows of the graph.
Reviewed By: quark-zju
Differential Revision: D19286350
fbshipit-source-id: a5649ca2f48e263ee24584339179655fb612d3d1
Summary:
A new implementation for rendering DAGs.
This new crate implements a generic DAG renderer, that can convert a
topologically sorted sequence of DAG nodes into a sequence of strings suitable
for rendering to a terminal.
The new renderer differs from the old renderer in a few important ways:
* It prioritizes keeping commits linear, and will allow gaps to form if
that will allow the history of the commits to be kept in a straight
line. This makes it easier to track long parallel histories.
* It supports octopus merges (nodes with more than two parents). Even
though Mercurial doesn't support octopus merge commits, summary DAGs
with omitted nodes can still end up with logical octopus merges.
* It supports reservation of columns for specific nodes. This can be
used to support smartlog-style indentation of draft stacks without
needing to hack around it by creating fake nodes.
* It separates out forming the graph from generating the lines. This
allows multiple back-ends for generating different styles of graph.
There are three back-ends implemented:
`AsciiRenderer` renders similar to the old graph renderer, using ASCII
characters. For example:
```
o F
|
| o E
| |
| | o D
.-----'
o | | C
+---'
o | B
:/
o A
```
`AsciiLargeRenderer` uses larger ASCII blocks to give a clearer picture of
complex graphs. For example:
```
o F
|
|
| o E
| |
| |
| | o D
______/
/ | |
o | | C
| ___/
|/ |
o | B
: /
:/
o A
```
`BoxDrawingRenderer` uses Unicode box drawing characters to give a more
continuous rendering of the graph, however requires support for these
characters in the terminal font. For example:
```
o F
│
│ o E
│ │
│ │ o D
╭─────╯
o │ │ C
├───╯
o │ B
├─╯
o A
│
~
```
Reviewed By: quark-zju
Differential Revision: D19272579
fbshipit-source-id: bb6fa4685c965544cc3b6b9261df3a3ec161b41f
Summary:
This can happen for "Reserved" diffs.
Reported By: hyr
Differential Revision: D19270860
fbshipit-source-id: 24e9b8c08fc16e3089e0dc3f8064a5e9671d9ace
Summary: adds a cli debug command to inspect the working copy parent. by default just returns eden's snapshot contents, but adds optional --hg flag to print Mercurial's dirstate information
Reviewed By: chadaustin
Differential Revision: D19167518
fbshipit-source-id: b65e112df6abe4e0e7a8a528a90b2e3d17297e66
Summary: This stack is deprecated in favor of the rocket stack.
Reviewed By: rhodo
Differential Revision: D18123600
fbshipit-source-id: db6e3bf4add9c49cf8f1d2f12f97b207e87c7302
Summary:
Sometimes GraphQL does not return JSON result. Do not crash on the client-side.
Context:
https://fb.workplace.com/groups/scm/permalink/2565497036833234/
Reviewed By: farnz
Differential Revision: D19254348
fbshipit-source-id: 75ca93b1dc8648d5c59915ecf450b9366df13c01
Summary: In Mononoke tests, we have failures because `hg book -fd` aborts if it's not given any bookmark names. Fix up this function to not run `hg book -fd` if there are no bookmarks
Reviewed By: quark-zju
Differential Revision: D19251930
fbshipit-source-id: 2f1696d9207ad64a94c28941e72806d9b0e686fe
Summary:
The test is broken by D19190496 because the test uses some weird test cases
about legacy Pythons.
The "parseindex2" test is revlog-related. The revlog is kind of stable, tested
indirectly in countless tests, and is also going away. We don't run the code
on legacy versions of Pythons.The "parseindex2" test does not provide much
value. Therefore just remove it.
Reviewed By: akushner
Differential Revision: D19221126
fbshipit-source-id: 5b904d97caa47f92098435ee85c1bb8b725c9f84
Summary:
This makes it possible to use the hgtime parsing utilities in Python.
Namely, patterns like `since 6 days ago to now` can now be parsed.
Reviewed By: sfilipco
Differential Revision: D18946335
fbshipit-source-id: c12ea3437301f20a8c457525c0798f471b8f174a
Summary:
The existing tests rely on this feature.
This requires replacing all `Local::` usage with `HgTime::now()`.
Reviewed By: sfilipco
Differential Revision: D18946336
fbshipit-source-id: 1a92183cc96f2fcb013b68a32e78f09f28a49452
Summary:
For example, `2018 to 2019` should be parsed as `2018-1-1 to 2019-12-31 23:59:59`,
not `2018-1-1 to 2019-1-1`.
Fix it to match the desired behavior.
Reviewed By: sfilipco
Differential Revision: D18946331
fbshipit-source-id: 640bbe49352d1f01855183ea9927bb0009646243
Summary:
The Python `datatime` stdlib has limitations:
- `datetime.datetime` can only express year 1 to 9999.
- `strptime` requires year >= 1900.
Limit the `HgTime` to 1900 ..= 9999 range to be consistent.
Note: Mercurial has an `i32` range limit, which seems to problematic
since `i32::MAX` is Jan 19, 2038. So I kept using `i64`.
This addressed some XXX comments about not returning errors.
The code change is mostly because added error handling.
Reviewed By: sfilipco
Differential Revision: D18946333
fbshipit-source-id: 0e4756457b0f13451dc5008ef19d4670a7aaa7fb
Summary:
It turns out the interaction between dag and idmap can be complex, especially
when a non-master id gets re-assigned a master id.
A high-level structure is helpful here to handle the corner cases correctly.
This diff adds basic definitions, and one of the most complex function - build.
The build function is ported from pydag.
Reviewed By: sfilipco
Differential Revision: D18838994
fbshipit-source-id: a2f6db3b188d5fd04d8e1d6df40ba985e33281f6
Summary: First step to implement segment and idmap rewrites for non-master ids.
Reviewed By: sfilipco
Differential Revision: D18838996
fbshipit-source-id: 0380c0193d9078a2a2d7fde9c5f565e8dbc8e713
Summary:
The index can delete all keys matching a prefix more efficiently than deleting
them one by one. Expose this feature.
The `dag` crate will use this feature to delete all "non-master" segments and
ids efficiently.
Reviewed By: sfilipco
Differential Revision: D18825296
fbshipit-source-id: b8531695609238a16913254af61004170f12954e
Summary: This makes it possible to define an index function that can delete data.
Reviewed By: sfilipco
Differential Revision: D18825299
fbshipit-source-id: 4eb03a001683092d99b439c8efb8e7909f543c70
Summary:
This makes it possible to actually delete keys from the index, which can be
handy for certain use-cases. For example, `dag` wants to rewrite non-master
ids and segments in certain cases.
Reviewed By: sfilipco
Differential Revision: D18825298
fbshipit-source-id: 3b69ca5fc763ba0c57bd16a28a349f2f829a0759
Summary:
This just moves parameters to a structure without changing the actual logic.
This is to make it possible to add a "remove" API to index.
Some variables are prefixed with `old_` or `new_` to make the code more
consistent.
Reviewed By: sfilipco
Differential Revision: D18825297
fbshipit-source-id: 27bd8438e66125c906ebf2044d6ff978b99c5986
Summary:
Make cpython-ext free from business logic. This adds a bit overhead.
But it should be fine given the error cases are considered rare.
Reviewed By: markbt
Differential Revision: D19186692
fbshipit-source-id: daaffc1369a36c781d1badea822bf62a144eb54e
Summary: We no longer use the `failure` crate. Rename related methods and traits.
Reviewed By: markbt
Differential Revision: D19186694
fbshipit-source-id: 1c83a90ee12db431b7d8e09a9c2cf1d43a4c0f93
Summary:
We no longer use the `failure` crate. Rename the module.
`cpython-failure` is dropped since it was merged into `cpython-ext`.
Reviewed By: markbt
Differential Revision: D19186693
fbshipit-source-id: 410d1491bcadd8d3272c7e2df08ecbe66fc0fef2
Summary:
The `map_pyerr<PE>` API is kind of hard to use correctly since the
code that calls `map_pyerr` does not really know the best suitable error type.
The existing code also "encourage"s the use of stdlib error types like
RuntimeError, ValueError, etc since the code is shorter, which is not great.
Error types from `mercurial.error` are better choices.
With previous diffs in this stack, we now decide the Python error type based
on the Rust error type. It's no longer necessary to specify an error type
with `map_pyerr<PE>`. So let's just drop the type parameter.
A `RustError` was added for error types that `cpython-ext` does not know how
to convert. We should avoid leaking it to the Python land by just implementing
how to convert `anyhow::Error` to `PyErr` in `cpython-ext`.
Reviewed By: markbt
Differential Revision: D19169527
fbshipit-source-id: f4563c36174cd51201b526bbc92a3f1c8a3da864
Summary:
Since we now have the indexedlog error type. Match it and print helpful
messages.
Reviewed By: markbt
Differential Revision: D19169207
fbshipit-source-id: cd874f651b681d2dbbb5aea613c56721c4ab6181
Summary:
The hgpython interpreter was used to run Python scripts in tests that might
rely on Mercurial modules.
The previous hgpython implementation is Python PAR based, which does not have
access to native Rust modules like bindings. Change it to use the native
"hg debugpython" implementation that is more compatible.
Reviewed By: markbt
Differential Revision: D19190496
fbshipit-source-id: 9791dbf9ba0ed92de702291faa9145f01b05ec40
Summary:
Expose the indexedlog error type to Python so it can be used in `except`
clause.
Reviewed By: markbt
Differential Revision: D19169208
fbshipit-source-id: c41f08ec11f78f7714b435dbfc6a28c36b34617d
Summary:
Convert indexedlog error and IOError to dedicated Python types so the Python
world can match them meaningfully by type and handle them accordingly.
Reviewed By: markbt
Differential Revision: D19169210
fbshipit-source-id: 1e7bfdf7cbaf917098efdcd63d600483a9427d33
Summary:
The current error message looks like fatal while it is not. Make it clear that
extension failures are not fatal.
Reviewed By: markbt
Differential Revision: D19166545
fbshipit-source-id: 7ce0231c90b0c5f50add15a12cb69b60b76dfcb1
Summary:
There are 3 extensions that can provide the `{gitnode}` template: "gitrevset",
"fbscmquery", "hggit". The last can read from commit extras. Fix the first two
to also read commit extras. At some point we need to unify the implementations.
Reviewed By: farnz
Differential Revision: D19198979
fbshipit-source-id: 24a0df0df4ceb7a9af9236a7f70babfd54e9802b
Summary:
When there are many processes (ex. import helpers). Writing to blackbox can be
slow due to flock contention.
Change blackbox "flush" decision from timestamp based to buffer-size based to
mitigate the issue.
Reviewed By: DurhamG
Differential Revision: D19198470
fbshipit-source-id: 71f87e5e3b45ef4db8efda22ca89fd8ce1fae7fa
Summary:
D17099990 changed the way status is computed and added a loop over the
nonnormal files to find any added, removed, or merged files. Unfortunately the
nonnormal file list can contain untracked ignored files in some cases, and if
that list is large (like if someone disabled a sparse profile) this loop can
take many seconds.
Let's updated the loop to filter out ignored files. This matches a similar loop
in fsmonitor, and we may want to think about removing nonnormal handling from
the fsmonitor code entirely now that it's handled in dirstate.py.
Reviewed By: quark-zju
Differential Revision: D19190259
fbshipit-source-id: 4ba4150507fdd72439bc4e5eb731a951c6100f5f
Summary:
I encountered a 19s `hg sl` run where 13s were spent on
`remotenames.updateaccessedbookmarks` (8 times).
{F225347899}
The filesystem is unusually slow since the it's nearly full. However, the
updates are not really necessary since I'm not accessing more remote names.
Accessing one bookmark via smartlog rewrites the entire accessed file, which is
O(N^2) and undesirable.
Improve it by skipping writing if nothing has changed. In the future we might
want to optimize it further by:
- Only update (and do ui.log) "accessedbookmarks" at most once per command.
- Potentially drop the "accessedbookmarks" tracking by using "remotenames"
directly - everything in "remotenames" is accessed.
Reviewed By: markbt
Differential Revision: D19191528
fbshipit-source-id: 46635d4e5c9d0034ace9cdafc1f42a4512aa8774
Summary:
Currently, hg's lfs extension used bytes concatenation to download a response
from the server-side. However, concatenating bytes in Python can be
inefficient, since it'll re-allocate the whole thing and copy it every time you
concatenate.
This makes repeated concatenation O(N^2) on the number of things you
concatenate, which is undesirable. This diff updates hg's lfs extension to not
do that.
Reviewed By: farnz
Differential Revision: D19194388
fbshipit-source-id: f4a38afd8ba41bc1c2e64e9127035675ecda6651
Summary:
HTTP makes no provision to tell your client that you failed halfway through
producing your response and won't have the answer they're looking for. So, if a
LFS server fails while producing a response, then we'll report an OID mismatch.
We can do a little better and disambiguate between "the server sent us the
wrong blob" (very scary) and "the server crashed" (merely annoying) by looking
at the content length of the response we got back. If it's not what was
advertised, we can reasonably safely assume the server crashed.
Reviewed By: farnz
Differential Revision: D19194389
fbshipit-source-id: 569bd20cffe2901e2801261ce783e99bcf8358e2
Summary:
The test is flaky in recent OSX builds.
The test is about a chg feature. Right now we only use chg on Linux. So limit
the test to Linux only.
Reviewed By: singhsrb
Differential Revision: D19191582
fbshipit-source-id: ce506f3f50c4368253caea0d73a647539c649735
Summary:
After D19043330, the test is failing on Windows (no execbit).
When converting a repo, we almost always want to preserve the execbit. So let's
just skip the test if execbit is not available.
Reviewed By: singhsrb
Differential Revision: D19191556
fbshipit-source-id: 8e65d0b4b03833216ac8f28e2ae25f87de72304f
Summary:
Reduce the log level for a few messages to help reduce some less important
clutter when running with an elevated debug log level.
Reviewed By: chadaustin
Differential Revision: D19188773
fbshipit-source-id: 396bb15e119fc12765ecbc707e7a1dadbdd02422
Summary:
Remove some half-baked, unnecessary logic for caching sizes separately
from SHA-1. Eden's backing stores do not support chunking large files
yet, so there's no value in caching content SHA-1 and size
separately. This fixes a scenario where fetching blob size and then
SHA-1 would result in two backing store imports.
Reviewed By: fanzeyi
Differential Revision: D19169096
fbshipit-source-id: dc32f3313e5f4230c06a5bbaa67da7bf0febaba8
Summary:
Troditionally, ObjectStore relied on HgBackingStore writing directly
to LocalStore in order to cache trees. This had the unfortunate side
effect that other backing store implementations did not benefit from
tree caching.
Move tree caching into ObjectStore so all backing stores benefit from
tree caching.
Reviewed By: simpkins, fanzeyi
Differential Revision: D19168211
fbshipit-source-id: b1019591ebb4760cc8b933b9adb82174b8e5fa1f
Summary:
The doctor tests for available disk space checked the host's actual
disk, causing them to fail when run on machines with full disks. Add a
fake and use it by default.
Reviewed By: simpkins
Differential Revision: D19145853
fbshipit-source-id: b4202a4eabffd8b7a968de9fb4370c0a0ca99d4d
Summary:
If a GC ran at the same time as HgBackingStore imported a commit, the
import would fail. Rather than round-tripping the tree through the
LocalStore, just return it.
Reviewed By: genevievehelsel
Differential Revision: D19167798
fbshipit-source-id: eddb280def349483289d2e300a58eff3ad7416e1
Summary: There is one instace of `getScmStatusBetweenRevisions` in use - it is used in the eden cli in a hacky way to check if a commit hash is valid. Since this is not used anywhere else in a meaningful way, this replaces that use case with a hg call and depreciates `getScmStatusBetweenRevisions`
Reviewed By: simpkins
Differential Revision: D18690026
fbshipit-source-id: 02bd2c20a0f631ec41116f9fd4e18d14369298ef
Summary:
We don't support any of these VCS's, and we're not running any of the
tests. The code is just bit rotting. Let's delete them.
Reviewed By: quark-zju
Differential Revision: D19043710
fbshipit-source-id: 6e0d625c755cbc875755dc09b394bc730186db1d
Summary:
This diff turns the return type of `BackingStore::getBlob` from `folly::Future` into `folly::SemiFuture` to prevent executor leaks.
This also enable us to remove the need of holding `serverThreadPool` from backing stores.
----
**Changes**
* `ObjectStore` now needs to hold a `folly::Executor::KeepAlive` that is used to turn `SemiFuture`s it gets from backing stores into `Future`.
* Signature changes of the implementations of `BackingStore` class.
* For tests, I chose to use `QueuedImmediateExecutor` in place of `UnboundedQueueExecutor` as it will basically execute tasks inline. I'm concerned introducing thread pool executor in tests may turn tests flaky.
Reviewed By: wez
Differential Revision: D18669664
fbshipit-source-id: 0cae89f365dcf8b345b49d64469a530cf25d4ac5
Summary:
This fixes a regression in `eden top` introduced this summer where it
would gradually add to the terminal's scrollback. The cause was
accidentally using `window.clear` instead of `window.erase`.
Reviewed By: genevievehelsel
Differential Revision: D19061316
fbshipit-source-id: 4ef4c7bcd00e30ff5d3b80406ea01a476346faf9
Summary:
The current implementation iterates through all entries, which will include
deleted entries. Rewrite it to use index lookups so it does not print
deleted entries.
Reviewed By: sfilipco
Differential Revision: D18838993
fbshipit-source-id: 58d3f2da27cd3e91bc10cbb04bcdca3b7ff07dbb
Summary:
Address review comment in D18640899. I went a bit further that makes COUNT,
ALL, and BITS consistent.
Reviewed By: sfilipco
Differential Revision: D18820721
fbshipit-source-id: e354a3aecf0d8cbe66c43d56198933cdc6420241
Summary:
Address review comment in D18640898.
Id::min_value and Id::max_value become unused and are removed.
Reviewed By: sfilipco
Differential Revision: D18820722
fbshipit-source-id: d9682bf5918307166571fa35aa15cc9eee1a7140
Summary:
Address review comment in D18640899. This makes the word `id` universally
refer to a same concept within the crate.
Reviewed By: sfilipco
Differential Revision: D18820723
fbshipit-source-id: 6803192db7e1304a72100568f8f29b90f25c7779
Summary: There is no need to insert duplicated entries to IdMap.
Reviewed By: sfilipco
Differential Revision: D18670158
fbshipit-source-id: 812ea421c9f36cd12b4aa2be5b2560abec627a52
Summary: This emulates what IdMap the server will send to the client at initial clone.
Reviewed By: sfilipco
Differential Revision: D18670160
fbshipit-source-id: 496142c1e149df9accf816e3d0a1b8316ff64c67
Summary: This makes it easier to check what's stored in IdMap.
Reviewed By: sfilipco
Differential Revision: D18670159
fbshipit-source-id: 91387d01da7d52a0d349e87ffa0aa8983bbe2213
Summary:
This diff implements the following part in a local-only environment:
Id -> Slice: Id -> Request --> Response -> Slice
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Slice -> Id: Slice -> Request --> Response -> Id
^^^^^^^^^^^^^^^^^^^^^^^^^^
Reviewed By: sfilipco
Differential Revision: D18664210
fbshipit-source-id: 740b9cdfe533e7bcc9bbbf2a53c8bcc5bc3648bf
Summary:
This diff implements the following part in a local-only environment:
Id -> Slice: Id -> Request --> Response -> Slice
^^^^^^^^^^^^^
Slice -> Id: Slice -> Request --> Response -> Id
^^^^^^^^^^^^^^^^
Reviewed By: sfilipco
Differential Revision: D18664205
fbshipit-source-id: 47a35c6f95a58ab29c014ac3c6799f16e6114da3
Summary:
In the future the client will have a sparse IdMap, and needs the server (who
has a complete IdMap)'s help to convert between Id and Slice (commit hash)
for commits in the MASTER group.
This diff defines basic abstractions for communications between a sparse IdMap
and a complete IdMap. The basic flows are:
To convert Id to Slice: Id -> Request --> Response -> Slice
To convert Slice to Id: Slice -> Request --> Response -> Id
(`->`: client-side; `-->`: server-side)
Reviewed By: sfilipco
Differential Revision: D18664206
fbshipit-source-id: 93a6b671be30ab1247c71d814817a2ea06b6bc66
Summary:
Add a config option to restrict the number of tree prefetches that can run in
parallel. Without this applications that recursively walk a checkout tree can
end up spawning a huge number of asynchronous prefetch operations. These
prefetch operations are quite expensive today, as we currently have to fetch
the full file contents in order to retrieve the file sizes. A recursive
directory walk can end up building up a huge backlog of in-progress tree
prefetches. This will slow down the directory walk, and it can take tens of
minutes to finish processing the prefetch backlog even after the directory
walk is aborted.
Reviewed By: chadaustin
Differential Revision: D19136685
fbshipit-source-id: cdc7a570d016fd7ca81a639cef83380b197acdfc
Summary:
On Windows, subprocess.Popen caused the current process to wait until
the background process was complete, which meant commit, amend, rebase, etc had extra long waits on the end. We have utility functions that correctly set up the background process, so let's use that instead.
Unfortunately we can't redirect stdin, stdout, stderr when using close_fds on
Windows, so we lose the ability to do logging.
Reviewed By: quark-zju
Differential Revision: D19149682
fbshipit-source-id: 6880ec7833e286b79b34c49c339aff3318f07295
Summary:
This is part of the logic of remotenames. Unfortunately our production setup
has some really annoying legacy code in multiple places that disables
remotenames and do `rm .hg/bookmarks` manually. They should really enable
remotenames and get rid of `rm .hg/bookmarks`. However that has complications.
With recent changes, `.hg/bookmarks` is moved to `.hg/store/bookmarks` and
later moved to `metalog`. That caused 10k+ bookmarks left in local bookmarks
after clone (or pull).
Writing remote bookmarks as local bookmarks makes no sense in modern setup.
Therefore this diff adds a config option to disable such feature.
The ideal state is the bookmark logic in exchange writes remotenames directly,
instead of silently ignoring it. But that's a risky and larger change. My
expectation is, the remotenames extension will be enabled and take care of the
rest.
The idea of not writing local bookmarks come from markbt.
Reviewed By: DurhamG
Differential Revision: D19148278
fbshipit-source-id: a03eac68378908586d6e6848ed43532c0aab06c3
Summary:
A future diff will remove the tags feature. Convert uses tags heavily
and it breaks a number of tests, so let's remove tags from the convert extension
before we remove tags entirely.
Reviewed By: quark-zju
Differential Revision: D19043330
fbshipit-source-id: 628d27fea1601931da45d1280eff981c5d12f93c
Summary:
The buck generated binary is 1.2GB in size. Drop dependency to reduce the size.
Dependent functions are re-invented.
New size is 31MB under `mode/opt` build, and 28MB under `mode/dev` build.
Reviewed By: wez
Differential Revision: D18900827
fbshipit-source-id: 536fa969d69f6261d812c2320795780d839b6ced
Summary:
In a future diff we'll be removing tags. The most prevalent tag is
'tip', which shows up in a ton of test output. Let's drop that tag first, so we
can safely update the tests before we drop tags entirely.
Reviewed By: xavierd
Differential Revision: D18995058
fbshipit-source-id: 8c63710cd4ed567ea24e32724b8660f9006a61f1
Summary:
A future diff will remove tags entirely, so let's move drawdag to be
bookmark based.
Reviewed By: quark-zju
Differential Revision: D18995059
fbshipit-source-id: 70ef67259b37ef9821009d0145aa1e03c09b1309
Summary:
Rolling back the entire transaction can causes Eden to get out of sync
with Mercurial and cause hangs. It can also causes users to lose the work
they've done splitting the commit.
Let's change split to commit the transaction after every commit. If a user exits
mid-split they will now be left in a state where the splits they've done so far
are still present, and we print a message telling them how to roll all the way
back if they want to.
Reviewed By: quark-zju
Differential Revision: D19114546
fbshipit-source-id: b67d5543f9c23a6299f4164c73662759661f59a9
Summary:
EdenMount offered both getRootTree and getRootTreeFuture. Replace the
former with the latter and adjust remaining callers.
Reviewed By: genevievehelsel
Differential Revision: D18666585
fbshipit-source-id: 961361a1fb545e48db0ece970bb5e6ffbca8efef
Summary:
The last remaining user of ObjectStore::getBlobMetadata was a debug
Thrift call. Remove it and update the Thrift call to use getBlobSize
and getBlobSha1.
Reviewed By: pkaush
Differential Revision: D18663376
fbshipit-source-id: 86baefc9004a07aac4ddf5849870431be04c75f2
Summary: This should allow us to show a spinning progress bar when repair() is working.
Reviewed By: xavierd
Differential Revision: D19056949
fbshipit-source-id: 8e058bad002bb2de29dbf8457bead0ebd1e99372
Summary: Fix some warnings that only show up in the macOS build.
Reviewed By: fanzeyi
Differential Revision: D19053236
fbshipit-source-id: 81f7187b263e0db6a57582677088519f9b97f1d7
Summary:
The current code crashes with `phaseroots` being an unknown attribute.
Reported By: DurhamG
Reviewed By: DurhamG
Differential Revision: D19048762
fbshipit-source-id: f59764ec04284cd643fbb5a3c319868df06c4912
Summary: There are some modules missed in the preimport list. Add them.
Reviewed By: xavierd
Differential Revision: D18957117
fbshipit-source-id: 254384f0bcafd52421fb22dc696b3a6a1fb98537
Summary:
Similar to D18926784.
Mercurial makes sure content written by `ui.write` is flushed, while there is
no such guarantee for `print`. When running using chg, it's more likely that
`print` content gets disappeared. Therefore let's use `ui.write` instead.
Reviewed By: ikostia
Differential Revision: D18926983
fbshipit-source-id: 960c189cc48bc3c89bb4f7993d2033dd45c52a67
Summary:
This is mostly result of:
```
find . -type f -exec sed -i "s/fbconduit/fbscmquery/g" {} \;
```
`fbconduit` extension doesn't use conduit anymore so the name is just misleading.
Reviewed By: ikostia
Differential Revision: D18748843
fbshipit-source-id: 0d59e61ba7a96d86d9d1333d81255108cc3141bc
Summary: This looks like a fallout of D18666054
Reviewed By: quark-zju
Differential Revision: D18992910
fbshipit-source-id: c2392553a0489097f82b4073bbbca37b43f53bba
Summary:
This is showing in profile for about 10% of the time, let's just disable it as
getting files should be fairly quick.
Reviewed By: quark-zju
Differential Revision: D17841315
fbshipit-source-id: 08a61e19351fb5217431a3ca8993b8074f6d6c74
Summary:
There is no need to duplicate the code.
The test change are due to commitpending being called 3 times, vs 1 before. The
total amount of times {fileslog,manifestlog}.commitpending is called is unchanged.
Reviewed By: quark-zju
Differential Revision: D17841316
fbshipit-source-id: be33a126f913af8824a6d5ade17351707d39ed2f
Summary: The smart traceback is generally more useful.
Reviewed By: markbt
Differential Revision: D18666054
fbshipit-source-id: a96bb3e2919ed6692c4a7b965ad74cd2c8a66241
Summary: Add type annotations to all functions in this file.
Reviewed By: xavierd
Differential Revision: D18949804
fbshipit-source-id: 4ac1789aa086e9e5f2397bc005a33cfb2f6c0b56
Summary: This keep failing due to single line changes. Let's make it more robust.
Reviewed By: kulshrax
Differential Revision: D18963816
fbshipit-source-id: c56748560b619c4c682e8cc9e5710c607aa363fb
Summary:
D18263067 updated the `CMD_FETCH_TREE` handler to fetch tree with a depth
of 1, rather than using fetching the entire recursive tree structure.
I think this has contributed to some of the recent user-reported slowness for
fetch operations, as fetching a large directory structure now triggers many
more individual requests for each separate tree.
This diff adds a configuration parameter to control the tree fetch depth. I
have set the default value at 3, which should hopefully provide a reasonable
level of tree prefetching without completely fetching all contents of deep
trees.
Reviewed By: wez
Differential Revision: D18942585
fbshipit-source-id: 6a8d749434520baee25a4277712c44b916adcb3f
Summary:
I made a mistake when editing the last version of the code. It uses
indexedlogdatastore to repair indexedloghistorystore, which is incorrect.
Fix it.
Reviewed By: xavierd
Differential Revision: D18945952
fbshipit-source-id: addec020c761e6c11d2fe485eb408f5b200afb08
Summary: This is some documentation I've taken while studying the `Takeover.*` classes. It could be helpful to live in this folder for future reference. If I missed anything or misrepresented anything, please let me know! I plan to keep this updated as changes are made to files in this folder.
Reviewed By: simpkins
Differential Revision: D18868291
fbshipit-source-id: 8fc65dd3dd6f1b5bcc344b5f4c2b5b9ceb11ed17
Summary:
Make util.smarttraceback:
- Support `traceback` object.
- Look more similar to normal traceback (headers, and indentation).
- Hide boring content.
- Show binary hashes in hex form.
Reviewed By: markbt
Differential Revision: D18666055
fbshipit-source-id: d61446604c673ec5e1a96912fa29935d4238ecc9
Summary:
See also D5271419. Decorators in `bundle2` and `exchange` are not side-effect
free. Move them to `uisetup` for better chg compatibility.
Reviewed By: StanislavGlebik
Differential Revision: D18926786
fbshipit-source-id: 2a3fa6ed4cef42c9bad4666ff75496d483991f88
Summary:
See also D5271419. Decorators in `bundle2` and `exchange` are not side-effect
free. Move them to `uisetup` for better chg compatibility.
Reviewed By: StanislavGlebik
Differential Revision: D18926782
fbshipit-source-id: b99e2882cb6530a648a065c6d739ee9a7aebb851
Summary:
See also D5271419. Decorators in `bundle2` and `exchange` are not side-effect
free. Move them to `uisetup` for better chg compatibility.
Reviewed By: StanislavGlebik
Differential Revision: D18926783
fbshipit-source-id: 9855b8debe3f809139a9fc3c6eea2775a1562cfd
Summary:
See also D5271419. Decorators in `bundle2` and `exchange` are not side-effect
free. Move them to `uisetup` for better chg compatibility.
Reviewed By: StanislavGlebik
Differential Revision: D18926787
fbshipit-source-id: cfbf8de6d8c841478d7aa3de180a0816349fa4e4
Summary: This looks like an omission from D18912817
Reviewed By: mitrandir77, ikostia
Differential Revision: D18935510
fbshipit-source-id: f72d95155c54d7eb36ff7c868f5e6b0ff6d6e39c
Summary:
The `reset` extension does its own obsmarker creation, rather than going
through `scmutil.cleanupnodes`. This means it doesn't support new-style
visibility tracking.
Fix this by making it use `scmutil.cleanupnodes`. This isn't completely
straightforward:
* The revset it uses to work out what to prune might accidentally include
public commits (e.g. when you reset to a different public branch). Make sure
these are filtered.
* The tests originally had strip and obsmarker based tests. The strip tests,
when converted to obsmarkers and using `scmutil.cleanupnodes` stop working
because they strip and revive the same commit over and over, which is an edge
case that obsmarkers can't handle well. Fix this by restoring the strip tests
as strip tests. A separate test handles the new-style visibility.
* Reset's behaviour is still a bit wonky. If an ancestor of the source commit
has other (non-bookmarked) descendants, then reset will try to prune those
commits. New-style visibility will ignore this, and this is tested in the
new test.
Reviewed By: farnz
Differential Revision: D18912817
fbshipit-source-id: cc115333407cf67d339c24fcd0807ddedce2660d
Summary:
D18820280 increased the log level for `globFiles()` calls from DBG3 to DBG2.
However, buck and other tools often end up huge numbers of glob calls, and
this ends up causing log spam.
This change reduces the log level of `globFiles()` back to DBG3. In a
subsequent diff I will update the code to increase the log level for slow
calls. This does mean that we will only get a log message for slow glob calls
after the call completes, but this is better than not getting a message at
all.
Reviewed By: pkaush
Differential Revision: D18924780
fbshipit-source-id: 86e83614f51424aca078e4de8a56c45018a6d286
Summary: was looking in this area and saw this TODO. moves the `setParentCommits()` call into `CheckoutConflict::finish()` so it is called while we are still holding the parents lock.
Reviewed By: chadaustin
Differential Revision: D18668329
fbshipit-source-id: 8415c792193e5b89737f15141f33c5a9799f527b
Summary:
This solve the following issue when running on devserver with demandimport enabled:
In [1]: ui.config?
abort: No module named windll!
Reviewed By: xavierd
Differential Revision: D18898009
fbshipit-source-id: bf4f3b12ac6f0a1628ed8ef805f6843868363542
Summary:
A spike in automatic GCs usually implies something has gone wrong. Log
an event for each one, recording the cache size prior to the GC and
the cache size after.
Reviewed By: simpkins
Differential Revision: D18902580
fbshipit-source-id: 158b2635733a415a9fcc7c412b2c0f44ed04aa01
Summary:
On Windows the error message is different. In this test we only need to verify
a command fails. Therefore remove the output and only keep the exit code.
Reviewed By: xavierd
Differential Revision: D18915580
fbshipit-source-id: bd1031170335918d507e6225b02541ec52bc62fa
Summary: Enabling getScmStatusBetweenRevisions to work on Windows and its tests.
Reviewed By: simpkins
Differential Revision: D18431271
fbshipit-source-id: eee82538e2fc3d7e371c96fc271cd9662ea6d737
Summary: This diff contains the tests for hg status. This is not comprehensive list of tests but the tests that we have working at this time. More tests will be added as we test and fix issues.
Reviewed By: simpkins
Differential Revision: D18454851
fbshipit-source-id: 80e3ff4d24873016dc420f202fbfe53fcffc24f2
Summary: The current implementation of EdenDispatcher is projectedfs specific and was linking against its dll. This was adding the need to add Prjfs dll for running our tests. Moving it to prjfs channel should fix that.
Reviewed By: simpkins
Differential Revision: D18462171
fbshipit-source-id: ff7bfcdd14d19552d49b61ff03cdd770badfcd68
Summary: This class provides an interface to create an EdenMount for testing and use it through the APIs. It's similar to TestMount for Eden on Posix.
Reviewed By: simpkins
Differential Revision: D18454852
fbshipit-source-id: 91d64b2c535060979ef634fe4e7779f50f169fc5
Summary: WinStore interface was taking wstring for path and we were converting wchar pointers to wstring which were later converted to RelativePath. In this diff we pass the path as is and save one copy per operation.
Reviewed By: simpkins
Differential Revision: D18444773
fbshipit-source-id: fa6a3efb6a7e91cf9b0dca919e80870e1bcf0804
Summary: Moved the current FsChannel implementation in to PrjfsChannel and converted the FsChannel.h into an abstract class. Created a TestFsChannel class for testing.
Reviewed By: simpkins
Differential Revision: D18440401
fbshipit-source-id: 30d6742cd18b4b76b664e3af7ed1a528cf876046
Summary:
A successful run of hg commit ends in call to resetParents() to reset the current parent commit.
We also need resetParent to run the Status tests.
Reviewed By: genevievehelsel
Differential Revision: D18439954
fbshipit-source-id: 44f7a009e050bd5e2130d8c4fc3a85a8ec3bfad1
Summary:
GenerateStatus will parse the CurrentState and compute the status. The algorithm here is little different than of Eden with FUSE, Eden with Prjfs will loop over the CurrentState and only compare the entries present in it. CurrentState has the list of all the entries modified and a complete status could be generated from it.
This doesn't solve all the scenarios. I have few unit tests running at the top of this stack and those cases are working. I need to test the other cases and make them work. One case that I already know doesn't work is when a user resets the parent and doesn't make any change. I will work on fixing the other cases and put up diffs for that.
Reviewed By: simpkins
Differential Revision: D18195573
fbshipit-source-id: 3650f26387f1a5f8d467fd81148c266fa1307a3c
Summary: This diff makes EdenFS to be able to use EdenApi to import blobs and trees when requested objects are not present in local hgcache.
Reviewed By: chadaustin
Differential Revision: D18605547
fbshipit-source-id: 4acd2e07cfd9de6b6775ded30ea22a4478b9f1e4
Summary: This will allow us to use EdenApi in EdenFS to fetch trees and blobs.
Reviewed By: xavierd
Differential Revision: D18622844
fbshipit-source-id: 59a9091e9f2fdbcae078da2fb24ee9c0dd18505b
Summary:
Add an option `experimental:use-edenapi` to `EdenConfig`.
See the next diff for usage.
Reviewed By: chadaustin
Differential Revision: D18605549
fbshipit-source-id: 2786c21bb38a76229078662cc5c1ddf906d1be4a
Summary:
I think we need to make `EdenConfig` available for testing as more things are using `EdenConfig` to configure behaviors.
Right now it does not do much things other than just construct an `EdenConfig` instance without manually providing these parameters.
Reviewed By: chadaustin
Differential Revision: D18798484
fbshipit-source-id: b4a85d966a9b4f425c547bef9eb6e7570d7e2170
Summary:
Two bugs conspired to cause edenfs after a graceful restart to think
the kernel supported FUSE_NO_OPENDIR_SUPPORT when it didn't: the
connection info struct wasn't zeroed, and FUSE connection capabilities
weren't properly mirrored into the Dispatcher upon graceful
restart. Fix both and add an integration test.
Reviewed By: simpkins
Differential Revision: D18903761
fbshipit-source-id: 23f4db3e240ee7d035f707820072c606a45f1138
Summary: Ideas for improvements. Put them in the code for future consideration.
Reviewed By: quark-zju
Differential Revision: D18870140
fbshipit-source-id: a2a84d94d72303adc64614b06e36588ff7778aab
Summary:
The list functionality is required by EdenFs. We want this functionality
to be well supported by the Manifest.
Reviewed By: quark-zju
Differential Revision: D18870143
fbshipit-source-id: 1ebaa713ff521226e6ace22cbd35cc841d967298
Summary:
Consistency in naming. The general idea is to have Metadata types that don't
contain paths. Then we will have File, Directory and eventually FsNode that
will contain paths.
Reviewed By: quark-zju
Differential Revision: D18870141
fbshipit-source-id: a1f09add7f1c3dd4fa0348693cd3ce2fd5767fa7
Summary:
This rename is going to make it easier to import and use outside of
manifest specific crates.
Reviewed By: quark-zju
Differential Revision: D18870142
fbshipit-source-id: 2f3ea460170308162ee834efc038b2dcedd9e233
Summary:
Since we got `repair` APIs in the Python land, use them to attempt to repair
things.
This would hopefully be able to fix some user-reported issues, usually
caused by hard reboots.
Reviewed By: xavierd
Differential Revision: D18737907
fbshipit-source-id: ef3deccbf4a1c959cff7e3eb78771acd9ae9bd01
Summary:
Register the logging and logging_full option handlers in the open
source and macOS builds of Eden.
Reviewed By: simpkins
Differential Revision: D18870545
fbshipit-source-id: 140debe77be071a9fa2d628ad305517273e095e3
Summary:
From Dulwich changelog:
0.19.0 2018-03-10
API CHANGES
GitClient.send_pack now accepts a generate_pack_data rather than a
generate_pack_contents function for performance reasons.
(Note: this ignores all push blocking failures!)
Reviewed By: quark-zju
Differential Revision: D18901269
fbshipit-source-id: 9510b3a516cb2fcbeae9f1f2072d8445e442a9ed
Summary: Adds a function which takes both the manifestID and the commitID to get a Tree. This will be used in `checkOutRevision()` and this allows us to skip looking up the manifestID since the caller can just pass it in themselves.
Reviewed By: wez
Differential Revision: D18719405
fbshipit-source-id: 919f0a7c84bff4a2f0bc20110c45bd272f9e9107
Summary:
It is very painful for Eden's automatic GC to delete trees from the
local store. An unfortunate number of programs scan the entire repo,
causing every tree to be loaded. If Eden's automatic GC kicks in, it
causes all trees to be downloaded again, blocking both FUSE and Thrift
requests.
Temporarily mark trees as persistent.
Reviewed By: fanzeyi
Differential Revision: D18842066
fbshipit-source-id: 4a22defcb1151912337d4167ac49852519c7789d
Summary:
`buck run :run_tests -- ...` was added to be a buck version of `run-tests.py`.
However it does not work because the watchman path provided was invalided.
Reviewed By: xavierd
Differential Revision: D18892361
fbshipit-source-id: 8745a33e0d19b14f042ef1cd255d10c5737dffb1
Summary:
Add `#chg-compatible` to 572 tests that seem to pass with chg enabled.
This should make them run faster.
Reviewed By: xavierd
Differential Revision: D18870507
fbshipit-source-id: fe895e733efffc9286cd3d17c7a156c803124395
Summary:
Previously, it's fine to migrate up to metalog, and fine to migrate down once
since we double write data. However, re-enabling metalog can be problematic
since there is no code path to do "migrate up" again.
This diff fixes the issue by tracking what files (or "keys") are using metalog
as the source of truth in metalog. So we can accurately figure out whether to
migrate svfs files to metalog on demand.
Reviewed By: xavierd
Differential Revision: D18864424
fbshipit-source-id: e61e1790c231f9c88de869f413f27bb954a29920
Summary: Expose the repair API to the Python world.
Reviewed By: xavierd
Differential Revision: D18737913
fbshipit-source-id: c31085727589b6938c2fafb28897925aea617bc4
Summary:
Provide a way to repair the MetaLog. It's not just repairing 2 indexedlog
structures, but also checking the relationships between them.
Reviewed By: xavierd
Differential Revision: D18737904
fbshipit-source-id: fcf8ae56a1fdbb0561765701d962dfad4a8b5bd4
Summary:
Detect `#chg-compatible` in test header and force enable chg. This would allow
us to mark tests as chg-compatible to get some speed ups.
Reviewed By: xavierd
Differential Revision: D18870508
fbshipit-source-id: 923d6cb10b0c621d9995809b60ae535126f6f2a5
Summary:
The chg logic is inlined in the "hg" binary so the old logic running "chg" as
an external binary needs to be changed.
The version check part can be annoying - it requires more frequent `make local`
to run the tests while `make local` is actually not necessary. So let's just
skip the version check if running in tests.
Reviewed By: ahornby
Differential Revision: D18870510
fbshipit-source-id: 2343549c4080a0425098698b5423c6cfc1f60c77
Summary:
We only support the "C" policy, therefore let's inline it and simplify the code
significantly.
Reviewed By: quark-zju
Differential Revision: D18819680
fbshipit-source-id: 0bc0a12b85ddc42af007d3a20c0f4e8497bedb21
Summary:
This is not available on Python3, instead stat.ST_MTIME and friends should be
used to get an integer time. This is achieved by wrapping the output of the
stat function and overriding the st_[acm]time properties.
Reviewed By: quark-zju
Differential Revision: D18819679
fbshipit-source-id: 2911f3d47506456e927872d57f69253d903618e2
Summary:
The environment variable `EDENSCM_TRACE_OUTPUT` specifies where to write the
tracing output for the current command. Environment variables are inherited
by subprocesses by default. That is undesirable because another hg command
(triggered by hook, or background maintaince) will rewrite the trace output.
Avoid it by unsetting EDENSCM_TRACE_OUTPUT for subprocesses.
Reviewed By: xavierd
Differential Revision: D18892973
fbshipit-source-id: 575b6c0df2e7a0775172893e4aa72ca33fa4658c
Summary:
Some users were reporting hg gc failures because it couldn't access a
hidden commit. It looks like there's a `repo.unfiltered().revs("draft()")`
line to get the list of draft commits, so we need to also use an unfiltered repo
when extracting all the pointers.
Reviewed By: quark-zju
Differential Revision: D18863471
fbshipit-source-id: 61220c1ec3dcbf55879211bb271b4d9b686d66a1
Summary:
D18732257 changed our fastlog integration from using conduit to using
scmquery. This changed the result type from a hash to a dictionary of {"hash":
"xxx"}, but the consumer code wasn't updated appropriately. This fixes it.
Reviewed By: mitrandir77
Differential Revision: D18833566
fbshipit-source-id: 2573cbf4f93c5e4aeb2515d42696526ceb51428e
Summary: Saves loadFileFromPath in the DiffContext. This allows the source control differ ( at `eden/fs/store/Diff.cpp`) to call this function while it is trying to load gitIgnores and resolve symlinks.
Reviewed By: simpkins
Differential Revision: D18647088
fbshipit-source-id: bddff0158130f0c96ac263bcce81dc51fc017800
Summary: Adds a helper function that can take a path to an Inode and load a file by simply getting the Inode then calling the existing `loadFile()` function
Reviewed By: simpkins
Differential Revision: D18647090
fbshipit-source-id: 89c25bef10a7a0d0ffd660293f85479ac0e9f9a9
Summary: Move the bulk of the gitignore loading to EdenMount, this is needed later in the stack to pass this function as a callback through the DiffContext to enable gitignore loading from the source control tree differ.
Reviewed By: simpkins
Differential Revision: D18565265
fbshipit-source-id: 062349d81bbfb78235ccb34d6ec72ad445a6e2b0
Summary:
The previously behavior silently uses the empty Root upon "id not found" cases,
which hides data corruption issues captured by the next changes.
Reviewed By: xavierd
Differential Revision: D18737906
fbshipit-source-id: 0f2cfe1b05f11e9d2584d96a14afcb1aed80da4d
Summary:
Recently many tests become unreliable due to MySQL errors like:
OperationalError: 2055: Lost connection to MySQL server at '<host:port>', system error: 32 Broken pipe
That makes the testing infra think our tests are broken and cause:
# Test Failure Bot
Test is now disabled because it has been consistently broken
Avoid that by detecting MySQL errors and mark tests as skipped.
Reviewed By: simpkins
Differential Revision: D18668303
fbshipit-source-id: 02cb09f31c712fed41903f1e8c4ad45a1d277c30
Summary:
It was used to highlight "more interesting" (aka. extension) code. Nowadays the
non-extension part of code is also interesting. Therefore drop the extension.
Reviewed By: simpkins
Differential Revision: D18666053
fbshipit-source-id: 027190f742015bdb9e7dba109e7a28987af3cf68
Summary: Otherwise the repair will fail with "unable to lock directory" errors.
Reviewed By: xavierd
Differential Revision: D18737911
fbshipit-source-id: 2c47b5b9a071e0db2d20fc385ef466aef1e732f6
Summary: This makes it possible for other crates to implement the `Repair` trait.
Reviewed By: xavierd
Differential Revision: D18737909
fbshipit-source-id: 3d098e583b45e84eb4bd01ef45569e9694b63ed6
Summary:
This simplifies the logic a bit by getting a free `repair` method from
indexedlog.
Reviewed By: xavierd
Differential Revision: D18737908
fbshipit-source-id: 4988c1a83b7709b751cd1899c5663acc0c42e313
Summary:
The goal is to make it easier for different structures to implement `repair` by
just providing `OpenOptions`.
Reviewed By: xavierd
Differential Revision: D18737903
fbshipit-source-id: af3c18038c5bb2f91f53ab53926ec097e1d87e6b
Summary: Uses `folly::IOBuf`'s `TakeOwnership` instead of `CopyBuffer` to achieve zero-copy of file content we get from Rust.
Reviewed By: chadaustin
Differential Revision: D18264900
fbshipit-source-id: 45baa70cb771a2bff0b31feaa1b85569625218eb
Summary:
This isolates the core types in manifest so that it is harder to
create unsound dependencies with specific implementations.
Reviewed By: quark-zju
Differential Revision: D18843133
fbshipit-source-id: 4b866ad84d2e7d0ff2dc4ec6bd65f66548c3fe4a
Summary: Signal more clearly that the iteration in the cursor is depth first.
Reviewed By: quark-zju
Differential Revision: D18843135
fbshipit-source-id: 8ee2c612963eaf6fec3ae48f75ebf16bd177fed4
Summary: Items feels a bit generic as a name.
Reviewed By: quark-zju
Differential Revision: D18843136
fbshipit-source-id: 29a904f3a8a1996760e71bd78610bdbfef4ad65b
Summary: No longer strictly related to files.
Reviewed By: quark-zju
Differential Revision: D18843128
fbshipit-source-id: c7853e08aa5d0256616e777c5bb6dd477b9ebc97
Summary:
`hgid` can be easily derived from the link member variable. Using link is
preferable because link is the owner of the data. It makes the logic more
specific, less bug prone to use the true source of the data.
Reviewed By: quark-zju
Differential Revision: D18843137
fbshipit-source-id: 443f1e5eb4498fa4f2ff9904dba7cc3dc4305add
Summary:
This structure is not public. It is private to the manifest crate so there is
no point to have those modifiers.
Reviewed By: quark-zju
Differential Revision: D18843134
fbshipit-source-id: 0794bbbf578863cabbd85ada87669b791ebcc72d
Summary:
The purpose of this change is to disambiguate from the other Directory struct
that we have in the Manifest. Right now that other is in manifest/lib.rs
Reviewed By: quark-zju
Differential Revision: D18843130
fbshipit-source-id: f25a60f3040cc19abcab99ac5f2cdbff83ccf7ee
Summary:
Directory<'a> which wraps a link is closely linked with the Link abstraction.
It can be seem as a convenience wrapper around Link.
Reviewed By: quark-zju
Differential Revision: D18843129
fbshipit-source-id: 2b677d71101bff3708a65e03bbcddf5074d51387
Summary:
Diff is a core operation for manifests. The Manifest trait should require that
it is implemented.
Reviewed By: quark-zju
Differential Revision: D18843132
fbshipit-source-id: 5e3febab669470e266878d934561dfc323b71c7f
Summary:
A previous change to the BFS iterator fixed it to return the root of the repository.
This influenced the `debugmanifestdirs` command which was manually
handling the root of the repository. This diff updates the command
to match the old behavior. Basically after the fix to BFS we are
not printing any characters for the path of the root. Forcing the debug command
to print of a '/'.
(Note: this ignores all push blocking failures!)
Reviewed By: quark-zju
Differential Revision: D18862167
fbshipit-source-id: 9ca6da707ae8ab037e88a2bbce3b3993b5d219e3
Summary:
In most cases nowadays people run into worse experiences when their operations
run through the fsmonitor path. Watchman is reliable enough that we can rely
on it to work better than the default crawling algorithms.
Reviewed By: quark-zju
Differential Revision: D18835322
fbshipit-source-id: 30873c7d4b1ec306e59b6152c20833cc0027cad5
Summary:
This is a test that aims to validate that commit manifest match between the C++
and Rust implementations.
Reviewed By: quark-zju
Differential Revision: D18521823
fbshipit-source-id: 678b82239e9be7ad078f1e04a6467e87adaac966
Summary: The conversion from tuple is not used much. I don't like them that much either.
Reviewed By: quark-zju
Differential Revision: D18843131
fbshipit-source-id: fb06b72a279b59ee8af5e43d83a10e39a576730b
Summary:
Querying all the files in a Manifest is core functionality that every Manifest
implementation should implement. The main question is how do we describe that.
We want to return an interator over the files of the manifest. The problem is
that the manifest is a trait and functions on traits can't return traits.
The solution is to Box the results. Not pretty but allows us to move the code
forward.
Reviewed By: quark-zju
Differential Revision: D18822854
fbshipit-source-id: ca44b24816203049f747ad67af48e33370861024
Summary:
The purpose of the rename is to signal that this function may potentially do
a lot of work, even calling and waiting on network requests.
Reviewed By: quark-zju
Differential Revision: D18822855
fbshipit-source-id: 98471944c8e6c7cffa7a2a11f2f733274802b376
Summary:
Removing the contructor for File that takes dependency on
tree implementation details.
Reviewed By: kulshrax
Differential Revision: D18822858
fbshipit-source-id: aed33b146dfdbdb23721a0f6e96977025d71a627
Summary: The types in file.rs being in a separate file confuse people.
Reviewed By: quark-zju
Differential Revision: D18822856
fbshipit-source-id: 29aa10c76bb5ef74f1fc0bf726e6b3cfaf19b428
Summary:
For now pushrebase updates working parent, bookmarks. This diff makes it also
update remote bookmarks.
This resolves a race condtion in remotenames where the pulled remote bookmarks
have unknown nodes and cause update to silently fail.
Reviewed By: sfilipco
Differential Revision: D18601035
fbshipit-source-id: 6f2c318cbf6b29a6427eeac6d374c1fb13e6155c
Summary:
Enable visibility and mutation to get modern features. Remove legacy configs.
As we're here, add more `log -G` outputs to make the test easier to reason about.
Reviewed By: xavierd
Differential Revision: D18601036
fbshipit-source-id: f0d656c696708c045c63f5b8e59b9a40f2dbbf20
Summary:
Move it to configuration.
This gives better control over how grep is executed. Previously the assumption
was that xargs was present in PATH. This updates removes that assumption.
Reviewed By: quark-zju
Differential Revision: D18715471
fbshipit-source-id: 09c3098984588cd7478172db0bcfb8d5cb8243fd
Summary:
`reachableroots(roots, heads, includepath=False)` returns a subset of `roots`.
I implemented `reachableroots` using the documented revset using segmented
changelog APIs and noticed smartlog graph is not correct. This diff fixes
the documentation.
This diff was also sent upstream as https://phab.mercurial-scm.org/D7518.
Reviewed By: ikostia
Differential Revision: D18723363
fbshipit-source-id: ed56ae85345dcf102159c4df0eb19c02cef2bf27
Summary:
The `update` hook gets called even for in-memory updates that does not affect
real filesystem (or edenfs). That hook is set to `edenfsctl redirect fixup`,
which adds 0.35s overhead per commit when doing in-memory rebase.
Define a edenfs-only update hook to remove the overhead.
This is part of a stack of optimization that makes a 20s rebase to 5s (0.6s per
commit to 0.07s per commit).
Reviewed By: xavierd
Differential Revision: D18838470
fbshipit-source-id: cbe81f9dd89bff6d7c4bec3063d6014211f9aac6
Summary:
With narrow-heads before the new dag structure, reading phases is expensive.
Avoid it.
This is part of a stack of optimization that makes a 20s rebase to 5s (0.6s per
commit to 0.07s per commit).
Reviewed By: xavierd
Differential Revision: D18838468
fbshipit-source-id: 0847d2508114203f1d16a7240a43375d325103a5
Summary:
With narrow-heads, rebase operation does not need to read existing phases to
decide new phases. Therefore remove it.
This is part of a stack of optimization that makes a 20s rebase to 5s (0.6s per
commit to 0.07s per commit).
Reviewed By: xavierd
Differential Revision: D18838469
fbshipit-source-id: 1298d4f4b0d72a764a286478e76c90b42b3fc569
Summary:
Use all available lines, now that we ignore the spurious errors that
can be thrown when calling `addnstr()` on the last line.
Reviewed By: wez
Differential Revision: D18826918
fbshipit-source-id: 7e5739207ba1a5bfa0d290da1345c9d9606b89fb
Summary:
Add a wrapper function around `addnstr()` that silently ignores any errors.
When writing to the last cell of a terminal curses will successfully display
the data, but report an error since the logical cursor cannot be moved to the
next cell. This error is not important and can be ignored.
The errors reported by the `curses` module unfortunately are not very useful
(they normally just report `addnwstr() returned ERR`). For now we silently
ignore all errors, although this diff does contain some stub code that can be
enabled to examine them during development.
Reviewed By: wez
Differential Revision: D18826917
fbshipit-source-id: a4c5394b49aea1334def1d047eec7b5e743fb9f2
Summary:
In the future we might have revlog or more complicated storage for maintaining
the IdMap. Define a trait for it.
Reviewed By: sfilipco
Differential Revision: D18664208
fbshipit-source-id: 242611d78852e8d82a52ab52192db87c13b848e7
Summary: This is the reverse of first_ancestor_nth. It will be used later.
Reviewed By: sfilipco
Differential Revision: D18664209
fbshipit-source-id: 8d3706cdc080086ec60da60870fde76c8933f6a8
Summary: This will be used in an upcoming change.
Reviewed By: sfilipco
Differential Revision: D18664207
fbshipit-source-id: 3030bb02e172b95c6dbabda74c318e7f16212f59
Summary:
With the ONLY_HEAD flag, ancestors can be sped up sometimes. Since ancestors
is used widely in other DAG calculations, perf improvements are visible across
the benchmark. The trade-off is to spend a bit more time on building segments
to figure out ONLY_HEAD.
After this diff:
building segments 576.625 ms
ancestors 47.119 ms
children 436.967 ms
common_ancestors (spans) 257.096 ms
descendants (small subset) 367.316 ms
gca_one (2 ids) 146.315 ms
gca_one (spans) 253.059 ms
gca_all (2 ids) 236.075 ms
gca_all (spans) 266.695 ms
heads 220.766 ms
heads_ancestors 36.258 ms
is_ancestor 98.967 ms
parents 219.173 ms
parent_ids 9.282 ms
range (2 ids) 99.859 ms
range (spans) 205.154 ms
roots 429.047 ms
Before this and previous diff:
building segments 542.564 ms
ancestors 316.075 ms
children 428.612 ms
common_ancestors (spans) 452.850 ms
descendants (small subset) 537.483 ms
gca_one (2 ids) 526.112 ms
gca_one (spans) 446.726 ms
gca_all (2 ids) 780.590 ms
gca_all (spans) 490.906 ms
heads 214.005 ms
heads_ancestors 217.558 ms
is_ancestor 256.769 ms
parents 213.170 ms
parent_ids 9.313 ms
range (2 ids) 259.435 ms
range (spans) 358.934 ms
roots 423.005 ms
Reviewed By: sfilipco
Differential Revision: D18642797
fbshipit-source-id: 27f1ee5797a92142b2c19195eb26baef58769553
Summary: The pre-calculated flag can be used for some fast paths.
Reviewed By: sfilipco
Differential Revision: D18642796
fbshipit-source-id: bc16b3517cc29e23478f2027e83e2a2282be0d0a
Summary:
The bindings APIs are out of date.
The latest idea is to write non-master commits to disk, in a different group.
There are no "segments in memory". Therefore:
- Rename `builddisk` to `build`.
- Make sure `build` updates internal states by calling `reload`.
- Drop `buildmem`.
- In case `builddisk` gets called very frequently, reduce its overhead by
doing nothing if all commits are built already.
The constructor is revised to match the latest interface (no
"max segment level" and "set_new_segment_size" needs to be called).
Reviewed By: sfilipco
Differential Revision: D18640897
fbshipit-source-id: 76a5d0db363713dc5b763503c238c1bfa61ea02e
Summary: Make it possible to build non-master segments.
Reviewed By: sfilipco
Differential Revision: D18641507
fbshipit-source-id: 4d0c46cdea240849af37d28219b6023578514323
Summary:
`build_segments_volatile(id)` should build flat segments so it includes `id`.
Right now it can fail that in a corner case. Fix it.
Reviewed By: sfilipco
Differential Revision: D18641509
fbshipit-source-id: ad1de694e4d31c51f29e9bc5800291066bf16dbb
Summary:
The IdMap has assertions about 1:1 mapping between Id and Slice.
Practically, a non-master commit can be merged into a master commit.
That means a commit hash might map to 2 Ids: [master-id, non-master-id].
Allow that to happen (temporarily).
Note: The non 1:1 mapping causes more headache than benefit and
D18838995 restores 1:1 mapping. But it still requires temporary
non-1:1 mapping to work.
Reviewed By: sfilipco
Differential Revision: D18641508
fbshipit-source-id: bc4bc831d5ab5ebfc2588bafd48114ae49d31e0f
Summary: fix the `build-eden-thrift-deps.py` to generate a new zip file, upload it to lfs, and point setup.py to use this new generated thrift code
Reviewed By: quark-zju
Differential Revision: D18836545
fbshipit-source-id: 52c58c50319c878f2f6f708b952f0aa158d2ef4e
Summary: panic on drop is a terrible idea. It can cause SIGILL kill without meaningful messages or meaningful backtrace. Remove it.
Reviewed By: xavierd
Differential Revision: D18640894
fbshipit-source-id: 3ab52ded9875e17e891acd8f9b45d82964615f01
Summary: This makes the code harder to crash in abnormal cases.
Reviewed By: sfilipco
Differential Revision: D18640895
fbshipit-source-id: ee81dd7739739d3d15052bdb7bb2ba80ed48cd0a
Summary:
This allows us to allocate Ids for non-master commits in a separate range
so they wouldn't cause fragmentation for commits in the master group.
Reviewed By: sfilipco
Differential Revision: D18640896
fbshipit-source-id: c19c53342583ded8795849584cf5f31bc86b2818
