Summary:
Update the chg code to correctly honor the `pager.stderr` setting, and avoid
piping stderr to the pager when it is disabled.
Since only the server-side code knows the hg config values, the server-side
chgserver.py code passes this config setting back to the client when sending
the pager request.
Reviewed By: quark-zju
Differential Revision: D17109106
fbshipit-source-id: 6b69b1a7de9f61db51af7b0ba00d65fa5053a795
Summary:
This refactors the code the code that sends system and pager requests from the
chg server to the client.
Previously these were both sent using the `S` channel code. The `S` channel
code was slightly unusual and had special handling: in general the code
assumed that upper-case channel codes did not include any request body data,
but this wasn't true for the `S` channel data. I changed this to lower-case
in order to eliminate this special case handling, and I also split up the
system and pager data into different channel codes, since they have fairly
different behavior. System requests are now sent with an `s` channel code,
and pager requests are sent with a `p` channel code.
I also changed the code to require that the server always adds a terminating
`\0` byte after each environment variable value. Previously the client code
was responsible for adding a nul terminator on the last string, which could
potentially require the client to copy the data into a larger buffer in order
to do so.
I also made a minor change to the client-side `readchannel()` code so that it
can read the channel type and body data length with a single system call
instead of making 2 separate `recv()` calls.
The main benefit of these changes is that they will let the server pass some
additional configuration information with pager requests. The change to make
use of this new field will come in a subsequent diff.
Reviewed By: quark-zju
Differential Revision: D17216291
fbshipit-source-id: c3044cf3d5f5e103f0b62d083e4ef3764160f20e
Summary:
For Eden we currently have a gap in our post-mount behavior;
we don't perform the update hooks for a freshly cloned and mounted repo.
The thought is that we'll explicitly trigger them by invoking `hg
debugedenrunpostupdatehook` at the end of the mount sequence.
Reviewed By: quark-zju
Differential Revision: D17237197
fbshipit-source-id: 9c2212c61735068c287eb98761503ce31bfee8a6
Summary: Also remove debug commands that are strictly related to remote lfs.
Reviewed By: markbt
Differential Revision: D17184902
fbshipit-source-id: da38a2150212500bab62191ddcfab0990276605e
Summary:
Instead of the lfs remote storage it was chosen to send the snapshot metadata via bundles.
Snapshot metadata consists of: the actual metadata blob + several other blobs (untracked files etc).
If we have several snapshot revisions in a single bundle, the blobs could repeat.
Then we should store each blob as a separate entry in a binary stream, keeping its id and contents.
Here we introduce a new bundle part type `"b2x:snapshotmetadataparttype"`.
```
1 byte of version info
[ # a list of binary entries, each corresponds to a separate file
# (either a metadata file itself or a related -- externally stored -- file)
<oid><length><data>
:oid: is a 64char string with the hash of the file
:length: is an unsigned int with length of the data
:data: is binary data of length <length>, the actual file contents
]
```
So far there is an ongoing discussion on the exact format of serialization.
Actual state is at [the quip doc](https://fb.quip.com/R5OVAzabX8oo).
Reviewed By: markbt
Differential Revision: D17184222
fbshipit-source-id: 90f833ec71556e90d513e3be3f3efa7f870b037d
Summary: That way it'll be easier to pack it into a blob.
Reviewed By: markbt
Differential Revision: D17183018
fbshipit-source-id: 44e21103f201aafb6f417a5b5a7b3d4735f32039
Summary: In the next diff I will replace remote lfs with bundle2
Reviewed By: markbt
Differential Revision: D17132405
fbshipit-source-id: a0dfff3ebad067abb0231cf31de08ae62affe7ce
Summary:
The `debugmutation` command uses the unfiltered repo to resolve the
user-provided revs. It shouldn't do this unless the user passes `--hidden`.
Reviewed By: mitrandir77
Differential Revision: D17156722
fbshipit-source-id: 5ab7704acc598cf8b7c1640a3096ba0ce6ac73e9
Summary:
Update the debugmutation format to collapse long chains. Add
`hg debugmutation -s` to follow the successor relationship (printing what the
commit became) instead of the predecessor relationship.
Reviewed By: mitrandir77
Differential Revision: D17156463
fbshipit-source-id: 44a68692e3bdea6a2fe2ef9f53b533199136eab1
Summary:
If treemanifest finds there are too many shared packs (more than
`packs.maxpackfilecount`) then it will purge them. This is a shame if there is
currently a repack in progress, as it will purge the packfiles from underneath
the repack, deleting lots of cache data that will be imminently repacked).
Skip the purge if there is a repack ongoing.
Reviewed By: mitrandir77
Differential Revision: D17155854
fbshipit-source-id: 20d46f29c252e508177b1fde08ca7a69841dcd7e
Summary:
Instead of hardcoding `--target <target>` and hardcode `$2` in the script.
Let's just use environment variables so `target` and other things have explicit
names.
Explicit set `REAL_CWD` so the script can learn the current directory before
it gets reset to the repo root.
Reviewed By: xavierd
Differential Revision: D17213186
fbshipit-source-id: 6a4fc4cf2cbf6e2c623400bc6bc13f7758a46c49
Summary: Let's consolidate these 2 to allow easy switching the the Rust based one.
Reviewed By: quark-zju
Differential Revision: D17187154
fbshipit-source-id: 5ccadabac2e2e4b684ca44917f1502e9a05d41d6
Summary:
While a corrupted packfile can be safely removed from the shared hgcache, the
same isn't true for local packfile. When building the packstore, let's allow
the behavior on corrupted packfile to be chosen. This is voluntarily made as an
explicit argument to DataPackStore and HistoryPackStore constructor so the
caller can take this into account.
Reviewed By: quark-zju
Differential Revision: D17187155
fbshipit-source-id: 658fce401f8902a74cfd92780013d1b96e20a590
Summary:
This makes metaedit support `-M / --reuse-message`, which I found handy when
rewriting prototype commits to formal commits.
Reviewed By: xavierd
Differential Revision: D17168991
fbshipit-source-id: fa768a2916ea3ef4db4c31a48989d10897379e92
Summary:
This just moves `-M / --reuse-message` handling from the `commit` command
to a lower layer, making it more reusable.
Reviewed By: xavierd
Differential Revision: D17168992
fbshipit-source-id: 4fe7e93ceae45eff281214dcff03ef3f9ee0c898
Summary:
Change `cmdutil.logmessage` to take a `repo` instead of `ui`. This makes the
next change easier.
Reviewed By: xavierd
Differential Revision: D17168990
fbshipit-source-id: 47c1707e5a9dbf06d07452b4c400903453992379
Summary:
I have seen multiple user complaints about slow hg commands that turned out to
be fsmonitor scanning the whole working copy. Print a warning in those cases.
Hopefully this can reduce our oncall burden a bit.
Reviewed By: xavierd
Differential Revision: D17170520
fbshipit-source-id: 8fd5721d123853136c84229d936c3e0c999f3d87
Summary:
I found this very slow. Log it so we can have some ideas about how long it
takes for others.
Reviewed By: xavierd
Differential Revision: D17066510
fbshipit-source-id: 3f9de9b816bcd2d062beb44bc03ea4114d829596
Summary:
`findrecenttrees` can take very long in my case - it only tests 1280 trees in
30 seconds. Log it so we can get some ideas about how long it takes.
As we're here, teach `util.timefunction` to figure out the `ui` object
automatically.
Reviewed By: xavierd
Differential Revision: D17066278
fbshipit-source-id: 7e59c8683359a7ce8d4e87fde92af36b95d37b2f
Summary:
Make sigtrace use smarttraceback so it prints more context.
As we're here, also make it print to stderr so we don't need to find the
traceback from /tmp.
Reviewed By: xavierd
Differential Revision: D17066277
fbshipit-source-id: 9a1000803fed27a71ec381b8ddbd76400dae99c9
Summary: It will be used by snapshot extension too.
Reviewed By: markbt
Differential Revision: D17132134
fbshipit-source-id: 6c9fc285e0f1eb445bfa0abe0b6f4de4a1bd1db0
Summary:
now blob vfs will be in core mercurial. It will be used by snapshot
extension too.
Reviewed By: quark-zju
Differential Revision: D17112671
fbshipit-source-id: e721749d27db37f55bb9eb6af3ea042e8036ddfa
Summary:
The Rust manifest does not allow for directories to be added if files with
the same name exist.
Reviewed By: quark-zju
Differential Revision: D17143550
fbshipit-source-id: fe2533b6f0c049d7b22f2fbb49f3e04959aea39c
Summary:
It is somewhat difficult to fetch the raw entry on the p1 side in the Rust
Manifests. These entries are used to write deltas to revlogs or to datapacks.
Reviewed By: xavierd
Differential Revision: D17143551
fbshipit-source-id: 6624116324664354d199d5f6ac55712c8ed29b9d
Summary:
The Rust code is almost at parity with the Python code, let's expose it to
Python.
Reviewed By: quark-zju
Differential Revision: D16794076
fbshipit-source-id: faf1da775b4e57328be62a06d0065c7becf1b9f4
Summary:
In general, mutation tracking doesn't care about divergence. However, in the
case of rebase, it doesn't make sense to allow divergence to occur if we can
avoid it by omitting some of the commits to rebase.
This makes rebase behaviour more like old obsmarker-based behaviour. This
breaks a test for mutation copying markers, so update those to use metaedit,
which has the copying behaviour for both obsmarkers and mutation. At some
point we should make rebase behave better in these cases.
Reviewed By: quark-zju
Differential Revision: D17136480
fbshipit-source-id: 9e465b7fc8bda33e7a746e4df68410713e2be37e
Summary: This plumbs it everywhere to be on-par with the historypack.
Reviewed By: quark-zju
Differential Revision: D17104465
fbshipit-source-id: 0e5b9343388b81f21609a38978816bd729102e1b
Summary:
Move the code that isn't proper to an historypack to a separate function. This
will be used to implement a similar functionality for the
IndexedLogHistoryStore
Reviewed By: quark-zju
Differential Revision: D17104466
fbshipit-source-id: 905a4c4c63d8bbb54611fcb9521ab3039d1b56fd
Summary:
This functionality will be needed by the IndexedLogHistoryStore, let's move the
code out of historypack.
Reviewed By: quark-zju
Differential Revision: D17104468
fbshipit-source-id: 33b6b36bb69ed09ca6f72f6f9b6531ffb15c2193
Summary:
1.) Windows doesn't support close_fds if stdin/stdout/stderr is redirected. It throws out this Exception: 'close_fds is not supported on Windows platforms if you redirect stdin/stdout/stderr'. This diff resolves this issue by importing mercurial util to use file_fd only on posix
2.) Originally, stable is implemented as a mercurial extension and gets expanded as the following (take update as an example):
```
hg up stable
-> calls getstablerev("--pick-best") in stable.py
-> calls a bash script in hg repo, e.g. ovrsource or fbsource
```
However, bash is not supported on Windows.
To resolve this, this diff sets up cwd in `subprocess.Popen` and then modifies stablerev to take in a command in string format instead of a script. This way, we can pass in either the bash script for fbsource(backward compatible) or `python <xxx.py>` for ovrsource.
Reviewed By: quark-zju
Differential Revision: D17140170
fbshipit-source-id: ea8ae76883cc34a0517fa7e9eae3cbb3ba901353
Summary:
Since the shared mutable packs contain fetched data from the network, not
commiting them means that we would need to redownload the data again. Let's
persist them on disk instead to avoid having to redownload them.
Reviewed By: quark-zju
Differential Revision: D17115796
fbshipit-source-id: 3e213461c7a864156ee4c6c68e6a042294883f9d
Summary:
The loosefiles repack was made incremental to greatly reduce the repack time
for users. Since the amount of local loosefiles should be way smaller than the
amount of shared ones, let's always run a full repack on the former. This
should allow us to kill all the local loosefiles, which will help in no longer
supporting them.
Reviewed By: quark-zju
Differential Revision: D17135975
fbshipit-source-id: 9480993b31aa57d0d6e6b7caffd282929183f782
Summary:
Now that we're past the point of being not confident about the Rust repack
code, we can remove the fallback code. Instead, let's fail hard if the Rust
repack fails.
Reviewed By: quark-zju
Differential Revision: D17135979
fbshipit-source-id: a521a4f9267361a167bba61968659557bed35e20
Summary:
Rust repack has been the default for a while now. Let's stop pretending we're
going to switch back to the Python repack by removing the config entirely.
Reviewed By: quark-zju
Differential Revision: D17135977
fbshipit-source-id: 5aaa0faa48e2b40a7314d5ab455f5eeaa4e4984d
Summary:
The inline revlog format merges `.i` and `.d` into one `.i` file. It was intended to reduce the
number of files for filelogs. For the changelog one extra file does not hurt.
This makes it easier to write native code parsing the changelog revlog index.
Reviewed By: xavierd
Differential Revision: D17125922
fbshipit-source-id: f48ffe0d2df71abec007a80e05b684dcbac71883
Summary:
I broke some svn tests in that commit, but while fixing forward
realized that we're blocked from making this change because we rely on
rolling back the last commit in the case where the filemap creates an
empty commit.
We can revisit this when we have convert this all to rust!
Reviewed By: sfilipco
Differential Revision: D17140172
fbshipit-source-id: c93911f599afd2d22e84b3327a2a9455204065a0
Summary:
We noticed that the runtime for convert was exponential
due to way that packs were managed.
This wraps the convert in a big transaction so that we reuse an
existing pack rather than writing out one for each commit.
Reviewed By: xavierd
Differential Revision: D17138073
fbshipit-source-id: 46e418a49f917e2ffc411920ace1a3d98ac6b8e4
Summary:
Wez noticed that when /var/cache/hgcache/<repo>/packs/manifest was missing, `hg
repack` would fail, not repacking anything. Let's fix the offending code to
properly ignore the missing directory.
Reviewed By: quark-zju
Differential Revision: D17120206
fbshipit-source-id: 7a7aebac717e5f128565ea2f8d50ffaeda4563f9
Summary:
`listdir` returns the contents a directory in the manifest. The format
is pretty simple, containing only the simple names of the files and or
directories. I don't know if this is something that eden can use because
it seems to simple. In other words, we have something but we may want
to iterate on it before we market it broadly.
Reviewed By: quark-zju
Differential Revision: D17098082
fbshipit-source-id: d6aa42c96781cf1f8b2e916fa10bb275593bdc65
Summary:
The C++ manifest implements walksubdirtrees which is used to compute the packs
that a "client" wants for a prefetch. In terms of interface the function is very
annoying and couples with storage and tree representations without being part
of any of them.
We reproduce that functionality as a means to replace the C++ implementation.
The long term goal is to do lazy fetches using an iteration style that plays
nicer with batching downloads.
This change also includes fastmanifest updates because they are required to
enable the walksubdirtrees functionality in our tests.
Reviewed By: quark-zju
Differential Revision: D17086669
fbshipit-source-id: 6c1f9fbf975814f0a2071f8d1c8e022e5ad58e29
Summary:
ignore-conflict-markers
This updates Mercurial to add a loggetpack option for wire proto logging that
allows capturing getpack responses. This will be useful to verify correctness
for ovrsource on Mononoke (right now, Mononoke isn't serving traffic for hosts
that use getpack).
Reviewed By: StanislavGlebik
Differential Revision: D17091537
fbshipit-source-id: 755a429949d7645010dddab95202c613025f2984
Summary:
While working on a stack, I noticed that `hg amend -e` would sometimes download
a lot of history information. For a 4 files change, I saw 738 history entry
fetched individually...
Looking at the profile, this pointed towards remotefilectx.parents requesting
the entire ancestormap. Since that function really only need the nodeinfo,
let's only get that.
Reviewed By: DurhamG
Differential Revision: D17104263
fbshipit-source-id: fae1f673b2d2a641ae4f22d1099317cc5abd8447
Summary:
Someone reported an error in this function being called with a missing
argument.
Reviewed By: quark-zju
Differential Revision: D17110694
fbshipit-source-id: 65adbae7e1380652afd1cec8bd3e2bbef27268dd
Summary:
After D7840236 stack, it's possible to have a single chg server that handles
different [extensions] configurations. The 'validate' step and 'hashstate' were
mainly designed to detect changes of [extensions], the source code of the
extensions. That becomes unnecessary with the latest design.
Remove them to simplify the logic.
chg no longer creates symlink `server2 -> server2-hash`. Bump the name
to `server3` to explicitly break compatibility.
Reviewed By: xavierd
Differential Revision: D16866463
fbshipit-source-id: 5e1d00e6f895d9b8ead0bcabefcea11756f57c94
Summary:
Use hgcommands via its bindings to run commands. This allows chg to run Rust
commands.
Reviewed By: xavierd
Differential Revision: D16866466
fbshipit-source-id: 0c264a6b5441d2d0b2cdd01beefda5606b00cc61
