Summary:
Change the default Python to prefer 3.11 instead of (CentOS 8 default) 3.8.
This matches the buck build more closely.
Reviewed By: sggutier
Differential Revision: D49710053
fbshipit-source-id: 6f521b994da32999eb977ebff63ade7d2dd012f5
Summary:
Previously, we defined our own `ModuleSpec` to avoid importing the `importlib`
on disk. That can cause issues with code that wants to set random attributes
on the `ModuleSpec` (heap) object. For example, Python 3.11 wants to set the
`_uninitialized_submodules` attribute, which fails with the current pymodules
implementation.
Actually, we can get the `ModuleSpec` from the frozen `_frozen_importlib`.
That does not hit the filesystem. Switch to the stdlib `ModuleSpec` for better
compatibility.
Reviewed By: sggutier
Differential Revision: D49710052
fbshipit-source-id: e0e5ad0267e29d659da25d167f9e7254adf198b9
Summary:
The `filter='data'` support needs Python 3.10.12 or 3.11:
241f2e54a6
Detect it and skip providing the filter keyword parameter.
Reviewed By: muirdm
Differential Revision: D49706544
fbshipit-source-id: 7b9d00c7bf5a915b3f21d172bfc8d543bf53dc72
Summary:
This diff introduces test for `GitDeltaManifest` that validate the following:
- The manifest gets generated without error for different types of commits and branching structures
- The manifest includes the entries for modifications wherever appropriate
- The manifest include multiple delta variants for merge commits
- The manifest excludes files in merge commits that are already present in both parents
- The manifest always includes an entry for root directory
- The manifest only include non-delta entries for files and directories in root commits
There were certain bugs discovered and fixed as part of this testing which I have included along with this diff. Note that this does not guarantee that `GitDeltaManifest` works 100% how we expect it to, but it does give us a good degree of confidence
Reviewed By: mitrandir77
Differential Revision: D49187571
fbshipit-source-id: d199de9b356fe9ad94e85338f2e8588f00ce906c
Summary:
I noticed we had no tests for some cases here, namely:
- deleting an untracked file
- bulk discarding all files
- bulk discarding a subset of files
This is stuff affected by the commits earlier in this stack, so let's add tests for this moving forward
Reviewed By: quark-zju
Differential Revision: D49617784
fbshipit-source-id: 2107c4f5e7481d14bbbcfcae322c9d8c989e5455
Summary:
Apparently `sl purge` is capable of deleting individual files. This is much nicer than what we did before, which was to just run `rm` ourselves. By running this through `sl`, we can queue it up, show optimistic state, handle cancelling, etc.
Plus, it's nice from a security perspective to not have an exposed way to delete arbitrary paths. I think we handled the security correctly, but it's better to not have to think about it at all.
Reviewed By: quark-zju
Differential Revision: D49616814
fbshipit-source-id: 8e1e4bef34310d357f4eaf7b2b38a8cf1932560e
Summary:
Bug:
- have some uncommitted changes, with some files untracked
- if you select a subset of files including untracked files
- then click the bulk discard button
- it would run `sl revert`, but that would skip the untracked files, causing them to not get reverted.
- what's worse, our optimistic state assumed we did delete these files, so it would hide them visually until we got the next refresh or invalidated the optimistic state.
The fix is to use `sl purge file1 file2 ...` to delete the files. I didn't think this command took a list of files, but it seems it actually does. I think it's intended to be used for directories, but it definitely works for files too.
Reviewed By: quark-zju
Differential Revision: D49616813
fbshipit-source-id: f488ce12f01b98424a04f21fe192a4762970c1ac
Summary: Small util to split an array into two halves, those which pass a predicate and those which don't.
Reviewed By: quark-zju
Differential Revision: D49616812
fbshipit-source-id: 03c33b6610de63262482ee517c783a241bd575fd
Summary:
It turns out `cmd` always wants the `/c` argument to be "(naively) quoted":
# This is what Python `subprocess.run(shell=True)` does:
cmd /c "{cmd}"
No need to worry about double quoting:
# Works fine
cmd /c ""\Program Files\nodejs\node.exe""
# Works fine
cmd /c ""\Program Files\nodejs\node.exe" "my script.js""
But not triple quoting:
cmd /c """\Program Files\nodejs\node.exe"""
# '""\Program' is not recognized
Without quoting it is problematic:
cmd /c \Program Files\nodejs\node.exe
# '\Program' is not recognized
Change system-command to quote on Windows, but avoid triple-quoting, to be
compatible with D49694880.
Reviewed By: muirdm
Differential Revision: D49697875
fbshipit-source-id: a0ad289c3d807bc686857b2ea39783cbf89c63a4
Summary:
This is quite error prone. So let's move it to a separate crate to avoid
reinventing it in wrong ways.
Reviewed By: muirdm
Differential Revision: D49697876
fbshipit-source-id: 1667ca29af2151438e9f537eac5d7824e3352f05
Summary:
Adding env vars support, so ISL can use env vars to pass correlator
and entry_point info
Reviewed By: liubov-dmitrieva
Differential Revision: D49660376
fbshipit-source-id: 3f2bf770a0642d223177a7a7b330f46cf20a8e71
Summary:
pass client request info into EdenFs
create an equivalent thrift type to the ClientRequestInfo type
we have to use thrift params (adding a new field into existing params) since thrift headers are not supported in the underlying transport used (tried in D49635948).
SocketTransport relies on FramedTransport which only transfers a payload, but no header metadata per frame.
Reviewed By: genevievehelsel
Differential Revision: D49656897
fbshipit-source-id: ac8b1ac22b20470cce903552c46672081478b59d
Summary:
Use of `*` in imports is discouraged, and doubly so when the import is re-exported with `pub use`. We only want to export the error type, so specify exactly that.
This fixes a new warning that shows up in the latest compiler version about the shadowing of `NonRootMPath` through the blanket import.
Differential Revision: D49680495
fbshipit-source-id: 7fd939d2a7c9e9a42e3f6d31679564a7aea7fcd0
Summary:
pass client info into thrift services (land service, derived data
service)
we need to extract the client info from the metadata and pass it in a
persistent header in thrift
Reviewed By: RajivTS
Differential Revision: D49640441
fbshipit-source-id: 512fcbde0ebe249770eac78f0766e0043f447a4a
Summary:
X-link: https://github.com/facebookincubator/velox/pull/6685
libmnl-static has not been used for fboss in centos8. It is not
available in centos9. fboss builds fine without limnl dependency
Reviewed By: srikrishnagopu
Differential Revision:
D49380161
Privacy Context Container: L1125642
fbshipit-source-id: e1382a9d04c0fc962b4083c4aa7edba6bc1e02ae
Summary: We ship Python 3.9+ everywhere, so now we can use `ZoneInfo` instead of `pytz`, [which is what the `pytz` developers recommend](https://github.com/stub42/pytz/blob/master/src/README.rst#issues--limitations).
Reviewed By: muirdm
Differential Revision: D49624253
fbshipit-source-id: 70f97ed350bde146197bb930c2280ed7c9b875d9
Summary: Recently we made changes to which Python version to prefer. In some cases, the `python3` binary that comes first in the path might be older than 3.8, which is the minimal Python version we support. This changes it so that 3.10 and 3.8 are preferred
Reviewed By: muirdm
Differential Revision: D49623917
fbshipit-source-id: ca1c17cf4694a90fc1f6e1641ab231cee6b7cae5
Summary:
I'm adding more test cases to cover the partial_commit_graph logic in the next diffs and the file was starting to get too big...
I prefer having smaller files because I can open related code side-by-side and more granular crates to improve buck build (see [the wiki](https://www.internalfb.com/intern/wiki/Rust-at-meta/Buck/rust-in-fbcode/#optimizing-build-time)).
Differential Revision: D49634613
fbshipit-source-id: bfd78f51b5ba970379ad6cb9404856f36705c4c7
Summary:
When I set up the unit tests, I didn't know that I could also use drawdag there. I thought it was available only for integration tests.
Drawdag makes everything more succint and easier to understand/maintain.
Differential Revision: D49536292
fbshipit-source-id: a3786e1d74f1f802ab1ff29bca15b2cd6c6fd610
Summary:
To display progress when processing very large/old directories.
I'm only displaying the progress bar when the log level is set to INFO or lower, so it doesn't show up on the integration tests.
Differential Revision: D49452508
fbshipit-source-id: 17988c9140df6f1219fd63dd8ae305a6b0aa5ba0
Summary:
## Bug
See D49366697 and D49369883 for context on the bug.
## Solution
The comments on the code should explain the solution (please LMK if that's not the case).
The **TL;DR** is that any files that are copied in a changeset `A` should only be copied from a changeset that are is being exported to the git repo, and thus have been rewritten before `A` and will be present in the `remapped_parents` map.
## What's next
Stop relying on the temporary hack of manually passing the name of the old directory as an export path by automatically getting the name of the old directories and all the changesets that affected it.
Differential Revision: D49416858
fbshipit-source-id: d084427a2db0c835ddb6ba39c4148bd2a4df82f7
Summary:
the `Config` usages were deleted from D49593287, let's remove the
unused dependency as well
Reviewed By: muirdm
Differential Revision: D49605511
fbshipit-source-id: 77921c814e9d94036d39f9e6773bfb6165bc5875
Summary:
## Context
D49366697 adds an integration test to cover a bug that was surfaced when running the tool in a real directory.
The **TL;DR** of the bug is that gitexport crashes when we pass `foo` as the export directory and at some point in history `foo` was created by renaming an entire directory named `bar`.
## Temporary solution
Manually pass `bar` as an export directory.
However, this doesn't work out of the box because of the way that the `paths_with_history` method works.
The documentation of `paths_with_history` says that it will not basically ignore any paths that don't exist in the changeset from which the method was called.
This means that passing `bar` as an export directory has no effect, since it doesn't exist in the starting changeset anymore.
By calling `path_with_history` for each path, we'll get the changesets affecting the old directories, even if they don't exist in the starting changeset anymore.
This unblocks the fix in D49416858.
Differential Revision: D49369883
fbshipit-source-id: dc3a44752cdcf76f03efdcd5e0e76a1435972026
Summary: Recently we changed the way we pick the preferred Python version. This affected parts of our OSS build, which this diff fixes.
Reviewed By: muirdm
Differential Revision: D49612380
fbshipit-source-id: ea39893f992025c2ad97c24eff03bc5dfe7b2581
Summary:
This skips the ISL build and the "other" Rust binaries (scm_daemon and mkscratch).
I tweaked things so "make oss" shares the same body as "make local".
Reviewed By: quark-zju
Differential Revision: D49605727
fbshipit-source-id: a29213a059e11876454b302324c1c5d7752feabe
Summary:
X-link: https://github.com/facebookincubator/velox/pull/6734
This is phase 1 of "sapling" rename. Later we will move out of the "eden" directory.
These were the high level steps I took:
1. "hg mv edenscm sapling" + "hg mv edenscmnative saplingnative"
2. s/edenscm/sapling/g
3. s/EdenSCM/Sapling/g
4. Update autocargo config
5. Update mercurialshim.py to support "edenscm" as legacy module (this is the critical part that keeps backwards compat w/ extensions and hooks that import from "edenscm")
Reviewed By: sggutier
Differential Revision: D49326692
fbshipit-source-id: 843db9ece03c7b80c666ff805bd855211614ac62
Summary:
When error messages are bouncing between processes it is hard to keep things categorized. For example, if "hg checkout" calls into EdenFS which calls back into the hg import helper, the "is-network-error" knowledge is long lost.
Add fallback detection that turns UncategorizedNativeErrors into TlsErrors if the error message looks cert related.
Reviewed By: zzl0
Differential Revision: D49607587
fbshipit-source-id: 51f72feda2c970feb5b7178ebcddf4c3f22a9eff
Summary: We previously had counters to provide visibility into the in-memory blob metadata cache's hit rate, but we lacked corresponding counters for the in-memory blob and tree caches. This adds BlobCache hit counters.
Reviewed By: genevievehelsel
Differential Revision: D49564626
fbshipit-source-id: 2fdd1e51fb8944607a032e0aef8610b9bf8dabcf
Summary: Before this change there was a bug where trying to read symlinks with POSIX-style paths (e.g., `/foo/bar`) would fail. This applied both to reading symlinks directory or trying to list the contents of their parent directories.
Reviewed By: genevievehelsel
Differential Revision: D49601495
fbshipit-source-id: 403d06c580d676d61fb6bd8233e5ac45b072c2df
