Summary:
The name `rev` is used for revision numbers. It was used as a hex node in the
code. Let's make it use binary `node` explicitly to avoid confusion.
Reviewed By: muirdm
Differential Revision: D45898261
fbshipit-source-id: 7f6fc04e1e5a6aaef46dd31c3aca9d494cbf488a
Summary:
Make fastlog uses a regular generator and just one generatorset, instead of a
background thread + queue + nested generatorsets. The "combinator" that takes
threads becomes unnecessary after D45898263 and is removed.
This avoids the SIGABRT issue which could be reproduced relatively easily like:
export CHGDISABLE=1
# likely SIGABRT in the first 10 runs
for i in `seq 100`; do hg log -r 'max(follow(fbcode/buck2,master))' --pager=off || { echo FAIL!; return 1 }; done
Example SIGABRT backtraces in gdb:
(gdb) r log -r 'max(follow(fbcode/buck2,master))' --pager=off --verbose --debug --config fastlog.debug=remote
Starting program: hg log -r 'max(follow(fbcode/buck2,master))' --pager=off --verbose --debug --config fastlog.debug=remote
[Thread debugging using libthread_db enabled]
...
thread '<unnamed>' panicked at 'panic in a function that cannot unwind', library/core/src/panicking.rs:123:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread caused non-unwinding panic. aborting.
Thread 14 "hg" received signal SIGABRT, Aborted.
[Switching to Thread 0x7fffc7a00700 (LWP 785327)]
0x00007ffff6a4eacf in raise () from /lib64/libc.so.6
(gdb) bt
0 0x00007ffff6a4eacf in raise () from /lib64/libc.so.6
1 0x00007ffff6a21ea5 in abort () from /lib64/libc.so.6
2 0x0000555557fb9677 in std::sys::unix::abort_internal () at library/std/src/sys/unix/mod.rs:350
3 0x0000555557fb1eaa in std::panicking::rust_panic_with_hook () at library/std/src/panicking.rs:711
4 0x0000555557fb1b42 in std::panicking::begin_panic_handler::{{closure}} () at library/std/src/panicking.rs:581
5 0x0000555557fb0226 in std::sys_common::backtrace::__rust_end_short_backtrace () at library/std/src/sys_common/backtrace.rs:150
6 0x0000555557fb1892 in rust_begin_unwind () at library/std/src/panicking.rs:579
7 0x0000555557fdb223 in core::panicking::panic_nounwind_fmt () at library/core/src/panicking.rs:93
8 0x0000555557fdb2d1 in core::panicking::panic_nounwind () at library/core/src/panicking.rs:123
9 0x0000555557fdb3f3 in core::panicking::panic_cannot_unwind () at library/core/src/panicking.rs:171
10 0x00005555563ae8c4 in cpython::conversion::ToPyObject::with_borrowed_ptr (self=0x7fffc79ff158, f=..., py=...) at ../../../third-party/rust/vendor/cpython-0.7.1/src/conversion.rs:55
11 cpython::objectprotocol::ObjectProtocol::call (py=..., args=..., kwargs=..., self=<optimized out>) at ../../../third-party/rust/vendor/cpython-0.7.1/src/objectprotocol.rs:214
12 pythreading::RGeneratorIter::next_internal (self=<optimized out>, py=..., iternext=<optimized out>) at src/lib.rs:405
13 0x00005555563b2264 in pythreading::RGeneratorIter::__next__ (py=...) at src/lib.rs:389
14 0x00005555563b3714 in pythreading::RGeneratorIter::create_instance::TYPE_OBJECT::wrap_unary::{{closure}} (py=...)
at ../../../third-party/rust/vendor/cpython-0.7.1/src/py_class/slots.rs:321
15 cpython::function::handle_callback::{{closure}} () at ../../../third-party/rust/vendor/cpython-0.7.1/src/function.rs:220
16 std::panicking::try::do_call (data=<optimized out>) at /rustc/84c898d65adf2f39a5a98507f1fe0ce10a2b8dbc/library/std/src/panicking.rs:487
17 std::panicking::try (f=...) at /rustc/84c898d65adf2f39a5a98507f1fe0ce10a2b8dbc/library/std/src/panicking.rs:451
18 0x00005555563b2609 in std::panic::catch_unwind (f=...) at /rustc/84c898d65adf2f39a5a98507f1fe0ce10a2b8dbc/library/std/src/panic.rs:140
19 cpython::function::handle_callback (_location=..., f=..., _c=...) at ../../../third-party/rust/vendor/cpython-0.7.1/src/function.rs:218
20 pythreading::RGeneratorIter::create_instance::TYPE_OBJECT::wrap_unary (slf=0x7fffefa613d0) at ../../../third-party/rust/vendor/cpython-0.7.1/src/py_class/slots.rs:318
21 0x00007ffff65e1518 in _PyEval_EvalFrameDefault () from /lib64/libpython3.8.so.1.0
22 0x00007ffff659f02f in _PyFunction_Vectorcall () from /lib64/libpython3.8.so.1.0
23 0x00007ffff65e11dd in _PyEval_EvalFrameDefault () from /lib64/libpython3.8.so.1.0
24 0x00007ffff659f02f in _PyFunction_Vectorcall () from /lib64/libpython3.8.so.1.0
25 0x00007ffff65e11dd in _PyEval_EvalFrameDefault () from /lib64/libpython3.8.so.1.0
26 0x00007ffff659f02f in _PyFunction_Vectorcall () from /lib64/libpython3.8.so.1.0
27 0x00007ffff659fb24 in method_vectorcall () from /lib64/libpython3.8.so.1.0
28 0x00007ffff659798f in PyObject_Call () from /lib64/libpython3.8.so.1.0
29 0x00007ffff662ec5e in t_bootstrap () from /lib64/libpython3.8.so.1.0
30 0x00007ffff65ee5a4 in pythread_wrapper () from /lib64/libpython3.8.so.1.0
31 0x00007ffff5c081ca in start_thread () from /lib64/libpthread.so.0
32 0x00007ffff6a39e73 in clone () from /lib64/libc.so.6
(gdb) info threads
Id Target Id Frame
1 Thread 0x7ffff7fe6d80 (LWP 784691) "hg" 0x00007ffff650f274 in visit_decref () from /lib64/libpython3.8.so.1.0
2 Thread 0x7ffff4a00700 (LWP 785063) "ctrl-c" 0x00007ffff6b1f9b4 in read () from /lib64/libc.so.6
3 Thread 0x7ffff4600700 (LWP 785066) "tokio-runtime-w" 0x00007ffff6b2ee87 in epoll_wait () from /lib64/libc.so.6
....
10 Thread 0x7fffee600700 (LWP 785073) "tokio-runtime-w" 0x00007ffff6a399bd in syscall () from /lib64/libc.so.6
11 Thread 0x7fffee200700 (LWP 785074) "rust-progress" 0x00007ffff6af9868 in nanosleep () from /lib64/libc.so.6
13 Thread 0x7fffd4a00700 (LWP 785135) "hg" 0x00007ffff6b2667f in select () from /lib64/libc.so.6
* 14 Thread 0x7fffc7a00700 (LWP 785327) "hg" (Exiting) 0x00007ffff6a4eacf in raise () from /lib64/libc.so.6
(gdb) thread 1
[Switching to thread 1 (Thread 0x7ffff7fe6d80 (LWP 784691))]
0 0x00007ffff650f274 in visit_decref () from /lib64/libpython3.8.so.1.0
(gdb) bt
0 0x00007ffff650f274 in visit_decref () from /lib64/libpython3.8.so.1.0
1 0x00007ffff650bc0f in func_traverse () from /lib64/libpython3.8.so.1.0
2 0x00007ffff6528e94 in collect.constprop () from /lib64/libpython3.8.so.1.0
3 0x00007ffff664f192 in _PyGC_CollectNoFail () from /lib64/libpython3.8.so.1.0
4 0x00007ffff664f43a in PyImport_Cleanup () from /lib64/libpython3.8.so.1.0
5 0x00007ffff66519ff in Py_FinalizeEx () from /lib64/libpython3.8.so.1.0
6 0x000055555627f9d5 in hgcommands::python::py_finalize () at src/python.rs:136
7 <hgcommands::hgpython::HgPython as core::ops::drop::Drop>::drop::{{closure}} () at src/hgpython.rs:237
8 tracing::span::Span::in_scope (self=0x7fffffffafc0, f=...) at ../../../third-party/rust/vendor/tracing-0.1.37/src/span.rs:1102
9 0x000055555615651a in <hgcommands::hgpython::HgPython as core::ops::drop::Drop>::drop (self=<optimized out>) at src/hgpython.rs:237
10 0x0000555556130936 in core::ptr::drop_in_place<hgcommands::hgpython::HgPython> () at /rustc/84c898d65adf2f39a5a98507f1fe0ce10a2b8dbc/library/core/src/ptr/mod.rs:490
11 hgcommands::run::dispatch_command (io=0x7fffffffcde0, dispatcher=..., cwd=..., in_scope=..., start_time=...) at src/run.rs:240
12 hgcommands::run::run_command::{{closure}} () at src/run.rs:148
13 0x000055555612dbbd in hgcommands::run::run_command (args=..., io=0x7fffffffcde0) at src/run.rs:133
14 0x00005555561242da in hgmain::main () at src/main.rs:114
Note it involves threads, GC, rust-cpython, and `Py_Finalize`.
See also https://github.com/dgrunwald/rust-cpython/pull/244.
Reviewed By: muirdm
Differential Revision: D45898265
fbshipit-source-id: ce94763c7249f83237472f1fc4b4abad1011230c
Summary:
It does not seem like the code path would ever need to handle len(paths) > 1
cases. So let's simplify the code accordingly.
For example, when you run `hg log PATH1 PATH2`, the underlying revset is
`follow(PATH1) or follow(PATH2)`. Each follow revset only gets 1 path.
I guess the multi-path support was needed before migrating to the follow revset
in D14764074.
Reviewed By: muirdm
Differential Revision: D45898264
fbshipit-source-id: 05d1aed8d08741fe555e7eda4e4b3a2fd797833e
Summary:
The `RGenerator` and `RGeneratorIter` types keep Python objects internally.
Let's be a good citizen and tell CPython the object graph so CPython can
properly GC related objects.
Reviewed By: muirdm
Differential Revision: D45898262
fbshipit-source-id: da2b04ec6f7928c486f3f691885bf2ed8cb06055
Summary: Add a function to query the (optional) ObjectId from VirtualInode.
Reviewed By: kmancini
Differential Revision: D45673165
fbshipit-source-id: 72dadaf4cbd5120c75548734aca2bb8ae715b8e2
Summary:
Add a function to query the (optional) ObjectId from an arbitrary
inode.
Reviewed By: kmancini
Differential Revision: D45673053
fbshipit-source-id: 23630638ba94bbd1925a39193fa125df04abebad
Summary:
I noticed some situations where we don't hold the root inode for the
duration of an asynchronous operation. To make that easier, I
introduced an EdenMountHandle.
Port EdenServer::getMount() to return EdenMountHandle, which
simplifies some Thrift service handlers.
Reviewed By: kmancini
Differential Revision: D45891887
fbshipit-source-id: 99dfaa4aaf609c8b237cceeaa15b91bd26981d02
Summary:
Asynchronous operations that must ensure an EdenMount (and its root)
are usable for the duration must hold a reference to the root's
InodePtr, even if they only need the EdenMount.
To make this relationship less implicit, introduce an EdenMountHandle
which is effectively the same as the previous
`pair<EdenMountPtr, TreeInodePtr>`, but more clearly documents the intent
is to preserve the usability of EdenMount and its root inode.
This also allows us to decouple that relationship in the future,
should we want to.
For now, only use EdenMountHandle in getMountPoints. Next, I'll switch
`getMount` to use it.
Reviewed By: kmancini
Differential Revision: D45830176
fbshipit-source-id: 832dd96b9a6fe04fabbb5af0ecbed0173f9d2a2b
Summary: This diff implements the `create_git_tag` endpoint introduced in the last diff. The primary logic is implemeted as part of the `mononoke_api::create_annotated_tag` method so no new logic is introduced here. The `create_annotated_tag` does not yet store the mapping of the created changeset object with the tag name
Reviewed By: mitrandir77
Differential Revision: D45654021
fbshipit-source-id: 3ebd8d7b67a59935f646f5ba4287e14a096f82d1
Summary: The `create_git_tag` endpoint introduced in this diff is responsible for creating a changeset that represents the metadata of a git tag and create a DB-level mapping between the changeset-representing-the-tag and the changeset that the tag bookmark points to. This diff just introduces the skeleton, the ne xt diff in the stack will implement the core logic
Reviewed By: mitrandir77
Differential Revision: D45643823
fbshipit-source-id: 3b25c8a9382a364667c262961ed8506344988d49
Summary: This test was failing on Windows due to `working_copy.remove` not failing silently anymore. Given that the exact thing that it is testing doesn't work, it's better to just remove it.
Reviewed By: quark-zju
Differential Revision: D45833748
fbshipit-source-id: 952ca8a49fcdde8b5345d27eab20cedc3b979d7a
Summary: There are some cases where the Curl crate might not support HTTP V2, but will still try to use V2 regardless. This diff makes it fall back to Any (whatever the Curl crate considers best and available) instead.
Reviewed By: muirdm
Differential Revision: D45778450
fbshipit-source-id: 7811d1f30f75fab09f7bb0b478f44c730bbfe12b
Summary: Sometimes the log size changes a bit, which was making this test flaky on Linux and fail in most cases on Windows. This diff fixes that issue.
Reviewed By: muirdm
Differential Revision: D45762283
fbshipit-source-id: d16aedc2e0bfd858f1c5d3060339ae6e106fcb5b
Summary:
Similar to calculateDepMap, cache the result of flatten() calculation. To avoid
accidentally modifying the cached value, we mark them as `Readonly`.
Reviewed By: zzl0
Differential Revision: D45800640
fbshipit-source-id: bcb70b3bbb3b6afaccded8dc93318a4bed5ee49f
Summary:
We can use an approach that is similar to calculateDepMap (D45232954) to reduce
its complexity from O(lines * revs) to O(lines).
Reviewed By: evangrayk
Differential Revision: D45800642
fbshipit-source-id: e5f62c83602332ce607dd73abee39cfd761d63c9
Summary:
Previously, clicking a file could crash the isl-server process like:
open file /home/quark/tmp/erepos/rs/isl/y.txt
node:events:491
throw er; // Unhandled 'error' event
^
Error: spawn xdg-open ENOENT
at ChildProcess._handle.onexit (node:internal/child_process:283:19)
at onErrorNT (node:internal/child_process:476:16)
at process.processTicksAndRejections (node:internal/process/task_queues:82:21)
Emitted 'error' event on ChildProcess instance at:
at ChildProcess._handle.onexit (node:internal/child_process:289:12)
at onErrorNT (node:internal/child_process:476:16)
at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
errno: -2,
code: 'ENOENT',
syscall: 'spawn xdg-open',
path: 'xdg-open',
spawnargs: [ '/home/quark/tmp/erepos/rs/isl/y.txt' ]
}
Node.js v18.16.0
Fix it by not letting the error propagate to the top-level handler.
Reviewed By: zzl0
Differential Revision: D45800641
fbshipit-source-id: 0f028f71d20c29dafd8395bc12ec23a937264224
Summary:
See the previous diff for context. This allows us to obtain file contents in
the working copy without running `hg addremove` first.
`debugexportstack` is better than `cat` because:
- It provides file content in the parent commit.
- It provides renaming information.
- It does not crash with non-utf8 content with JSON.
Reviewed By: muirdm
Differential Revision: D45787978
fbshipit-source-id: 69d37ae5d5b6cf27dda7a8ab0c0a443e83709d06
Summary:
By default, `debugexportstack -r wdir()` exports tracked file changes. In the
ISL use-case, we'd like to add a "chunk selector" for files in the working copy.
The "chunk selector" would like to know the file contents regardless of whether
files are in `?`, `M`, `A`, `!`, `R` states.
This diff adds a `--assume-tracked` flag to support this use-case. Basically,
this flag can be used to make files in `!` or `?` states exported as if they
were `R` and `A` states.
Reviewed By: muirdm
Differential Revision: D45787979
fbshipit-source-id: 812a389ffe6d01b8a4200754bdc38c49b775c67e
Summary:
I noticed that `workingfilectx` handles renames properly. There is no need to
use `wctx`.
Reviewed By: muirdm
Differential Revision: D45787980
fbshipit-source-id: b46364305bd551b462445a8719fde1d77b694581
Summary:
The EdenMount::getObjectStore gave the assumption to callers that the
ObjectStore lived longer than the EdenMount. Since this assumption is implicit
and not explicit, there is no way to guarantee it by just reading the code in a
local fashion. It forces the reader/auditor to understand the code in its
entirety. Unfortunately, EdenFS contains more of these than can be kept in
memory which leads to bugs and use-after-free.
This can be avoided by breaking the implicit lifetime assumption and simply
returning the shared_ptr. If the caller needs to keep the ObjectStore around,
they can do so explicitely which makes reading the code easier.
Reviewed By: chadaustin
Differential Revision: D45831168
fbshipit-source-id: 8bdb8e708308a8a220e3130d2fc51854c8556cb1
Summary:
This will be used in a later diff outside of EdenMount.cpp, thus let's move it
to its own file.
Reviewed By: chadaustin
Differential Revision: D45831170
fbshipit-source-id: 1cc55e1ae580ee7f02e3d9c6a717da4751c955e6
Summary:
When a BlobMetadata fails to fetch, the fetch promise is set to a `nullptr` to
force the higher level BackinStore to fetch the full blob. Unfortunately,
`HgImportRequestQueue::markImportAsFinished` attempts to deep copy that
`nullptr` leading to a crash.
Reviewed By: chadaustin
Differential Revision: D45873681
fbshipit-source-id: 3f5b777a3671cf53dd9d605dca2e119eabe3e625
Summary: Here we implement the logic needed to bump QPS counters so we can shift traffic cross regionally with cslb
Reviewed By: mzr
Differential Revision: D45731719
fbshipit-source-id: d5df034d40588d47f19e121b21d606637dcdc41f
Summary: Add fallback logic for MetadataRenameFinder in case users forget to record copy information in the header
Reviewed By: muirdm
Differential Revision: D45865166
fbshipit-source-id: 991a71718b9e149fa1cd23fe33dd254f22e2f366
Summary: Those functions will be used for fallback logic in next diff
Reviewed By: muirdm
Differential Revision: D45865165
fbshipit-source-id: 73a4df33ff1dc407c53bf98fc95c10e1bff2501c
Summary:
RenameFinderInner is like a base struct for MetadataRenameFinder and ContentSimilarityRenameFinder.
It is introduced for code reuse between those two file based rename finders.
Reviewed By: muirdm
Differential Revision: D45865167
fbshipit-source-id: 66302c7dcad87fcd36e19ad2649d7fc4d42098b6
Summary: We already have a ContentSimilarityRenameFinder, MetadataRenameFinder sounds more appropriate than the original SaplingRenameFinder, since both rename finders now describes what information they are using for finding renames.
Reviewed By: muirdm
Differential Revision: D45827713
fbshipit-source-id: 9d9e5576cec676afbdd661333591085bef70d996
Summary: We have timeout in seconds (not a limit of number of tries), and it's usually around a minute. Doesn't make a lot of sense to sleep with 0.1. It also spams the logs a lot.
Reviewed By: YousefSalama
Differential Revision: D45810132
fbshipit-source-id: b638d086526da2450251fe412acead20c0bf5579
Summary:
The user doesn't really care if a runlog update failed. This is expected to happen sometimes on Windows since the update will fail if the file is underlying runlog file is open (by another process).
Also, add a couple retries to the final runlog update on Windows to increase the chance we close out the entry.
Reviewed By: sggutier
Differential Revision: D45831199
fbshipit-source-id: 664e14695fdf3a7363dafc7523cef78c9fafd84c
