Summary:
These tracing points will help us understand the memcache hit rate as well as
the fetching speed.
Reviewed By: quark-zju
Differential Revision: D19836499
fbshipit-source-id: 1936c44efc3e7715069e6a959f5331139d591d5c
Summary:
Everytime a cache miss is seen, the data fetched from the server will be sent
directly to memcache for future use. Unfortunately, doing so in a blocking
manner severely impact the overall fetching speed from the server. Since
memcache is purely an optimization, we can afford to send data to it
asynchronously.
Let's move as much as possible of the code to a background thread to reduce the
overhead of memcache.
Reviewed By: DurhamG
Differential Revision: D19836011
fbshipit-source-id: 68e506ef7464d6e99d98457d0d37178f514be1a9
Summary:
Instead of fetching data one-by-one, let's prefetch data concurrently by using
the new get_iter function.
Reviewed By: DurhamG
Differential Revision: D19836009
fbshipit-source-id: 4a50328c0cbbba677c2de3777ebe4c34cb10c1e2
Summary:
Even when memcache would be able to prefetch everything, this would always call
into the underlying remote store with an empt key set. For things like `hg
prefetch` and a large number of keys, the effect of doing that is minimum, but
for EdenFS or `hg log -p`, the roundtrip to the server for every file/revision
would add a significant amount of overhead. Let's simply stop iterating when we
no longer need to fetch anything.
Reviewed By: DurhamG
Differential Revision: D19835797
fbshipit-source-id: 54ad704428c3b20d973cfa87f7171899ec44b3f9
Summary:
See also https://github.com/serde-rs/bytes/.
This will be used in the `dag` crate.
Reviewed By: DurhamG
Differential Revision: D19770858
fbshipit-source-id: 2a870a564e0ceecdc7a4667853b2b2a5ea4ce6e3
Summary:
This crate provides the core features of the commonly known `Bytes` crate:
zero-copy slicing and cloning, while also supports mmap-backed buffers.
The main motivation is to replace `Mmap` in `indexedlog`. That has multiple
benefits:
- Handles 0-sized mmap more cleanly.
- Handles clones more cleanly.
- Gain the flexibility to zero-copy data without lifetime / reference.
- Gain the flexibility to switch to non-mmap data.
The `bytes::Bytes` crate does not yet support mmap buffers as of its latest
release (0.5.4).
Implementation wise, `minibytes::Bytes` uses `Option<Arc<dyn Trait>>` for the
"trait object". This makes implementing the mmap storage just one line.
`bytes 0.5.4` re-invents the "trait object" manually using unsafe code. It requires
about 50 lines to implement the mmap storage (in D19756122).
Reviewed By: xavierd
Differential Revision: D19770856
fbshipit-source-id: 8cfa7052a18ac2e0cd6348b77d5e2a4acc61195c
Summary: This makes the output more readable even if the "name" of a span is very long.
Reviewed By: DurhamG
Differential Revision: D19780536
fbshipit-source-id: dce0d3777409c32b0752db51341a572addb823ea
Summary:
As initializing the memcache client takes ~0.7s, let's move it to a background
thread as to not impact Mercurial startup time. This diff uses ArcSwap in
order to reduce the overhead of the very common read paths as much as possible.
Using Mutex or RwLock instead would have caused unecessary contention.
Reviewed By: DurhamG
Differential Revision: D19518693
fbshipit-source-id: 886e9b86813fda6ff005ccce99659890026f643a
Summary:
This allows the Python code to build a memcache client and build ContentStore
and MetadataStore with it.
Reviewed By: DurhamG
Differential Revision: D19518694
fbshipit-source-id: d932fd5223ccfdf37db69cbb54a11a6571312709
Summary:
This enables an in-process memcache client for the Rust
ContentStore/MetadataStore. For now, this implementation is lacking several
necessary optimization:
- Start-up time is always slowed down by ~0.7s, the initialization will be
moved to a background thread
- Writing data to memcache is blocking and will be moved to a background
thread too.
- Prefetching data does a roundtrip to memcache for every key, batching
memcache APIs will be added.
Compared to the existing hg_memcache_client, this implementation is both
significantly shorter and do not exhibit some of the pathological behavior of
having to flush the indexedlog for every fetched blob when used in Eden.
Reviewed By: DurhamG
Differential Revision: D19518696
fbshipit-source-id: 4725447d13e7eddd9586135c2511e13ddb921771
Summary:
The library already has a way to wrap a Python object into a Rust object that
exposes the Rust Read/Write interface. This is the reverse direction for
the Write interface.
The initial intention is to expose Rust stdout as described in D19702533.
However, I found Python's `sys.stdout.buffer` also enforces utf-8 encoding
on Windows (unless PYTHONLEGACYWINDOWSSTDIO is set). So Python's
stdout actually behaves similarly with Rust's stdout on Windows and is okay
to use. That said, it's still useful to have this abstraction, for streampager [1]
integration.
[1]: https://github.com/markbt/streampager/
Reviewed By: sfilipco
Differential Revision: D19716127
fbshipit-source-id: ba39898122561d9a49b7080ee95d7c940540eb40
Summary:
Drop stdoutbytes/stdinbytes. They make things unnecessarily complicated
(especially for chg / Rust dispatch entry point).
The new idea is IO are using bytes. Text are written in utf-8 (Python 3) or
local encoding (Python 2). To make stdout behave reasonably on systems not
using utf-8 locale (ex. Windows), we might add a Rust binding to Rust's stdout,
which does the right thing:
- When writing to stdout console, expect text to be utf-8 encoded and do proper decoding.
- Wehn writing to stdout file, write the raw bytes without translation.
Note Python's `sys.stdout.buffer` does not do translation when writing to stdout console
like Rust's stdout.
For now, my main motivation of this change is to fix chg on Python 3.
Reviewed By: xavierd
Differential Revision: D19702533
fbshipit-source-id: 74704c83e1b200ff66fb3a2d23d97ff21c7239c8
Summary:
All diff functions are (bytes, bytes) -> bytes to preserver
the original file encoding.
Because of that I had to add ui.writebytes output function that accepts
bytes for terminal output.
Reviewed By: farnz
Differential Revision: D19656673
fbshipit-source-id: b9a1e4361e825fc8c2313e8402c2bbe00f490dd4
Summary: `PyPath` is to `PyPathBuf` as `Path` is to `PathBuf` and `str` is to `String`.
Reviewed By: quark-zju
Differential Revision: D19647995
fbshipit-source-id: 841a5f6fea295bc72b00da028ae256ca38578504
Summary: Username as utf8, so let's make mutationmarker treat them as such.
Reviewed By: xavierd
Differential Revision: D19649887
fbshipit-source-id: 3f8b2db434a57ee8ee3017de8d925c19a2002b20
Summary: Add `PyNone`. This is a marker struct that indicates that a python function can only return `PyNone`.
Reviewed By: xavierd
Differential Revision: D19644338
fbshipit-source-id: f846b146237ebf7de996177494934fec662cde0f
Summary:
`PyPath` is the type that owns the data. Rename it to `PyPathBuf` for analogy
with `PathBuf` and `RepoPathBuf`, and to allow us to introduce a reference type
named `PyPath`.
Reviewed By: xavierd
Differential Revision: D19643797
fbshipit-source-id: 56d80fea5677f7223e967b0723039d1763b26f68
Summary:
Make the type easier to use. Namely, the treestate bindings want PyPath <->
bytes since treestate internally uses bytes.
Reviewed By: xavierd
Differential Revision: D19635357
fbshipit-source-id: 37d1889b5da1d7f3869bb7820de0219b87b71a8b
Summary:
Set up the `cpython-ext` and `hgcommands` libraries so that they can compile
against py2 and py3 versions of rust-cpython. Make py2 the default so
that cargo test still works.
Reviewed By: singhsrb
Differential Revision: D19615656
fbshipit-source-id: 3403e7077deb3c0a9dfe0e3b7d4f4ad1da73bba3
Summary:
Update the Rust hgcommands code to pass the command line arguments into the
Python logic as `Str` types, so that this will be Unicode `str` objects when
using Python 3.
Reviewed By: xavierd
Differential Revision: D19596739
fbshipit-source-id: 7cdfd44a1c4ce8b0f86d20b634d9b27eab822b2d
Summary: This is incorrectly removed due to a bad rebase / merge.
Reviewed By: DurhamG
Differential Revision: D19607801
fbshipit-source-id: a6ee7a3f184ff1882eb1f1513f7fed74a7108727
Summary:
This makes `make hg3` work. It requires cleaning up the `build` directory when
switching between py2 and py3 build, which will be fixed later.
Reviewed By: DurhamG
Differential Revision: D19604824
fbshipit-source-id: 060ff313420126a5dba935c4451b45dc9af45f13
Summary: Add methods to convert to a `Str` object from `String` and from `Vec[u8]`
Reviewed By: xavierd
Differential Revision: D19596743
fbshipit-source-id: 6499f7f1b8329f4d14ce8179a41ed46982a85c8e
Summary: Update to the new version of rust-cpython. This supports `list.append`, so make use of it.
Reviewed By: xavierd
Differential Revision: D19590905
fbshipit-source-id: 03609d4f698ae8e4380e82b8144caaa205b4c2d4
Summary:
Also, add a util::path::strip_unc function that is more clear than the
normalize_for_display
Reviewed By: DurhamG
Differential Revision: D19595961
fbshipit-source-id: 330bcb708bf64320a3562d79db685d6cb1e14f16
Summary:
For Python3 compatibility, let's use PyPath, it hides the logic of encoding for
Python2
Reviewed By: DurhamG
Differential Revision: D19590024
fbshipit-source-id: 7bed134a500b266837f3cab9b10604e1f34cc4a0
Summary:
This is optional, but it helps investigating Python errors chained with other
Rust errors.
For example:
error.RustError: failed fetching from store (, cc38739855a7f356b4a2aaac0a0a858fd646e6bf)
Caused by:
TypeError()
Traceback (most recent call last):
File "scm3/edenscm/hgext/remotefilelog/contentstore.py", line 53, in get
chain = self.getdeltachain(name, node)
File "scm3/edenscm/hgext/remotefilelog/contentstore.py", line 91, in getdeltachain
chain = self._getpartialchain(name, node)
File "scm3/edenscm/hgext/remotefilelog/contentstore.py", line 125, in _getpartialchain
return store.getdeltachain(name, node)
TypeError
Without this diff there is only "TypeError()" without the traceback.
This can be turned off by unsetting RUST_BACKTRACE.
Reviewed By: markbt
Differential Revision: D19581173
fbshipit-source-id: 74605b78146b6b1c9ddd5ad720dcd19ff73908a8
Summary: The format_err is used in shared code too, we need to import it.
Reviewed By: quark-zju
Differential Revision: D19592591
fbshipit-source-id: bd344bf3c295473f4647235a98432d11c9678bf9
Summary:
This will be used as an argument to the Rust bindings when using paths. This
type is either a PyBytes in Python2 and uses the various encoding function to
convert into a String, or a PyUnicode in Python3 with no encoding change.
Reviewed By: farnz
Differential Revision: D19587890
fbshipit-source-id: 58903426585693193754691fe3c756b9097b35f6
Summary:
Without this, Rust code using the feature (ex. lz4, used by lz4revlog) will
panic.
Reviewed By: sfilipco
Differential Revision: D19581188
fbshipit-source-id: b499449df4fede27fe66cf8e5af57e8347a0dd48
Summary: This converts to bytes on Python 2, but unicode on Python 3.
Reviewed By: markbt
Differential Revision: D19581180
fbshipit-source-id: 0de9056a01ae30810a72352387de5a940b37d7ab
Summary:
In a future diff, I have RepoPath in Rust and want to send unicode path to
Python.
Reviewed By: sfilipco
Differential Revision: D19581184
fbshipit-source-id: 73a03707a6bdae4a497a8ee2c14314aa4ffefb6d
Summary: The docs promise that both `<` and `>` bounds are inclusive, so let's fix that.
Reviewed By: markbt
Differential Revision: D19580840
fbshipit-source-id: 13770a8e9351fe62f58e9a701b526a167752543a
Summary:
Separate Segment and IdDag in two individual files. This is preparation for
refactoring IdDag to be more flexible in terms of storage. That will probably
involve moving stuff out of IdDag into a new file that deals with the storage
abstractions.
Reviewed By: quark-zju
Differential Revision: D19559127
fbshipit-source-id: b3b9b18e2653157e69148b1f29292a57b30016ec
Summary:
I wrote it to understand how renderdag draws the same graph with different
orders. It seems useful for future optimization that tries to reduce the number
of columns. So let's check it in.
Reviewed By: xavierd
Differential Revision: D19440713
fbshipit-source-id: 8bc580799f6b24c87886d5ac306020f50bb694e5
Summary: This gives us some confidence about octopus merge handling.
Reviewed By: DurhamG
Differential Revision: D19540726
fbshipit-source-id: e84de74aecae54429483edd185d39fd1bd858f87
Summary:
TestContext uses ParentRevs. That limits parents to at most 2.
Use a type parameter so we can opt-in Vec<usize> for octopus merge support,
at the cost of worse cache efficiency.
Reviewed By: DurhamG
Differential Revision: D19540727
fbshipit-source-id: f9e8de151b7b296fd6f0fd89be9de2b8996634c7
Summary:
Our new algorithms support octopus merges. However there were no tests using
octopus merges. This diff adds a simple one.
Reviewed By: DurhamG
Differential Revision: D19540728
fbshipit-source-id: 8411024f0b7e27c2ebfabbe1935496124c25df7b
Summary:
The test runs the old and new algorithm and compares their result. This is more
interesting than using random numbers, since the fuzzing framework will try to
explore new code paths.
Reviewed By: sfilipco
Differential Revision: D19511576
fbshipit-source-id: e9a2066769b54a60bb92643e5715f91a6fccbcb5
Summary:
The ported algorithm will work as a comparison to verify dag's range
implementation.
Reviewed By: sfilipco
Differential Revision: D19511574
fbshipit-source-id: 589353d6e6c91b8d6707c977eeb8558ac733b525
Summary: This can prevent potential moves and clones on the caller of prefetch.
Reviewed By: quark-zju
Differential Revision: D19518697
fbshipit-source-id: 63839fc3f4bb9ca420e290eabaffb481a3584f7b
Summary:
The test runs the old and new algorithm and compares their result. This is more
interesting than using random numbers, since the fuzzing framework will try to
explore new code paths.
This cannot run on stable Rust yet. I added a README for how to run it.
Reviewed By: sfilipco
Differential Revision: D19504096
fbshipit-source-id: 621da02c50a771dee9932f9d7a407cb1f412a543
Summary: Sometimes the graph is too large. Provide a way to slice it.
Reviewed By: sfilipco
Differential Revision: D19511575
fbshipit-source-id: 504317d6894764043b23ea49dcf09c8cdea96961
Summary:
As we plan to test the dag crate with some other DAG implementation,
add a convenient structure that setups both DAG implementations.
Reviewed By: sfilipco
Differential Revision: D19503371
fbshipit-source-id: 3e9933ad37301bfac36eb1af6d82b4298af778b6
Summary:
The ported algorithm will work as a comparison to verify dag's gca
implementation.
Reviewed By: sfilipco
Differential Revision: D19503373
fbshipit-source-id: f5253db89fbcdc2fd02f3fdaa0796e24338b1fba
Summary:
This is similar to D17581248. It will make the old linear-scan algorithm (which
will be added later) about 5x faster.
Reviewed By: sfilipco
Differential Revision: D19503372
fbshipit-source-id: c65d7217e7b144603dadd57f54a5e70f513c8e51
Summary: This allows bindag to be used outside benches.
Reviewed By: sfilipco
Differential Revision: D19503374
fbshipit-source-id: 131061f7d1d28125875a86afc330dbb9634249cf
Summary:
We've seen a case where a datapack contains a circular delta chain, causing
Mercurial to fall into a infinite loop when trying to read it. Let's fail when
the chain is over 1000 entries.
Reviewed By: quark-zju
Differential Revision: D19458453
fbshipit-source-id: bfa503f7807122eca72cf94418abda161dafa41c
Summary: This ensures IdMap and IdDag are guaranteed consistent in the storage layer.
Reviewed By: DurhamG
Differential Revision: D19432658
fbshipit-source-id: 00f1a9b4c747baa1f14d78c31d925682317463b4
Summary: This makes it possible to do extra sanity checks.
Reviewed By: DurhamG
Differential Revision: D19443783
fbshipit-source-id: 254c2537a6aadd25a67c5e48a768187ce65aa686
Summary: This makes the code overall shorter.
Reviewed By: DurhamG
Differential Revision: D19443552
fbshipit-source-id: abd1db227830a88549c7eca1cfd08b67c4914518
Summary:
As reported by JT, EdenFS may hold the file descriptor of mapped pack files too long even when it is deleted by external processes, thus taking more disk spaces.
This diff fixes this problem by making EdenFS periodically rescan the pack files.
Reviewed By: chadaustin
Differential Revision: D19395439
fbshipit-source-id: 4bfd6a7ac13dceb3099d2704d62b3825433aff4b
Summary:
In both cases (clone with or without dirty content), the external key buffers
used by indexes should be re-created, since mem_buf location has changed.
Reviewed By: DurhamG
Differential Revision: D19432657
fbshipit-source-id: fe6f76e7ccfd16ccd2f5c1d89866687a3503603e
Summary: This allows us to build the multi-log structure.
Reviewed By: DurhamG
Differential Revision: D19431786
fbshipit-source-id: e0e09b0d5a73d293a80626924b74ddf2ce6d6fa3
Summary: Migrate to the new API. This is more compatible with the next change.
Reviewed By: DurhamG
Differential Revision: D19431788
fbshipit-source-id: dba1a88fd003d17fc1774b0cec9157d32c5acdb0
Summary:
This allows us to incrementally abstract away parts related to filesystem from
Log. For example, instead of using std::fs to access filesystem directly, use
methods on the GenericPath instead.
Right now the main motivation is to add support for "multi-logs" sharing a single meta
file. To do that, methods reading and writing metas are moved to the
GenericPath type.
This also unifies the open functions. Now OpenOptions::create_in_memory can be
private. Empty in-memory Logs can be opened via `open(())`.
Reviewed By: DurhamG
Differential Revision: D19431784
fbshipit-source-id: dbdf94f60261e09f131c6fdd9fe3b99242a28af5
Summary: This makes an upcoming change easier.
Reviewed By: DurhamG
Differential Revision: D19432656
fbshipit-source-id: 65adc883c3c3937aa7022196b83c75715b820721
Summary: The goal is to split log.rs to smaller modules so it's easier to reason about.
Reviewed By: DurhamG
Differential Revision: D19431787
fbshipit-source-id: 23b8346c461da322e7a2240b8224e6194867aaee
Summary:
The goal is to split log.rs to smaller modules so it's easier to reason about.
OpenOptions has many dependencies. Move the most complicated one (repair) to
another standalone module.
Reviewed By: DurhamG
Differential Revision: D19431783
fbshipit-source-id: 28c3af7cd6e19588bfe9c395e2648244faf3179d
Summary: The goal is to split log.rs to smaller modules so it's easier to reason about.
Reviewed By: DurhamG
Differential Revision: D19431785
fbshipit-source-id: b9a598b3018267ff6e43ef57dc807cb4e880467c
Summary:
In order to avoid pathological linkrevfixup after pushing a change to a
pushrebase repo, we need to be able to fetch history data that is already
present locally from the server. Since the Rust stores will always check
whether the data is present locally before fetching it, we would not be
fetching anything, causing the pathological linkrevfixup to kick in.
By allowing the stores to be built without a local component, the prefetch
function will not find the local history, and will thus be able to fetch it
properly.
Reviewed By: DurhamG
Differential Revision: D19412619
fbshipit-source-id: 421c59c63634ead7f98e6ba89da0532067f7b412
Summary:
In the cpython bindings, the Rust String can take both PyBytes and PyUnicode
strings, which is perfect for Python3 compatibility as string literals are
PyBytes in Python2 and PyUnicode in Python3. The return values are kept as
Bytes for now as changing this is a much larger change in itself.
Other approaches tried:
- Using PyUnicode as input/output: an extremely large codemod had to be done,
with very little benefits
- Using String as output: since we do have some configs that are unicode, on
the Python side, the output might either be bytestrings or unicode, leading
to weird bugs.
Reviewed By: DurhamG
Differential Revision: D18650466
fbshipit-source-id: aebdf30590dcae40b7df2787e5ece88e2ec9395c
Summary:
Mercurial assumes some specific ordering, let's comment about this to make sure
we don't re-order and introduce subtle bugs.
Reviewed By: quark-zju
Differential Revision: D19394352
fbshipit-source-id: 0f9e02d2c6addf040311a54b8161b06bbeaa6be9
Summary:
The error message and the exception type are slightly different between the
Rust and Python ContentStore. For now, let's just fix this up manually.
Reviewed By: quark-zju
Differential Revision: D19394350
fbshipit-source-id: e432094a9dfcf605568a1890c0303b733e98d203
Summary: The rust telemetry wrapper and the python code were not in sync in how they presented errors to the user on failure. This makes them identical in message and error code (however the "abort" from python is red in color).
Reviewed By: quark-zju
Differential Revision: D19313848
fbshipit-source-id: d5b95e8d5130f20fc85c6664933ee18702f7cc5e
Summary:
Previously, NameDag only supports writing directly to disk. That is not easy
for hg to use, since hg knows commits before bookmark names during pull, while
the API requires both commits and bookmarks (to decide which commits belong to
the "master" group) at the same time.
The old API also does not match other storage layers like indexedlog and
metalog, where writes are buffered until explicitly flushed.
This diff adds the "write-in-memory" (named "add_heads"), and "flush" APIs to
make NameDag easier to use. Under the hood, it just copies the "added" portion
of the DAG, and re-use the old API ("build", renamed to "add_heads_and_flush")
to do the job.
Note: The buffered changes pattern cannot be used in IdDag, since Ids might
have to be re-assigned on flush. But NameDag does not expose the raw Ids in its
"normal" interface, so it's fine to reassign Ids on flush.
Reviewed By: markbt
Differential Revision: D19405474
fbshipit-source-id: 75e5e5815c78c3577a0138f48185f6c4b5a80891
Summary:
This is more consistent with the name "IdDag", because both "Name" and "Id" are
noun.
Reviewed By: singhsrb
Differential Revision: D19405473
fbshipit-source-id: a3b7546dd0ddcae5d9ed562838bd6be10a47063c
Summary: This is more consistent with the name "NamedDag".
Reviewed By: singhsrb
Differential Revision: D19405472
fbshipit-source-id: f7023307acaf96bf77c9fa9704dcaf6fc59b56f2
Summary:
The git repo has a lot of merges. So it's interesting as a testing graph.
The file was generated by:
git clone https://github.com/git/git --bare
hg --config extensions.hggit= clone git.git git-hg
hg --cwd git-hg debugbindag -r 'all()' -o git.bindag
Reviewed By: DurhamG
Differential Revision: D19411825
fbshipit-source-id: 0fec8a16786dc8e6790986bca5c62a76276aa942
Summary:
This changes the pattern commonly seen in smartlog:
╷ o e7f5f529 ...
╭─╯
│ o a1b2773d ...
╭─╯
o ecbb4eaa
╷
to:
╷ o e7f5f529 ...
├─╯
│ o a1b2773da ...
├─╯
o ecbb4eaa
╷
The change only applies to commits with a single parent. Vertical lines from
merge commits stay disconnected intentionally. For example:
│ │ o I
│ │ │
│ o │ H
╭─┼─┬─┬─╮
│ │ │ │ o G
│ │ │ │ │
This makes it more obvious that `H` has 5 parents while `I` only has 1 parent -
`I` does not "borrow" `H`'s parents. This problem does not exist if `H` only
has 1 parent.
Reviewed By: quark-zju
Differential Revision: D19407687
fbshipit-source-id: 1046c8e2309f50e3f1620ed21f1b10573759a5f8
Summary:
Moving `store_element` and `get_hgid` from testutil.rs to lib.rs in order to
get rid of some buck build warnings. They are only used in lib.rs.
Reviewed By: markbt
Differential Revision: D19350343
fbshipit-source-id: 081783d753425daaae6fadc38589da5e4e8b745d
Summary:
It makes sense for manifest-tree to expose a testutil version for building a
TreeManifest. No reason for the testutil function to be limited to the crate.
The only issue is that `make_tree` is to generic so we rename the function to
`make_tree_manifest`.
Reviewed By: markbt
Differential Revision: D19350351
fbshipit-source-id: 06fe04d99bf820d3e81417f5a5514aa4b0d282e6
Summary:
Types that are defined by the manifest core crate should have test utilities
defined in the same crate.
This is motivated by various warning in the buck build.
Reviewed By: markbt
Differential Revision: D19350350
fbshipit-source-id: a6a7c3fb54b465aa09a28ff8b70b49a355b328fc
Summary:
Previously, if the rust walker encountered any issues with individual
files it would stop and throw an exception or in some cases panic. To replace
the python walker we need to handle errors more gracefully, so let's add logic
for reporting which files had issues and what those issues were.
This also contains a minor one-line fix to prevent the walker from traversing
directories that contain a '.hg' directory.
Reviewed By: quark-zju
Differential Revision: D19376540
fbshipit-source-id: ede40f2b31aa1e856c564dd9956b9f593f21cf27
Summary:
This simplifies the error handling and makes it more compatible with things that
might capture the output, including the Python testing framework.
Reviewed By: DurhamG
Differential Revision: D19325642
fbshipit-source-id: 53de8b9a8294219e2b8e62831dce236841bd4cbb
Summary: Make `hg status` use the EdenFS Thrift path, similar to the telemetry wrapper.
Reviewed By: DurhamG
Differential Revision: D19325641
fbshipit-source-id: 14777a252d7cb433316511a2a1f1a6649e9cb020
Summary:
Make it possible to define reusable flags like walkopts, templateopts that can
be shared among commands.
Reviewed By: DurhamG
Differential Revision: D19325640
fbshipit-source-id: 89a9ad0e59672b1f8cfa22daaacd3c75d8cff61a
Summary:
Add a crate aiming for talking to EdenFS via Thrift.
This is mostly ported from telemetry. There are lots of things that can be
cleaned up like moving the command portion to hgcommands, or using a better
abstracted rendering layer for outputting, etc. But for now, I just ported
the code as-is.
Reviewed By: DurhamG
Differential Revision: D19325639
fbshipit-source-id: ac70315ed166024857a90aab070e4d521f3e3b58
Summary:
The main motivation is to replace (super slow) Python Thrift serialization.
It also makes it possible to port the native "status" command from telemetry
wrapper to the main executable.
The crate is made to be built with either buck or cargo in fbcode environment:
- Cargo build uses `build.rs` to re-generate (checked-in) Thrift code.
- Buck build uses TARGETS to build Thrift code on the fly.
Reviewed By: DurhamG
Differential Revision: D19325638
fbshipit-source-id: 1d3287d8d6c674065500929827fa13ba9d448708
Summary:
Enhance the tests to test whether the width of the graph matches the width that
was given by the `width` method before `next_row` was called.
The width we are interested in is the number of cells the string would occupy,
not the string length, so use the `unicode-width` crate to determine this.
Reviewed By: quark-zju
Differential Revision: D19282571
fbshipit-source-id: d9852c4c9e0f76c78db047f0da5dd34723a62a2a
Summary: Switch from `:` to `.` for the ancestor lines. I think they look better.
Reviewed By: quark-zju
Differential Revision: D19371721
fbshipit-source-id: e18f1a62e23620a82007e2c377607a0a61623830
Summary:
Box-drawing characters with curves aren't reliably renderable on Windows. Add
a "square" glyph set that uses right-angle characters. These characters are
available in cp437 and cp850, so should be available on most Windows systems.
For completeness, add a "dec" glyph set that uses DEC line drawing characters,
for use in old terminals like xterm.
Reviewed By: quark-zju
Differential Revision: D19371722
fbshipit-source-id: 35887243cceab66c702e2b5278b572f77946805f
Summary:
Using RepoPath::arbitrary() has a high chance or producing a lot of different
top level directories. The helper method added by this change,
`generate_repo_paths` simulates directories having various files and several
directories.
Reviewed By: quark-zju
Differential Revision: D19321366
fbshipit-source-id: 5c1aec78b6157f3cbea3d0673b29b3a676de88c0
Summary:
The TestStore can be leveraged by methods outside the manifest-tree crate.
Anything that wants to instantiate a manifest for test purposes could
leverage it.
Reviewed By: quark-zju
Differential Revision: D19204894
fbshipit-source-id: 4ac42d09855c70f829feefc6c71dcdbf7211cae3
Summary:
If the user specifies a timezone-less date, ex. `2000-7-1`, it's incorrect
to use the same offset of `Local::now()`, because of daylight savings.
Fix it by going through proper type conversion.
Reviewed By: DurhamG
Differential Revision: D19179800
fbshipit-source-id: 6fd433757e7f6e4baf4625944d4061548a436f11
Summary:
Previously, between 100 to 500 runs, the test is likely to fail:
---- log::tests::test_repair_and_delete_content stdout ----
thread 'log::tests::test_repair_and_delete_content' panicked at 'assertion failed: `(left == right)`
left: `"Verified first 141 entries, 999 of 1400012 bytes in log\nReset log size to 999\nIndex \"c\" is incompatible with (truncated) log\nRebuilt index \"c\""`,
right: `"Rebuilt metadata\nVerified first 141 entries, 999 of 1400012 bytes in log\nReset log size to 999\nRebuilt index \"c\""`', src/log.rs:3414:9
The "Rebuilt metadata" part is missing.
This is because after D17742003, the end of `meta` is the `epoch`, which is a
random number. The test is rewriting the end of `meta` to corrupt it. That
has a chance to not corrupt the file. Change the test to corrupt the fixed
header of meta so it can always corrupt the file and therefore stabilize
the test.
Reviewed By: xavierd
Differential Revision: D19325087
fbshipit-source-id: 53d46c8a6cb9771582f8f37e27f185d303fde0ce
Summary:
Per discussion, we decided to use "VertexName" as the struct name for things
like commit hashes, or the string names in tests (or the Mozilla DAG in tests).
Therefore, introduce a dedicated VertexName type and repalce all callsites to
use it.
`bytes::Bytes` is used so copying the `VertexName` is somewhat considered cheap.
This adds some overhead copying slices (and `Bytes` has some overhead). It
regresses the "building segments" benchmark from 673ms to 773ms, which seems
okay given the cleaner interface.
Reviewed By: markbt
Differential Revision: D19154905
fbshipit-source-id: 4c6d4eca67c11c10ed5f21999ccdc3f1b01695e8
Summary:
Redefine Id::{MIN, MAX} using Group so Id::MAX.group() is a valid group.
Previously Id::MAX has an invalid group.
Reviewed By: sfilipco
Differential Revision: D19154908
fbshipit-source-id: 6108f02da786eb228a7d1b772ca5134c1ebb6f6b