Summary:
Also, add a util::path::strip_unc function that is more clear than the
normalize_for_display
Reviewed By: DurhamG
Differential Revision: D19595961
fbshipit-source-id: 330bcb708bf64320a3562d79db685d6cb1e14f16
Summary:
For Python3 compatibility, let's use PyPath, it hides the logic of encoding for
Python2
Reviewed By: DurhamG
Differential Revision: D19590024
fbshipit-source-id: 7bed134a500b266837f3cab9b10604e1f34cc4a0
Summary:
This is optional, but it helps investigating Python errors chained with other
Rust errors.
For example:
error.RustError: failed fetching from store (, cc38739855a7f356b4a2aaac0a0a858fd646e6bf)
Caused by:
TypeError()
Traceback (most recent call last):
File "scm3/edenscm/hgext/remotefilelog/contentstore.py", line 53, in get
chain = self.getdeltachain(name, node)
File "scm3/edenscm/hgext/remotefilelog/contentstore.py", line 91, in getdeltachain
chain = self._getpartialchain(name, node)
File "scm3/edenscm/hgext/remotefilelog/contentstore.py", line 125, in _getpartialchain
return store.getdeltachain(name, node)
TypeError
Without this diff there is only "TypeError()" without the traceback.
This can be turned off by unsetting RUST_BACKTRACE.
Reviewed By: markbt
Differential Revision: D19581173
fbshipit-source-id: 74605b78146b6b1c9ddd5ad720dcd19ff73908a8
Summary: The format_err is used in shared code too, we need to import it.
Reviewed By: quark-zju
Differential Revision: D19592591
fbshipit-source-id: bd344bf3c295473f4647235a98432d11c9678bf9
Summary:
This will be used as an argument to the Rust bindings when using paths. This
type is either a PyBytes in Python2 and uses the various encoding function to
convert into a String, or a PyUnicode in Python3 with no encoding change.
Reviewed By: farnz
Differential Revision: D19587890
fbshipit-source-id: 58903426585693193754691fe3c756b9097b35f6
Summary:
Without this, Rust code using the feature (ex. lz4, used by lz4revlog) will
panic.
Reviewed By: sfilipco
Differential Revision: D19581188
fbshipit-source-id: b499449df4fede27fe66cf8e5af57e8347a0dd48
Summary: This converts to bytes on Python 2, but unicode on Python 3.
Reviewed By: markbt
Differential Revision: D19581180
fbshipit-source-id: 0de9056a01ae30810a72352387de5a940b37d7ab
Summary:
In a future diff, I have RepoPath in Rust and want to send unicode path to
Python.
Reviewed By: sfilipco
Differential Revision: D19581184
fbshipit-source-id: 73a03707a6bdae4a497a8ee2c14314aa4ffefb6d
Summary: The docs promise that both `<` and `>` bounds are inclusive, so let's fix that.
Reviewed By: markbt
Differential Revision: D19580840
fbshipit-source-id: 13770a8e9351fe62f58e9a701b526a167752543a
Summary:
Separate Segment and IdDag in two individual files. This is preparation for
refactoring IdDag to be more flexible in terms of storage. That will probably
involve moving stuff out of IdDag into a new file that deals with the storage
abstractions.
Reviewed By: quark-zju
Differential Revision: D19559127
fbshipit-source-id: b3b9b18e2653157e69148b1f29292a57b30016ec
Summary:
I wrote it to understand how renderdag draws the same graph with different
orders. It seems useful for future optimization that tries to reduce the number
of columns. So let's check it in.
Reviewed By: xavierd
Differential Revision: D19440713
fbshipit-source-id: 8bc580799f6b24c87886d5ac306020f50bb694e5
Summary: This gives us some confidence about octopus merge handling.
Reviewed By: DurhamG
Differential Revision: D19540726
fbshipit-source-id: e84de74aecae54429483edd185d39fd1bd858f87
Summary:
TestContext uses ParentRevs. That limits parents to at most 2.
Use a type parameter so we can opt-in Vec<usize> for octopus merge support,
at the cost of worse cache efficiency.
Reviewed By: DurhamG
Differential Revision: D19540727
fbshipit-source-id: f9e8de151b7b296fd6f0fd89be9de2b8996634c7
Summary:
Our new algorithms support octopus merges. However there were no tests using
octopus merges. This diff adds a simple one.
Reviewed By: DurhamG
Differential Revision: D19540728
fbshipit-source-id: 8411024f0b7e27c2ebfabbe1935496124c25df7b
Summary:
The test runs the old and new algorithm and compares their result. This is more
interesting than using random numbers, since the fuzzing framework will try to
explore new code paths.
Reviewed By: sfilipco
Differential Revision: D19511576
fbshipit-source-id: e9a2066769b54a60bb92643e5715f91a6fccbcb5
Summary:
The ported algorithm will work as a comparison to verify dag's range
implementation.
Reviewed By: sfilipco
Differential Revision: D19511574
fbshipit-source-id: 589353d6e6c91b8d6707c977eeb8558ac733b525
Summary: This can prevent potential moves and clones on the caller of prefetch.
Reviewed By: quark-zju
Differential Revision: D19518697
fbshipit-source-id: 63839fc3f4bb9ca420e290eabaffb481a3584f7b
Summary:
The test runs the old and new algorithm and compares their result. This is more
interesting than using random numbers, since the fuzzing framework will try to
explore new code paths.
This cannot run on stable Rust yet. I added a README for how to run it.
Reviewed By: sfilipco
Differential Revision: D19504096
fbshipit-source-id: 621da02c50a771dee9932f9d7a407cb1f412a543
Summary: Sometimes the graph is too large. Provide a way to slice it.
Reviewed By: sfilipco
Differential Revision: D19511575
fbshipit-source-id: 504317d6894764043b23ea49dcf09c8cdea96961
Summary:
As we plan to test the dag crate with some other DAG implementation,
add a convenient structure that setups both DAG implementations.
Reviewed By: sfilipco
Differential Revision: D19503371
fbshipit-source-id: 3e9933ad37301bfac36eb1af6d82b4298af778b6
Summary:
The ported algorithm will work as a comparison to verify dag's gca
implementation.
Reviewed By: sfilipco
Differential Revision: D19503373
fbshipit-source-id: f5253db89fbcdc2fd02f3fdaa0796e24338b1fba
Summary:
This is similar to D17581248. It will make the old linear-scan algorithm (which
will be added later) about 5x faster.
Reviewed By: sfilipco
Differential Revision: D19503372
fbshipit-source-id: c65d7217e7b144603dadd57f54a5e70f513c8e51
Summary: This allows bindag to be used outside benches.
Reviewed By: sfilipco
Differential Revision: D19503374
fbshipit-source-id: 131061f7d1d28125875a86afc330dbb9634249cf
Summary:
We've seen a case where a datapack contains a circular delta chain, causing
Mercurial to fall into a infinite loop when trying to read it. Let's fail when
the chain is over 1000 entries.
Reviewed By: quark-zju
Differential Revision: D19458453
fbshipit-source-id: bfa503f7807122eca72cf94418abda161dafa41c
Summary: This ensures IdMap and IdDag are guaranteed consistent in the storage layer.
Reviewed By: DurhamG
Differential Revision: D19432658
fbshipit-source-id: 00f1a9b4c747baa1f14d78c31d925682317463b4
Summary: This makes it possible to do extra sanity checks.
Reviewed By: DurhamG
Differential Revision: D19443783
fbshipit-source-id: 254c2537a6aadd25a67c5e48a768187ce65aa686
Summary: This makes the code overall shorter.
Reviewed By: DurhamG
Differential Revision: D19443552
fbshipit-source-id: abd1db227830a88549c7eca1cfd08b67c4914518
Summary:
As reported by JT, EdenFS may hold the file descriptor of mapped pack files too long even when it is deleted by external processes, thus taking more disk spaces.
This diff fixes this problem by making EdenFS periodically rescan the pack files.
Reviewed By: chadaustin
Differential Revision: D19395439
fbshipit-source-id: 4bfd6a7ac13dceb3099d2704d62b3825433aff4b
Summary:
In both cases (clone with or without dirty content), the external key buffers
used by indexes should be re-created, since mem_buf location has changed.
Reviewed By: DurhamG
Differential Revision: D19432657
fbshipit-source-id: fe6f76e7ccfd16ccd2f5c1d89866687a3503603e
Summary: This allows us to build the multi-log structure.
Reviewed By: DurhamG
Differential Revision: D19431786
fbshipit-source-id: e0e09b0d5a73d293a80626924b74ddf2ce6d6fa3
Summary: Migrate to the new API. This is more compatible with the next change.
Reviewed By: DurhamG
Differential Revision: D19431788
fbshipit-source-id: dba1a88fd003d17fc1774b0cec9157d32c5acdb0
Summary:
This allows us to incrementally abstract away parts related to filesystem from
Log. For example, instead of using std::fs to access filesystem directly, use
methods on the GenericPath instead.
Right now the main motivation is to add support for "multi-logs" sharing a single meta
file. To do that, methods reading and writing metas are moved to the
GenericPath type.
This also unifies the open functions. Now OpenOptions::create_in_memory can be
private. Empty in-memory Logs can be opened via `open(())`.
Reviewed By: DurhamG
Differential Revision: D19431784
fbshipit-source-id: dbdf94f60261e09f131c6fdd9fe3b99242a28af5
Summary: This makes an upcoming change easier.
Reviewed By: DurhamG
Differential Revision: D19432656
fbshipit-source-id: 65adc883c3c3937aa7022196b83c75715b820721
Summary: The goal is to split log.rs to smaller modules so it's easier to reason about.
Reviewed By: DurhamG
Differential Revision: D19431787
fbshipit-source-id: 23b8346c461da322e7a2240b8224e6194867aaee
Summary:
The goal is to split log.rs to smaller modules so it's easier to reason about.
OpenOptions has many dependencies. Move the most complicated one (repair) to
another standalone module.
Reviewed By: DurhamG
Differential Revision: D19431783
fbshipit-source-id: 28c3af7cd6e19588bfe9c395e2648244faf3179d
Summary: The goal is to split log.rs to smaller modules so it's easier to reason about.
Reviewed By: DurhamG
Differential Revision: D19431785
fbshipit-source-id: b9a598b3018267ff6e43ef57dc807cb4e880467c
Summary:
In order to avoid pathological linkrevfixup after pushing a change to a
pushrebase repo, we need to be able to fetch history data that is already
present locally from the server. Since the Rust stores will always check
whether the data is present locally before fetching it, we would not be
fetching anything, causing the pathological linkrevfixup to kick in.
By allowing the stores to be built without a local component, the prefetch
function will not find the local history, and will thus be able to fetch it
properly.
Reviewed By: DurhamG
Differential Revision: D19412619
fbshipit-source-id: 421c59c63634ead7f98e6ba89da0532067f7b412
Summary:
In the cpython bindings, the Rust String can take both PyBytes and PyUnicode
strings, which is perfect for Python3 compatibility as string literals are
PyBytes in Python2 and PyUnicode in Python3. The return values are kept as
Bytes for now as changing this is a much larger change in itself.
Other approaches tried:
- Using PyUnicode as input/output: an extremely large codemod had to be done,
with very little benefits
- Using String as output: since we do have some configs that are unicode, on
the Python side, the output might either be bytestrings or unicode, leading
to weird bugs.
Reviewed By: DurhamG
Differential Revision: D18650466
fbshipit-source-id: aebdf30590dcae40b7df2787e5ece88e2ec9395c
Summary:
Mercurial assumes some specific ordering, let's comment about this to make sure
we don't re-order and introduce subtle bugs.
Reviewed By: quark-zju
Differential Revision: D19394352
fbshipit-source-id: 0f9e02d2c6addf040311a54b8161b06bbeaa6be9
Summary:
The error message and the exception type are slightly different between the
Rust and Python ContentStore. For now, let's just fix this up manually.
Reviewed By: quark-zju
Differential Revision: D19394350
fbshipit-source-id: e432094a9dfcf605568a1890c0303b733e98d203
Summary: The rust telemetry wrapper and the python code were not in sync in how they presented errors to the user on failure. This makes them identical in message and error code (however the "abort" from python is red in color).
Reviewed By: quark-zju
Differential Revision: D19313848
fbshipit-source-id: d5b95e8d5130f20fc85c6664933ee18702f7cc5e
Summary:
Previously, NameDag only supports writing directly to disk. That is not easy
for hg to use, since hg knows commits before bookmark names during pull, while
the API requires both commits and bookmarks (to decide which commits belong to
the "master" group) at the same time.
The old API also does not match other storage layers like indexedlog and
metalog, where writes are buffered until explicitly flushed.
This diff adds the "write-in-memory" (named "add_heads"), and "flush" APIs to
make NameDag easier to use. Under the hood, it just copies the "added" portion
of the DAG, and re-use the old API ("build", renamed to "add_heads_and_flush")
to do the job.
Note: The buffered changes pattern cannot be used in IdDag, since Ids might
have to be re-assigned on flush. But NameDag does not expose the raw Ids in its
"normal" interface, so it's fine to reassign Ids on flush.
Reviewed By: markbt
Differential Revision: D19405474
fbshipit-source-id: 75e5e5815c78c3577a0138f48185f6c4b5a80891
Summary:
This is more consistent with the name "IdDag", because both "Name" and "Id" are
noun.
Reviewed By: singhsrb
Differential Revision: D19405473
fbshipit-source-id: a3b7546dd0ddcae5d9ed562838bd6be10a47063c
Summary: This is more consistent with the name "NamedDag".
Reviewed By: singhsrb
Differential Revision: D19405472
fbshipit-source-id: f7023307acaf96bf77c9fa9704dcaf6fc59b56f2
Summary:
The git repo has a lot of merges. So it's interesting as a testing graph.
The file was generated by:
git clone https://github.com/git/git --bare
hg --config extensions.hggit= clone git.git git-hg
hg --cwd git-hg debugbindag -r 'all()' -o git.bindag
Reviewed By: DurhamG
Differential Revision: D19411825
fbshipit-source-id: 0fec8a16786dc8e6790986bca5c62a76276aa942
