Summary:
- Implement a basic grep.
- Implement a naive `sedi`.
- Replace some shell for loops using Python for loops.
- Replace some shell code in Python.
Reviewed By: mitrandir77
Differential Revision: D17616051
fbshipit-source-id: 2fcc19a928989b66e5e9502e10aa07c37b8d1056
Summary:
$TESTTMP wasn't being expanded due to being in between '', since we're
already in the right directory, there is no need to use $TESTTMP.
Reviewed By: DurhamG
Differential Revision: D17687911
fbshipit-source-id: d9f7048cc95955f59e8a492335a90d48773bc6d2
Summary: The test test-check-code.t was complaining about it.
Reviewed By: quark-zju, singhsrb
Differential Revision: D17687037
fbshipit-source-id: b61351a239554d73fd3285dedca613e72d1d3e59
Summary: Now help text for the snapshot extension doesn't have any TODOs.
Reviewed By: markbt
Differential Revision: D17627630
fbshipit-source-id: 3e02f6540ab84010712a146a069dbb4a3968182f
Summary:
If some bundle consisted of several snapshots, and the client requests a part of that data, the server can trigger the rebundling process.
This diff teaches snapshots to the rebundling process.
The `bundlerepo` object should store data in memory, so a `unionstore` class is made up.
It allows to "pair" in-memory store with a regular local store and query them via the same API.
Reviewed By: markbt
Differential Revision: D17684945
fbshipit-source-id: 0bdb1f70b777fb2d8193b3bfa46b5915c2019663
Summary:
`util.dirs` does not have a defined order. This stablizes the doctest in
`_rootsanddirs`.
Reviewed By: xavierd
Differential Revision: D17679690
fbshipit-source-id: f3e92c0cde1440e6cb225881dbc0de4220c399b9
Summary: Allow switching between HTTP gettreepack based prefetching vs client-drive BFS HTTP prefetching via a config option. Will be used to make performance comparisons.
Reviewed By: xavierd
Differential Revision: D17381174
fbshipit-source-id: 30a029378b050dd40cb73a616f6d3ebd8e3f3e1d
Summary:
Add a client-driven tree prefetching implementation to the Rust manifest code. Unlike the existing prefetch implementation in Python, this one does all computation of which nodes to fetch on the client side using the BFS logic from BfsDiff. The trees are then bulk fetched layer-by-layer using EdenAPI.
This initial version is fairly naive, and omits some obvious optimizations (such as performing fetches of multiple trees concurrently), but is sufficient to demonstrate HTTP tree prefetching in action.
Reviewed By: xavierd
Differential Revision: D17379178
fbshipit-source-id: f17fe99834ad4fec07b4a4ab196928cc4fe91142
Summary:
Change the `Files` iterator in the Rust manifest code to traverse the tree in BFS order, allowing for layer-by-layer prefetching similar to `Diff`. This can substantially speed up walks over the tree when the cache is cold.
As a side-effect, this changes the order in which paths are reported during a manifest walk. (In particular, they are now reported in breadth-first order rather than depth-first order.) This may break things that rely on the existing ordering; as such, we may need to add a sort somewhere if this turns out to be a problem.
Reviewed By: xavierd
Differential Revision: D17645389
fbshipit-source-id: 624e426094a93e206bde4523ea8bd034fe5aeb90
Summary:
This diff makes the code closer to shell behavior.
For example, globs are disabled for both single and double quotes:
~ % echo /bin/bash*
/bin/bash /bin/bashbug
~ % echo "/bin/bash*"
/bin/bash*
~ % echo '/bin/bash*'
/bin/bash*
Environment variables are not expanded for single qutoes:
~ % echo $PWD
/home/quark
~ % echo "$PWD"
/home/quark
~ % echo '$PWD'
$PWD
Tests using single quoted environment varialbes are updated to use double
quotes, mostly by using this vim command:
%s/\(sh % ".*\)'\([^'$]*\$[^']*\)'/\1\\"\2\\"
The translation script was updated to prefer double quotes to preserve
environment variable expansion behavior.
Reviewed By: xavierd
Differential Revision: D17675351
fbshipit-source-id: d5c8d5f23ea8e29fe093c4e6ae89ddacda97141e
Summary: This allows customization about how to expand an argument with different quotes.
Reviewed By: xavierd
Differential Revision: D17675350
fbshipit-source-id: 1dc4b2e2b7ea0296dceda0d8654c2278a602e4f8
Summary:
We need to get "whether an argument is single-quoted or not" information to
decide whether to run extra logic (ex. expanding globs) on it. `shlex` does not
really have public interface for that and it's hard to `wrapfunction` patch it.
Therefore vendor the script so we can customize it.
Some Python 2 compatibility work was done:
- use `cStringIO` instead of `io`
- mark the file as utf-8
- remove `quote`, which uses an incompatible `re.ASCII`
Reviewed By: xavierd
Differential Revision: D17675349
fbshipit-source-id: eb99a9b69ca15f770c410bd3845052cb0c6f1b17
Summary: The variable was used but sometimes not initialized.
Reviewed By: quark-zju
Differential Revision: D17667256
fbshipit-source-id: 9efa3b160cb23205baaa614827e5990475b956c9
Summary:
`arg` can be empty in this context. Avoid crashing with IndexError.
Also fix test-sparse-issues-t.py so it quotes the pattern correctly.
Reviewed By: xavierd
Differential Revision: D17669371
fbshipit-source-id: ea9e4e7c09a88308a650442c22e133684805738e
Summary: Avoid using `rm -rf` but just create new repos aggressively.
Reviewed By: singhsrb
Differential Revision: D17669063
fbshipit-source-id: f5ea142b08f16dcbdcf99a937fba803e1d8d9958
Summary: This makes it clear that the new behavior is different.
Reviewed By: sfilipco
Differential Revision: D17584605
fbshipit-source-id: 59573dfa14b4eb7a4a9c5bce6ae7340f408035da
Summary:
When migrating down from narrow-heads, we need to restore phase roots. Use a
repo requirement to track the migration state, and perform migrations on
demand.
Reviewed By: sfilipco
Differential Revision: D17584604
fbshipit-source-id: 36c04b0385dd94481f5340a3ecd8eeef3506c765
Summary:
I think we moved the last perforce based repository into `fbsource`.
From the Scuba logs, this doesn't seem to be used any more:
https://fburl.com/scuba/lc6vgy3v. We can revive the extension if required for a
future Perforce repository import.
Reviewed By: quark-zju
Differential Revision: D17630314
fbshipit-source-id: e3ca1a97974aff5fc262b8c724b70d4548280325
Summary: Move ipdb logic so it wouldn't be triggered for common errors like error.Abort.
Reviewed By: xavierd
Differential Revision: D17628968
fbshipit-source-id: fae390d56203d8362a10a75acaa67901279656cf
Summary:
The test is mostly automatic conversion and easy fixes.
No refactoring to simplify code, on the contrary, more
direct printing.
Reviewed By: quark-zju
Differential Revision: D17630869
fbshipit-source-id: 45e57de5fd965d0cc913c2612a6f52cb3c8bd977
Summary:
Filename expansion is probably used a lot in our tests. I think that our
migration path is going to be simplified if we do filename expansion as
part of shobj. The current implementation works for the test that I was
migrating, test-revert.
Reviewed By: quark-zju
Differential Revision: D17630872
fbshipit-source-id: b8312e3d1fee9d331a661849a4cc5ef06ed5b091
Summary: Following a similar implementation to `chmod +x`.
Reviewed By: quark-zju
Differential Revision: D17630871
fbshipit-source-id: 115fe8fd0cca214f7f739bec66301c9116edb250
Summary: I would like to call into this script directly from dott tests.
Reviewed By: quark-zju
Differential Revision: D17630870
fbshipit-source-id: 169ab3f1706d77cb1cdc1461b2a6fce2bc77a877
Summary:
Renaming it to `generateworkingcopystates`. This is a valid module name.
The goal is to convert this script to a module. While we are converting
tests it needs to be accessible in both script form and module.
Reviewed By: singhsrb
Differential Revision: D17611370
fbshipit-source-id: 0b2d0de71f0da7cd6efd7874c505d9aa77f7b296
Summary:
This test checks that a directory without modifications between tree
and parent is not going to be materialized.
Reviewed By: quark-zju
Differential Revision: D17540173
fbshipit-source-id: 465f1e0410c42a55665bcd6903d75266c61d5e80
Summary:
`util.hgcmd()` can report an incorrect hg path in the buck build case, since
the buck target is a wrapper script.
Fix it by setting the executable path explicitly.
Reviewed By: simpkins
Differential Revision: D17632684
fbshipit-source-id: d6c208fa1981180f6248bf7b57cc57031f53946e
Summary: Remove the DFS diff implementation and replace it with the BFS implementation.
Reviewed By: xavierd
Differential Revision: D17618818
fbshipit-source-id: d486642caae924f866a200d3c82fa5a4cb7d5286
Summary: In order to support efficient data fetching, eventually BFS diff will need to be the default diff algorithm. As an initial step towards this, enable `treemanifest.bfsdiff` by default to validate that the BFS diff algorithm works as a drop-in replacement for the existing diff algorithm. Once we have confidence in this, we can land the next diff in this stack wherein we remove DFS diff entirely.
Reviewed By: xavierd
Differential Revision: D17618819
fbshipit-source-id: 444cb05874913e4eb267a24bd0596e6dde2e099e
Summary: To ensure feature parity between BFS diff and DFS diff, copy the DFS tests into the BFS module and ensure they pass.
Reviewed By: xavierd
Differential Revision: D17618820
fbshipit-source-id: b516abbfa4e231fdc383293d94d8965333f2ab99
Summary:
This test is currently failing and its not worth fixing since we only
needed this during the `WWW` Subversion deprecation which has already happened.
Reviewed By: quark-zju
Differential Revision: D17629893
fbshipit-source-id: 60f56b84a3cdcd8ddc56d46a155b1f14f2c00646
Summary:
The use of `fdopen` was to change the buffer size. It's not fatal if buffer
size cannot be changed. This fixes a crash in `nohup hg root`.
Reviewed By: aditya7fb
Differential Revision: D17628883
fbshipit-source-id: 58f75e8e99fbe4cea8da2e3b48a092ee1331db6f
Summary: Python binding for working copy rust library. Walker is initialized with the root of the repo and the python matcher, iteratively returns the matching files in the working copy. Modified Walker and PythonMatcher to allow Walker to have Send trait.
Reviewed By: xavierd
Differential Revision: D17403235
fbshipit-source-id: b8b84928aac7c79c4388a8ba8aa5475aac0c5219
Summary:
Remove py from python matcher, because of RC, so python matcher will have send trait.
The python matcher needs to be stored by the python walker, and pyclasses can only
store data that is Send + 'static. Python pathmatcher methods should only be called
by python methods.
Reviewed By: xavierd
Differential Revision: D17511705
fbshipit-source-id: 00a4938fb00c30244ae04cb38362e8875c72fa47
Summary:
Define the `with_chg` feature when building hg_rust, and link in the chg C
code.
Reviewed By: ikostia
Differential Revision: D17109105
fbshipit-source-id: b8e210df19ff5695553d54047ed561cf7397738f
Summary:
Change the `//scm/hg:hg` target to use an `sh_binary()` rule that invokes the
`:hg_rust` binary with the proper environment so it can find its dependencies,
rather than copying the binary and all of its dependencies into a new
subdirectory.
In dev mode builds the `hg_rust` binary isn't guaranteed to work anywhere
other than its original location, due to the way that dev mode builds use
`$ORIGIN` in the binary's `RPATH` setting. This happened to work up until now
as the hg_rust binary did not have any separate libraries, but I plan to add
one on the `chg` library.
Reviewed By: quark-zju
Differential Revision: D17109104
fbshipit-source-id: ae8bb1126969f012d1d2fb7d04e80867a310b9a8
Summary:
According to the [manifest specification][1], the `path` attribute of a
`<project>` is "An optional path ... If not supplied the project name is
used". Respect this when we parse manifests ourselves.
We may want to consider importing the manifest-parsing module from `repo`
itself and using that instead of rolling our own parsing code. That would
prevent bugs like this from happening at all.
[1] https://gerrit.googlesource.com/git-repo/+/master/docs/manifest-format.md
Reviewed By: mwdevine
Differential Revision: D17542715
fbshipit-source-id: 13d963f4d9b8bf30bcd5f5593aef0ad9da90e7ec
Summary:
Python doesn't recursively import submodules, so we need to specify the
exact module we're using if we want to be able to access it. I don't know
why this works in Michael's development environment, but running the
converter fails locally for me unless I do this.
Reviewed By: mwdevine
Differential Revision: D17542716
fbshipit-source-id: 5b769e193a964f8499ee35033e61d71cf0ccef90
Summary:
Both xavierd and I noticed that characters typed in the terminal sometimes got
eaten. In my case, killing the background cloud sync processes solves it.
It turns out that the cloud sync erroed out like:
ServiceError: service error: 500 ...: Mysql error 1062. Duplicate entry 'fbsource-user/quark/default-remote-fbobjc/stable' for key 'PRIMARY' to db ...
And the `cloud sync` command started ipdb, which starts eating characters :/.
D17621846 should have fixed it. Let's just make it safer by redirecting
stdin of background processes to /dev/null.
Reviewed By: xavierd
Differential Revision: D17622250
fbshipit-source-id: d804aea4aa0aeb50bba0307f93a41eb94e8529fc
Summary:
xavierd encountered an issue where the user name becomes `test`. It turned out
to be chgserver was started using the `TESTTMP` environment and `util.getuser`
got defined wrong.
Fix it by making `util` redefine those functions for every chg invocation.
Reviewed By: xavierd
Differential Revision: D17622095
fbshipit-source-id: e12665317957eeeb85a1796c2bc252a487805cef
Summary:
It turns out that `ui.interactive()` only checks if input is a tty or not,
we also need to check output. That's `ui.formatted()`.
Reviewed By: xavierd
Differential Revision: D17621846
fbshipit-source-id: 5ed30e3dfd59ed8e50261d0b085fbfded43b1c19
Summary:
We want to get rid of `hgsubversion` now that Subversion is almost
dead. Lets move the functionality still provided by the `hgsubversion`
extension to the `globalrevs` extension.
Reviewed By: quark-zju
Differential Revision: D17611443
fbshipit-source-id: 31e1b5a80ecf453e70a7e4ca929d805865579ab3
Summary:
Make the Python test pass, mainly:
- avoid `grep`, use `hg log -T` instead
- rewrite shell `for` loops
- rewrite `python foo.py` to inline python function
Remove a few less important, and hard-to-translate cases.
Reviewed By: xavierd
Differential Revision: D17611805
fbshipit-source-id: a2cd5de8815b60e632b3f46b8d4493575f9388f8