Commit Graph

30 Commits

Author SHA1 Message Date
Jun Wu
41ccbe9f8d hg.rust: add BuildInfo symbols
Summary:
BuildInfo symbols are generated for fbcode binaries, and scanned by the
internal procprint tool. The procprint tool also scans hg binaries, but cannot
extract any build information.

The procprint data is used by Push4Push to find out job owners running ancient
software.

This diff inserts BuildInfo symbols so Push4Push (and we) can use them to
figure out people running old versions of hg and send tasks to them.

Note rpmbuild strips symbols by default and is hard to customize. For now, I
just disable the debuginfo packages to preserve the symbols. If space usage or
runtime performance becomes an issue, we can try other workarounds (ex. provide
a shim of `%_find_debuginfo_dwz_opts` that skips `hg.rust`, or provide a shim
of `%__strip`, etc).

By default, the symbols only get added when `setup.py` is run by `rpmbuild`.
So `make local` won't rebuild `hg.rust` every time.

Reviewed By: singhsrb

Differential Revision: D14464056

fbshipit-source-id: ac6f9dedf28cee5fc8d65e144e3d7d1c544d1eda
2019-03-20 23:44:10 -07:00
Wez Furlong
ce8a548d96 hg: rename scratch to mkscratch and include in the hg build
Summary:
My goal is to have `mkscratch` get deployed with the mercurial
release on all platforms.   This is currently deployed via packman
as `scratch` on devservers, and that packman build should continue to
succeed because the TARGETS file in here is left unchanged.

That will allow both `scratch` and `mkscratch` to coexist for a while
until we've made it through a few mercurial releases, and then we can
either fixup existing users of `scratch` to use `mkscratch`, or re-do
the `fb-scratch` rpm so that it symlinks to `mkscratch`.

I don't really know if this diff is sufficient to include the binary
in the mercurial package; I'm cargo culting the `scm_daemon` stuff
and that seems to be included in the omnibus, so maybe this is it?

Reviewed By: quark-zju

Differential Revision: D14060864

fbshipit-source-id: 5fd74a877437d84fe7676819c2cefb902fb7002d
2019-02-14 17:28:14 -08:00
Wez Furlong
6e445711e6 hg: move scratch into hg code base
Summary:
This shouldn't change any behavior, but should allow
deploying scratch to mac and windows in D14060864

Reviewed By: simpkins

Differential Revision: D14060865

fbshipit-source-id: 8daf65739fa6835c6df682484a8f2dba432977dd
2019-02-13 17:19:46 -08:00
Jun Wu
443a8f33b3 indexedlog: move binary indexedlog_dump out
Summary:
It makes testing duplicated - now `cargo test` would try running tests on 2 entry points:
lib.rs and indexedlog_dump.rs.  Move it to a separate crate to solve the issue.

Reviewed By: markbt

Differential Revision: D13498266

fbshipit-source-id: 8abf07c1272dfa825ec7701fd8ea9e0d1310ec5f
2018-12-18 08:17:21 -08:00
Kostia Balytskyi
a07b4a97bc hg.rust: ensure we rebuild if chg changes
Summary: For the sake of incremental builds and `make local`.

Reviewed By: quark-zju

Differential Revision: D13058780

fbshipit-source-id: f59dc235e75232b475ca0c2504b47ff0d4a64c8a
2018-11-16 02:35:09 -08:00
Jun Wu
61f0a3da45 tests: add a test-check test that runs fix-code.py
Summary:
Add "--dry-run" for fix-code.py and use it in test-check.
This avoids license header and version = "*" issues.

Reviewed By: ikostia

Differential Revision: D10213070

fbshipit-source-id: 9fdd49ead3dfcecf292d5f42c028f20e5dde65d3
2018-11-15 18:54:06 -08:00
Jun Wu
616306543b codemod: use explicit versions in Cargo.toml
Summary:
This is done by running `fix-code.py`. Note that those strings are
semvers so they do not pin down the exact version. An API-compatiable upgrade
is still possible.

Reviewed By: ikostia

Differential Revision: D10213073

fbshipit-source-id: 82f90766fb7e02cdeb6615ae3cb7212d928ed48d
2018-11-15 18:54:06 -08:00
Jun Wu
a411ecb012 hgmain: remove default features
Summary:
The default feature requires additional environment setup. Since environment
setup cannot be avoided, let's rely on the "environment setup" logic to specify
"with_chg" feature correctly. So "cargo build" without specifying features can
work.

This might fix the Windows CI build.

Reviewed By: DurhamG

Differential Revision: D12948723

fbshipit-source-id: cf2609bccace69cd22a9e4ef4fd2c1fc1bf73338
2018-11-06 15:40:25 -08:00
Kostia Balytskyi
475f7f8d90 hg.rust: add support for passing control to chg
Summary:
This is needed before we start moving builtins into `hg.rust`. The decision
to run or not run a builtin needs to be made before the decision to run or not
run a `chg`.

Reviewed By: quark-zju

Differential Revision: D10859033

fbshipit-source-id: 04018904684fc3c8ca2b927bfb828e40a8d09938
2018-11-05 10:08:29 -08:00
Kostia Balytskyi
c5aa7a01b3 chg: learn to build a static lib
Summary:
In order to be able to run `chg` from the main `hg.rust` binary, we decided
that we will turn it into a static lib. This diff teaches our current build
scripts to do this.

Reviewed By: quark-zju

Differential Revision: D10853906

fbshipit-source-id: 8e0f37aa7e52d4a0610f36d7903eb0a318c193ad
2018-11-05 10:08:29 -08:00
Jun Wu
3adc813687 codemod: add copyright headers
Summary: This is just the result of running `./contrib/fix-code.py $(hg files .)`

Reviewed By: ikostia

Differential Revision: D10213075

fbshipit-source-id: 88577c9b9588a5b44fcf1fe6f0082815dfeb363a
2018-10-26 15:09:12 -07:00
Kostia Balytskyi
f2b8b4571f hgpython: rename hgenv to be buildenv
Summary: As per quark-zju's request in the earlier diff.

Reviewed By: quark-zju

Differential Revision: D10173168

fbshipit-source-id: 20ab1fbc597b8329bbfec5dabd501d202571bdec
2018-10-12 14:55:09 -07:00
Kostia Balytskyi
0946205e68 hgmain/hgpython: add copyright headers
Reviewed By: farnz

Differential Revision: D10145635

fbshipit-source-id: 0d88c18a44a86a8eb19f40ddba0c13f9570f3a76
2018-10-12 14:55:09 -07:00
Kostia Balytskyi
682e4bed1a hgpython: extract hgpython from hgmain
Summary:
Following the conversation with quark-zju, this in future will help us conditionally dynamically load
the `hgpython` `.dll`/`.so` only if we need it.

Reviewed By: quark-zju

Differential Revision: D10084949

fbshipit-source-id: c20ef014ad9922913ee36d1ec28b0555b64f7d1f
2018-10-12 14:55:09 -07:00
Kostia Balytskyi
fd6e0cb3f6 hg.rust: make sure build/env is respected when determining hg-python location
Summary:
This is important to facilitate `make local`-like builds. For those builds on Windows,
we can override some run-time behaviors of Mercurial binary via the `build/env` file.

Reviewed By: quark-zju

Differential Revision: D9919807

fbshipit-source-id: 1d83a4457d0bba4f5f9e901e01b29cf43a737259
2018-10-12 14:55:09 -07:00
Kostia Balytskyi
593d138e75 hg.rust: add hgenv wrapper
Summary:
`hgenv::HgEnv` is a wrapper functionality around environment variables reading
logic. This wrapper is capable of determining that Mercurial is run from within
the repo and that `./build/env` file exists. In this case, the wrapper will
use the contents of the file as a fallback source for environment variables.

Reviewed By: quark-zju

Differential Revision: D9945195

fbshipit-source-id: 17dcea32c26868b66a3f506e2c2c0816bc452d45
2018-10-12 14:55:09 -07:00
Kostia Balytskyi
d2056d4c86 hg.rust: add some function docs
Summary: Just adding docs.

Reviewed By: StanislavGlebik

Differential Revision: D9945194

fbshipit-source-id: 4420a31acc458ad996d827b57e752c23ef643ef1
2018-10-12 14:55:09 -07:00
Kostia Balytskyi
0aa67ea4c2 hg.rust: support embedded entry points
Summary:
To facilitate `py2exe`-like loading of Python's stdlib, we need to adjust
code in multiple places, one of which is the logic of entry point search
by the `hg.rust` binary.

Things, considered after this commit are:
1. `HGPYENTRYPOINT` env var, which points to a zip file => append `mercurial/entrypoint.py` and proceed
2. `HGPYENTRYPOINT` env var, which points to a normal file (maybe in the form of `.../library.zip/mercurial/entrypoint.py`)
3. `lib/library.zip/mercurial/entrypoint.py` under the same dir as the executing binary
4. `mercurial/entrypoint.py` under the same dir as the executing binary
5. A list of locations, provided by the `HGPYENTRYPOINTSEARCHPATH` env var, inspected at compile time. Each of these is expected to contain `mercurial/entrypoint.py`
6. A list hardcoded locations, each of which is expected to contain `mercurial/entrypoint.py`

The way decision is made:
After the candidates list is constructed in an order, described above, we take the first candidate that satisfies the condition: it either exists or some zip file exists somewhere in the candidate path.

Reviewed By: quark-zju

Differential Revision: D9919808

fbshipit-source-id: 06cb736867432c39f75019f5b44a48918c57a8e5
2018-10-12 14:55:09 -07:00
Kostia Balytskyi
92685331b1 hg: move to more fine-grained python execution from the binary
Summary:
This steals lots of code from [1] and uses dev versions of crates from [2]
as well.

This approach is better than calling things via `Py_Main` directly because
it allows us to pre-populate the container with some data (pre-population
needs to happen after `Py_Initialize` and before passing control to
current Python-based hg).

NB: returning `1` when the exception instance does not have a `code` attribute is an experimentally-determined decision. With `255` tests fail, so `1` just matches Python behavior. I guess we can change it anytime if we come up with exit code globbing for the tests.

Reviewed By: quark-zju

Differential Revision: D9482436

fbshipit-source-id: 9ca63f2358ddc1de60228c489aa9d02b4b7bc482
2018-08-30 04:42:11 -07:00
Kostia Balytskyi
59e00ccf47 hg: add some convenient panicking conversion to encoding
Summary:
Local bytes `&[u8]` or `Vec<u8>` is frequently wrapped into a `CString`,
because `CString` includes a trailing 0. Let's add a helper for that.

Reviewed By: quark-zju

Differential Revision: D9482435

fbshipit-source-id: 096ba725d83acc9c5fc1fe836dce509fe36e49e9
2018-08-30 04:42:11 -07:00
Kostia Balytskyi
67ab33a170 hg: migrate hgmain to scm/hg/lib/encoding conversions
Summary: Let's use the centrally provided stuff.

Reviewed By: quark-zju

Differential Revision: D9457398

fbshipit-source-id: 2c493e7466118f49a9ea483c9d90211694bfed6b
2018-08-30 04:42:11 -07:00
Kostia Balytskyi
4629e800fa hgmain: implement a binary entry point for Mercurial
Summary:
This crate compiles in a binary entry-point for Mercurial.
The way the binary works at first:
- dynamically link to the Python's shared library (Linux, Windows) or Python
  Framework (OSX)
- find the location of `hgexe.py` (this is what's currently known as `hg` and
  what serves as a shebang entrypoint on OSX and Linux today)
- call `Py_Main` with `hgexe.py` and the rest of the supplied arguments

A couple of decisions that someone might find controversial:
- all strings are expected to be correctly conversible to/from Rust String
- in particular, this means that we're limiting the command line arguments (and
  therefore the repo file names) to the valid Unicode strings. This is not much
  of a limitation on Windows, but it is on Linux.
- `build.rs` takes linker path hints from the environment, to be supplied by
  the `setup.py` later
- `hgexe.py` is expected to live either alongside the resulting binary or in
  the `libexec` dir, that is a sister to one of the binary's parents.

Reviewed By: quark-zju

Differential Revision: D9218066

fbshipit-source-id: ff6e01d06f118cd6bc1a20fbab6b5d17fe51b531
2018-08-17 10:51:47 -07:00
Jun Wu
cce43b11c1 scmdaemon: use io::Write on osx
Summary:
D9030136 removed the `use` statement and broke OSX build:

```
   Compiling scm_daemon v0.1.0 (file:///.../exec/scm_daemon)
error[E0599]: no method named `write_fmt` found for type `std::fs::File` in the current scope
  --> src/main.rs:62:33
   |
62 |             File::create(path)?.write_fmt(format_args!("{}", std::process::id()))?;
   |                                 ^^^^^^^^^
   |
   = help: items from traits can only be used if the trait is in scope
help: the following trait is implemented but not in scope, perhaps add a `use` for it:
   |
14 | use std::io::Write;
   |
```

Reviewed By: DurhamG, singhsrb

Differential Revision: D9066112

fbshipit-source-id: f685b75c4032e8388529f6cf8a9724426bd15baa
2018-07-30 11:36:51 -07:00
Hugh Harris
1ac21f90b5 Remove unused import from scm_daemon
Summary: `std::io::Write` was imported but not used causing a warning

Differential Revision: D9030136

fbshipit-source-id: 64e8e2f03cd73ecf77113f6edf7d62c2372833ea
2018-07-27 11:34:32 -07:00
Liubov Dmitrieva
e22322c2e5 commit cloud subscriber: skip cloud sync if can't resolve interngraph host
Summary:
this will reduce cloud sync errors and unnecessary cloud sync calls

the daemon triggers cloud sync on service start/restart
it is not always the time when the machine online (and connected to correct network), so we get cloud sync errors

Reviewed By: markbt

Differential Revision: D8692972

fbshipit-source-id: 59033fd4c3e7c30100d82b908442bbf1ebea9322
2018-06-29 12:20:11 -07:00
Liubov Dmitrieva
f14c984f62 scm_daemon fix broken build
Summary:
std::process::id is stable feature but not for old compilers

https://doc.rust-lang.org/std/process/fn.id.html

hopefully we will revert this diff soon

Reviewed By: markbt

Differential Revision: D8332282

fbshipit-source-id: cd6ed573a7bd1294ac666f267b09a2121c386a34
2018-06-08 07:52:01 -07:00
Liubov Dmitrieva
2d82868f9f scm daemon, write pid file (to be used for restart after log rotation)
Summary: write pid file

Reviewed By: markbt

Differential Revision: D8331445

fbshipit-source-id: 47492ffc9cc87a2ef228ce9581873dec6e01e369
2018-06-08 06:31:10 -07:00
Liubov Dmitrieva
ae7ece9cd5 scm daemon: refactoring
Summary:
just refactoring to improve the code quality

the main improvement is that I separated TcpReceiver to a different service,
any other services can register callbacks with TcpReceiver service.

For WorkspaceSubscriberService callbacks are implemented using mpsc channel to notify the main WorkspaceSubscriberService thread and single atomic flag that allows running subscriptions to join.

Another improvement is that I added logic to run cloud sync on the first keep alive after connection errors

Reviewed By: markbt

Differential Revision: D8226109

fbshipit-source-id: 3fe513da9273b28b2262948ecdf620821e7ab313
2018-06-05 07:11:55 -07:00
Liubov Dmitrieva
12796cbac0 code fixes to support Rust < 1.26
Summary: This changes to support Scm Daemon on dev machines

Reviewed By: farnz

Differential Revision: D8139892

fbshipit-source-id: b6df53d6ce6615d24822b739d4d1705e0f572660
2018-05-24 12:19:55 -07:00
Liubov Dmitrieva
39ccc28933 Scm Daemon initial implementation
Summary: Scm Daemon initial implementation that currently just listen to Commit Cloud Live Notifications and trigger `hg cloud sync` on notifications

Reviewed By: markbt

Differential Revision: D8119768

fbshipit-source-id: a0d86624fe4b81b3adc89990640916d3da279b8c
2018-05-24 12:19:55 -07:00