Summary:
This diff changes our versioning scheme from `MAJOR.MINOR-%Y%m%d-%H%M%S-rHASH` to `MAJOR.MINOR-%Y%m%d-%H%M%S-hHASH`
At the moment we cannot send another PR to Homebrew-core since our lastest release (`0.1.20221212-142634-r7ae28228`) gets detected [as an Erlang version](9c88c39bae/Library/Homebrew/version.rb (L411)) instead of a [hyphenated version](9c88c39bae/Library/Homebrew/version.rb (L427)). This was not an issue in the past since the hashes of our previous releases didn't happen to be of the form (`[Rr]\d+[AaBb]\d*(?:-\d+)?)`).
Reviewed By: bolinfest
Differential Revision: D42006425
fbshipit-source-id: 8dd4c52e1f49b79763bcc5863f7578a0f36dda73
Summary:
The latest Homebrew bottles for Apple Sillicon macOS built by our Github Actions were broken, as mentioned in https://github.com/facebook/sapling/issues/315 . This was caused due to updating the Formula template used by our Github actions to 3.11 but not updating the Github actions themselves to Python 3.11. This commit fixes that last part.
Pull Request resolved: https://github.com/facebook/sapling/pull/319
Test Plan:
Triggered a [build on a fork of the sapling repo](https://github.com/sggutier/sapling/releases/tag/0.1.20221211-120017-rcd410769), downloaded the bottle built by it, and checked that it ran properly on my M1 mac:
```
$ sl --version
Sapling 0.1.20221211-120017-rcd410769
$ file $(which sl)
/Users/sggutier/homebrew/bin/sl: Mach-O 64-bit executable arm64
$ otool -L $(which sl)
/Users/sggutier/homebrew/bin/sl:
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1311.100.3)
/usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0)
/System/Library/Frameworks/Security.framework/Versions/A/Security (compatibility version 1.0.0, current version 60158.100.133)
/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1858.112.0)
/Users/sggutier/homebrew/opt/openssl@1.1/lib/libssl.1.1.dylib (compatibility version 1.1.0, current version 1.1.0)
/Users/sggutier/homebrew/opt/openssl@1.1/lib/libcrypto.1.1.dylib (compatibility version 1.1.0, current version 1.1.0)
/System/Library/Frameworks/SystemConfiguration.framework/Versions/A/SystemConfiguration (compatibility version 1.0.0, current version 1163.100.19)
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
/System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices (compatibility version 1.0.0, current version 1141.1.0)
/Users/sggutier/homebrew/opt/python@3.11/Frameworks/Python.framework/Versions/3.11/Python (compatibility version 3.11.0, current version 3.11.0)
/usr/lib/libresolv.9.dylib (compatibility version 1.0.0, current version 1.0.0)
$ sl dbsh -c "import sys; print(sys.version)"
3.11.0 (main, Nov 28 2022, 13:49:33) [Clang 14.0.0 (clang-1400.0.29.202)]
$ sl clone https://github.com/sggutier/sapling/ saplingtest && cd saplingtest && sl
remote: Enumerating objects: 677771, done.
remote: Counting objects: 100% (2847/2847), done.
remote: Compressing objects: 100% (1469/1469), done.
remote: Total 677771 (delta 1396), reused 2692 (delta 1261), pack-reused 674924
Receiving objects: 100% (677771/677771), 175.60 MiB | 2.59 MiB/s, done.
Resolving deltas: 100% (454743/454743), done.
From https://github.com/sggutier/sapling
* [new ref] 2857ac6b96 -> remote/main
6535 files updated, 0 files merged, 0 files removed, 0 files unresolved
@ 2857ac6b9 Today at 11:33 mbolin https://github.com/facebook/sapling/issues/317 remote/main
│ Add build instructions for Windows (https://github.com/facebook/sapling/issues/317)
~
$ touch something && sl st
warning: watchman has recently started (pid 1093) - operation will be slower than usual
? something
$ cd eden/scm && sl root
/Users/sggutier/saplingtest
```
Reviewed By: bolinfest
Differential Revision: D41921132
Pulled By: sggutier
fbshipit-source-id: 0ed4f2d6f214f02669e45c9c4b8cced7de9caa2e
Summary:
This changes our versioning schema from `MAJOR.MINOR-%Y%m%d-%H%M%S-HASH` to `MAJOR.MINOR-%Y%m%d-%H%M%S-rHASH`
Previously we had changed our versioning scheme for better compatibility with semantic versioning and for trying to get merged into Homebrew-core. Homebrew is capable of detecting the version number based on the URL of some formula. However, it seems that at the moment Homebrew doesn't support semantic versioning, although [it supports a number of different versioning schemas](9c88c39bae/Library/Homebrew/version.rb).
With these changes (and with our current Formula we have), this will make `sl --version` appear as ``MAJOR.MINOR-%Y%m%d-%H%M%S` on systems using Homebrew, but it will show as ``MAJOR.MINOR-%Y%m%d-%H%M%S-rHASH` everywhere else.
Reviewed By: bolinfest
Differential Revision: D41655159
fbshipit-source-id: f0da45ee45e57e83e2cf018a73559ece52d09fe2
Summary:
Stack created with [Sapling]
* __->__ https://github.com/facebook/sapling/issues/208
[sl] ci: change versioning scheme
This changes our versioning scheme to `VERSION-%Y%m%d-%H%M%S-HASH`, where `VERSION` is defined in the VERSION file, `%Y%m%d-%H%M%S` is the current date, hour, minutes, and seconds, and `HASH` is the hash of the current commit.
Pull Request resolved: https://github.com/facebook/sapling/pull/208
Test Plan: Tested on my personal account
Reviewed By: bolinfest
Differential Revision: D41418387
Pulled By: sggutier
fbshipit-source-id: f89bce7cc842c6ca77c3a8330565fa19d41e39cc
Summary:
X-link: https://github.com/facebook/sapling-staging/pull/72
Modifies the homebrew formula and separates the current macOS github action into an x86 one and an arm64 one.
Reviewed By: bolinfest
Differential Revision: D41201780
fbshipit-source-id: 956152026115c886c5a32678db7ca23ee3ae91e1
Summary:
Creates a macOS Homebrew bottle that can be used in x86 systems.
This also updates the Homebrew formula template so that it statically links openssl and it is easier to edit by the homebrew action.
X-link: https://github.com/facebook/sapling-staging/pull/67
Reviewed By: bolinfest
Differential Revision: D41106453
Pulled By: sggutier
fbshipit-source-id: 2a16ef4a22639bb0b15d21eb0c88c7ca90e45bf7
Summary:
Before landing D41062213 (2736052d39), I got a lint warning that I
had an unnecessary use of an f-string because there
weren't any expressions evaluated as part of the f-string.
I removed the `f` and just hit Land without re-running
`gen_workflows.py`.
In this case, removing the `f` changed the behavior of
the code because the string originally included double `{{`
to escape it, which is necessary as an f-string, but is
interpreted literally in an ordinary string literal.
This diff removes the double `{{` so that `gen_workflows.py`
produces the desired output again.
Reviewed By: sggutier
Differential Revision: D41105429
fbshipit-source-id: f4bc023cd42a7f265325fd9262e7c602e900bb99
Summary: To be used to propagate through the Sapling version to the build script.
Reviewed By: bolinfest
Differential Revision: D40943082
fbshipit-source-id: aacb88c7a0c0113f240a020f994b5eebd56faef9
Summary:
I took this idea from Glean's comparable workflow:
5e2e569572/.github/workflows/glean-docker.yml
`workflow_dispatch` means we can always trigger a one-off build,
but the schedule ensures the Docker image is rebuilt every
Monday morning. This helps ensure things like the Yarn offline-mirror
don't drift too far from the set of dependencies we are actually using.
Reviewed By: evangrayk
Differential Revision: D40659812
fbshipit-source-id: 39fb85bc34b20d8c8558d2b1be9f7eb8d16fc9aa
Summary:
Recall that `sapling-cli-ubuntu-20.04.Dockerfile` and
`sapling-cli-ubuntu-22.04.Dockerfile` are used as part of the
respective `sapling-cli-ubuntu-xx.yy-image.yml` GitHub actions
that are used to create Docker images for the other actions so
that tools and dependencies do not have to be installed from
scratch each time.
Note that this existing line in the `Dockerfile`:
```
RUN yarn config set yarn-offline-mirror "$HOME/npm-packages-offline-cache"
```
was meant to set up an offline cache so `yarn install` in subsequent
builds would be faster. Unfortunately, it appears we were not
actually populating the offline cache because we were not
specifying the `--prefer-offline` flag to `yarn install`, which this diff fixes.
Previously, we were also defensive in the call to `yarn install`:
```
RUN [ -f /tmp/repo/addons ] && yarn --cwd /tmp/repo/addons install || true
```
But now the build will fail if `install` does not succeed, as the previous
approach made it too easy for problems to go unnoticed.
I updated the `.yml` files by running:
```
~/fbsource/fbcode/eden/oss/ci$ buck2 run :gen_workflows -- ../.github/workflows/
```
Reviewed By: evangrayk
Differential Revision: D40658823
fbshipit-source-id: 2813641b746654ac06e15ac151f1c017d8e39622
Summary:
Updated `verify_release.sh` so that:
- It runs `hg isl` and verifies that it runs successfully. Hopefully this will help
us catch things like introducing changes that do not work on Node v10,
which would break `hg isl` on Ubuntu 20.04 LTS.
- It drops the `--git` flag that is no longer necessary.
Reviewed By: quark-zju
Differential Revision: D39527960
fbshipit-source-id: e97a44103962c37107d2444d4ce8062438e6c9f5
Summary:
Add `Recommends: nodejs` to the `control` file for our `.deb` as Node is
not necessary for the core functionality of Sapling, but it is necessary for ISL.
Note that recommended packages are installed by default, so things should
still "just work" for ordinary users, though Ubuntu/Debian users have been
known to do:
```
sudo apt install --no-install-recommends nodejs
```
Though I assume they want to manage their own Node installation via `nvm`
or somesuch in those situations.
Note that now that we are installing the `nodejs` package, we need to update
`verify_release.sh` to set `DEBIAN_FRONTEND=noninteractive` so we don't
get stuck with timezone prompts when setting up the Docker image.
Reviewed By: jordanwebster
Differential Revision: D39493888
fbshipit-source-id: a8b62674273cb97dfeaf695fbbe5e1ff7e6a5501
Summary:
For now, I use this to verify releases manually, but once we fix the
current set of issues with `sl clone`, we can then extend this script
to invoke more `sl` commands to sanity-check a release build.
Ultimately, it should become part of our CI in GitHub Actions.
Reviewed By: quark-zju
Differential Revision: D39185995
fbshipit-source-id: 6f2eeb454610bfdf465cca159cf276a45362e6f0
Summary:
Simplifies the build logic so the only thing `make deb` does is
run the `./packaging/debian/build_deb.sh` script. Note that the
role of `build_deb.sh` has expanded so that it:
- reads the Ubuntu version from `/etc/os-release`
- sets `PY_VERSION` and `GIT_DEB_DEP` accordingly
- runs `DESTDIR=install make install-oss`
This means that someone on either Ubuntu 20.04 or Ubuntu 22.04
can just run `VERSION=0.0.0 make deb` and it should do the right thing.
Reviewed By: quark-zju
Differential Revision: D39182615
fbshipit-source-id: 073d569a1bcecfb9fbeb78e8042ac18b0c1bb879
Summary:
Currently, the open source build contains this line in a `Cargo.toml` file:
c403b32adb/eden/scm/Cargo.toml (L7)
For posterity, the contents of the line are:
```
graphql-parser = { git = "https://github.com/vmagro/graphql-parser", rev = "1d155d96e6052767380ab5e67c57e3d6608a31ac" }
```
The `cargo fetch` calls I am removing in this diff *used to* work. But today when
I ran the workflow that used this `Dockerfile`, they failed with:
```
#13 4.589 error: failed to resolve patches for `https://github.com/rust-lang/crates.io-index`
#13 4.589
#13 4.589 Caused by:
#13 4.589 failed to load source for dependency `graphql-parser`
#13 4.589
#13 4.589 Caused by:
#13 4.589 Unable to update https://github.com/vmagro/graphql-parser?rev=1d155d96e6052767380ab5e67c57e3d6608a31ac#13 4.589
#13 4.589 Caused by:
#13 4.589 revspec '1d155d96e6052767380ab5e67c57e3d6608a31ac' not found; class=Reference (4); code=NotFound (-3)
#13 ERROR: process "/bin/sh -c cargo fetch --manifest-path eden/scm/exec/hgmain/Cargo.toml" did not complete successfully: exit code: 101
------
> [ 8/17] RUN cargo fetch --manifest-path eden/scm/exec/hgmain/Cargo.toml:
#13 4.589 error: failed to resolve patches for `https://github.com/rust-lang/crates.io-index`
#13 4.589
#13 4.589 Caused by:
#13 4.589 failed to load source for dependency `graphql-parser`
#13 4.589
#13 4.589 Caused by:
#13 4.589 Unable to update https://github.com/vmagro/graphql-parser?rev=1d155d96e6052767380ab5e67c57e3d6608a31ac#13 4.589
#13 4.589 Caused by:
#13 4.589 revspec '1d155d96e6052767380ab5e67c57e3d6608a31ac' not found; class=Reference (4); code=NotFound (-3)
```
What happened? Here's what it looks like:
- After discovering that our `Cargo.toml` files created by `autocargo` contained massive
`[patch.crates-io]` sections that appeared to be slowing down our open source build,
I went and posted about it in the Source Control Team group:
https://fb.workplace.com/groups/sourcecontrolteam/posts/5310340079087295
- One way to improve the situation is to eliminate these patches from `//third-party/rust`,
so I spot-checked a number of the patches, tracking down the diff that introduced them,
and attempted to follow-up with the author to upstream the patch.
- D37532244 (199330fc39) appeared to be the diff that pulled in the patch for `graphql-parser`.
In the comments, I pinged the author (vmagro) to see if he could upstream his patch.
- Vinnie agreed and updated his PR: https://github.com/graphql-rust/graphql-parser/pull/66
- *what appears to have happened* is that Vinnie updated the PR by doing a "force push"
where the previous head was `1d155d96e6052767380ab5e67c57e3d6608a31ac` and
the new head is `c778917f57f6b2c26d9291819b9bb341a2f5948a`.
- GitHub...does not like force pushes. Indeed, if you visit
1d155d96e6
you see a yellow warning banner at the top that says
**This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.**
I don't know what call `cargo` makes to resolve these `git` dependencies, but I assume
GitHub is refusing to respond as it normally would for orphaned commits such as these.
D39190591 (f500204c06) is the for fbsource overall, but based on this experience, I still think it is best
to remove these `cargo fetch` calls. Of note, **even though the `cargo fetch` calls failed,
the GitHub action to build Sapling still succeeds** because Sapling does not depend
on `graphql-parser`, so `cargo build` only builds what it needs whereas `cargo fetch`
fetches everything in the `Cargo.toml` whether it needs it or not.
In sum, given the current state of `autocargo` and the `[patch.crates-io]` section it generates,
`cargo fetch` just seems like a giant liability, so we are better off without it.
Reviewed By: fanzeyi
Differential Revision: D39189595
fbshipit-source-id: cbdf3c51e39b7ed2e49a4373e7f9fc66337766cc
Summary:
Although D39042765 (a04fc2e9b3) appeared to successfully produce `.deb` files, it turns
out that they would only work if you already had the requisite package
dependencies installed. For example, running `hg --version` on a clean
Ubuntu instance would yield:
```
hg: error while loading shared libraries: libpython3.8.so.1.0: cannot open shared object file: No such file or directory
```
It turned out that our `DEBIAN/control` file was missing a key `Depends:` line
to tell `dpkg` what dependencies to install. Once again, I decided to look and
see how wezterm deals with this, and the solution appears to be `dpkg-shlibdeps`:
https://github.com/wez/wezterm/blob/97eaa58112a4/ci/deploy.sh#L234,L240
`dpkg-shlibdeps` looks at your executables and, based on the symbols, determines
what the appropriate packages are. For our Ubuntu 22.04 `.deb`, this turns out to be:
```
Depends: libc6 (>= 2.34), libgcc-s1 (>= 4.2), libpython3.10 (>= 3.10.0), libssl3 (>= 3.0.0~~alpha1), zlib1g (>= 1:1.1.4)
```
Apparently you cannot have a package in the `Depends:` line that comes from a PPA,
so if you want to make your `.deb` easy for users to install, that means you should
limit your dependencies to "standard" packages for the distro. In our case, that meant
that our Ubuntu 22.04 `.deb` should use Python 3.10 instead of Python 3.8 (which we
previously fetched from `ppa:deadsnakes/ppa`).
In order to support this, this diff makes a number of changes:
- Adds logic to `setup.py` to check the `PY_VERSION` environment variable to decide
whether to use Python 3.10 or something else.
- Updates `gen_workflows.py` to define different Python deps based on the Ubuntu version.
- Updates `gen_workflows.py` to remove the logic that uses the `ppa:deadsnakes/ppa`.
- Ran `buck2 run //eden/oss/ci:gen_workflows -- eden/oss/.github/workflows/` to update the GitHub Actions
- Moved the logic for the `deb` target in the `Makefile` into a separate shell script because
it was getting too complex to express directly in Make.
- Introduced separate `deb-ubuntu-20.04` and `deb-ubuntu-22.04` targets in the `Makefile`.
- Updated `bytearrayobject.rs` to support Python 3.10 in addition to Python 3.9.
- Updated `pick_python.py` to consider `python3.10` if `python3.8` is not found before going
down the rest of the list.
As you can see in the Test Plan, we are *really close* to things "just working," but we also have
to register `git` as a dependency, which is not discovered by `dpkg-shlibdeps`. This will be
addressed in D39156794.
Reviewed By: DurhamG
Differential Revision: D39156794
fbshipit-source-id: ca1e0a73096e0de97230804a97f316114b8bfc2e