Summary:
Add a devcontainer config
My laptop is full of weird versions of weird tools. Sapling requires specific versions of specific tools. It's nice to be able to open the sapling folder in vscode and get a prompt for "would you like to do your development inside a container with all the correct tools pre-installed?" :) This also allows people on github to click a button to open in github's web-based container-IDE and start developing without installing anything at all, though I don't know if we'd expect anybody to do that
For starters I'm using the CI build image without changes to try and keep the maintenance burden as small as possible, though it would be possible to use a separate layer within the Dockerfile (eg if we wanted to install extra tools that are useful for dev but not needed for building), or have a totally separate Dockerfile
---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/facebook/sapling/pull/807).
* __->__ https://github.com/facebook/sapling/issues/807
* https://github.com/facebook/sapling/issues/806
* https://github.com/facebook/sapling/issues/804
Pull Request resolved: https://github.com/facebook/sapling/pull/807
Differential Revision: D52575911
Pulled By: quark-zju
fbshipit-source-id: b24e1a21493f18a06355c984a72936cd11fef2ea
Summary:
[CI] shrink builder-images by 60% / several GB
If we copy the repo into docker, save the state, then delete the repo, then the repo is still there in the history
These changes use docker staged builds - having a "base" stage which installs the tools needed, a "populate-cache" stage which uses ~3GB of temporary disk space to populate the 162MB cache, and then a "main" stage which copies the 162MB cache but leaves the rest of the temporary files behind
Before - around 5GB:
```
$ docker build -t sapling-build . -f .github/workflows/sapling-cli-ubuntu-22.04.Dockerfile
$ docker history sapling-build
IMAGE CREATED CREATED BY SIZE COMMENT
33b251176a7d 26 seconds ago RUN |1 DEBIAN_FRONTEND=noninteractive /bin/s… 0B buildkit.dockerfile.v0
<missing> 28 seconds ago WORKDIR /root 0B buildkit.dockerfile.v0
<missing> 28 seconds ago RUN |1 DEBIAN_FRONTEND=noninteractive /bin/s… 0B buildkit.dockerfile.v0
<missing> 28 seconds ago RUN |1 DEBIAN_FRONTEND=noninteractive /bin/s… 1.56GB buildkit.dockerfile.v0
<missing> About a minute ago RUN |1 DEBIAN_FRONTEND=noninteractive /bin/s… 2.19MB buildkit.dockerfile.v0
<missing> About a minute ago RUN |1 DEBIAN_FRONTEND=noninteractive /bin/s… 8.97MB buildkit.dockerfile.v0
<missing> About a minute ago WORKDIR /tmp/repo 0B buildkit.dockerfile.v0
<missing> About a minute ago COPY . /tmp/repo # buildkit 1.17GB buildkit.dockerfile.v0
<missing> About a minute ago ENV PATH=/root/.cargo/bin:/usr/local/sbin:/u… 0B buildkit.dockerfile.v0
<missing> About a minute ago RUN |1 DEBIAN_FRONTEND=noninteractive /bin/s… 1.49GB buildkit.dockerfile.v0
<missing> 2 minutes ago RUN |1 DEBIAN_FRONTEND=noninteractive /bin/s… 421MB buildkit.dockerfile.v0
<missing> 2 minutes ago RUN |1 DEBIAN_FRONTEND=noninteractive /bin/s… 632kB buildkit.dockerfile.v0
<missing> 2 minutes ago RUN |1 DEBIAN_FRONTEND=noninteractive /bin/s… 100B buildkit.dockerfile.v0
<missing> 2 minutes ago RUN |1 DEBIAN_FRONTEND=noninteractive /bin/s… 1.19kB buildkit.dockerfile.v0
<missing> 2 minutes ago RUN |1 DEBIAN_FRONTEND=noninteractive /bin/s… 0B buildkit.dockerfile.v0
<missing> 2 minutes ago RUN |1 DEBIAN_FRONTEND=noninteractive /bin/s… 81.2MB buildkit.dockerfile.v0
<missing> 2 minutes ago RUN |1 DEBIAN_FRONTEND=noninteractive /bin/s… 43.1MB buildkit.dockerfile.v0
<missing> 2 minutes ago ENV TZ=Etc/UTC 0B buildkit.dockerfile.v0
<missing> 2 minutes ago ARG DEBIAN_FRONTEND=noninteractive 0B buildkit.dockerfile.v0
<missing> 3 weeks ago /bin/sh -c #(nop) CMD ["/bin/bash"] 0B
<missing> 3 weeks ago /bin/sh -c #(nop) ADD file:50f947da69b3b6c63… 69.3MB
<missing> 3 weeks ago /bin/sh -c #(nop) LABEL org.opencontainers.… 0B
<missing> 3 weeks ago /bin/sh -c #(nop) LABEL org.opencontainers.… 0B
<missing> 3 weeks ago /bin/sh -c #(nop) ARG LAUNCHPAD_BUILD_ARCH 0B
<missing> 3 weeks ago /bin/sh -c #(nop) ARG RELEASE 0B
```
After - around 2.2GB:
```
$ docker build -t sapling-build . -f .github/workflows/sapling-cli-ubuntu-22.04.Dockerfile
$ docker history sapling-build
IMAGE CREATED CREATED BY SIZE COMMENT
1496dd42165f About a minute ago COPY /root/npm-packages-offline-cache /root/… 162MB buildkit.dockerfile.v0
<missing> 2 minutes ago RUN |1 DEBIAN_FRONTEND=noninteractive /bin/s… 2.19MB buildkit.dockerfile.v0
<missing> 2 minutes ago RUN |1 DEBIAN_FRONTEND=noninteractive /bin/s… 8.97MB buildkit.dockerfile.v0
<missing> 11 minutes ago ENV PATH=/root/.cargo/bin:/usr/local/sbin:/u… 0B buildkit.dockerfile.v0
<missing> 11 minutes ago RUN |1 DEBIAN_FRONTEND=noninteractive /bin/s… 1.49GB buildkit.dockerfile.v0
<missing> 12 minutes ago RUN |1 DEBIAN_FRONTEND=noninteractive /bin/s… 421MB buildkit.dockerfile.v0
<missing> 12 minutes ago RUN |1 DEBIAN_FRONTEND=noninteractive /bin/s… 632kB buildkit.dockerfile.v0
<missing> 12 minutes ago RUN |1 DEBIAN_FRONTEND=noninteractive /bin/s… 100B buildkit.dockerfile.v0
<missing> 12 minutes ago RUN |1 DEBIAN_FRONTEND=noninteractive /bin/s… 1.19kB buildkit.dockerfile.v0
<missing> 12 minutes ago RUN |1 DEBIAN_FRONTEND=noninteractive /bin/s… 0B buildkit.dockerfile.v0
<missing> 12 minutes ago RUN |1 DEBIAN_FRONTEND=noninteractive /bin/s… 81.2MB buildkit.dockerfile.v0
<missing> 12 minutes ago RUN |1 DEBIAN_FRONTEND=noninteractive /bin/s… 43.1MB buildkit.dockerfile.v0
<missing> 12 minutes ago ENV TZ=Etc/UTC 0B buildkit.dockerfile.v0
<missing> 12 minutes ago ARG DEBIAN_FRONTEND=noninteractive 0B buildkit.dockerfile.v0
<missing> 3 weeks ago /bin/sh -c #(nop) CMD ["/bin/bash"] 0B
<missing> 3 weeks ago /bin/sh -c #(nop) ADD file:50f947da69b3b6c63… 69.3MB
<missing> 3 weeks ago /bin/sh -c #(nop) LABEL org.opencontainers.… 0B
<missing> 3 weeks ago /bin/sh -c #(nop) LABEL org.opencontainers.… 0B
<missing> 3 weeks ago /bin/sh -c #(nop) ARG LAUNCHPAD_BUILD_ARCH 0B
<missing> 3 weeks ago /bin/sh -c #(nop) ARG RELEASE 0B
```
---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/facebook/sapling/pull/804).
* https://github.com/facebook/sapling/issues/807
* https://github.com/facebook/sapling/issues/806
* __->__ https://github.com/facebook/sapling/issues/804
Pull Request resolved: https://github.com/facebook/sapling/pull/804
Reviewed By: sggutier
Differential Revision: D52484298
fbshipit-source-id: c4f446007722950bd127dff79a2308edc4e718e1
Summary: have more context about the error for debugging
Reviewed By: quark-zju, sggutier
Differential Revision: D51282332
fbshipit-source-id: 7d6666ff16b3181f5a9c70303eca1fc6f6ddb631
Summary:
This diff fixed the openssl compile errror by setting the env variables
```
error: failed to run custom build command for `openssl-sys v0.9.95`
Caused by:
process didn't exit successfully: `D:\a\sapling\sapling\eden\scm\build\cargo-target\release\build\openssl-sys-dbaf6a9d88c8a4e6\build-script-main` (exit code: 101)
--- stdout
cargo:rerun-if-env-changed=X86_64_PC_WINDOWS_MSVC_OPENSSL_LIB_DIR
X86_64_PC_WINDOWS_MSVC_OPENSSL_LIB_DIR unset
cargo:rerun-if-env-changed=OPENSSL_LIB_DIR
OPENSSL_LIB_DIR unset
cargo:rerun-if-env-changed=X86_64_PC_WINDOWS_MSVC_OPENSSL_INCLUDE_DIR
X86_64_PC_WINDOWS_MSVC_OPENSSL_INCLUDE_DIR unset
cargo:rerun-if-env-changed=OPENSSL_INCLUDE_DIR
OPENSSL_INCLUDE_DIR unset
cargo:rerun-if-env-changed=X86_64_PC_WINDOWS_MSVC_OPENSSL_DIR
X86_64_PC_WINDOWS_MSVC_OPENSSL_DIR unset
cargo:rerun-if-env-changed=OPENSSL_DIR
OPENSSL_DIR unset
note: vcpkg did not find openssl: Could not look up details of packages in vcpkg tree could not read status file updates dir: The system cannot find the path specified. (os error 3)
```
Reviewed By: muirdm
Differential Revision: D51115009
fbshipit-source-id: d021991de1c80099a5f2ae12fe486ec49b7fb318
The internal and external repositories are out of sync. This Pull Request attempts to brings them back in sync by patching the GitHub repository. Please carefully review this patch. You must disable ShipIt for your project in order to merge this pull request. DO NOT IMPORT this pull request. Instead, merge it directly on GitHub using the MERGE BUTTON. Re-enable ShipIt after merging.
Summary:
[sl] fix compatibility with PEP 440
Our current versioning schema isn't compatible with PEP 440. Luckily, changing the last `-` to a `+` works.
Credit for the idea goes to ZhongRuoyu as mentioned in https://github.com/facebook/sapling/issues/598
Pull Request resolved: https://github.com/facebook/sapling/pull/606
Test Plan:
- Ran `sl --version` to verify that it outputted a sane format (got `Sapling 0.2.20230417-165855+6db1e950`)
- Verified that this was compliant with PEP 440: https://regex101.com/r/sv6aUZ/1
- Verified that this was compliant with Semver 2.0: https://regex101.com/r/RQsynA/1
- Also, verified that this was still compatible with Homebrew in a similar way to D42006425:
```
$ git --no-pager diff
diff --git a/Library/Homebrew/test/version_spec.rb b/Library/Homebrew/test/version_spec.rb
index 647fb5dcc..ac35b920a 100644
--- a/Library/Homebrew/test/version_spec.rb
+++ b/Library/Homebrew/test/version_spec.rb
@@ -841,6 +841,12 @@ describe Version do
.to be_detected_from("https://opam.ocaml.org/archives/easy-format.1.0.2+opam.tar.gz")
end
+ specify "sapling-better" do
+ expect(described_class.create("0.2.20230417-165855"))
+ .to be_detected_from("https://github.com/facebook/sapling/archive/refs/tags/0.2.20230417-165855+6db1e950.tar.gz")
+ end
+
+
specify "no extension version" do
expect(described_class.create("1.8.12"))
.to be_detected_from("https://waf.io/waf-1.8.12")
%
```
```
$ ./bin/brew tests --only version -d -v
Randomized with seed 48812
bundle exec parallel_rspec --nice -- -I /Users/sggutier/homebrew/Library/Homebrew/test --seed 48812 --color --require spec_helper --tag ~needs_linux --tag ~needs_network --tag ~needs_ci --tag ~needs_svn -- test/version_spec.rb test/version/parser_spec.rb
2 processes for 2 specs, ~ 1 spec per process
...................
.....................................................................................................................
Took 2 seconds
```
---
Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/facebook/sapling/pull/606).
* __->__ https://github.com/facebook/sapling/issues/606
* https://github.com/facebook/sapling/issues/605
Reviewed By: zzl0
Differential Revision: D45072716
Pulled By: sggutier
fbshipit-source-id: 6fea731486e8b7ea4606a96b064dc3f724420649
Summary:
[sl] bump macOS python CI to 3.11.3
We still depend on pointing to the specific Python combination that GitHub CI and Homebrew have, which currently is 3.11.3.
Hashes were taken from d39875e212/Formula/python@3.11.rb
Pull Request resolved: https://github.com/facebook/sapling/pull/605
Test Plan:
Downloaded bottles from:
https://github.com/sggutier/sapling/actions/runs/4727040284https://github.com/sggutier/sapling/actions/runs/4727040323
and installed the arm64 version. Also, verified that the dumps from `otool -L` looked correct (they usually don't when there is some when building)
Dump from arm64 version:
```
$ otool -L sapling\ 8/0.2.20230417-165855+6db1e950/bin/sl && file sapling\ 8/0.2.20230417-165855+6db1e950/bin/sl
sapling 8/0.2.20230417-165855+6db1e950/bin/sl:
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1311.100.3)
/usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0)
/System/Library/Frameworks/Security.framework/Versions/A/Security (compatibility version 1.0.0, current version 60158.100.133)
/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1858.112.0)
@HOMEBREW_PREFIX@@/opt/openssl@1.1/lib/libssl.1.1.dylib (compatibility version 1.1.0, current version 1.1.0)
@HOMEBREW_PREFIX@@/opt/openssl@1.1/lib/libcrypto.1.1.dylib (compatibility version 1.1.0, current version 1.1.0)
/System/Library/Frameworks/SystemConfiguration.framework/Versions/A/SystemConfiguration (compatibility version 1.0.0, current version 1163.100.19)
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
@HOMEBREW_PREFIX@@/opt/python@3.11/Frameworks/Python.framework/Versions/3.11/Python (compatibility version 3.11.0, current version 3.11.0)
sapling 8/0.2.20230417-165855+6db1e950/bin/sl: Mach-O 64-bit executable arm64
```
Dump from x86 version:
```
$ otool -L sapling\ 9/0.2.20230417-165855+6db1e950/bin/sl && file sapling\ 9/0.2.20230417-165855+6db1e950/bin/sl
sapling 9/0.2.20230417-165855+6db1e950/bin/sl:
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1311.100.3)
/usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0)
/System/Library/Frameworks/Security.framework/Versions/A/Security (compatibility version 1.0.0, current version 60158.100.133)
/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1858.112.0)
@HOMEBREW_PREFIX@@/opt/openssl@1.1/lib/libssl.1.1.dylib (compatibility version 1.1.0, current version 1.1.0)
@HOMEBREW_PREFIX@@/opt/openssl@1.1/lib/libcrypto.1.1.dylib (compatibility version 1.1.0, current version 1.1.0)
/System/Library/Frameworks/SystemConfiguration.framework/Versions/A/SystemConfiguration (compatibility version 1.0.0, current version 1163.100.19)
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
@HOMEBREW_PREFIX@@/opt/python@3.11/Frameworks/Python.framework/Versions/3.11/Python (compatibility version 3.11.0, current version 3.11.0)
sapling 9/0.2.20230417-165855+6db1e950/bin/sl: Mach-O 64-bit executable x86_64
```
---
Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/facebook/sapling/pull/605).
* https://github.com/facebook/sapling/issues/606
* __->__ https://github.com/facebook/sapling/issues/605
Reviewed By: zzl0
Differential Revision: D45072742
Pulled By: sggutier
fbshipit-source-id: 4ced2d609d9cf4098cd4bbea5d5df45659588ad8
Summary:
After upgrading openssl along with Python in D43640261 (b671ca71f4), I broke the build since it doesn't match the openssl version that GitHub actions has.
This commit downgrades the version for fixing the build
Reviewed By: zzl0
Differential Revision: D43675103
fbshipit-source-id: 11754aab77a086770b3c6a20fd5b09203a4515cb
Summary:
[sl] ci: fix Windows OS CI due to path length
Currently our OSS CI on Windows is failing due to some paths being too long (e.g., `C:/Users/runneradmin/.cargo/git/checkouts/fbthrift-abf000ee5c7fcc50/c757ec4/thrift/website/src/json/ref/cpp/f/struct/special/structapache_1_1thrift_1_1op_1_1detail_1_1AnyOp_3_01type_1_1cpp__type_3_01T_00_01type_1_1map_3_01KTag_00_01VTag_01_4_01_4_01_4/assign.json`)
Since the length of that path is just above the Windows max length (263 vs. 260). This diff tries to alleviate that by setting the `core.longpaths` option for the git system config.
Pull Request resolved: https://github.com/facebook/sapling/pull/500
Test Plan:
Ran GitHub actions on a personal fork of the repo.
---
Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/facebook/sapling/pull/500).
* __->__ https://github.com/facebook/sapling/issues/500
Reviewed By: bolinfest
Differential Revision: D42727420
Pulled By: sggutier
fbshipit-source-id: 3cf908f8e9537f20d796dcaa97292512f1172919
Summary: One of our previous commits (D42611397 (5bb56fc6a2)) introduced the ability to pick between `sl` and `git` for the tag name, but currently this is breaking or CI on macOS. This diff fixes that issue.
Reviewed By: zzl0
Differential Revision: D42719150
fbshipit-source-id: 7e8f74322dc7bfe8d022e4f65b40d130221399bb
Summary:
This diff changes our versioning scheme from `MAJOR.MINOR-%Y%m%d-%H%M%S-rHASH` to `MAJOR.MINOR-%Y%m%d-%H%M%S-hHASH`
At the moment we cannot send another PR to Homebrew-core since our lastest release (`0.1.20221212-142634-r7ae28228`) gets detected [as an Erlang version](9c88c39bae/Library/Homebrew/version.rb (L411)) instead of a [hyphenated version](9c88c39bae/Library/Homebrew/version.rb (L427)). This was not an issue in the past since the hashes of our previous releases didn't happen to be of the form (`[Rr]\d+[AaBb]\d*(?:-\d+)?)`).
Reviewed By: bolinfest
Differential Revision: D42006425
fbshipit-source-id: 8dd4c52e1f49b79763bcc5863f7578a0f36dda73
Summary:
The latest Homebrew bottles for Apple Sillicon macOS built by our Github Actions were broken, as mentioned in https://github.com/facebook/sapling/issues/315 . This was caused due to updating the Formula template used by our Github actions to 3.11 but not updating the Github actions themselves to Python 3.11. This commit fixes that last part.
Pull Request resolved: https://github.com/facebook/sapling/pull/319
Test Plan:
Triggered a [build on a fork of the sapling repo](https://github.com/sggutier/sapling/releases/tag/0.1.20221211-120017-rcd410769), downloaded the bottle built by it, and checked that it ran properly on my M1 mac:
```
$ sl --version
Sapling 0.1.20221211-120017-rcd410769
$ file $(which sl)
/Users/sggutier/homebrew/bin/sl: Mach-O 64-bit executable arm64
$ otool -L $(which sl)
/Users/sggutier/homebrew/bin/sl:
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1311.100.3)
/usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0)
/System/Library/Frameworks/Security.framework/Versions/A/Security (compatibility version 1.0.0, current version 60158.100.133)
/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1858.112.0)
/Users/sggutier/homebrew/opt/openssl@1.1/lib/libssl.1.1.dylib (compatibility version 1.1.0, current version 1.1.0)
/Users/sggutier/homebrew/opt/openssl@1.1/lib/libcrypto.1.1.dylib (compatibility version 1.1.0, current version 1.1.0)
/System/Library/Frameworks/SystemConfiguration.framework/Versions/A/SystemConfiguration (compatibility version 1.0.0, current version 1163.100.19)
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
/System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices (compatibility version 1.0.0, current version 1141.1.0)
/Users/sggutier/homebrew/opt/python@3.11/Frameworks/Python.framework/Versions/3.11/Python (compatibility version 3.11.0, current version 3.11.0)
/usr/lib/libresolv.9.dylib (compatibility version 1.0.0, current version 1.0.0)
$ sl dbsh -c "import sys; print(sys.version)"
3.11.0 (main, Nov 28 2022, 13:49:33) [Clang 14.0.0 (clang-1400.0.29.202)]
$ sl clone https://github.com/sggutier/sapling/ saplingtest && cd saplingtest && sl
remote: Enumerating objects: 677771, done.
remote: Counting objects: 100% (2847/2847), done.
remote: Compressing objects: 100% (1469/1469), done.
remote: Total 677771 (delta 1396), reused 2692 (delta 1261), pack-reused 674924
Receiving objects: 100% (677771/677771), 175.60 MiB | 2.59 MiB/s, done.
Resolving deltas: 100% (454743/454743), done.
From https://github.com/sggutier/sapling
* [new ref] 2857ac6b96 -> remote/main
6535 files updated, 0 files merged, 0 files removed, 0 files unresolved
@ 2857ac6b9 Today at 11:33 mbolin https://github.com/facebook/sapling/issues/317 remote/main
│ Add build instructions for Windows (https://github.com/facebook/sapling/issues/317)
~
$ touch something && sl st
warning: watchman has recently started (pid 1093) - operation will be slower than usual
? something
$ cd eden/scm && sl root
/Users/sggutier/saplingtest
```
Reviewed By: bolinfest
Differential Revision: D41921132
Pulled By: sggutier
fbshipit-source-id: 0ed4f2d6f214f02669e45c9c4b8cced7de9caa2e
Summary:
This changes our versioning schema from `MAJOR.MINOR-%Y%m%d-%H%M%S-HASH` to `MAJOR.MINOR-%Y%m%d-%H%M%S-rHASH`
Previously we had changed our versioning scheme for better compatibility with semantic versioning and for trying to get merged into Homebrew-core. Homebrew is capable of detecting the version number based on the URL of some formula. However, it seems that at the moment Homebrew doesn't support semantic versioning, although [it supports a number of different versioning schemas](9c88c39bae/Library/Homebrew/version.rb).
With these changes (and with our current Formula we have), this will make `sl --version` appear as ``MAJOR.MINOR-%Y%m%d-%H%M%S` on systems using Homebrew, but it will show as ``MAJOR.MINOR-%Y%m%d-%H%M%S-rHASH` everywhere else.
Reviewed By: bolinfest
Differential Revision: D41655159
fbshipit-source-id: f0da45ee45e57e83e2cf018a73559ece52d09fe2
Summary:
Stack created with [Sapling]
* __->__ https://github.com/facebook/sapling/issues/208
[sl] ci: change versioning scheme
This changes our versioning scheme to `VERSION-%Y%m%d-%H%M%S-HASH`, where `VERSION` is defined in the VERSION file, `%Y%m%d-%H%M%S` is the current date, hour, minutes, and seconds, and `HASH` is the hash of the current commit.
Pull Request resolved: https://github.com/facebook/sapling/pull/208
Test Plan: Tested on my personal account
Reviewed By: bolinfest
Differential Revision: D41418387
Pulled By: sggutier
fbshipit-source-id: f89bce7cc842c6ca77c3a8330565fa19d41e39cc
Summary:
X-link: https://github.com/facebook/sapling-staging/pull/72
Modifies the homebrew formula and separates the current macOS github action into an x86 one and an arm64 one.
Reviewed By: bolinfest
Differential Revision: D41201780
fbshipit-source-id: 956152026115c886c5a32678db7ca23ee3ae91e1
Summary:
Creates a macOS Homebrew bottle that can be used in x86 systems.
This also updates the Homebrew formula template so that it statically links openssl and it is easier to edit by the homebrew action.
X-link: https://github.com/facebook/sapling-staging/pull/67
Reviewed By: bolinfest
Differential Revision: D41106453
Pulled By: sggutier
fbshipit-source-id: 2a16ef4a22639bb0b15d21eb0c88c7ca90e45bf7
Summary:
Before landing D41062213 (2736052d39), I got a lint warning that I
had an unnecessary use of an f-string because there
weren't any expressions evaluated as part of the f-string.
I removed the `f` and just hit Land without re-running
`gen_workflows.py`.
In this case, removing the `f` changed the behavior of
the code because the string originally included double `{{`
to escape it, which is necessary as an f-string, but is
interpreted literally in an ordinary string literal.
This diff removes the double `{{` so that `gen_workflows.py`
produces the desired output again.
Reviewed By: sggutier
Differential Revision: D41105429
fbshipit-source-id: f4bc023cd42a7f265325fd9262e7c602e900bb99
Summary: To be used to propagate through the Sapling version to the build script.
Reviewed By: bolinfest
Differential Revision: D40943082
fbshipit-source-id: aacb88c7a0c0113f240a020f994b5eebd56faef9
Summary:
I took this idea from Glean's comparable workflow:
5e2e569572/.github/workflows/glean-docker.yml
`workflow_dispatch` means we can always trigger a one-off build,
but the schedule ensures the Docker image is rebuilt every
Monday morning. This helps ensure things like the Yarn offline-mirror
don't drift too far from the set of dependencies we are actually using.
Reviewed By: evangrayk
Differential Revision: D40659812
fbshipit-source-id: 39fb85bc34b20d8c8558d2b1be9f7eb8d16fc9aa
Summary:
Recall that `sapling-cli-ubuntu-20.04.Dockerfile` and
`sapling-cli-ubuntu-22.04.Dockerfile` are used as part of the
respective `sapling-cli-ubuntu-xx.yy-image.yml` GitHub actions
that are used to create Docker images for the other actions so
that tools and dependencies do not have to be installed from
scratch each time.
Note that this existing line in the `Dockerfile`:
```
RUN yarn config set yarn-offline-mirror "$HOME/npm-packages-offline-cache"
```
was meant to set up an offline cache so `yarn install` in subsequent
builds would be faster. Unfortunately, it appears we were not
actually populating the offline cache because we were not
specifying the `--prefer-offline` flag to `yarn install`, which this diff fixes.
Previously, we were also defensive in the call to `yarn install`:
```
RUN [ -f /tmp/repo/addons ] && yarn --cwd /tmp/repo/addons install || true
```
But now the build will fail if `install` does not succeed, as the previous
approach made it too easy for problems to go unnoticed.
I updated the `.yml` files by running:
```
~/fbsource/fbcode/eden/oss/ci$ buck2 run :gen_workflows -- ../.github/workflows/
```
Reviewed By: evangrayk
Differential Revision: D40658823
fbshipit-source-id: 2813641b746654ac06e15ac151f1c017d8e39622
Summary:
Updated `verify_release.sh` so that:
- It runs `hg isl` and verifies that it runs successfully. Hopefully this will help
us catch things like introducing changes that do not work on Node v10,
which would break `hg isl` on Ubuntu 20.04 LTS.
- It drops the `--git` flag that is no longer necessary.
Reviewed By: quark-zju
Differential Revision: D39527960
fbshipit-source-id: e97a44103962c37107d2444d4ce8062438e6c9f5
Summary:
Add `Recommends: nodejs` to the `control` file for our `.deb` as Node is
not necessary for the core functionality of Sapling, but it is necessary for ISL.
Note that recommended packages are installed by default, so things should
still "just work" for ordinary users, though Ubuntu/Debian users have been
known to do:
```
sudo apt install --no-install-recommends nodejs
```
Though I assume they want to manage their own Node installation via `nvm`
or somesuch in those situations.
Note that now that we are installing the `nodejs` package, we need to update
`verify_release.sh` to set `DEBIAN_FRONTEND=noninteractive` so we don't
get stuck with timezone prompts when setting up the Docker image.
Reviewed By: jordanwebster
Differential Revision: D39493888
fbshipit-source-id: a8b62674273cb97dfeaf695fbbe5e1ff7e6a5501
Summary:
For now, I use this to verify releases manually, but once we fix the
current set of issues with `sl clone`, we can then extend this script
to invoke more `sl` commands to sanity-check a release build.
Ultimately, it should become part of our CI in GitHub Actions.
Reviewed By: quark-zju
Differential Revision: D39185995
fbshipit-source-id: 6f2eeb454610bfdf465cca159cf276a45362e6f0
Summary:
Simplifies the build logic so the only thing `make deb` does is
run the `./packaging/debian/build_deb.sh` script. Note that the
role of `build_deb.sh` has expanded so that it:
- reads the Ubuntu version from `/etc/os-release`
- sets `PY_VERSION` and `GIT_DEB_DEP` accordingly
- runs `DESTDIR=install make install-oss`
This means that someone on either Ubuntu 20.04 or Ubuntu 22.04
can just run `VERSION=0.0.0 make deb` and it should do the right thing.
Reviewed By: quark-zju
Differential Revision: D39182615
fbshipit-source-id: 073d569a1bcecfb9fbeb78e8042ac18b0c1bb879
Summary:
Currently, the open source build contains this line in a `Cargo.toml` file:
c403b32adb/eden/scm/Cargo.toml (L7)
For posterity, the contents of the line are:
```
graphql-parser = { git = "https://github.com/vmagro/graphql-parser", rev = "1d155d96e6052767380ab5e67c57e3d6608a31ac" }
```
The `cargo fetch` calls I am removing in this diff *used to* work. But today when
I ran the workflow that used this `Dockerfile`, they failed with:
```
#13 4.589 error: failed to resolve patches for `https://github.com/rust-lang/crates.io-index`
#13 4.589
#13 4.589 Caused by:
#13 4.589 failed to load source for dependency `graphql-parser`
#13 4.589
#13 4.589 Caused by:
#13 4.589 Unable to update https://github.com/vmagro/graphql-parser?rev=1d155d96e6052767380ab5e67c57e3d6608a31ac#13 4.589
#13 4.589 Caused by:
#13 4.589 revspec '1d155d96e6052767380ab5e67c57e3d6608a31ac' not found; class=Reference (4); code=NotFound (-3)
#13 ERROR: process "/bin/sh -c cargo fetch --manifest-path eden/scm/exec/hgmain/Cargo.toml" did not complete successfully: exit code: 101
------
> [ 8/17] RUN cargo fetch --manifest-path eden/scm/exec/hgmain/Cargo.toml:
#13 4.589 error: failed to resolve patches for `https://github.com/rust-lang/crates.io-index`
#13 4.589
#13 4.589 Caused by:
#13 4.589 failed to load source for dependency `graphql-parser`
#13 4.589
#13 4.589 Caused by:
#13 4.589 Unable to update https://github.com/vmagro/graphql-parser?rev=1d155d96e6052767380ab5e67c57e3d6608a31ac#13 4.589
#13 4.589 Caused by:
#13 4.589 revspec '1d155d96e6052767380ab5e67c57e3d6608a31ac' not found; class=Reference (4); code=NotFound (-3)
```
What happened? Here's what it looks like:
- After discovering that our `Cargo.toml` files created by `autocargo` contained massive
`[patch.crates-io]` sections that appeared to be slowing down our open source build,
I went and posted about it in the Source Control Team group:
https://fb.workplace.com/groups/sourcecontrolteam/posts/5310340079087295
- One way to improve the situation is to eliminate these patches from `//third-party/rust`,
so I spot-checked a number of the patches, tracking down the diff that introduced them,
and attempted to follow-up with the author to upstream the patch.
- D37532244 (199330fc39) appeared to be the diff that pulled in the patch for `graphql-parser`.
In the comments, I pinged the author (vmagro) to see if he could upstream his patch.
- Vinnie agreed and updated his PR: https://github.com/graphql-rust/graphql-parser/pull/66
- *what appears to have happened* is that Vinnie updated the PR by doing a "force push"
where the previous head was `1d155d96e6052767380ab5e67c57e3d6608a31ac` and
the new head is `c778917f57f6b2c26d9291819b9bb341a2f5948a`.
- GitHub...does not like force pushes. Indeed, if you visit
1d155d96e6
you see a yellow warning banner at the top that says
**This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.**
I don't know what call `cargo` makes to resolve these `git` dependencies, but I assume
GitHub is refusing to respond as it normally would for orphaned commits such as these.
D39190591 (f500204c06) is the for fbsource overall, but based on this experience, I still think it is best
to remove these `cargo fetch` calls. Of note, **even though the `cargo fetch` calls failed,
the GitHub action to build Sapling still succeeds** because Sapling does not depend
on `graphql-parser`, so `cargo build` only builds what it needs whereas `cargo fetch`
fetches everything in the `Cargo.toml` whether it needs it or not.
In sum, given the current state of `autocargo` and the `[patch.crates-io]` section it generates,
`cargo fetch` just seems like a giant liability, so we are better off without it.
Reviewed By: fanzeyi
Differential Revision: D39189595
fbshipit-source-id: cbdf3c51e39b7ed2e49a4373e7f9fc66337766cc
Summary:
Although D39042765 (a04fc2e9b3) appeared to successfully produce `.deb` files, it turns
out that they would only work if you already had the requisite package
dependencies installed. For example, running `hg --version` on a clean
Ubuntu instance would yield:
```
hg: error while loading shared libraries: libpython3.8.so.1.0: cannot open shared object file: No such file or directory
```
It turned out that our `DEBIAN/control` file was missing a key `Depends:` line
to tell `dpkg` what dependencies to install. Once again, I decided to look and
see how wezterm deals with this, and the solution appears to be `dpkg-shlibdeps`:
https://github.com/wez/wezterm/blob/97eaa58112a4/ci/deploy.sh#L234,L240
`dpkg-shlibdeps` looks at your executables and, based on the symbols, determines
what the appropriate packages are. For our Ubuntu 22.04 `.deb`, this turns out to be:
```
Depends: libc6 (>= 2.34), libgcc-s1 (>= 4.2), libpython3.10 (>= 3.10.0), libssl3 (>= 3.0.0~~alpha1), zlib1g (>= 1:1.1.4)
```
Apparently you cannot have a package in the `Depends:` line that comes from a PPA,
so if you want to make your `.deb` easy for users to install, that means you should
limit your dependencies to "standard" packages for the distro. In our case, that meant
that our Ubuntu 22.04 `.deb` should use Python 3.10 instead of Python 3.8 (which we
previously fetched from `ppa:deadsnakes/ppa`).
In order to support this, this diff makes a number of changes:
- Adds logic to `setup.py` to check the `PY_VERSION` environment variable to decide
whether to use Python 3.10 or something else.
- Updates `gen_workflows.py` to define different Python deps based on the Ubuntu version.
- Updates `gen_workflows.py` to remove the logic that uses the `ppa:deadsnakes/ppa`.
- Ran `buck2 run //eden/oss/ci:gen_workflows -- eden/oss/.github/workflows/` to update the GitHub Actions
- Moved the logic for the `deb` target in the `Makefile` into a separate shell script because
it was getting too complex to express directly in Make.
- Introduced separate `deb-ubuntu-20.04` and `deb-ubuntu-22.04` targets in the `Makefile`.
- Updated `bytearrayobject.rs` to support Python 3.10 in addition to Python 3.9.
- Updated `pick_python.py` to consider `python3.10` if `python3.8` is not found before going
down the rest of the list.
As you can see in the Test Plan, we are *really close* to things "just working," but we also have
to register `git` as a dependency, which is not discovered by `dpkg-shlibdeps`. This will be
addressed in D39156794.
Reviewed By: DurhamG
Differential Revision: D39156794
fbshipit-source-id: ca1e0a73096e0de97230804a97f316114b8bfc2e
