Michał Wawrzyniec Urbańczyk e930738e02 [CI] Engine CI Rework, Part 1 (#9295) ... I have created PR with the first set of changes for the Engine CI. The changes are small and effectively consist of: 1. Spltting the `verifyLicensePackages`. It is now run only on Linux. There are hardly any time benefits, as the actual job cost is dominated by the overhead of spinning a new job — but it is not expensive in the big picture. 2. Splitting the Scala Tests into separate job. This is probably the biggest "atomic" piece of work we have. 3. Splitting the Standard Library Tests into a separate job. The time is nicely split across the jobs now. The last run has: * 27 min for Scala tests; * 25 min for Standard Library tests; * 24 min for the "rest": the old job containing everything that has not been split. While total CPU time has increased (as jobs are not effectively reusing the same build context), the wall time has decreased significantly. Previously we had ~1 hour of wall time for the old monolithic job, so we are getting more than 2x speedup. The now-slowest Scala tests job is currently comparable with the native Rust tests (and they should improve when the old gui is gone) — which are the slowest job across all CI checks. The PR is pretty minimal. Several future improvements can be made: * Reorganizing and splitting other "heavy" jobs, like the native image generation. * Reusing the built Engine distribution. However, this is probably a lower priority than I initially thought. * Building package takes several minutes, so duplicating this job is not that expensive. * The package is OS-specific. * Scala tests don't really benefit from it, they'd need way more compilation artifacts. It'd make sense to reuse the distribution if we, for example, decided to split more jobs that actually benefit from it, like Standard Library tests. * Reusing the Rust build script binary. * As our self-hosted runners reuse environment, we effectively get this for free. Especially when Rust part of codebase is less frequently changed. * This is however significant cost for the GitHub-hosted runners, affecting our macOS runners. Reusing the binary does not save wall time for jobs that are run in parallel (as we have enough runners), but if we introduce job dependencies that'd force sequential execution of jobs on macOS, this would be a significant need.		2024-03-06 18:56:13 +00:00
..
src	[CI] Engine CI Rework, Part 1 (#9295)	2024-03-06 18:56:13 +00:00
Cargo.toml	Build script fixes & improvements (#8285)	2023-11-23 21:19:31 +01:00