Commit Graph

531 Commits

Author SHA1 Message Date
Daniel Lockyer
5a72c5ad91 Updated Nx to v19
refs https://github.com/nrwl/nx/releases/tag/19.8.0

- this commit updates Nx to v19
- we need to add some extra commands to the dev script to stop and
  restart the Nx daemon, so it's ready and running before we execute a
  bunch of Nx commands concurrently
- this also updates nx.json to the format needed for the latest version
2024-09-25 10:16:08 +02:00
Daniel Lockyer
0854f8a531 Exported Git commit hash from update script
ref https://linear.app/tryghost/issue/DEV-25/move-version-bumping-logic-into-ghost-repo

- this allows us to re-use the value from outside the script in CI
2024-09-24 14:08:38 +02:00
Daniel Lockyer
ca691e99e8 Fixed branch name in canary workflow
- `github.ref_name` is a more reliable way to find the branch name than
  we were previously doing
2024-09-24 10:46:44 +02:00
Daniel Lockyer
24447f438e Passed along branch name to canary job
ref https://linear.app/tryghost/issue/DEV-20/faster-builds

- we should pass along the branch name in the metadata field so it's
  more DRY for canary builds in the future
2024-09-24 10:17:22 +02:00
Daniel Lockyer
ae8f8f128b Added version bumping script to repo
ref https://linear.app/tryghost/issue/DEV-25/move-version-bumping-logic-into-ghost-repo

- we're slowly migrating our build code into the OSS repo, which means
  we need to move scripts over
- we have this as a bash script, but I've rewritten it to JS so it's a
  little more maintainable
- this script will just bump the version in the package.json files and
  set the GHA output
2024-09-24 09:11:54 +02:00
Daniel Lockyer
9093ffbf98 Fixed browser tests incorrectly executing Stripe CLI
- we shouldn't try and load the Stripe CLI via the dev script because
  it's done in the browser tests and involves more setup than the dev
  script contains
- this cuts 2mins from the browser tests because they're no longer
  waiting for the Stripe CLI to be auth'd
2024-09-23 17:43:53 +02:00
Daniel Lockyer
7c346c28eb Enabled Nx caching on main
- we should be able to trust Nx enough that we can sustain the build
  cache across commits, which will speed up the workflow because we
  don't need to rebuild our TS projects all the time
2024-09-23 15:19:31 +02:00
Daniel Lockyer
5791be4937 Merged setup steps in CI
- we don't need these to be separate steps and having them separate
  actually makes CI slower because it takes ~10-12s for GitHub to start
  new jobs
2024-09-23 14:48:17 +02:00
Chris Raible
b90aca2816
Removed jaeger container from docker compose (#20994)
no issue

- OpenTelemetry has been problematic in a number of ways (boot time,
breaking the frontend). May revisit it at some point in the future, but
for now it is only exporting metrics via prometheus and not traces, so
there's currently nothing sending data to this jaeger container
- Cleaning it up for now as it's just sitting there idly consuming
resources
2024-09-12 10:37:54 -07:00
Chris Raible
2a0d49c539
Added MySQL data volume to docker compose (#20982)
no issue

- This allows us to run `docker-compose down` or to restart docker
desktop without losing all our local databases
- Added a data volume to the MySQL service in the `docker-compose.yml`
file to persist the data between container restarts
- The `yarn docker:reset` command will still reset all the data in the
database since it uses `down -v` to remove the volumes as well
2024-09-12 09:38:24 -07:00
Sam Lord
625c89e37f
Added the ability to run browser tests using local Portal (#20990)
ref DOGM-32

Using the dev script as a template, this script runs the tests with
local copies of the applications needed instead of the released CDN
versions
2024-09-12 15:55:47 +01:00
Daniel Lockyer
77df6186f0 Removed auto-labeller for PRs
- this was an early attempt to group PRs together by labels, so we can
  triage PRs easier, but it's not finished and actually producing more
  noise than signal
- we might want to re-add this in the future, but for now, silence 🧘
2024-08-29 11:06:00 +02:00
Hannah Wolfe
2720791434
Updated CI workflow to run on PR label/unlabel
- We have browser tests which only run if the browser tests flag is added to the PR
- The label has to be present on PR creation, which is hard to remember/doesn't fit with various workflows
- The default type of action for the pull_request trigger are opened, synchronize, reopened
- This PR adds labeled and unlabeled to those, which I think will help us to run the tests as expected
- The expectation is that adding the browser test label will now trigger the tests to run
2024-08-29 09:24:37 +01:00
Princi Vershwal
f984fbd47e
🎨 Improved the performance of the /members/events/ aggregated_click_event endpoint (#20790)
Ref https://linear.app/tryghost/issue/ONC-216/improve-the-performance-of-the-membersevents-aggregated-click-event
2024-08-22 18:26:10 +05:30
Daniel Lockyer
0f3805e096 Changed color of adminX prefix for yarn dev
- red makes it look like an error, which is very misleading
- I've changed this to a random purple I found
- credits to @vershwal and @dvdwinden
2024-08-20 12:35:24 +02:00
Chris Raible
f147167a29
Added SQLite and MySQL check to migration review checklist (#20708)
no issue

- knex can behave differently with SQLite and MySQL, which can cause
migrations to behave differently in each database. This PR adds a check
to the migration review checklist to remind us to test the migration in
both databases before merging.
2024-08-01 13:38:59 -07:00
renovate[bot]
b754513658 Update jaegertracing/all-in-one Docker tag to v1.58 2024-06-21 09:38:38 +01:00
Chris Raible
417c9c49ea
Added OpenTelemetry instrumentation to Ghost backend (#20144)
This commit adds OpenTelemetry instrumentation to Ghost's backend, which
allows us to view traces similar to what we see in Sentry Performance
locally.

OpenTelemetry is enabled if `NODE_ENV === 'development'` or if it is
explicitly enabled via config with `opentelemetry:enabled`.

It also adds a [Jaeger](https://www.jaegertracing.io/) container to
Ghost's docker-compose file for viewing the traces. There's no setup
required (beyond running `yarn docker:reset` to pickup the changes in
the docker-compose file the first time — but this will also reset your
DB so be careful). This will launch the Jaeger container, and you can
view the UI to see the traces at `http://localhost:16686/search`.
2024-06-19 13:56:51 -07:00
renovate[bot]
54bd9a1ab4 Update benchmark-action/github-action-benchmark action to v1.20.3 2024-05-20 10:39:02 +01:00
renovate[bot]
15569145f3 Update benchmark-action/github-action-benchmark action to v1.20.1 2024-05-09 10:23:48 +02:00
Michael Barrett
af92297ca9
Added Redis via Docker (#20085)
no refs

Redis can be utilised for various caching purposes within Ghost. This PR
adds a Redis service to the docker-compose file to allow for easier
local development when Redis is required
2024-05-07 11:02:36 +01:00
Joe Grigg
c744740761 Updated canary build CI to use the main Moya build pipeline
ref ENG-807
2024-05-01 13:22:01 +01:00
Djordje Vlaisavljevic
7a3bbfde10
Added ActivityPub playground (#20081)
ref MOM61

- Adds admin-x react app we’ll use as ActivityPub playground to the
sidebar nav behind the feature flag.
- Wired up routing to Ember
- Setup the project as `admin-x-activitypub`

---------

Co-authored-by: Ronald Langeveld <hi@ronaldlangeveld.com>
2024-04-25 16:44:29 +08:00
Daniel Lockyer
10e81aeed8
ℹ️ Added support for Node 20
ref https://linear.app/tryghost/issue/ENG-765/add-support-for-node-20

- this adds support for Node 20 to Ghost and CI, as Node 20 is an LTS
  version and we should pick it up
2024-04-18 13:17:21 +02:00
renovate[bot]
e8ea2e4db0 Update GitHub Artifact Actions to v4 2024-04-18 12:36:39 +02:00
renovate[bot]
4ab31122a4 Update actions/cache action to v4 2024-04-18 12:36:07 +02:00
renovate[bot]
2d6a361bb5 Update dorny/paths-filter action to v2.12.0 2024-04-16 09:44:01 +02:00
Joe Grigg
d0c1814a1c Fixed incorrect branch for canary build
no-issue
2024-03-25 15:02:10 +00:00
Joe Grigg
7c5b8355e4 Switched canary build to use the new faster build pipeline
no-issue
2024-03-25 14:36:02 +00:00
Ronald Langeveld
bc7ff4be9c
Added no-seed flag to yarn setup. (#19697)
refs https://ghost.slack.com/archives/C02G9E68C/p1707994528206559

- When running Ghost development and using MySQL + Docker, you can run
the command `yarn setup --no-seed` to not have the database seeded with
test data.
2024-02-15 14:48:08 +02:00
Michael Barrett
47439b3b94
Updated docker-compose.yml to include a name for the project (#19588)
no refs

Added a name for the project in `docker-compose.yml` as it was using the
`scripts` directory as the name

Before:

```sh
❯ docker-compose ls
NAME                STATUS              CONFIG FILES
scripts             running(1)          /path/to/ghost/.github/scripts/docker-compose.yml
```

After:

```sh
❯ docker-compose ls
NAME                STATUS              CONFIG FILES
ghost               running(1)          /path/to/ghost/.github/scripts/docker-compose.yml
```
2024-02-01 09:06:44 +00:00
Sam Lord
167a442ffe Free ubuntu-latest machines now also have 4 cores
refs: https://github.blog/2024-01-17-github-hosted-runners-double-the-power-for-open-source/

This should still allow us to run tests in 8~ minutes without paying for the VM time
2024-01-18 09:35:30 +00:00
Simon Backx
ea7290c0da
Added xxl options to data generation (#19488)
no issue

Allows switching between 100k members vs 2M members
2024-01-16 10:20:33 +00:00
Simon Backx
d2cb23c3fa
Wired up Docker setup script and increased data generation performance (#19420)
ref PROD-233

- Stored whether Docker is used in the config files
- When running `yarn setup`, any existing Docker container will be
reset. Run with `-y` to skip the confirmation step.
- `yarn setup` will always init the database and generate fake data
- Increased amount of default generated data to 100,000 members + 500
posts.
- Made lots of performance improvements in the data generator so we can
generate the default data in ±170s
2024-01-05 13:42:30 +00:00
Simon Backx
eebf0e2eaf Added knex-migrator init to setup script
ref PROD-233

When clearing the Docker database container, you still need to be able to run `yarn setup` to create the tables.
2023-12-14 18:30:45 +01:00
Daniel Lockyer
b67176431b Removed manual list of Typescript projects from dev script
fixes https://github.com/TryGhost/DevOps/issues/122

- this introduces some code to get all the Typescript projects
  programatically instead of relying on developers to manually add their
  project to the list, which is clearly better
2023-12-13 13:18:11 +01:00
Daniel Lockyer
a9c9202b5c Added defaults to Nx build step
refs https://github.com/TryGhost/DevOps/issues/122

- this allows us to DRY up some of the other configuration we have
  dotted around and fixes a niche bug with overlapping builds that we
  were seeing before, because Nx should only allow one build at a time
2023-12-13 11:54:31 +01:00
Daniel Lockyer
d65a95836b Revert "Enabled app build artifact caching in CI"
This reverts commit 617d3a6ffc.
2023-12-11 15:32:46 +01:00
Daniel Lockyer
617d3a6ffc Enabled app build artifact caching in CI
fixes https://github.com/TryGhost/DevOps/issues/120

- we've seen errors in CI where the unit tests for the apps folder are
  being run concurrently, but the build assets are not present so
  multiple builds are happening for the same folder
- this leads to errors because Vite tries to cleanup or build folders
  at the same time
- this should fix that by eagerly building the apps and caching the
  output so unit tests run quicker
2023-12-11 12:28:27 +01:00
Daniel Lockyer
398f96249c Added MySQL setup via Docker
refs https://github.com/TryGhost/DevOps/issues/118

- we should standardize how we run MySQL, so there aren't massive gaps
  in developer experience
- Docker is great for this, and it's pretty easy to set up
- this adds a docker-compose.yml file with a small MySQL setup
- also configures `yarn setup` to spin up the Docker container and add
  it to config.local.json
- in the near future, we'll provide a base SQL file in .github/scripts/mysql-preload
  to ensure the DB is in a known state
2023-12-07 10:57:47 +01:00
Daniel Lockyer
d617891e46 Configured Nx watch to run in an infinite loop
refs https://ghost.slack.com/archives/C02G9E68C/p1701939292474999?thread_ts=1701881060.000299&cid=C02G9E68C

- when you change branch or do a `git pull`, Nx will kill the watch
  command
- this becomes annoying when you're doing local development because it
  tears down the whole `yarn dev` command
- this puts the nx watch command in an infinite loop so it's just
  restart again and concurrently doesn't detect it as a failure
2023-12-07 10:34:20 +01:00
Simon Backx
17ec1e8937
Added email address alignment protections (#19094)
ref GRO-54
fixes GRO-63
fixes GRO-62
fixes GRO-69

When the config `hostSettings:managedEmail:enabled` is enabled, or the
new flag (`newEmailAddresses`) is enabled for self-hosters, we'll start
to check the from addresses of all outgoing emails more strictly.

- Current flow: nothing changes if the managedEmail config is not set or
the `newEmailAddresses` feature flag is not set
- When managedEmail is enabled: never allow to send an email from any
chosen email. We always use `mail.from` for all outgoing emails. Custom
addresses should be set as replyTo instead. Changing the newsletter
sender_email is not allowed anymore (and ignored if it is set).
- When managedEmail is enabled with a custom sending domain: if a from
address doesn't match the sending domain, we'll default to mail.from and
use the original as a replyTo if appropriate and only when no other
replyTo was set. A newsletter sender email addresss can only be set to
an email address on this domain.
- When `newEmailAddresses` is enabled: self hosters are free to set all
email addresses to whatever they want, without verification. In addition
to that, we stop making up our own email addresses and send from
`mail.from` by default instead of generating a `noreply`+ `@` +
`sitedomain.com` address

A more in depth example of all cases can be seen in
`ghost/core/test/integration/services/email-addresses.test.js`

Includes lots of new E2E tests for most new situations. Apart from that,
all email snapshots are changed because the from and replyTo addresses
are now included in snapshots (so we can see unexpected changes in the
future).

Dropped test coverage requirement, because tests were failing coverage
locally, but not in CI

Fixed settings test that set the site title to an array - bug tracked in
GRO-68
2023-11-23 10:25:30 +01:00
Jono M
a93c665d20
Created a skeleton AdminX demo app (#19005)
refs https://github.com/TryGhost/Product/issues/4152

---

<!-- Leave the line below if you'd like GitHub Copilot to generate a
summary from your commit -->
<!--
copilot:summary
-->
### <samp>🤖[[deprecated]](https://githubnext.com/copilot-for-prs-sunset)
Generated by Copilot at a28462f</samp>

This pull request adds a new admin-x app called `admin-x-demo`, which
demonstrates how to use the shared packages `admin-x-framework` and
`admin-x-design-system` to create a simple app that renders a button and
a modal. It also improves the development workflow, the vite
integration, the dependency management, and the type checking for the
admin-x apps and packages. It modifies some files in the
`admin-x-framework` and `admin-x-design-system` packages to make the
modals prop optional, to introduce a new type for the props from the
Ember app, to fix the z-index of the modal backdrop, and to use
consistent file extensions and module syntax.
2023-11-20 13:30:15 +00:00
Daniel Lockyer
6bc703e235 Fixed glob for Admin-X projects
refs c695497598

- the previous glob didn't match correctly
2023-11-20 12:49:34 +01:00
Daniel Lockyer
b1666f596f Removed GitHub Copilot for PRs templating value
refs https://githubnext.com/copilot-for-prs-sunset

- unfortunately the technical preview will end in mid December so we
  need to cleanup the templating var
2023-11-20 11:43:58 +01:00
Daniel Lockyer
c695497598 Switched to actions/labeler for PR labeling
fixes https://github.com/TryGhost/DevOps/issues/111

- this allows us to add labels based upon the contents in a smarter way
  than the previous workflow
2023-11-20 11:43:58 +01:00
Simon Backx
e3cdc24d6b
Added test to test default theme against linked gscan version (#19020)
fixes GRO-32

If we ever introduce errors in the linked source theme and linked gscan
version combination, this test will catch it.
2023-11-16 12:03:00 +00:00
Daniel Lockyer
e2be6dbf14 Configured submodules to clone in performance tests
refs https://github.com/TryGhost/DevOps/issues/105

- we should include submodules in the checkout so the Ghost boot is
  representative of booting Ghost with a theme
2023-11-15 14:08:32 +01:00
Daniel Lockyer
566f2055d0 Moved GitHub context logging to job
- this stops the entire context from being output in the env vars, so it
  cleans up the output at the top of each job
2023-11-15 13:54:20 +01:00
Daniel Lockyer
18afd39193 Removed NODE_ENV env var from perf tests
refs 05977d5c26

- this seems to break things, will investigate once this is merged
- also adds showing output flag so debugging is easier
2023-11-15 13:35:41 +01:00