ref
https://linear.app/tryghost/issue/ENG-1591/add-prometheus-and-grafana-services-to-docker-compose
This commit adds 2 new services to the docker compose file to enable
monitoring metrics from Ghost locally in real-time:
1. Prometheus - a service that scrapes Ghost's new `/metrics` endpoint
introduced in this
[commit](768336efad).
2. Grafana - a service that consumes the metrics from prometheus and
exposes them in a dashboard that you can view locally at
`localhost:3000`.
# Usage
Both of these services are selectively enabled using docker compose
[profiles](https://docs.docker.com/compose/how-tos/profiles/). This way,
if you don't opt-in to using these monitoring tools, they won't start
and consume resources on your host machine. To enable these services,
enable the `monitoring` profile by either setting the `COMPOSE_PROFILES`
environment variable to `monitoring`, or specifying the `--profile
monitoring` CLI argument to any `docker compose ...` commands.
I've found the easiest way to configure this in an 'always on' fashion
is to create a `.env` file in the project's root directory and add
`COMPOSE_PROFILES=monitoring` to it. As an added convenience, you can
also set `COMPOSE_FILE=.github/scripts/docker-compose.yml`, which will
allow you to run `docker compose ...` commands from the root directory
without specifying the full path each time.
# Intended for development only
These services are meant for local development only, and are not
configured for a production use-case. For example, the Grafana instance
is configured to have _no authorization_ so you won't need a
username/password to login at `localhost:3000`. Prometheus is also
configured to scrape the metrics once every second, which is likely
excessive for production use-cases, but may be useful for getting more
granular metrics while e.g. load testing locally.
# Dashboards
The Grafana instance includes a default dashboard including most of the
main default metrics provided by our prometheus client integration. The
dashboard is defined in a JSON file at
`.github/scripts/docker/grafana/dashboards/main-dashboard.json' and can
be modified & committed to add new visualizations that will be available
to anyone work on Ghost locally. You can also add other dashboards to
the same directory for specific use-cases, which should be picked up and
made available in the Grafana UI. [Read
more](https://grafana.com/docs/grafana/latest/dashboards/build-dashboards/view-dashboard-json-model/)
about Grafana's JSON schema for dashboards.
refs https://ghost.slack.com/archives/C02G9E68C/p1727704490753759
- if you open a PR and it becomes outdated enough such that the base
commit was 100 commits ago, the workflow starts to fail
- to help prevent this, we can increase it by 1000, which should more
than cover enough use-cases but still keep checkout quick
- users like `renovate[bot]` have brackets in the username
- this breaks the command and it exits with `exit code 3.`
- to fix this, we can encode the username before passing it in
ref https://linear.app/tryghost/issue/DEV-31/staging-deploys-of-feature-branchesprs
- we want the ability to ship a PR to staging, so we can test and QA
without merging to `main`
- most of the infrastructure is already in place for this, so it's
mostly a case of wiring it all up
- this commit will send a slightly different payload to the build
process, to indicate it's coming from a PR
- I've also added a check that the user is a member of the org, so we
don't get random builds from non-members
- to trigger this, we should be able to add the `deploy-to-staging`
label and it Just Works :TM:
refs https://github.com/nrwl/nx/releases/tag/19.8.0
- this commit updates Nx to v19
- we need to add some extra commands to the dev script to stop and
restart the Nx daemon, so it's ready and running before we execute a
bunch of Nx commands concurrently
- this also updates nx.json to the format needed for the latest version
ref https://linear.app/tryghost/issue/DEV-25/move-version-bumping-logic-into-ghost-repo
- we're slowly migrating our build code into the OSS repo, which means
we need to move scripts over
- we have this as a bash script, but I've rewritten it to JS so it's a
little more maintainable
- this script will just bump the version in the package.json files and
set the GHA output
- we shouldn't try and load the Stripe CLI via the dev script because
it's done in the browser tests and involves more setup than the dev
script contains
- this cuts 2mins from the browser tests because they're no longer
waiting for the Stripe CLI to be auth'd
- we should be able to trust Nx enough that we can sustain the build
cache across commits, which will speed up the workflow because we
don't need to rebuild our TS projects all the time
no issue
- OpenTelemetry has been problematic in a number of ways (boot time,
breaking the frontend). May revisit it at some point in the future, but
for now it is only exporting metrics via prometheus and not traces, so
there's currently nothing sending data to this jaeger container
- Cleaning it up for now as it's just sitting there idly consuming
resources
no issue
- This allows us to run `docker-compose down` or to restart docker
desktop without losing all our local databases
- Added a data volume to the MySQL service in the `docker-compose.yml`
file to persist the data between container restarts
- The `yarn docker:reset` command will still reset all the data in the
database since it uses `down -v` to remove the volumes as well
ref DOGM-32
Using the dev script as a template, this script runs the tests with
local copies of the applications needed instead of the released CDN
versions
- this was an early attempt to group PRs together by labels, so we can
triage PRs easier, but it's not finished and actually producing more
noise than signal
- we might want to re-add this in the future, but for now, silence 🧘
- We have browser tests which only run if the browser tests flag is added to the PR
- The label has to be present on PR creation, which is hard to remember/doesn't fit with various workflows
- The default type of action for the pull_request trigger are opened, synchronize, reopened
- This PR adds labeled and unlabeled to those, which I think will help us to run the tests as expected
- The expectation is that adding the browser test label will now trigger the tests to run
no issue
- knex can behave differently with SQLite and MySQL, which can cause
migrations to behave differently in each database. This PR adds a check
to the migration review checklist to remind us to test the migration in
both databases before merging.
This commit adds OpenTelemetry instrumentation to Ghost's backend, which
allows us to view traces similar to what we see in Sentry Performance
locally.
OpenTelemetry is enabled if `NODE_ENV === 'development'` or if it is
explicitly enabled via config with `opentelemetry:enabled`.
It also adds a [Jaeger](https://www.jaegertracing.io/) container to
Ghost's docker-compose file for viewing the traces. There's no setup
required (beyond running `yarn docker:reset` to pickup the changes in
the docker-compose file the first time — but this will also reset your
DB so be careful). This will launch the Jaeger container, and you can
view the UI to see the traces at `http://localhost:16686/search`.
no refs
Redis can be utilised for various caching purposes within Ghost. This PR
adds a Redis service to the docker-compose file to allow for easier
local development when Redis is required
ref MOM61
- Adds admin-x react app we’ll use as ActivityPub playground to the
sidebar nav behind the feature flag.
- Wired up routing to Ember
- Setup the project as `admin-x-activitypub`
---------
Co-authored-by: Ronald Langeveld <hi@ronaldlangeveld.com>
no refs
Added a name for the project in `docker-compose.yml` as it was using the
`scripts` directory as the name
Before:
```sh
❯ docker-compose ls
NAME STATUS CONFIG FILES
scripts running(1) /path/to/ghost/.github/scripts/docker-compose.yml
```
After:
```sh
❯ docker-compose ls
NAME STATUS CONFIG FILES
ghost running(1) /path/to/ghost/.github/scripts/docker-compose.yml
```
ref PROD-233
- Stored whether Docker is used in the config files
- When running `yarn setup`, any existing Docker container will be
reset. Run with `-y` to skip the confirmation step.
- `yarn setup` will always init the database and generate fake data
- Increased amount of default generated data to 100,000 members + 500
posts.
- Made lots of performance improvements in the data generator so we can
generate the default data in ±170s
fixes https://github.com/TryGhost/DevOps/issues/122
- this introduces some code to get all the Typescript projects
programatically instead of relying on developers to manually add their
project to the list, which is clearly better
refs https://github.com/TryGhost/DevOps/issues/122
- this allows us to DRY up some of the other configuration we have
dotted around and fixes a niche bug with overlapping builds that we
were seeing before, because Nx should only allow one build at a time
fixes https://github.com/TryGhost/DevOps/issues/120
- we've seen errors in CI where the unit tests for the apps folder are
being run concurrently, but the build assets are not present so
multiple builds are happening for the same folder
- this leads to errors because Vite tries to cleanup or build folders
at the same time
- this should fix that by eagerly building the apps and caching the
output so unit tests run quicker
