ref https://linear.app/ghost/issue/ENG-1782/move-docker-related-files-to-a-better-location-that-githubscripts
- The docker files are currently located in `.github/scripts`. This location doesn't make a lot of sense — you wouldn't think to look there unless you already knew they were there. This also requires you to specify the path to the `compose.yml` file whenever running a `docker compose ...` command.
- This commit moves the `compose.yml` file to the root of the repo, so you can simply run `docker compose up` and it will automatically find the file in the root, without having to specify `-f .github/scripts/docker-compose.yml`. This is a major win for convenience over the current setup.
- It also moves all the related files, including the `Dockerfile` used by the Dev Container setup and configuration files for supporting services into a new `.docker` directory, which is a more logical location, and should be easier to find.
- Also updated the current convenience commands in the `package.json` scripts block (`yarn docker:reset` and `yarn docker:down`
no issue
- Dev Containers let you work on Ghost in a consistent, isolated
environment with all the necessary development dependencies
pre-installed. VSCode (or Cursor) can effectively run _inside_ the
container, providing a local quality development environment while
working in a well-defined, isolated environment.
- For now the default setup only works with "Clone repository in
Container Volume" or "Clone PR in Container Volume" — this allows for a
super quick and simple setup. We can also introduce another
configuration to allow opening an existing local checkout in a Dev
Container, but that's not quite ready yet.
- This PR also added the `yarn clean:hard` command which: deletes all
node_modules, cleans the yarn cache, and cleans the NX cache. This will
be necessary for opening a local checkout in a Dev Container.
- To learn more about Dev Containers, read this guide from VSCode:
https://code.visualstudio.com/docs/devcontainers/containers#_personalizing-with-dotfile-repositories
---------
Co-authored-by: Joe Grigg <joe@ghost.org>
Co-authored-by: Steve Larson <9larsons@gmail.com>
- Added yarn command to update TB CLI, as that needs doing frequently and I can never remember the command
- Improved safety & usability of tinybird test script by ensuring branches are correctly created before running & adding optional delete
- Updated tinybird test to warn only for sanity check as that's not always a valid check (Will prob remove soon)
- Improved output of tinybird test script on failure, so that the diff is readable and closer to what git shows you
- Added tool to convert tinybird ndjson to csv to make it easier to bring the data into google sheets for verifying numbers
refs https://github.com/nrwl/nx/releases/tag/19.8.0
- this commit updates Nx to v19
- we need to add some extra commands to the dev script to stop and
restart the Nx daemon, so it's ready and running before we execute a
bunch of Nx commands concurrently
- this also updates nx.json to the format needed for the latest version
ref https://linear.app/tryghost/issue/DEV-25/move-version-bumping-logic-into-ghost-repo
- we're slowly migrating our build code into the OSS repo, which means
we need to move scripts over
- we have this as a bash script, but I've rewritten it to JS so it's a
little more maintainable
- this script will just bump the version in the package.json files and
set the GHA output
ref DOGM-32
Using the dev script as a template, this script runs the tests with
local copies of the applications needed instead of the released CDN
versions
- we don't really need to run `yarn` here, as it pulls down all the
dependencies for the 2 themes
- we rarely need these locally, and it's also slowing down CI, so we can
just remove it from here for now
ref https://linear.app/tryghost/issue/ANAL-27/setup-tinybird-project-and-cicd
- Tinybird has a system for managing it's configuration as code, with full ci/cd support
- The tinybird CLI tool uses python, so we'll run that using docker, via `yarn tb`
- Some of the files tinybird adds should not be in source control, so we've added those to git ignore
- Everything in /ghost/tinybird is tinybird's init config
ref https://linear.app/tryghost/issue/ENG-1266
- Mexico changed tz to not participate in DST
- our package was a couple years behind, so we likely have fixes for
other countries/regions, too
no issue
When testing Stripe migrations, it is useful to be able to clear the
database quickly without deleting admins and tokens. This is possible
with the data generator.
ref https://linear.app/tryghost/issue/CFR-4/
- added request queueing middleware (express-queue) to handle high
request volume
- added new config option `optimization.requestQueue`
- added new config option `optimization.requestConcurrency`
- added logging of request queue depth - `req.queueDepth`
We've done a fair amount of investigation around improving Ghost's
resiliency to high request volume. While we believe this to be partly
due to database connection contention, it also seems Ghost gets
overwhelmed by the requests themselves. Implementing a simple queueing
system allows us a simple lever to change the volume of requests Ghost
is actually ingesting at any given time and gives us options besides
simply increasing database connection pool size.
---------
Co-authored-by: Michael Barrett <mike@ghost.org>
- this version is written in TS, but was published a few months ago and
needs to be bumped here
- also updates a previous deep include into the library, which was
unnecessary anyway
refs https://github.com/TryGhost/Toolbox/issues/501
- at this point, we have no real reason to keep this behind as it wasn't
proven what the cause of the high CPU was, and it's just causing more
lockfile issues with the resolution
no issue
The data generator went out of memory when trying to generate fake data
for > 2M members. This adds some improvements to make sure it doesn't go
out of memory.
---------
Co-authored-by: Fabien "egg" O'Carroll <fabien@allou.is>
ref PROD-233
- Stored whether Docker is used in the config files
- When running `yarn setup`, any existing Docker container will be
reset. Run with `-y` to skip the confirmation step.
- `yarn setup` will always init the database and generate fake data
- Increased amount of default generated data to 100,000 members + 500
posts.
- Made lots of performance improvements in the data generator so we can
generate the default data in ±170s
fixes https://github.com/TryGhost/DevOps/issues/123
- ember-auto-import supports webpack v5 and v5 supports persistent file
caching
- this stores a cache of built files across warm boots, so Admin built
via `yarn dev` should be faster next time
- also ensures that recursive `node_modules` folders are emptied upon
`yarn fix`, as the webpack cache is stored in `node_modules` under
`ghost/admin`
- locally, this speeds up a warm build by 2x (!)
refs https://github.com/TryGhost/DevOps/issues/118
- we should standardize how we run MySQL, so there aren't massive gaps
in developer experience
- Docker is great for this, and it's pretty easy to set up
- this adds a docker-compose.yml file with a small MySQL setup
- also configures `yarn setup` to spin up the Docker container and add
it to config.local.json
- in the near future, we'll provide a base SQL file in .github/scripts/mysql-preload
to ensure the DB is in a known state
