closes https://github.com/TryGhost/Toolbox/issues/497
- The classification of fatal/non-fatal errors has been updated to only be fatal when causing page renders with 5xx or 4xx responses.
- Some of the rules checking Ghost 5.x compatibility have been relaxed to only be "error" with the gscan version bump
- You can find more details on which exact rules were relaxed in the gscan's commit log - https://github.com/TryGhost/gscan/compare/v4.35.1...v4.36.0
refs https://github.com/TryGhost/Toolbox/issues/488
- Node 18 is now LTS so we're adding support for it
- this adds Node 18.12.1 (the latest security release) to our supported
ranges and CI
- this was all getting terribly behind so I've done several things:
- majority of `@tryghost/*` except Lexical packages
- gscan + knex-migrator to remove old `@tryghost/errors` usage
- bumped lockfile
refs https://github.com/TryGhost/Team/issues/2370
Due to a possible bug in either `@sentry/node` (mainly the Express
middlewares and the usage of deprectated Domain) and Node v16+,
unhandled promise rejections are transformed into uncaught exceptions
and cause Ghost to crash in unexpected situations.
Reverting to `v7.11.1` fixes this (but definitely not ideal at all)
because errors are caught in the Express middleware.
Reproduction repo:
https://github.com/SimonBackx/sentry-node-unhandled-rejection-crash
refs https://github.com/TryGhost/Toolbox/issues/479
- this includes a handful of improvements to get Playwright working on a
local environment including:
- adding `testing-browser` environment so we don't nuke `development`
environments, and makes all the necessary changes to get Ghost to
behave when this is running
- stopped running one global instance of Ghost as this doesn't provide
a clean environment
- copies a few default fixtures that are needed for the new
environment
no issue
Local tests can now setup Stripe during the global setup process, and the webhook server is run out-of-process.
Running tests in CI against localhost will use environment variables to setup Stripe.
Providing a test URL will avoid setting up Stripe and will assume that it is already done.
fixes https://github.com/TryGhost/Team/issues/2346
- Adds email batch browse endpoint
- Adds email recipient failures browse endpoint
- Adds new fixtures and E2E tests for the new API
- Added support for snapshot tests to have 'nullable' types.
refs https://github.com/TryGhost/Team/issues/2225
- updated the `formatOnWrite` transform map for posts to include the new `nodes` and `transformMap` options used by `urlUtils` for transforming node payload data
- added `nodes` to the `lexicalLib` module that pulls in our default nodes to be passed in to the URL transform utilities
- added `urlTransformMap` to the `lexicalLib` module that maps transform type and data type to URL transform utility functions that accept a single URL argument
refs: https://github.com/TryGhost/Toolbox/issues/479
Framework includes:
* command to run tests
* command to record tests
* mechanism for starting and stopping Ghost before and after each suite of tests
* mechanism for loading fixtures into Ghost before starting tests
* sample test for controlling Ghost Admin
fixes https://github.com/TryGhost/Team/issues/2282
Added a new email service package that is used when the email stability
flag is enabled. Currently not yet implemented so will throw an error
for all entry points (if flag enabled).
Removed usage of `labs.isSet.bind` across the code, because that breaks
the stubbing of labs by `mockManager.mockLabsEnabled` and
`mockManager.mockLabsDisabled`. `flag => labs.isSet(flag)` should be
used instead.
All email depending tests now disable the `emailStability` feature flag
to keep the tests passing + make sure we still run all the tests for the
old flow while the email stability package is being built.
refs https://github.com/TryGhost/Team/issues/2267
This will eventually be replaced by a Mailgun specific implementation,
but for now we're using mock one which responds with fake data for
local development.
refs https://github.com/TryGhost/Toolbox/issues/475
- Having a single coverage config file for unit and e2e type of tests doesn't play well for tracking coverage progress. The unit test ones are run often when developing, so one can observe the changes in coverage. The e2e tests are ran mostly on CI environment and have different purposes/reach comparing to unit tests.
closes https://github.com/TryGhost/Toolbox/issues/475
- We did not have visibility and history into test coverage statistics in non-unit test suites. This data is useful identifying problematic areas and can be used to keep the code quality under control
- Enabled test coverage for e2e tests, including integration and regression tests
- Decreased the "branches" coverage requirement for the tests to pass (did not want to introduce an additional c8 configuration file just yet)
refs: https://github.com/TryGhost/Ghost/issues/14882
This commit totally removes Bluebird from the importer. Updated `@tryghost/promise` to use native async/await and refactored importer logic to avoid the need of `reflect()`.
fixes https://github.com/TryGhost/Team/issues/2175
- New event type `aggregated_click_event` that is disabled by default in all the existing activity feeds
- This returns click events, but only the first click events for each member/post combination.
- It includes the total count of unique link clicks for that member on that post combination
- Had to resort to some custom knex queries to make this work easily
- Requires `@tryghost/bookshelf-pagination@0.1.31`, included in `@tryghost/bookshelf-plugins@0.6.1` (this fixes an issue with custom selects breaking the total count query of pages)
- Went a bit overboard with the pagination tests to cover as much unknown edge cases as possible
refs: https://github.com/TryGhost/Toolbox/issues/440
This was working locally where the dependency is resolved implicitly, but when deployed there is no @tryghost/data-generator in the node_modules folder.
- up until this commit, git hooks were only used by a handful of people
because they were a pain:
- they'd only be set up when you did `yarn setup`
- the existing hooks ran `yarn lint` on all projects, which was
incredibly slow
- as a result, not many of us actually had them enabled, but this would
cause issues in CI because people were pushing un-linted commits
- other JS projects tend to use husky to automate the git hook setup and
lint-staged to speed up linting on changed files
- this commit switches to using them both
- `lint-staged` only runs `eslint` on staged JS files that are about to
be committed - if there's a linting error, it will stop the commit
- I've configured the pre-commit hook to successfully exit in CI because we
don't want to run pre-commit hooks right now
- this means we can remove Grunt - yay!
refs https://github.com/TryGhost/Toolbox/issues/320
- Header snapshot matching was missing from webhook e2e tests. With a bumped version of webhook-mock-receiver it's now possible to record and match webhook request headers.
closes https://github.com/TryGhost/Team/issues/1877
- bumped `@tryghost/kg-default-cards` which includes updated "should render" dependencies that adds a fully enabled button to the list of possible requirements for the product card to render. Now any one of the following will render the product card:
- title is present
- description is present
- button url is enabled and button text+url are present
refs https://github.com/TryGhost/Toolbox/issues/410
- The 'private' value in 'Cache-Control' response header for all errors made it impossible for shared caches (e.g.: Fastly, Cloudflare) to cache 404 responses efficiently.
- The change substitutes 'max-age=0' which should not effect the browser cache behavior but would allow shared caches to process such requests efficiently.
- A more loose caching logic only applies to 404 responses from GET requests that are not user-specific (non-authenticated, non-cookie containing requests)
refs https://github.com/TryGhost/Toolbox/issues/426
- we're going to need to support more complex combinations of dev
commands soon, with other packages optionally running and env
variables being altered
- this command pulls out a lot of the dev env scripting into a single
scripts
- also cleans up the use of grunt-shell so we can remove the dependency
refs https://github.com/TryGhost/Team/issues/1949
- bumps `@tryghost/kg-default-cards` which updates the rendered output for emails
- added `height: auto` style to the img element so clients don't render the image at the fixed image height retrieved from the `height="x"` attribute
closes https://github.com/TryGhost/Team/issues/1916
closes https://github.com/TryGhost/Team/issues/1917
- Added database storage for link redirects and click events via repositories (hides away database layer) defined in the wrapper services
- Added LinkClickRepository to store click events to database
- Added LinkRedirectRepository to store link redirects to database
- Added PostLinkRepository to link LinkRedirects with posts
- Renamed link-replacement package to link-replacer, and made it dependency less (it only replaces links now, doesn't do anything else)
- The link-tracking service has a new `addTrackingToUrl` which returns a new URL that includes tracking. The new `addRedirectToUrl` method does the same but without tracking for now.
- MEGA service now uses the link-replacer to replace links in the emails using a combination of different services (member attribution + link-tracking service)
no issue
- bumped `@tryghost/url-utils` to get access to the new lexical transform utilities
- updated the Post model's `parse()` and `formatOnWrite()` methods to transform the `lexical` field contents when reading/writing to ensure any links in content point at the correct place with `site.url` config changes
no issue
- added `@tryghost/kg-lexical-html-renderer` dependency
- added `lexical` lib following the same pattern as our `mobiledoc` lib
- updated the Post model's `onSaving` hook to generate the `html` value from `lexical` when present
no issue
- bumped `@tryghost/admin-api-schema` to allow passthrough of the `lexical` property on post and page API endpoints
- prevented saving of blank document in the `mobiledoc` field if `lexical` is provided
- prevented API input containing both `mobiledoc` and `lexical` fields to avoid issues when both are present:
- not possible to know which content is latest/has precedence
- not possible to know which editor should be displayed in Admin
no issue
- By bumping the version of adapter-base-cache I'm expecting `yarn` command to pick up this package. I suspect the failures on CI are due to some caching issue.
https://github.com/TryGhost/Toolbox/issues/364
- When the adapter base class lives deep inside Ghost's codebase it is pretty hard for other developers to extend it. With the goal of making Ghost easier to use and deploy by others, this kind of functionality should be as easy to extend as possible.
- The base adapters should live in the TryGhost/SDK repository. Next ones to move are Scheduling, SSO, and Storage base adapters.
refs https://github.com/TryGhost/Team/issues/1871
This commit adds a test to the serialize method of `post-emaiserializer`. It checks whether the generated email HTML is valid and standard HTML5 and that all properties are escaped.
To do this validation, I depend on the new `html-validate` dev dependency. Just parsing the HTML with a HTML parser is not enough to guarantee that the HTML is okay.
Apart from that this fixes:
- Removed the sanitizeHTML method and replaced it with normal HTML escaping. We don't want to allow any HTML in the escaped fields. Whereas `sanitizeHTML` still allows valid HTML, but we don't want that and want the same behaviour as on the site. E.g., a post with a title `All your need to know about the <br /> tag` should actually render the same title and non-html content, being `All your need to know about the <br /> tag`
- The file, nft and audio card didn't (always) escape the injected HTML fields (new version @tryghost/kg-default-cards)
- `@tryghost/string` is bumped because it contains the new escapeHtml method
closes https://github.com/TryGhost/Team/issues/1873
- bumps `@tryghost/kg-default-cards` which amends the product card rendering to output adjusted `width` and `height` attributes and a resized `src` attribute on the `<img>` element
fixes https://github.com/TryGhost/Ghost/issues/15190
refs https://github.com/TryGhost/framework/pull/76
- log output always uses UTC timestamps, but it may be desirable to
configure logs to use the local machine timezone
- a new config option has been added to `@tryghost/logging` so you can
switch the logs to the local timezone
- this commit bumps the package and sets the default config option to
`false`, so it doesn't suddenly change the timezone of the logs
- docs will be updated soon but if you'd like to use the
timezone-altered timestamps, you can set `logging.useLocalTime` to
`true`
- credits to https://github.com/levee223 for the implementation and PR
- in its current form, bundling will happen before we build Admin
- Admin complains because the version in its package.json for
`@tryghost/members-csv` is different to the one linked in the monorepo
- by putting bundling at the end, we write the new package versions
after we've already built Admin, so this issue should go away
refs TryGhost/Team#1826
- adds new service package that manages all the email alert notifications for free members and paid subscriptions
- includes email templates for free member signup and paid subscription start/cancel
- initializes staff service before members to allow managing email alert notifications
- passes staff service to members api for triggering alerts
refs https://github.com/TryGhost/Team/issues/1808
refs https://github.com/TryGhost/Team/issues/1809
refs https://github.com/TryGhost/Team/issues/1820
refs https://github.com/TryGhost/Team/issues/1814
### Changes in `member-events` package
- Added MemberCreatedEvent (event, not model)
- Added SubscriptionCreatedEvent (event, not model)
### Added `member-attribution` package (new)
- Added the AttributionBuilder class which is able to convert a url history to an attribution object (exposed as getAttribution on the service itself, which handles the dependencies)
```
[{
"path": "/",
"time": 123
}]
```
to
```
{
"url": "/",
"id": null,
"type": "url"
}
```
- event handler listens for MemberCreatedEvent and SubscriptionCreatedEvent and creates the corresponding models in the database.
### Changes in `members-api` package
- Added urlHistory to `sendMagicLink` endpoint body + convert the urlHistory to an attribution object that is stored in the tokenData of the magic link (sent by Portal in this PR: https://github.com/TryGhost/Portal/pull/256).
- Added urlHistory to `createCheckoutSession` endpoint + convert the urlHistory to attribution keys that are saved in the Stripe Session metadata (sent by Portal in this PR: https://github.com/TryGhost/Portal/pull/256).
- Added attribution data property to member repository's create method (when a member is created)
- Dispatch MemberCreatedEvent with attribution
### Changes in `members-stripe-service` package (`ghost/stripe`)
- Dispatch SubscriptionCreatedEvent in WebhookController on subscription checkout (with attribution from session metadata)
- cleaned up unused dependencies
- adds missing dependencies that are used in the code
- this should help us be more explicit about the dependencies a package
uses
refs https://github.com/TryGhost/Toolbox/issues/363
- the oembed service is completely standalone and could do with some
individual unit tests
- moving it out to a package allows us to draw the boundaries better and
allows us to remove some dependencies from the core package.json
- we need to initialize the submodules in case they weren't done when
cloning, otherwise Casper will be missing
- we can avoid doing an initial build because it'll get wiped when we do
`yarn dev` anyway
refs https://github.com/TryGhost/Toolbox/issues/363
- this API framework is standalone and should be pulled out into a
separate package so we can define its boundaries more clearly, and
promote better testing of smaller parts
refs https://github.com/TryGhost/Toolbox/issues/363
- this middleware is standalone and I suspect we're going to be touching
it further when we work on Ghost's caching in the near future
refs https://github.com/TryGhost/Toolbox/issues/363
- this commit pulls all code involving the Mailgun client SDK into one
new package called `mailgun-client`
- this means we should be able to replace `mailgun-js` (deprecated) with
`mailgun.js` (the new, official one) without editing code all over the
place
- this also lays some groundwork for better testing of smaller
components
refs https://github.com/TryGhost/Team/issues/1723
- Added count.replies to comments
- Added replies endpoint
- Limited returned replies to 3.
- Replaced likes_count with count.likes in comments
- Instead of fetching all the likes of a comment to determine the total count, we'll now use count.likes
- Instead of fetching all the likes of a comment to determine whether a member liked a comment, we'll now use count.liked (which returns the amount of likes of the current member, being 0 or 1). This is mapped to `liked` to make it more natural to work with.
The `members.test.snap` file changed because we no longer include `liked: false` if we didn't fetch the liked relation. And in the comments events of the activity feed the liked property is therefore removed.
These changes requires an update to the `bookshelf-include-count` plugin:
- Updated to also work for nested relations
- This moves the count queries from the `bookshelf-include-count` plugin to the `countRelations` method of each model.
- Updated to keep the counts after saving a model (crud.edit didn't return the counts before)
- in order to pack the tarball correctly, we need monobundle to run
beforehand
- up until now, it's the responsibility of the release script to make sure
to run it before `npm pack`
- this commit ensures it gets run, so the generated .tgz file is valid
refs https://github.com/TryGhost/Toolbox/issues/345
- this commit bumps `eslint-plugin-ghost`, which bumps compatiblity to
2022
- this also removes a lot of the manually-added
`parserOptions.ecmaVersion` that we had in imported packages, in favor
of the value set in `eslint-plugin-ghost`
fixes https://github.com/TryGhost/Toolbox/issues/370
- we no longer need `bthreads` because we can use native
`worker_threads` now we don't have to support Node 10 any longer
- this allows us to clean up a dependency and stick with native
libraries
- the referenced node-sqlite3 issue should be fixed (or at least, we now
maintain it so we can fix it if not)
refs https://github.com/TryGhost/Ghost/commit/a58efd6b
The references commit updated the admin-api-schema to require the `trail_days`
property, which is not yet supported by the Admin. When saving membership
setting we also save all the Tiers, which then causes the validation to fail.
Until the Admin supports the property we should remove validation from the API
- most of these tasks were duplicates or had been extracted out into
npm/yarn scripts
- in order to get closer to removing Grunt, I've cleaned up the majority
of those script
- this also removes an intermediate command to building Admin, so it starts
ever so slightly quicker
refs 82dcc042cd
- `coffeescript` was only added in the first place because of an update
to `grunt-bg-shell`, which required it
- since then, we've removed `grunt-bg-shell`, so we don't need this
dependency any further
- this commit switches our `yarn dev` workflow from heavily relying on
Grunt, to using `nodemon` and `concurrently`
- we're doing this to reduce reliance on Grunt, but also to fix several
nits with the way `yarn dev` works in the monorepo
- we now use `nodemon` to run the Ghost backend, and it should
auto-refresh whenever you change a file in any of the packages (except
`admin`)
- we use `concurrently` to simultaneously run `ghost` and `admin` at
the same time. it seems to handle process cleanup well and has nice
colored prefixes to help with differentiating between log output
- this commit ends up removing a handful of Grunt dependencies and
reduces the functionality stored in the Gruntfile
- on the whole, it should keep existing functionality but there may be
some small underlying changes to get used to
- this is only a temporary fix until we have time to go and investigate
some performance issues in the tests
- we keep seeing random failures due to timeouts in the integration
tests, and it's incredibly distracting
- as part of our effort to reduce our usage of Grunt in favor of more
maintainable (and maintained) alternatives, this commit removes Grunt
from Admin
- the main difference here is switching from subgrunt to shell, which
should be a nice stepping stone to removing Grunt altogether one day
refs https://github.com/TryGhost/Toolbox/issues/365
- some of this was barely used anyway, but now Admin is a package in the
monorepo, we don't need to be installing dependencies twice
refs https://github.com/TryGhost/Toolbox/issues/363
- this shared library is standalone, and it used in various places of
Ghost core, so we can pull it out to keep it easier to reason about
- we also use the `html-to-text` dependency in another package but it's
outdated and could now switch to this new package
refs https://github.com/TryGhost/Team/issues/1695
This updates the comment_count helper from a block to inline, and the
frontend script to replace the entire element with the comment count
text. This means that theme designers will have the most flexibility
as they can choose whether or not to wrap the text in an element, as
well as which element.
refs 0a34be4012
- the admin html is no longer stored in core/server, and we don't need to
copy the production file to default.html
- this commit cleans up the grunt command to do this, and removes the
plugin whose only use was doing this
- this takes us another step closer to removing grunt
- this commit switches out the Grunt postcss plugin with the official
postcss CLI
- this means we can remove yet another step from Grunt, which helps
towards our goal of removing Grunt entirely
- I've confirmed the minified output is exactly the same as before
fixes https://github.com/TryGhost/Ghost/issues/14323
- Fixed support for resizing images from Unsplash using the `img-url` helper (previously the size property was ignored for images from Unsplash)
- Added support for `avif` file formats (supported by sharp out of the box)
- Added support for setting the format of images, with a new `format` option:
E.g. to convert an image to webp (only works in combination with size for now, except for Unsplash where you can use it without size):
```
{{img_url @site.cover_image size="s" format="webp"}}
```
This can help improve the performance of a theme, by serving assets in `<picture>` elements with webp and fallback image formats.
Usage example:
```html
<picture>
<source
srcset="{{img_url feature_image size="s" format="avif"}} 300w,
{{img_url feature_image size="m" format="avif"}} 600w,
{{img_url feature_image size="l" format="avif"}} 1000w,
{{img_url feature_image size="xl" format="avif"}} 2000w"
sizes="(min-width: 1400px) 1400px, 92vw"
type="image/avif"
>
<source
srcset="{{img_url feature_image size="s" format="webp"}} 300w,
{{img_url feature_image size="m" format="webp"}} 600w,
{{img_url feature_image size="l" format="webp"}} 1000w,
{{img_url feature_image size="xl" format="webp"}} 2000w"
sizes="(min-width: 1400px) 1400px, 92vw"
type="image/webp"
>
<img
srcset="{{img_url feature_image size="s"}} 300w,
{{img_url feature_image size="m"}} 600w,
{{img_url feature_image size="l"}} 1000w,
{{img_url feature_image size="xl"}} 2000w"
sizes="(min-width: 1400px) 1400px, 92vw"
src="{{img_url feature_image size="xl"}}"
alt="{{#if feature_image_alt}}{{feature_image_alt}}{{else}}{{title}}{{/if}}"
>
</picture>
```
