fixes https://github.com/TryGhost/Team/issues/481
This change fixes an issue when multiple images with the same name are
uploaded in parallel. The current system does not guarantee that the
original filename is stored under NAME+`_o`, because the upload for the
original file and the resized file are happening in parallel.
Solution:
- Wait for the storage of the resized image (= the image without the _o
suffix) before storing the original file.
- When that is stored, use the generated file name of the stored image
to generate the filename with the _o suffix. This way, it will always
match and we don't risk both files to have a different number suffix.
We'll also set the `targetDir` argument when saving the file, to avoid
storing the original file in a different directory (when uploading a
file around midnight both files could be stored in 2023/01 and 2023/02).
Some extra optimisations needed with this fix:
- Previously when uploading image.jpg, while it already exists, it would
store two filenames on e.g., `image-3.jpg` and `image_o-3.jpg`. Note the
weird positioning of `_o`. This probably caused bugs when uploading
files named `image-3.jpg`, which would store the original in
`image-3_o.jpg`, but this original would never be used by the
handle-image-sizes middleware (it would look for `image_o-3.jpg`). This
fix would solve this weird naming issue, and make it more consistent.
But we need to make sure our middlewares (including handle-image-sizes)
will be able to handle both file locations to remain compatible with the
old format. This isn't additional work, because it would fix the old bug
too.
- Prevent uploading files that end with `_o`, e.g. by automatically
stripping that suffix from uploaded files. To prevent collisions.
Advantage(s):
- We keep the original file name, which is better for SEO.
- No changes required to the storage adapters.
Downside(s):
- The storage of both files will nog happen parallel any longer. But I
expect the performance implications to be minimal.
- Changes to the routing: normalize middleware is removed
refs https://github.com/TryGhost/Toolbox/issues/503
- The Dynamic URL service no longer generates "url.added" event when only a partial resource update happened - only non-url forming properties were modified. The sitemaps service still needs to know when to update the lastmod ("Last Modified") field associated with specific URL.
refs https://github.com/TryGhost/Toolbox/issues/501
- this reverts commit 48dda23554
- also includes a resolution for `@elastic/elasticsearch` so we don't
run a version that is potentially problematic - see referenced issue
for context
refs https://github.com/TryGhost/Team/issues/2416
This extends the mock API to use a more formal pattern of moving our
entity code into a separate package, and use the service/repository
patterns we've been work toward.
The repository is currently in memory, this allows us to start using
the API without having to make commitments to the database structure.
We've also injected a single fake webmention for testing. I'd expect
the Mention object to change a lot from this initial definition as we
gain more information about the type of data we expect to see.
refs https://github.com/TryGhost/Toolbox/issues/499
refs 6bcc47a0ad
- Using module directly caused issues with snapshots manager instance initialization (mocha hooks did not apply to a correct instance)
- See refed commit for more
refs https://github.com/TryGhost/Toolbox/issues/499
- Outgoing emails have been a weak point of Ghost's stability recently. The concept of "emailMockReceiver" similarly to "webhookMockReceiver", allows to test side-effects like outgoing emails.
- This is a first iteration which should lay groundwork for testing all outgoing emails in the future
- The change adds a new concept of "email mock receiver" which is very similar to how the "webhook mock receiver" works. The email mock receiver exposes two methods to record and verify snapshots:
- matchHTMLSnapshot - records and verifies only the HTML content of the outgoint email
- matchMetadataSnapshot - records and verifies all the non-HTML properties sent along an email content, e.g.: to address, plaintext, subject, etc.
- What's missing is matching content based on dynamic content like dates, links with JWT tokens, etc.
closes https://github.com/TryGhost/Toolbox/issues/497
- The classification of fatal/non-fatal errors has been updated to only be fatal when causing page renders with 5xx or 4xx responses.
- Some of the rules checking Ghost 5.x compatibility have been relaxed to only be "error" with the gscan version bump
- You can find more details on which exact rules were relaxed in the gscan's commit log - https://github.com/TryGhost/gscan/compare/v4.35.1...v4.36.0
refs https://github.com/TryGhost/Toolbox/issues/488
- Node 18 is now LTS so we're adding support for it
- this adds Node 18.12.1 (the latest security release) to our supported
ranges and CI
- this was all getting terribly behind so I've done several things:
- majority of `@tryghost/*` except Lexical packages
- gscan + knex-migrator to remove old `@tryghost/errors` usage
- bumped lockfile
refs https://github.com/TryGhost/Team/issues/2370
Due to a possible bug in either `@sentry/node` (mainly the Express
middlewares and the usage of deprectated Domain) and Node v16+,
unhandled promise rejections are transformed into uncaught exceptions
and cause Ghost to crash in unexpected situations.
Reverting to `v7.11.1` fixes this (but definitely not ideal at all)
because errors are caught in the Express middleware.
Reproduction repo:
https://github.com/SimonBackx/sentry-node-unhandled-rejection-crash
refs https://github.com/TryGhost/Toolbox/issues/479
- this includes a handful of improvements to get Playwright working on a
local environment including:
- adding `testing-browser` environment so we don't nuke `development`
environments, and makes all the necessary changes to get Ghost to
behave when this is running
- stopped running one global instance of Ghost as this doesn't provide
a clean environment
- copies a few default fixtures that are needed for the new
environment
no issue
Local tests can now setup Stripe during the global setup process, and the webhook server is run out-of-process.
Running tests in CI against localhost will use environment variables to setup Stripe.
Providing a test URL will avoid setting up Stripe and will assume that it is already done.
fixes https://github.com/TryGhost/Team/issues/2346
- Adds email batch browse endpoint
- Adds email recipient failures browse endpoint
- Adds new fixtures and E2E tests for the new API
- Added support for snapshot tests to have 'nullable' types.
refs https://github.com/TryGhost/Team/issues/2225
- updated the `formatOnWrite` transform map for posts to include the new `nodes` and `transformMap` options used by `urlUtils` for transforming node payload data
- added `nodes` to the `lexicalLib` module that pulls in our default nodes to be passed in to the URL transform utilities
- added `urlTransformMap` to the `lexicalLib` module that maps transform type and data type to URL transform utility functions that accept a single URL argument
refs: https://github.com/TryGhost/Toolbox/issues/479
Framework includes:
* command to run tests
* command to record tests
* mechanism for starting and stopping Ghost before and after each suite of tests
* mechanism for loading fixtures into Ghost before starting tests
* sample test for controlling Ghost Admin
fixes https://github.com/TryGhost/Team/issues/2282
Added a new email service package that is used when the email stability
flag is enabled. Currently not yet implemented so will throw an error
for all entry points (if flag enabled).
Removed usage of `labs.isSet.bind` across the code, because that breaks
the stubbing of labs by `mockManager.mockLabsEnabled` and
`mockManager.mockLabsDisabled`. `flag => labs.isSet(flag)` should be
used instead.
All email depending tests now disable the `emailStability` feature flag
to keep the tests passing + make sure we still run all the tests for the
old flow while the email stability package is being built.
refs https://github.com/TryGhost/Team/issues/2267
This will eventually be replaced by a Mailgun specific implementation,
but for now we're using mock one which responds with fake data for
local development.
refs https://github.com/TryGhost/Toolbox/issues/475
- Having a single coverage config file for unit and e2e type of tests doesn't play well for tracking coverage progress. The unit test ones are run often when developing, so one can observe the changes in coverage. The e2e tests are ran mostly on CI environment and have different purposes/reach comparing to unit tests.
closes https://github.com/TryGhost/Toolbox/issues/475
- We did not have visibility and history into test coverage statistics in non-unit test suites. This data is useful identifying problematic areas and can be used to keep the code quality under control
- Enabled test coverage for e2e tests, including integration and regression tests
- Decreased the "branches" coverage requirement for the tests to pass (did not want to introduce an additional c8 configuration file just yet)
refs: https://github.com/TryGhost/Ghost/issues/14882
This commit totally removes Bluebird from the importer. Updated `@tryghost/promise` to use native async/await and refactored importer logic to avoid the need of `reflect()`.
fixes https://github.com/TryGhost/Team/issues/2175
- New event type `aggregated_click_event` that is disabled by default in all the existing activity feeds
- This returns click events, but only the first click events for each member/post combination.
- It includes the total count of unique link clicks for that member on that post combination
- Had to resort to some custom knex queries to make this work easily
- Requires `@tryghost/bookshelf-pagination@0.1.31`, included in `@tryghost/bookshelf-plugins@0.6.1` (this fixes an issue with custom selects breaking the total count query of pages)
- Went a bit overboard with the pagination tests to cover as much unknown edge cases as possible
refs: https://github.com/TryGhost/Toolbox/issues/440
This was working locally where the dependency is resolved implicitly, but when deployed there is no @tryghost/data-generator in the node_modules folder.
- up until this commit, git hooks were only used by a handful of people
because they were a pain:
- they'd only be set up when you did `yarn setup`
- the existing hooks ran `yarn lint` on all projects, which was
incredibly slow
- as a result, not many of us actually had them enabled, but this would
cause issues in CI because people were pushing un-linted commits
- other JS projects tend to use husky to automate the git hook setup and
lint-staged to speed up linting on changed files
- this commit switches to using them both
- `lint-staged` only runs `eslint` on staged JS files that are about to
be committed - if there's a linting error, it will stop the commit
- I've configured the pre-commit hook to successfully exit in CI because we
don't want to run pre-commit hooks right now
- this means we can remove Grunt - yay!
refs https://github.com/TryGhost/Toolbox/issues/320
- Header snapshot matching was missing from webhook e2e tests. With a bumped version of webhook-mock-receiver it's now possible to record and match webhook request headers.
closes https://github.com/TryGhost/Team/issues/1877
- bumped `@tryghost/kg-default-cards` which includes updated "should render" dependencies that adds a fully enabled button to the list of possible requirements for the product card to render. Now any one of the following will render the product card:
- title is present
- description is present
- button url is enabled and button text+url are present
refs https://github.com/TryGhost/Toolbox/issues/410
- The 'private' value in 'Cache-Control' response header for all errors made it impossible for shared caches (e.g.: Fastly, Cloudflare) to cache 404 responses efficiently.
- The change substitutes 'max-age=0' which should not effect the browser cache behavior but would allow shared caches to process such requests efficiently.
- A more loose caching logic only applies to 404 responses from GET requests that are not user-specific (non-authenticated, non-cookie containing requests)
refs https://github.com/TryGhost/Toolbox/issues/426
- we're going to need to support more complex combinations of dev
commands soon, with other packages optionally running and env
variables being altered
- this command pulls out a lot of the dev env scripting into a single
scripts
- also cleans up the use of grunt-shell so we can remove the dependency
refs https://github.com/TryGhost/Team/issues/1949
- bumps `@tryghost/kg-default-cards` which updates the rendered output for emails
- added `height: auto` style to the img element so clients don't render the image at the fixed image height retrieved from the `height="x"` attribute
closes https://github.com/TryGhost/Team/issues/1916
closes https://github.com/TryGhost/Team/issues/1917
- Added database storage for link redirects and click events via repositories (hides away database layer) defined in the wrapper services
- Added LinkClickRepository to store click events to database
- Added LinkRedirectRepository to store link redirects to database
- Added PostLinkRepository to link LinkRedirects with posts
- Renamed link-replacement package to link-replacer, and made it dependency less (it only replaces links now, doesn't do anything else)
- The link-tracking service has a new `addTrackingToUrl` which returns a new URL that includes tracking. The new `addRedirectToUrl` method does the same but without tracking for now.
- MEGA service now uses the link-replacer to replace links in the emails using a combination of different services (member attribution + link-tracking service)
no issue
- bumped `@tryghost/url-utils` to get access to the new lexical transform utilities
- updated the Post model's `parse()` and `formatOnWrite()` methods to transform the `lexical` field contents when reading/writing to ensure any links in content point at the correct place with `site.url` config changes
no issue
- added `@tryghost/kg-lexical-html-renderer` dependency
- added `lexical` lib following the same pattern as our `mobiledoc` lib
- updated the Post model's `onSaving` hook to generate the `html` value from `lexical` when present
no issue
- bumped `@tryghost/admin-api-schema` to allow passthrough of the `lexical` property on post and page API endpoints
- prevented saving of blank document in the `mobiledoc` field if `lexical` is provided
- prevented API input containing both `mobiledoc` and `lexical` fields to avoid issues when both are present:
- not possible to know which content is latest/has precedence
- not possible to know which editor should be displayed in Admin
no issue
- By bumping the version of adapter-base-cache I'm expecting `yarn` command to pick up this package. I suspect the failures on CI are due to some caching issue.
https://github.com/TryGhost/Toolbox/issues/364
- When the adapter base class lives deep inside Ghost's codebase it is pretty hard for other developers to extend it. With the goal of making Ghost easier to use and deploy by others, this kind of functionality should be as easy to extend as possible.
- The base adapters should live in the TryGhost/SDK repository. Next ones to move are Scheduling, SSO, and Storage base adapters.
refs https://github.com/TryGhost/Team/issues/1871
This commit adds a test to the serialize method of `post-emaiserializer`. It checks whether the generated email HTML is valid and standard HTML5 and that all properties are escaped.
To do this validation, I depend on the new `html-validate` dev dependency. Just parsing the HTML with a HTML parser is not enough to guarantee that the HTML is okay.
Apart from that this fixes:
- Removed the sanitizeHTML method and replaced it with normal HTML escaping. We don't want to allow any HTML in the escaped fields. Whereas `sanitizeHTML` still allows valid HTML, but we don't want that and want the same behaviour as on the site. E.g., a post with a title `All your need to know about the <br /> tag` should actually render the same title and non-html content, being `All your need to know about the <br /> tag`
- The file, nft and audio card didn't (always) escape the injected HTML fields (new version @tryghost/kg-default-cards)
- `@tryghost/string` is bumped because it contains the new escapeHtml method
closes https://github.com/TryGhost/Team/issues/1873
- bumps `@tryghost/kg-default-cards` which amends the product card rendering to output adjusted `width` and `height` attributes and a resized `src` attribute on the `<img>` element
