fixes https://linear.app/tryghost/issue/CORE-45/
- this commit fixes two email related issues:
- SES transport: the auth mechanism was set up wrong and so none of
the requests would go through. This now follows the docs on https://nodemailer.com/transports/ses/
- SMTP transport: the latest versions of Nodemailer don't seem to
allow overriding of options if a service is present. I've filed
https://github.com/nodemailer/nodemailer/issues/1327 but in the
mean time, I assign the options back to the transporter object
to ensure they always get applied
- I've fixed this in our `@trghost/nodemailer` package and so this commit
bumps that here
refs https://github.com/TryGhost/Ghost/security/advisories/GHSA-wfrj-qqc2-83cm
refs https://github.com/advisories/GHSA-48ww-j4fc-435p
- a vulnerability in `nodemailer` means that the `sendmail` transport is
vulnerable to command injection for flags passed to the `sendmail`
binary
- updating to the latest version of Nodemailer required creating
`@tryghost/nodemailer`, which is a wrapper around Nodemailer and
several plugins that used to be in the core
- this commit switches to using that package, and fixes up some small
code + test changes
no issue
- this package has been bumped to support Node 12 + 14
- AFAICT I added it to the Renovate list back when we had some timezone
issues with moment, but we've since pinned the version of moment so we
shouldn't experience that now
- therefore this commit also removes it from the Renovate ignore list
no-issue
The webhook handler did not correctly check for whether or not Stripe
was connected, and would attempt to handle webhooks anyway, resulting in
errors due to missing Stripe config. This fixes the handler to exit
early.
refs https://github.com/TryGhost/Team/issues/1006
The @tryghost/members-api module has been updated to remove webhooks
from Stripe when disconnecting. This will ensure that we do not leave
around old/invalid webhooks that will not be handled and generate
errors.
refs https://github.com/TryGhost/Team/issues/1006
Moving the logic of disconnecting Stripe into the members-api module
decouples the Ghost API from the Members API internals. This method can
now be updated independently of Ghost, to implement the deletion of
webhooks from Stripe.
refs https://github.com/TryGhost/Team/issues/995
Since we reintroduced the comped status, we did not update the
subscription handling to correctly set members to a status of comped
when they were on a 'Complimentary' plan. This meant that 'comped' members
had a status of 'paid'. The changes to @tryghost/members-api ensure that
handling subscriptions going forward will not result in this error.
Since we handle the Complimentary plan correctly now, we do not need to
manually check for the existence of one, we can instead rely on the
status to set the `comped` flag.
no-issue
The ProductRepository changed to require the options parameter which is
not passed when created complimentary subscriptions. This updates the
code to no longer require the options parameter and instead provide a
default.
no-issue
The ProductRepository changed to require the options parameter which is
not passed when created complimentary subscriptions. This updates the
code to no longer require the options parameter and instead provide a
default.
no-issue
When updating products we make many reads and writes to the database,
some of these reads were not happening inside of a transaction which was
causing issues when loading the members setting page. This bumps the
@tryghost/members-api dependency to ensure that all of the database
operations happen inside of the transaction
closes https://github.com/TryGhost/Team/issues/860
refs 5405b6ca7c
- The slow test was running slow because it's not a "unit test" it is testing much more. Moved it to a correct suite - regression which simplified the logic a lot (no need to mock db calls).
- Brought back the 2000ms limit as the bottleneck has been solved
refs https://github.com/TryGhost/Team/issues/860
- Lifting it slightly just so the timeout doesn't waste team's time on false-positive failures. A follow up commit should put the threshold back to the original value and solve the slow test
refs https://github.com/TryGhost/Team/issues/892
- When a published email-only post is edited on the client it sends through a "sent" status over to the backend over PUT endpoint. It's a valid new status that should be accepted by the validation
refs https://github.com/TryGhost/Team/issues/982
When creating a Product with invalid data for Stripe, e.g. a price of
one gazillion dollars - the Stripe API requests would fail, but we would
end up with a broken product created in the database. This updates
@tryghost/members-api to wrap these calls in a database transaction, and
will roll back any operations if one of them fails.
refs https://github.com/TryGhost/Team/issues/986
This updates the @tryghost/members-api module to return the full member
object from getMemberIdentityData, which is used to populate req.member
used by themes to construct the `@member` template data.
The full object is read from the service which handles all additional
properties and logic for retrieving members, including the dummy
subscriptions for comped members.
refs https://github.com/TryGhost/Team/issues/979
This bump to @tryghost/members-api includes a fix for handling a
subscription updating to a new price. Previously we would add the new
product to the member, but the old one would still be attached. Now we
check that there are no other active subscriptions for the product
associated with the old price, and remove it.
refs https://github.com/TryGhost/Team/issues/873
This includes the update to @tryghost/members-api which includes the new
MemberBREADService which is used to handle the logic for controller
methods outside of the controller.
With it, we've introduced the concept of a dummy subscription for comped
members. This gives API consumers a way to get the created_at date for a
comped members access to a product.
no issue
- by default, `c8` will only show stats for files that were loaded during execution
- this means the coverage stats are too high because we're not taking
into account all the files not loaded by tests
- this commit append the `--all` flag to `c8` which fixes this
refs https://github.com/TryGhost/Team/issues/1007
- bumped `@tryghost/kg-default-cards` with updated rendering
- aligns text as well as button
- has single payload toggle for divider display
- adds toggle for button display
- bumps other @tryghost/kg-*` packages that had minor dependency bumps
refs https://github.com/TryGhost/Team/issues/873
This version of @tryghost/members-api handles creating
MemberProductEvents when a member is created, updated or has their
subscription updated. This populates the members_product_events table
and can be used to determine when a member was given access to a
product, which is useful with the concept of comped access to a product,
where we do not have a subscription as a record.
refs https://github.com/TryGhost/Team/issues/946
This adds the initial bulk actions endpoint used for the members
filtering feature. The idea is to eventually move bulk destroy into this
endpoint to and provide a consistent interface for applying bulk actions
to members.
The @tryghost/members-api package has been bumped to include the new
bulkEdit method.
The sinon.restore in tests was moved to an afterEach so that stubs did
not effect other tests.
refs https://github.com/TryGhost/Team/issues/906
- The feature has moved to GA from behind alpha flag. It's skipping the beta phase as it's not needed in this specific situation
no issue
- coverage allows us to see how much of our code we're hitting in tests
and it's time we started to get visibility on this in Ghost
- we can then make informed decisions on which well-tested internal libraries
can be pulled out into their own packages
- this commit:
- adds the c8 dev dependency to Ghost
- prepends the `test:unit` command with c8 with the `text-summary` reporter
- adds a `coverage:unit` command to get the `text` report
- you can do `yarn coverage:unit --reporter html` to get a HTML report etc
- uses this new test coverage reporter command in CI tests
no-issue
The first implementation of bulkDestroy did not pass the search option
unless the filter was also passed. This was incorrect and has been
fixed.
no-issue
This moves the logic out of the controller and into the members-api
member repository. Removing complexity from the controllers and
out into services is desirable to reduce code in the Ghost codebase
and move logic into modules which can be tested easier.
refs https://github.com/TryGhost/Team/issues/944
This updates both libraries to include support for custom foreign keys
on relations, as well as relations with a "virtual" table, defined by a
query.
This will allow us to filter members based on aggregates of their
events, as well as on properties of their stripe subscriptions which are
linked not by the `id` column but by thir external stripe id, e.g. `customer_id`.
refs https://github.com/TryGhost/framework/pull/19
The @tryghost/bookshelf-filter plugin no longer bundles hardcoded
relations and expansion definitions, instead leaving it up to the
library consumer to implement.
This PR adds the preexisting relations and expansions to the relevant
models, in order to preserve our existing filtering functionality.
refs https://github.com/TryGhost/Team/issues/928
- duplicated email template so email-cta changes can go into the labs version
- added `accentContrastColor` to template settings for using white/black depending on the accent color
- added `.gh-btn-accent` styles to the email template (email-cta card already uses those for the button)
refs https://github.com/TryGhost/Team/issues/928
- bumped `kg-default-cards` to version with a fix for `payload.html` being `undefined`
- bumped other `kg-*` packages to that had sub-dependency bumps
refs https://github.com/TryGhost/Team/issues/912
- When the improt acceedes the threshold for the first time we need a way to notify configured escalationAddress to verify the instance owner's email address.
refs https://github.com/TryGhost/Team/issues/927
- bumped `@tryghost/kg-default-cards` to version with button output when rendering
- bumped `@tryghost/kg-card-factory` and `@tryghost/kg-markdown-html-renderer` as they had (unrelated) sub-dependency updates
refs 81b89e9289
- The mark-used-partials util was recording `undefined` partials, it's not anymore
- Also fixed the `normalizePath` function that was breaking when given `undefined`
- This solves the two ends of a bug that prevented some Ghost instances to boot due to a theme with dynamic partials
- this commit bumps GScan to the fixed version
refs https://github.com/TryGhost/Team/issues/909
The request for fetching logged-in member data was making several extra DB queries to fetch data points that are not used on frontend, like `labels`, `products`, `stripe customer`. This refactor removes these extra data points and cuts DB queries in half for fetching logged in member.
refs https://github.com/TryGhost/Team/issues/919
This version of @tryghost/members-api has a patch to the loading of the
benefits relation for products, which previously was erroring and
causing a 500 error.
refs https://github.com/TryGhost/Team/issues/664
The new WellKnownController and middleware handles exposing a JSON Web
Key Set for us.
In order to serve the keys on /members/.well-known/jwks.json without a
trailing slash, we must mount the wellKnown middleware before the
frontend.
refs https://github.com/TryGhost/Team/issues/908
The `cookies` module inside `@tryghost/members-ssr` was being used incorrectly
and resulted in cookies being set with `'[Object object]'`
Co-authored-by: Renovate Bot <bot@renovateapp.com>
issue https://github.com/TryGhost/Team/issues/901
- The main issue was in the AST linter, where partials could have either forward- or backward-slash separators depending on the operating system. We have normalized all partial names to only have forward-slashes, which matches the handlebars syntax, like `{{> icons/avatar}}`.
- The second issue was in the 005-template-compile, where we wouldn't check partials even though they were used. This also comes from the path separator difference.
- We have also updated all regular expressions to account for paths with backward-slashes
refs https://github.com/TryGhost/Team/issues/778
- cleans up the stripe migration to add default monthly/yearly prices for sites, which had a possibility of using complimentary (0 amount prices) in edge cases
- adds missing return in the same migration for an unlikely failure to parse stripe plans
refs 20aea8c819
refs bf587d4055
- The 1000 ms limit was not enough to pass consistently for some slow unit tests. Doubling the previous threshold to be on the safe side of the slow CIs.
- We should limit this back down to 1000ms or less during next cleanup!
refs https://github.com/TryGhost/Team/issues/542
The members-api has been updated to use the passed `created_at`
property when creating members for the status change events, as well as
using the subscription start_date for members paid subscription events
This is a duplicate of 7eb821b3f5 but
with updated dependencies.
refs https://github.com/TryGhost/Team/issues/542
The members-api has been updated to use the passed `created_at`
property when creating members for the status change events, as well as
using the subscription start_date for members paid subscription events
closes https://github.com/TryGhost/Team/issues/607
- patch was added in `bookshelf-include-count@0.1.3` to fix member counts for labels
- bumps `bookshelf-plugins` to include the patch
refs https://github.com/TryGhost/Team/issues/864
- gscan was breaking when parsing double-quoted helpers like `{{"pagination"}}`.
- re-used a function to convert StringLiterals to PathExpressions, which fixes the issue.
refs https://github.com/TryGhost/Team/issues/864
- gscan was breaking when parsing double-quoted helpers like `{{"pagination"}}`.
- re-used a function to convert StringLiterals to PathExpressions, which fixes the issue.
- Move test:regression out of yarn test:all
- Moved lint to be explicitly part of yarn test:all
- Got rid of posttest as that is implicit and hard to read
- postest only runs after exactly yarn test… not after any of the other commands
- but if you’re using yarn test xxx it’s probably to quickly debug something and is when you least want to wait for linting
- Meanwhile, if you’re coding and running unit or acceptance tests before pushing, you’re not seeing lint output
- But you also don't need it twice...
- So for now, running lint just after yarn test:unit - it should be better than the current method
- In an ideal world, our acceptance tests would be much, much faster
- ATM we output how much time is spent on reloading Ghost for each suite, but this is output with console
- Changed this to use DEBUG, so we don't clutter the UI normally
- Added further debug statements, and a cumulative time, so we can see where time is spent/wasted
- Added a DEBUG command for running acceptance tests with this output
- This shows us that reloading Ghost accounts for 50% of the test time
refs https://github.com/TryGhost/Team/issues/858
Previously when complimentary access used a Stripe subscription under
the hood, we were able to restrict access to Stripe Checkout based on
whether or not a Member had an active Stripe subscription. This updates
the logic in members-api to instead check for whether or not the member
has 1-or-more products.
refs: 7e6800b2b8
- I kept flip-flopping on making grunt dev into grunt and removing it so we have to remember to use `yarn dev` and fucked it up entirely
- Do the simple thing for now - and think about how we can make grunt dev better overall
This commit achieves a few things:
- ☑️ No longer having to remember whether a command is yarn something or grunt something
- ☑️ Simplification of tools hopefully making them easier to remember and use
- ☑️ Complete removal of the need for grunt from our test tooling
Several of the tools still use grunt under the hood, but the **entrypoint** should aways be `yarn xxx`.
- `grunt main` -> `yarn main`
- `grunt dev` -> `yarn dev`
- `grunt build` -> `yarn build`
- `grunt test:file-or-folder` -> `yarn test file-or-folder`
- `grunt test-unit` -> `yarn test:unit`
- `grunt test-acceptance` -> `yarn test:acceptance`
- `grunt test-regression` -> `yarn test:regression`
- `grunt validate` -> removed due to lack of use
There is now also `yarn test:all` to run all 3 classes of tests
This PR also reorders & restructures the Gruntfile extensively so that:
- The remaining useful commands are all at the top of the file
- Config and other blah happens after all the useful commands
- All release-only config happens in the release task at the very end of the file
---
DONE:
* Removed all references to npm/bower
* Removed all references to lint / deprecated command
* Moved debug to yarn dev:debug
* Removed all references to travis
* Removed broken help task + useless comment
* Removed unused knex-migrator and clean:test setup tasks
* Added new test commands, removed grunt validate
* Moved stubClientFiles to test utility and use in the few tests that need it
* Used mocha in yarn directly except grunt test:x
* Swapped grunt test for yarn test
* extensive cleanup and reshuffling
refs https://github.com/TryGhost/Team/issues/849
With multiple products back behind alpha flag, we want to bring back segmented post access for members which allows site owners to set post access to specific products. This change -
- updates admin-api-schema to allow `visibility_filter` attribute to post/page response
- We have a lot of tests, so it can be hard to see if any are problematic
- We have a problem with some tests being too slow, this highlights these in particular
- Very slow tests that take seconds are not really unit tests, we should look for different ways to run these
- Also we should tailor our mocha settings more, so that we have more lenient settings for acceptance/regression tests but not for unit tests
- Note: You have to wait for all tests to run to see the output
closes https://github.com/TryGhost/Team/issues/781
refs 076ad99593
refs https://github.com/TryGhost/Ghost/commit/2b73d0f66
refs https://github.com/TryGhost/Ghost/commit/bab12ca3d
- With refs 076ad99593
landing in main there is no more 500 error stopping us from bumping the version
- Previous versions of gscan wasn't able to detect partial usage inside
of a helper, e.g.: "{{#if author}}{{> "missing_partial"}}{{/if}}". It's
important to be able to detect these on theme upload/validation stage to
avoid errors during the runtime.
- The updated version provides additional information about used and
unused partials and helpers.
- With this change users might start seeing a "fatal" error during an
upload/activation of a theme previously passing validation. Currently
active themes with partials used in the context will still allow for a
boot, but will log a 422 error pointing to missing partial.
refs https://github.com/TryGhost/Team/issues/827
- Ghost needs an ability to insert email-only content for defined segments of members - at the moment distinguishing only free&paid ones.
- The card with a "segment" payload should contain a valid NQL statement to target specific members
- Makes the logic for determining the admin and frontend vhost args independent and easier to test
- Moved the tests to specifically test the vhost utils & removed proxyquire as a dependency
- We want to breakdown the current parent app into the existing core/app.js and boot code, allowing us to decouple the backend and frontend further
- This is all part of the refactoring to separate server and frontend completely
- We keep having test issues caused by yarn, these fix commands help quickly check it's not a yarn problem
- Have renamed fixmodulenotrequired to fix:client to make a nice consistent naming pattern that's easy to remember and type
refs https://github.com/TryGhost/Team/issues/765
This includes the changes to @tryghost/members-csv so that we can read
and write the `product` column from/to csv files. Allowing us to include
products in exports, as well as use them in imports.
- getSubdir, getSiteUrl & getAdminUrl were currently part of @tryghost/url-utils
- They have been split out into their own library, and refactored so that they expect to be bound to nconf
- With this commit we can do e.g. config.getSubdir() rather than needing @tryghost/url-utils
- These functions will be passed to url-utils via DI
- This is the first step in breaking down url-utils into smaller pieces
- This commit only does a single change in Gruntfile.js to use the new funtions - this will be rolled out slowly
refs https://github.com/TryGhost/Team/issues/748
This updates the @tryghost/members-api MemberRepository to stop ignoring
the `products` data passed to write operations, and to attach products
directly to members. As this logic is part of a new feature, we are
maintaining existing functionality by deleting the products data when
the feature flag is not enabled.
This functionality allows us to give members complimentary access to a
product without needing to use a Stripe Subscription internally.
refs https://github.com/TryGhost/Team/issues/781
refs https://github.com/TryGhost/Team/issues/774
- Previous versions of gscan weren't able to detect partial usage inside
of a heprer, e.g.: "{{#if author}}{{> "missingpartial"}}{{/if}}". It's
important to be able to detect these on theme upload/validation stage to
avoid errors during the runtime.
- The updated version provides additional information about used and
unused partials and helpers.
- With this change users might start seeing a "fatal" error during an
upload/activation of a theme previously passing validation. Currently
active themes with partials used in the context will still allow for a
boot, but will log a 422 error pointing to missing partial.
no issue
Fixes the build - as test checks for instanceof ValidationError, and
we were using 2 versions of @tryghost/errors between Ghost and
@tryghost/validator.
refs https://github.com/TryGhost/Team/issues/775
As we currently do not delete canceled subscriptions and they are
exposed via the API, this functionality has been added to the
editSubscription controller method under the PUT HTTP method.
The cancelSubscription method in @tryghost/members-api was updated to
handle deleting by member id
- Part of the effort to split Ghost down into smaller, decoupled pieces
- Moved out our internal validator tooling to a separate library
- Replaced all usage of our own tooling and validatorjs directly with @tryghost/validator
- Removed the validatorjs dependency and removed the renovate pin
- This gives us a consistant, smaller, clearer public API for validations
- It will eventually be used on Ghost Admin too
- This way we can start getting up to date with validator whilst not increasing build size
no issue
The only pieces of Ghost-Ignition used in Ghost were debug and
logging. Both of these modules have been superceded by the Framework
monorepo, and all usages of Ignition have now been removed, replaced
with @tryghost/debug and @tryghost/logging.
- validator has seen a lot of change, this one bump is safe
- after this we need to go through and make API updates as isURL has changed quite significantly!
no issue
Part of the effort to split ghost into smaller, decoupled parts. The
@root-utils package lets us avoid hard-coding a path to package.json,
and means that the ghost-version.js file could eventually be moved
into a separate module.
This commit uses a patched version of @tryghost/root-utils which
checks for the existence of a `current` directory, as used in
Ghost-CLI. Since this is very specific to Ghost and Ghost CLI, there's
a new method called "getGhostRoot" for this purpose.
refs:
- cf15f60085
- dd20cc649b
- ccf27f7009
- abf146d61f
- 2b54c92a14
- bb029a53f6
- 95bd7ee675
- 9018b4df22
- df01a6e5f4
- d313726b34
- these plugins were in a state where they were independent enough to be
pulled out into their own packages, which is what we did in the
referenced commits above
- each package is named like `@tryghost/bookshelf-<plugin>`
- to avoid requiring multiple packages into Ghost, we've also created a
wrapper package called `@tryghost/bookshelf-plugins` which re-exports
all these plugins, so the changes in Ghost are very simple - dbebdd43b5
- this commit deletes the plugins + tests, and replaces with our new
package with some minor code changes
no issue
Part of the effort to split ghost into smaller, decoupled parts. The
@root-utils package lets us avoid hard-coding a path to package.json,
and means that the ghost-version.js file could eventually be moved
into a separate module.
refs https://github.com/TryGhost/Team/issues/770
We want post feature image functionality to better match what's available inside the editor, to do that we'll need somewhere to store alt and caption meta data. `posts_meta` chosen because even though we want to make this generic for other tables in the future those tables also have a `feature_image` (or closely related) field.
- updated schema with new columns
- added migration to create columns
- cleaned new columns from API output
- not output on v2/v3
- conditionally output on v4/canary output based on labs flag
- bumped `@tryghost/admin-api-schema` to allow new columns through in canary API requests
- silently clean properties from input when labs flag is disabled
- updated acceptance tests so they fail if `admin-api-schema` is not letting the new fields through
- The underlying package-json package has had i18n ripped out using the new tpl utility instead
- It's also then been refactored to not be a class that needs instantiating
- This means it can be required directly and its public interface methods used where needed
- This is a much nicer, neater pattern for what is a mature utility library :)
refs 971ac479dc
- `@tryghost/tpl` was accidentally placed in `devDependencies` but
Ghost needs this to run so it would cause Ghost-CLI to fail when it
only installs `dependencies`
- this commit moves the dependency to `dependencies`
- also pins the dependency to `0.1.0`
- We are going to get rid of the internal i18n tool because it doesn't solve a real use case
- Instead, we have a new tpl utility that does basic string interpolation
- This pattern will make it easier for us to decouple the codebase, and the new tool helps to keep the refactor surface area really small
- This is the first example of using the new tpl helper, so it also adds @tryghost/tpl
refs https://github.com/TryGhost/Team/issues/753
This bumps members-api to handle migration to revert portal settings to named values again as well as handle migration cleanup for old portal settings.
no issue
- `cssnano` requires `postcss` as a peer dependency which was causing warnings when running `yarn`
- `postcss` was available because it's a sub-dependency of `sanitize-html` so nothing was broken but explicitly adding it to `package.json` resolves the warnings
refs https://github.com/TryGhost/Team/issues/712
The ProductRepository in members-api has been updated to accept
monthly_price & yearly_price when creating and updating products, as
well as including them when reading.
refs https://github.com/TryGhost/Team/issues/728
- The code of update check has been extracted into it's own package as a part of TryGhost/Core monorepo. This commit is a cleanup of the leftover files
refs https://github.com/TryGhost/Team/issues/711
Since we need to support multiple products, which will each have a
monthly and yearly price, we are moving away from storing these ids in
settings, and instead in a column on the products table. The migration
depends on the settings being prepopulated, which is done outside of
knex-migrator, in members-api, so this migration must also be done there.
no issue
- includes sub-dependency bumps for `markdown-it` packages for markdown renderer bug fixes and to keep Ghost and Admin markdown rendering in sync
refs https://github.com/TryGhost/Team/issues/704
Previously a customer message was not passed, and the default message
was used. This made a reference to, "someone else is editing the post"
which is incorrect in this context! The members-api has been updated
with an appropriate error message, with a link to our documentation.
refs https://github.com/TryGhost/Team/issues/704
Currently when attempting to create stripe_prices without a Stripe
connection, it will fail silently. This is an issue when initially
configuring Members as the Stripe connection can take some time to be
established. By erroring we allow the client to be notifed that the
connection does not yet exist, so that it can be retried later.
refs f51c640fb0
- This should not affect any of the current code but should improve edge case errors related to "allowlist" types of limits
- See referenced commit for details when this edgecase was happening
refs https://github.com/TryGhost/Team/issues/635
This is to ensure we don't break migrations for any sites which have
imported external subscriptions which have an interval of 'week' or
'day'
The bump to members-api includes the handling of these intervals for
ongoing population of mrr events
refs https://github.com/TryGhost/Team/issues/698
The migrations to populate these settings are handled by the members-api
module, as they depend on the stripe_prices table being populated. This
cannot be guarunteed at boot, so we're unable to do this as a standard
migration.
refs https://github.com/TryGhost/Team/issues/687
- The frontmatter field has leaked into the API layer unintentionally when it was introduced into the DB schema during 4.0 release.
- The fix add the field to "trim" list in all API. A proper validation and handling will be add per API as usecase for the field becomes clear
