Commit Graph

582 Commits

Author SHA1 Message Date
Nazar Gargol
d2fbe327e4 Update Stripe concurrency in members importer
no issue

- There were many failed import records due to rate-limit errors. With concurrency of 9 imports go through with 100% success
- Would need to verify these limits with live API to make the most of it
2020-08-13 21:04:57 +12:00
Nazar Gargol
78b4dff656 Moved batching logic inside the members importer module
no issue

- This way importer is more self contained and controller logic doesn't have to know about batch sizes and other unecessary variables
2020-08-13 20:31:11 +12:00
Kevin Ansfield
c434666ba2 🐛 Removed [http://url/] output in member email preview text
no issue

- we output the post excerpt in a hidden div in the email template so that email clients pick it up as the "preview" text when listing emails
- when no custom excerpt is provided the preview text is grabbed from post.excerpt which is the first 500 chars of the post.plaintext value
- post.plaintext formats links as "Link [http://url/]" which is unwanted in html email previews

- add a basic replacement to the post email serializer to remove any `[http://url/]` occurrences from the post excerpt before rendering the email content
2020-08-12 20:14:06 +01:00
Kevin Ansfield
163092f377 🐛 Fixed apostrophes not displaying correctly in Outlook for member emails
refs https://github.com/TryGhost/Ghost/issues/11536

- Outlook supports `'` as a special char for apostrophes but not `&#apos;` which is what cheerio/juiced render
- adds a basic string placement to the email serializer to switch to the older style of special char
2020-08-12 20:01:42 +01:00
Kevin Ansfield
c7ff4c9e93 Moved email sending to the background job queue
no issue

- moves the meat of `pendingEmailHandler()` code into a new function `sendEmailJob()` that is passed over to the new job service
- lets the server keep processing email generation and sending when it receives a shutdown request rather than halting processing mid-send and ending up in a partial state
2020-08-12 17:02:14 +01:00
Fabien 'egg' O'Carroll
1294e3f92c
Replaced all usage of member models with members-api (#12117)
no-issue

* Added stripeSubscriptions relation to member model

This allows us to fetch the subscriptions for a member via standard
model usage, e.g. `withRelated: ['stripeSubscriptions']` rather than
offloading to loops and `decorateWithSubscriptions` functions, this is
more performant and less non-standard than the existing method.

* Updated serialize methods to match existing format

The current usage of `decorateWithSubscriptions` and the usage of
members throughout the codebase has a subscriptions array on a stripe
object on the member, this ensures that when we serialize members to
JSON that we are using the same format.

There is definitely room to change this in future, but this is an
attempt to create as few breaking changes as possible.

* Installed @tryghost/members-api@0.26.0

This includes the required API changes so that everywhere can use
members-api directly rather than models and/or helper methods
2020-08-12 14:17:44 +01:00
Nazar Gargol
c696d715c1 Extracted batched member import into separate module
no issue

- The code in controller was becoming hard to reason about.
- Having a single module shows exactly how many dependencies are there to do an import for single batch.
- Having a separate module would make it easier to extract into it's own package in Members monorepo
2020-08-12 20:18:30 +12:00
Hannah Wolfe
bbbd011074 Added job-manager & wired up shutdown and testmode
- Bottom line - we need to manage shutting down gracefully when doing long-running tasks
- To achieve that, we're going to use job queues

In this commit:
- added new @tryghost/job-manager dependency
- added a minimal job service, that handles in passing things like logging and (maybe later) config
- job service is wired up to server shutdown, so that the queue finishes before the server exits
- also added a new job endpoint to testmode so that it's easy to test job behaviour without needing to do real work
2020-08-11 21:31:34 +01:00
Daniel Lockyer
5b471e1bbe Extracted promise libs and history into @tryghost/promise
- deleted files under `core/server/lib/promise` and related test files
- added `@tryghost/promise` as a dependency
- fixed all local requires to point to the new package
2020-08-11 18:44:21 +01:00
Daniel Lockyer
c9a5b28669 Extracted core/server/lib/security to @tryghost/security package
- code and tests were extracted out to this package
- deletes these files
- replaces all local requires, and adds it as a dependency
2020-08-11 14:06:50 +01:00
Daniel Lockyer
8799feb801 Replaced constants file with @tryghost/constants
- extracted constants file into a new package
- replaced all local requires of the file with new package
2020-08-11 12:51:16 +01:00
Kevin Ansfield
2efcf94645
Improved performance of sending newsletter emails (#12091)
no-issue

- switch from `membersService.api.members.list` to using bookshelf `Member.findPage()` with the `{paid: true}` filter to avoid per-member queries (N+1) to decorate members with subscriptions and a heavy post-fetch filter via `contentGating`
- add concurrency to the Mailgun API requests in `bulk-email` service to reduce overall time submitting API requests
- add debug statements with timing output for easier measurements
2020-08-06 15:19:39 +02:00
Gerardo Lopez Dueñas
275bf41ba3
Fixed bookmark card image size in fastmail (#11970)
closes #11907

The image in the bookmark card was being shown out of the bounds of
the card because of a general style `height: auto !important`.

I added a new `max-height` property to the image to avoid exceeding
parent height.
2020-08-03 18:38:45 +01:00
Kevin Ansfield
29d94e7814 Fixed mailgun config not allowing custom hosts with ports
no issue

- `mailgun()` expects the `host` option not to include a port but `url.host` will include the port, we instead want to use `url.hostname` which skips the port
2020-07-30 17:28:51 +01:00
Hannah Wolfe
436db4ec3b Updated host limit message to be clearer
- limits are based on total members not number of members that will be emailed
2020-07-26 20:49:30 +01:00
Hannah Wolfe
92446d85ea Changed member limit to be DRY & use raw query
- Member limit code was duplicated in 2 places unnecessarily
- Also used member api code that fetched members and subscriptions fully hyrated when we only need a count
- Using a raw query significantly improves performance here
2020-07-26 20:49:30 +01:00
Hannah Wolfe
3491e60c9d Added config to send bulk email in testmode
- mailgun has a testmode flag we can use to get email to be accepted but not delivered
- this is useful for developers testing general bulk email code - not for users - so it is only available via config
2020-07-24 11:55:34 +01:00
Rish
2ac69e637e Added publication icon to members site data
refs https://github.com/TryGhost/members.js/issues/72

- Portal is using using publication logo from settings for signup/signin pages
- Instead, we are switching to using publication icon from settings, which also needs to be passed in site data API
2020-07-23 17:04:44 +05:30
Fabien 'egg' O'Carroll
69080824de
🐛 Ensured webhooks are created once (#12075)
refs #12074

Since we've split members settings into multiple keys the
reconfiguration of the members-api has been happening in quick
succession as the stripe_connect_* settings are all set at once.

This debounces the call to reconfigure the members-api so that we only
need to instantiate it once.
2020-07-22 12:27:05 +02:00
Rish
3d164d222b Updated error name check in GhostMailer
no issue
refs e8511d0568

- Adds extra check for empty `err` object while checking error name for status code in GhostMailer
2020-07-21 01:08:19 +05:30
Rish
e8511d0568 Updated error code for incorrect recipients in GhostMailer
no issue

- By default, GhostMailer throws EmailError with statusCode as `500` for any failure in sending mail
- In case of failure due to `RecipientError`, status code as now correctly sent as `400` as its a bad request and not an error we can't handle.
2020-07-21 01:05:21 +05:30
Rish
90b39fbb9a Updated status and error message for newsletter email failures
refs https://github.com/TryGhost/Ghost/issues/11971

- Added statusCode from bulk email provider to API response
- Updated error messages for different bulk email(mailgun) failure states
- Added `context` to preview mail API error message with mail provider's error message
2020-07-17 13:54:09 +05:30
Rishabh Garg
ead073ad68
Updated sender email to existing "from-address" (#12044)
refs #12043

- On updating From-address from Members settings in Labs, we send a confirmation email to the updated address with magic link for verification
- Previously, no explicit sender email was being set for this so fallback config address was used
- This updates the sender address to use the current from address for "from-address" update emails
2020-07-13 17:57:31 +05:30
Fabien O'Carroll
06c7152f4b 🐛 Fixed Stripe webhooks for subdirectory setups
no-issue

- Added breaking test for webhook url including subdirectory

- Previously the webhook handler URL was generated incorrectly when
  running Ghost on a subdirectory, appending the path to the root of the
  host, this fix ensures that the subdirectory is included before the
  path.
2020-07-10 17:43:46 +02:00
Fabien O'Carroll
832ecad43b Improved handling of Stripe webhooks on boot
no-issue

This version of members-api includes changes to how webhooks are
managed, previously they would be deleted and recreated on every boot of
Ghost. Now they are created and the secret is persisted, on boot the
webhook is updated to the most current url and events. If the api
version is wrong or the update fails, the webhook is deleted and
recreated and the settings updated.

 - Installed @tryghost/members-api@0.24.0
 - Updated config to work with 0.24.0
2020-07-10 17:43:46 +02:00
Fabien 'egg' O'Carroll
e14188807d Fixed express-jwt config
no-issue

express-jwt expects an array of valid algorithms, not a single algorithm.
2020-07-08 15:09:44 +02:00
Rish
c5d360e647 Fixed members plan currency symbol
no issue

The refactor to new settings lost the uppercasing of the currency string, this puts it back
2020-07-08 10:47:28 +05:30
Fabien O'Carroll
694d92d270 Added guards for stripe_plans setting being null
no-issue

There is concern that the settings cache can return `null` for values
which it cannot parse correctly, this just ensures that we always have
an array where we expect one
2020-07-07 13:10:17 +02:00
Rish
8d989bd3c3 Updated members site data to include new portal settings
no issue

- Adds new portal button settings to members site data for portal script
- Updates settings input/output serializers to handle portal icon image url
2020-07-07 14:06:04 +05:30
Daniel Lockyer
8956a9df35 Merged 3.22.2 into master
* tag '3.22.2':
  v3.22.2
  Updated Ghost-Admin to v3.22.2
  Emitted all settings events on reinit of cache (#12012)
  🐛 Updated access to be true by default in v3 API
  Hardened members subscription migration against missing data (#12009)
2020-07-06 16:19:55 +01:00
Fabien 'egg' O'Carroll
8b6ec4d922
Emitted all settings events on reinit of cache (#12012)
closes #12003

There are a few parts of Ghost that rely on the settings events being
emitted anytime a setting is changed, so that the data is kept in sync.

When a setting is renamed in a migration essentially what happens is
that the settings value is changed from a default value to its actual
value, but this does no emit an event.

Anything that is initialised before migrations have run that relies on
the events to keep it up to date will have stale data - e.g. the themes
i18n service.

This change ensures that when we `reinit` after migrations have been
run, we emit events for every setting to tell the rest of Ghost that it
has changed.
2020-07-06 17:09:43 +02:00
Fabien O'Carroll
5e6a4f6f7d Updated bulk-email service to use mailgun settings
refs #10318
2020-07-03 11:48:47 +02:00
Kevin Ansfield
fd4a011995
🐛 Fixed settings cache being out of sync after migrations (#11987)
refs https://github.com/TryGhost/Ghost/issues/10318

- re-initialize settings cache after migrations by shutting down to clean up event listeners then and calling `init` again
- important to ensure `db.ready` event is not emitted until settings have finished re-initializing to avoid problems with background processes using the db connection which is disconnected/re-connected or being kicked off with out-of-date settings
2020-07-01 17:16:57 +01:00
Fabien 'egg' O'Carroll
ee786aaa5d
Cleaned up members & stripe settings (#11957)
* Updated members default settings

ref #10318

This pulls out the members_subscription_settings & stripe_connect_intgration settings into separate keys

* Updated usage of members_from_address

* Updated stripe_connect usage

* Updated members config to use new settings

* Updated members middleware to use isStripeConnected

* Updated members service to reload correctly

We reload the members-api instance when the related settings change, so
this makes sure we're listening to the correct settings changes

* Updated ghost_head helper to use new settings

* Updated theme middleware to use new settings

* Renamed members_allow_signup -> members_allow_free_signup

* Fixed tests after settings refactor

* Removed  from direct key settings key

* Fixed regression tests for settings api
2020-06-29 16:22:42 +02:00
Nazar Gargol
2de1c07dd4 Renamed settings key brand.primaryColor to accent_color
refs #10318
2614565d5a

- Changes naming throughout the codebase matching the change in the
referenced commit
2020-06-24 14:18:13 +12:00
Nazar Gargol
f0a811e9fe Renamed ghost_head/ghost_foot to codeinjection_head/codeinjection_foot
refs #10318
refs 2614565d5a

- Renamed ghost_head/ghost_foot in settings to match the new names
introduced in migrations
- Above change lead to reshufling in the mappings in input/output
serializers
- Makes sure change is compatible with v2 API
2020-06-24 14:18:13 +12:00
Nazar Gargol
5f1060a8bf Renamed settings keys active_timezone to timezone
refs https://github.com/TryGhost/Ghost/issues/10318
refs 2614565d5a

- Renames to match referenced migration renames
- Fixed API responses so they are consistent with newly renamed fields
- Not returning lang and timezone keys from settings in API v2 ther rest should be returned in API v3/canary
2020-06-24 14:11:20 +12:00
Nazar Gargol
c4dff2d50e Renamed settings keys from default_locale to lang
refs #10318
refs 2614565d5a

- Renames default_locale settings key as introduced in referenced
migration
2020-06-24 14:07:24 +12:00
Kevin Ansfield
8a817050c5
Organised default settings into new groups (#11952)
refs https://github.com/TryGhost/Ghost/issues/10318

- precursor to migrating from `settings.type` to `settings.group`
- renames `blog` type to `site`
- renames `bulk_email` type to `email`
- moves settings out of `site` (previously `blog`) into more appropriate groups such as `core` or individual feature groups
2020-06-23 20:49:08 +01:00
Rish
77e1ada6c6 Added new settings for members modal
no issue

-  reverts commit 87c31444fd but with modifications to settings naming
-  Adds new settings for members modal customization to default settings
- `portal_button` controls the visibility of beacon in members modal
- `portal_name` controls the visibility of name field in signup
- `portal_plans` controls the visibility of plans allowed for member to signup with
-  Adds stripe connect check to determine if stripe is setup or not
-  Adds the 3 new settings to members site data
-  Updates to snake case naming for members site API data
2020-06-19 21:35:49 +05:30
Rish
87c31444fd Revert "Added new settings for members modal settings (#11927)"
This reverts commit ea7d450a9b.
2020-06-18 19:06:13 +05:30
Rishabh Garg
ea7d450a9b
Added new settings for members modal settings (#11927)
no issue

-  Adds new settings for members modal customization to default settings
- `membersjs_show_beacon` controls the visibility of beacon in members modal
- `membersjs_show_signup_name` controls the visibility of name field in signup
- `membersjs_allowed_plans` controls the visibility of plans allowed for member to signup with
-  Adds stripe connect check to determine if stripe is setup or not
-  Adds the 3 new settings to members site data
2020-06-18 17:11:08 +05:30
Hannah Wolfe
27066ce910
🐛 Fixed missing text version in bulk email (#11919)
closes #11917

- Pass text-only version to mailgun as `text` not `plaintext`
- This ensures we send a text-only version of the email, and this in turn should help to improve spam scores
2020-06-15 15:31:09 +01:00
Hannah Wolfe
edff7ac853
Adding INR currency support (#11911)
- We have many customers asking for INR as there are special rules in Stripe for this currency
- As well as a desire for local-selling
- Meaning it's not valid to use e.g. USD instead
2020-06-12 19:36:25 +01:00
Fabien O'Carroll
63f1e57036 Fixed Members Stripe config for Stripe Connect
no-issue

This was initially missed as local settings always had the original
stripeDirect keys, this ensures that regardless of Connect vs Direct vs
Both vs Neither that the config is correct.

Also ensures that the Members API instance is reloaded when the Stripe
Connect settings are changed.
2020-06-12 19:22:36 +02:00
Rish
41b3ecc744 Removed extra redirect on Admin for from address update
no issue

- Ghost-Admin redirects all paths to `/ghost/settings...` as `/ghost/#/settings/...`, this updates the admin redirect on successful magic link validation to directly use the latter to avoid extra redirect
2020-06-11 19:49:19 +05:30
Rish
9c478d778a Fixed email copy for update email template
no issue

- Fixes email copy changes for update-email mails sent for from address update
2020-06-11 18:19:53 +05:30
Rish
82cf095600 Added stripe config check to members site data
refs https://github.com/TryGhost/members.js/issues/43

- Adds new `isStripeConfigured` flag to public members site data which denotes if stripe setup is completed
- Helps clients like members.js/themes to configure payment behavior based on this flag
2020-06-10 21:15:13 +05:30
Fabien O'Carroll
5f808dfbee Encoded the mode in the Stripe Connect OAuth state
no-issue

The service at stripe.ghost.org must know which client_secret to use,
either the test, or live one. By encoding a JSON object as the state we
are able to pass data through the flow to inform this decision at the
end.

Note, that we still keep a random value in the state to protect against
CSRF attacks.
2020-06-10 14:06:57 +02:00
Fabien O'Carroll
2ff151a893 Added support for stripe connect test mode
no-issue

This adds the ability to pass a `mode` param of 'test' to the members stripe
connect service, which will ensure we use a testmode client_id for the
Stripe Connect OAuth flow.

This will allow users to connect their account in testmode.
2020-06-10 14:06:57 +02:00