Commit Graph

139 Commits

Author SHA1 Message Date
Nazar Gargol
8418c829de Improved error handling for batch inserted records
no issue

- Similar to 3a594ce22e this adds error handling to batch operations done outside models
2020-08-20 19:01:17 +12:00
Nazar Gargol
f8adb4a672 Refactored bulk insert/delete operations into separate module
no issue

- Moved bulk db operations outside of importer module to create clearer separation of responsibilities
2020-08-20 18:42:38 +12:00
naz
3a594ce22e
Improved error handling for batch inserted member records (#12146)
no issue

- When batch insert fails handling should be more granular and aim to retry and insert as many records from the batch as possible.  
- Added retry logic for failed member's batch inserts. It's a sequential insert for each record in the batch. This implementation was chosen to keep it as simple as possible
- Added filtering of "toCreate" records when member fails to insert. We should not try inserting related members_labels/members_stripe_customers/members_stripe_customer_subscriptions records because they would definitely fail insertion without associated member record
2020-08-20 17:41:47 +12:00
Nazar Gargol
2e769e3122 Fixed handling for Stripe connected members import
no issue

- When stripe is disconnected and there are Stripe-connected records present in imported set they should not be processed and proper error should be thrown
2020-08-20 17:08:19 +12:00
Nazar Gargol
4498b4624a Fixed label serialization in members bulk importer
no issue

- After a refactor logic was missing trimming logic and handling for empty labels
2020-08-20 16:43:42 +12:00
Nazar Gargol
32fd913ac3 Fixed default subscribed value for member model in the importer
no issue

- The default value should not be a string but rather a boolean
2020-08-19 23:42:48 +12:00
Fabien 'egg' O'Carroll
cada88975a
Updated bulk importer to improve performance (#12128)
no-issue

* Added bulkAdd method to Member,Customer&Subscription model
  This allows us to keep the db access in the model layer
* Updated @tryghost/members-api to 0.27.2
  This includes fixes for rate-limiting of requests, and exposes necessary
  Stripe methods for creating customers and complimentary subscriptions,
  without affecting the database.
* Refactored importer to parallelise tasks where possible
  By parallelising our tasks we are able to improve the speed at which the
  entire import completes.
2020-08-18 13:39:45 +01:00
Nazar Gargol
4126187df3 Fixed parameter naming for members importer 2020-08-13 21:08:34 +12:00
Nazar Gargol
d2fbe327e4 Update Stripe concurrency in members importer
no issue

- There were many failed import records due to rate-limit errors. With concurrency of 9 imports go through with 100% success
- Would need to verify these limits with live API to make the most of it
2020-08-13 21:04:57 +12:00
Nazar Gargol
78b4dff656 Moved batching logic inside the members importer module
no issue

- This way importer is more self contained and controller logic doesn't have to know about batch sizes and other unecessary variables
2020-08-13 20:31:11 +12:00
Fabien 'egg' O'Carroll
1294e3f92c
Replaced all usage of member models with members-api (#12117)
no-issue

* Added stripeSubscriptions relation to member model

This allows us to fetch the subscriptions for a member via standard
model usage, e.g. `withRelated: ['stripeSubscriptions']` rather than
offloading to loops and `decorateWithSubscriptions` functions, this is
more performant and less non-standard than the existing method.

* Updated serialize methods to match existing format

The current usage of `decorateWithSubscriptions` and the usage of
members throughout the codebase has a subscriptions array on a stripe
object on the member, this ensures that when we serialize members to
JSON that we are using the same format.

There is definitely room to change this in future, but this is an
attempt to create as few breaking changes as possible.

* Installed @tryghost/members-api@0.26.0

This includes the required API changes so that everywhere can use
members-api directly rather than models and/or helper methods
2020-08-12 14:17:44 +01:00
Nazar Gargol
c696d715c1 Extracted batched member import into separate module
no issue

- The code in controller was becoming hard to reason about.
- Having a single module shows exactly how many dependencies are there to do an import for single batch.
- Having a separate module would make it easier to extract into it's own package in Members monorepo
2020-08-12 20:18:30 +12:00
Hannah Wolfe
436db4ec3b Updated host limit message to be clearer
- limits are based on total members not number of members that will be emailed
2020-07-26 20:49:30 +01:00
Hannah Wolfe
92446d85ea Changed member limit to be DRY & use raw query
- Member limit code was duplicated in 2 places unnecessarily
- Also used member api code that fetched members and subscriptions fully hyrated when we only need a count
- Using a raw query significantly improves performance here
2020-07-26 20:49:30 +01:00
Rish
2ac69e637e Added publication icon to members site data
refs https://github.com/TryGhost/members.js/issues/72

- Portal is using using publication logo from settings for signup/signin pages
- Instead, we are switching to using publication icon from settings, which also needs to be passed in site data API
2020-07-23 17:04:44 +05:30
Fabien 'egg' O'Carroll
69080824de
🐛 Ensured webhooks are created once (#12075)
refs #12074

Since we've split members settings into multiple keys the
reconfiguration of the members-api has been happening in quick
succession as the stripe_connect_* settings are all set at once.

This debounces the call to reconfigure the members-api so that we only
need to instantiate it once.
2020-07-22 12:27:05 +02:00
Rishabh Garg
ead073ad68
Updated sender email to existing "from-address" (#12044)
refs #12043

- On updating From-address from Members settings in Labs, we send a confirmation email to the updated address with magic link for verification
- Previously, no explicit sender email was being set for this so fallback config address was used
- This updates the sender address to use the current from address for "from-address" update emails
2020-07-13 17:57:31 +05:30
Fabien O'Carroll
06c7152f4b 🐛 Fixed Stripe webhooks for subdirectory setups
no-issue

- Added breaking test for webhook url including subdirectory

- Previously the webhook handler URL was generated incorrectly when
  running Ghost on a subdirectory, appending the path to the root of the
  host, this fix ensures that the subdirectory is included before the
  path.
2020-07-10 17:43:46 +02:00
Fabien O'Carroll
832ecad43b Improved handling of Stripe webhooks on boot
no-issue

This version of members-api includes changes to how webhooks are
managed, previously they would be deleted and recreated on every boot of
Ghost. Now they are created and the secret is persisted, on boot the
webhook is updated to the most current url and events. If the api
version is wrong or the update fails, the webhook is deleted and
recreated and the settings updated.

 - Installed @tryghost/members-api@0.24.0
 - Updated config to work with 0.24.0
2020-07-10 17:43:46 +02:00
Rish
c5d360e647 Fixed members plan currency symbol
no issue

The refactor to new settings lost the uppercasing of the currency string, this puts it back
2020-07-08 10:47:28 +05:30
Fabien O'Carroll
694d92d270 Added guards for stripe_plans setting being null
no-issue

There is concern that the settings cache can return `null` for values
which it cannot parse correctly, this just ensures that we always have
an array where we expect one
2020-07-07 13:10:17 +02:00
Rish
8d989bd3c3 Updated members site data to include new portal settings
no issue

- Adds new portal button settings to members site data for portal script
- Updates settings input/output serializers to handle portal icon image url
2020-07-07 14:06:04 +05:30
Fabien 'egg' O'Carroll
ee786aaa5d
Cleaned up members & stripe settings (#11957)
* Updated members default settings

ref #10318

This pulls out the members_subscription_settings & stripe_connect_intgration settings into separate keys

* Updated usage of members_from_address

* Updated stripe_connect usage

* Updated members config to use new settings

* Updated members middleware to use isStripeConnected

* Updated members service to reload correctly

We reload the members-api instance when the related settings change, so
this makes sure we're listening to the correct settings changes

* Updated ghost_head helper to use new settings

* Updated theme middleware to use new settings

* Renamed members_allow_signup -> members_allow_free_signup

* Fixed tests after settings refactor

* Removed  from direct key settings key

* Fixed regression tests for settings api
2020-06-29 16:22:42 +02:00
Nazar Gargol
2de1c07dd4 Renamed settings key brand.primaryColor to accent_color
refs #10318
2614565d5a

- Changes naming throughout the codebase matching the change in the
referenced commit
2020-06-24 14:18:13 +12:00
Rish
77e1ada6c6 Added new settings for members modal
no issue

-  reverts commit 87c31444fd but with modifications to settings naming
-  Adds new settings for members modal customization to default settings
- `portal_button` controls the visibility of beacon in members modal
- `portal_name` controls the visibility of name field in signup
- `portal_plans` controls the visibility of plans allowed for member to signup with
-  Adds stripe connect check to determine if stripe is setup or not
-  Adds the 3 new settings to members site data
-  Updates to snake case naming for members site API data
2020-06-19 21:35:49 +05:30
Rish
87c31444fd Revert "Added new settings for members modal settings (#11927)"
This reverts commit ea7d450a9b.
2020-06-18 19:06:13 +05:30
Rishabh Garg
ea7d450a9b
Added new settings for members modal settings (#11927)
no issue

-  Adds new settings for members modal customization to default settings
- `membersjs_show_beacon` controls the visibility of beacon in members modal
- `membersjs_show_signup_name` controls the visibility of name field in signup
- `membersjs_allowed_plans` controls the visibility of plans allowed for member to signup with
-  Adds stripe connect check to determine if stripe is setup or not
-  Adds the 3 new settings to members site data
2020-06-18 17:11:08 +05:30
Hannah Wolfe
edff7ac853
Adding INR currency support (#11911)
- We have many customers asking for INR as there are special rules in Stripe for this currency
- As well as a desire for local-selling
- Meaning it's not valid to use e.g. USD instead
2020-06-12 19:36:25 +01:00
Fabien O'Carroll
63f1e57036 Fixed Members Stripe config for Stripe Connect
no-issue

This was initially missed as local settings always had the original
stripeDirect keys, this ensures that regardless of Connect vs Direct vs
Both vs Neither that the config is correct.

Also ensures that the Members API instance is reloaded when the Stripe
Connect settings are changed.
2020-06-12 19:22:36 +02:00
Rish
41b3ecc744 Removed extra redirect on Admin for from address update
no issue

- Ghost-Admin redirects all paths to `/ghost/settings...` as `/ghost/#/settings/...`, this updates the admin redirect on successful magic link validation to directly use the latter to avoid extra redirect
2020-06-11 19:49:19 +05:30
Rish
9c478d778a Fixed email copy for update email template
no issue

- Fixes email copy changes for update-email mails sent for from address update
2020-06-11 18:19:53 +05:30
Rish
82cf095600 Added stripe config check to members site data
refs https://github.com/TryGhost/members.js/issues/43

- Adds new `isStripeConfigured` flag to public members site data which denotes if stripe setup is completed
- Helps clients like members.js/themes to configure payment behavior based on this flag
2020-06-10 21:15:13 +05:30
Fabien O'Carroll
5f808dfbee Encoded the mode in the Stripe Connect OAuth state
no-issue

The service at stripe.ghost.org must know which client_secret to use,
either the test, or live one. By encoding a JSON object as the state we
are able to pass data through the flow to inform this decision at the
end.

Note, that we still keep a random value in the state to protect against
CSRF attacks.
2020-06-10 14:06:57 +02:00
Fabien O'Carroll
2ff151a893 Added support for stripe connect test mode
no-issue

This adds the ability to pass a `mode` param of 'test' to the members stripe
connect service, which will ensure we use a testmode client_id for the
Stripe Connect OAuth flow.

This will allow users to connect their account in testmode.
2020-06-10 14:06:57 +02:00
Fabien O'Carroll
9348134afe Added display_name & account_id to stripe connect
no-issue

This will allow us to display this information in the Admin to make it
easier for the user to see what the current state is.
2020-06-09 17:11:17 +02:00
Nazar Gargol
81fc5f8eda Added special handling for member import with Stripe connection
no issue

- When imported member contains stripe_customer_id data but there is no Stripe configured on the Ghost instance such import should faiil. The logic is consistent with one where import fails after not being able to find customer in linked Stripe account
- Fixed import stats to show import failures instead of "duplicate" when the validation error is of "Stripe" origin
2020-06-09 23:02:38 +12:00
Rish
c63f7f2b13 Updated copy for email address change template
no issue

- Fixes typo and title for email address update mails
2020-06-09 12:11:57 +05:30
Rish
bca41e1877 Allowed updating from address domain for member emails
refs https://github.com/TryGhost/Ghost/issues/11414

Confirms if the fromAddress for sending member emails is valid and accessible using magic link flow, allowing owners to update full from address including domain change.

- Extends member service to handle magic link generation and validation for email update
- Updates existing setting endpoint to not directly update from address
- Adds new endpoint to send magic link to new address
- Adds new endpoint for validating the magic link when clicked and update the new email for from address
- Adds new email template for from address update email
2020-06-09 00:06:07 +05:30
Fabien O'Carroll
37a1947fc5 Fixed broken regression tests from #11861
no-issue

The config is now a class and the context of `this` was not bound
2020-06-02 16:20:57 +02:00
Fabien O'Carroll
44f1c0a7b6 Updated members config to respect stripeDirect
no-issue

If the stripeDirect config value is NOT set / false, then connect token
should be used over the API keys if both are present, or else use
whichever is present.  If the stripeDirect config value is set /
true,then API keys should be used and stripe connect token should be
completely ignored.

The reason for this is that we want to use Stripe Connect as the primary
auth method going forward, but we want to keep the direct version
availiable should the Ghost Foundation ever dismantle. This will allow
people to use all the same features with no dependency on an external
service.
2020-06-02 15:28:42 +02:00
Fabien O'Carroll
a1f883edbc Refactored members config to use DI
no-issue

This makes testing it much easier
2020-06-02 15:28:42 +02:00
Fabien O'Carroll
eb962d8e5c Added stripe connect module to member service
no-issue

This module handles the creation of a url used for authorization of
Stripe Connect, and also the parsing of the data eventually received
from the authorization flow.
2020-06-02 15:28:42 +02:00
Rish
d2cf7111ec Added new mail type for member email update
refs https://github.com/TryGhost/members.js/issues/30

- Added new `updateEmail` type for sending email address update confirmation mail to member
- The link in email updates member's email address
2020-06-02 14:27:03 +05:30
Rish
9948e85154 Updated member subscribe email template
no issue

- Updated button copy to "Confirm email address"
- Added missing color to copy link address
2020-06-01 12:55:47 +05:30
Vikas Potluri
00c324fa4e
Moved core/server/lib/common/logging to core/shared/logging (#11857)
- Represents that logging is shared across all parts of Ghost at present
  * moved core/server/lib/common/logging to core/shared/logging
  * updated logging path for generic imports
  * updated migration and schema imports of logging
  * updated tests and index logging import
  * 🔥 removed logging from common module
  * fixed tests
2020-05-28 19:30:23 +01:00
Rish
3471a21bc3 Removed email update directly using member session auth
refs https://github.com/TryGhost/members.js/issues/30

- Member cannot update their email directly but need to do it via magic link sent to new email address
- Previous profile update change had allowed email to be updated directly as well for authenticated member
2020-05-28 19:31:00 +05:30
Vikas Potluri
1bd8c18a16
Moved core/server/lib/url-utils to core/shared/url-utils (#11856)
* moved url-utils from server to shared
* updated imports of url-utils
2020-05-28 11:57:02 +01:00
Vikas Potluri
15d9a77092
Moved config from server to shared (#11850)
* moved `server/config` to `shared/config`
* updated config import paths in server to use shared
* updated config import paths in frontend to use shared
* updated config import paths in test to use shared
* updated config import paths in root to use shared
* trigger regression tests
* of course the rebase broke tests
2020-05-27 18:47:53 +01:00
Rishabh Garg
fe3eab1836
Added member update endpoint with session auth (#11824)
- Allows member logged in with valid session to update their profile info - name, email, subscribed(newsletter subscription status)
- Adds new util method for formatted member response on the endpoints
- Adds common middlewares for body/bool parser and maintenance
- Adds `subscribed` status to member response
2020-05-20 14:37:58 +05:30
Hannah Wolfe
998eb62e22 Added success indicator for members magic links
- Add a query param that indicates whether signin/up succeeded or failed
- Add unit tests for all 3 possible cases for the createSessionFromMagicLink middleware
- Added an acceptance test to show the behaviour works in principle
2020-05-08 13:17:51 +01:00