Commit Graph

990 Commits

Author SHA1 Message Date
Kevin Ansfield
22306c4b19 Fixed type errors in post-email-serializer
no issue

- fixed errors showing in editor, no bugs just minor expected type differences
2021-06-18 15:37:42 +01:00
Kevin Ansfield
557ef28ae5 Added psmRedesign as an allowed labs flag
refs https://github.com/TryGhost/Team/issues/786

- flag will be used by Admin to toggle between old (popover) and new (sidebar) post settings menu behaviour
2021-06-17 10:49:20 +01:00
Sam Lord
3f0bab4389 Replaced request module with @tryghost/request
no issue
Part of the effort to break up Ghost into smaller, decoupled modules.
2021-06-16 13:16:15 +01:00
Sam Lord
24332c3d24 Replaced ghost-version.js with @tryghost/version
no issue
Part of the effort to break up the Ghost codebase into smaller, decoupled modules.
2021-06-16 13:16:15 +01:00
Hannah Wolfe
526993965a
Switch to @trghost/validator, remove validator
- Part of the effort to split Ghost down into smaller, decoupled pieces
- Moved out our internal validator tooling to a separate library
- Replaced all usage of our own tooling and validatorjs directly with @tryghost/validator
- Removed the validatorjs dependency and removed the renovate pin
- This gives us a consistant, smaller, clearer public API for validations
- It will eventually be used on Ghost Admin too
- This way we can start getting up to date with validator whilst not increasing build size
2021-06-16 08:11:22 +01:00
Sam Lord
35e51e364b Switch to @tryghost/debug, remove ghost-ignition
no issue
The only pieces of Ghost-Ignition used in Ghost were debug and
logging. Both of these modules have been superceded by the Framework
monorepo, and all usages of Ignition have now been removed, replaced
with @tryghost/debug and @tryghost/logging.
2021-06-15 17:24:22 +01:00
Sam Lord
caea330647 Change to use @tryghost/logging
no issue

Logging is now controlled by a logginrc.js file in the root of the project - and now we can just import @tryghost/logging everywhere
2021-06-15 15:59:11 +01:00
Rishabh
0766a19afd Updated products data in portal site endpoint
refs https://github.com/TryGhost/Team/issues/763
refs https://github.com/TryGhost/Team/issues/718

- Removes redundant `plans` data from site data
- Updates products data to include monthly/yearly price
- Filters multiple products on site data based on labs flag
2021-06-11 12:41:46 +05:30
Kevin Ansfield
1bc57b584a
Added posts_meta.feature_image_{alt,caption} columns (#13030)
refs https://github.com/TryGhost/Team/issues/770

We want post feature image functionality to better match what's available inside the editor, to do that we'll need somewhere to store alt and caption meta data. `posts_meta` chosen because even though we want to make this generic for other tables in the future those tables also have a `feature_image` (or closely related) field.

- updated schema with new columns
- added migration to create columns
- cleaned new columns from API output
  - not output on v2/v3
  - conditionally output on v4/canary output based on labs flag
- bumped `@tryghost/admin-api-schema` to allow new columns through in canary API requests
  - silently clean properties from input when labs flag is disabled
  - updated acceptance tests so they fail if `admin-api-schema` is not letting the new fields through
2021-06-10 20:35:56 +01:00
Kevin Ansfield
f49f7699aa
Enabled all labs flags when testing (#13036)
no issue

Shows impact of new code behind labs flags through the existing acceptance/regression tests. Allows for existing tests to be updated to match new behaviour rather than requiring separate tests where individual flags are enabled. Should result in minimal test updating once code reaches GA.

- adds a forced `'labs:enabled'` fixture op that edits the `labs` setting to enable all flags then restarts the settings service to pick up the new setting
- modifies labs service to not remove ALPHA_FEATURE labs settings when running in a testing environment
2021-06-10 12:54:34 +01:00
Hannah Wolfe
1f11bd9012
Updated package-json lib to not need DI
- The underlying package-json package has had i18n ripped out using the new tpl utility instead
- It's also then been refactored to not be a class that needs instantiating
- This means it can be required directly and its public interface methods used where needed
- This is a much nicer, neater pattern for what is a mature utility library :)
2021-06-09 16:48:19 +01:00
Naz
0f6ea35d7e Added 'alpha' feature support to labs
refs https://github.com/TryGhost/Team/issues/772

- When the feature is introduced into Ghost at it's first lifecycle stage - "alpha" the rule is to have a "enableDeveloperExperiments" flag along with labs toggle turned on before it's usagble in the codebase
- The changeset introduced a "ALPHA_KEYS" concept which should allow distinguishing alpha flags from beta flags.
2021-06-09 19:30:34 +04:00
Rishabh
136fb093a3 Added labs flag for multiple products
refs https://github.com/TryGhost/Team/issues/763

- Adds new labs feature flag in the allow list to enable multiple products
2021-06-09 15:39:05 +05:30
Rishabh
d2e4f30b5b Handled empty prices for product in Portal settings
no refs

Monthly/yearly price values on a product can be `null` when stripe is not connected, this change handles the prices passed to Portal settings to ignore null prices in the array.
2021-06-08 19:41:55 +05:30
Sanne de Vries
a028f7e706 Added padding to newsletter header image 2021-06-08 11:33:07 +02:00
Kevin Ansfield
97fec8443b Fixed email header divider showing when title+icon are disabled but no publication icon is set
refs https://github.com/TryGhost/Team/issues/755

- the default value for `show_header_icon` is `true` but if there's no publication icon set then it should be read as `false` when rendering the email
2021-06-08 10:18:18 +01:00
Hannah Wolfe
7069cc1221 Added initial match helper
refs: https://github.com/TryGhost/Team/issues/759

- wired up a matchHelper feature flag & used the labsEnabledHelper tool to gate the helper
- added a first version of the match helper, which is intended to replace the has helper
- this is an experimental helper and may or may not make it to GA
- match is a simple comparison helper, right now it does a very basic equals or not equals comparison
- much more functionality is needed to reach parity with has
2021-06-07 21:15:05 +01:00
Naz
7a21522804 Restored labs' getAll function
refs https://github.com/TryGhost/Team/issues/757
refs 37ef40b46e/core/server/services/labs.js

- As more labs flags have been added to allowlist the `getAll` method should be returning members+all allowlisted flags
- The changeset restored the state of the method to the one reffed in the commit
2021-06-07 20:51:37 +04:00
Kevin Ansfield
8dc4f183e2 Updated newsletter email template for new settings
refs https://github.com/TryGhost/Team/issues/755

Make use of the new settings in the email template when `enableDeveloperExperiments` flag is enabled.

- added header image output if set
- hide all header output if both show publication title+icon are disabled
- hide individual header output for title and logo based on individual settings
- add left-align and serif classes to title based on individual settings
- hide feature image when disabled
2021-06-07 16:31:50 +01:00
Naz
cd35358fdb Added labs setting input validation
refs https://github.com/TryGhost/Team/issues/757

- To safeguard from mise of a very permissing "object" value of the "labs" setting this change introduces an "allowlist" approach to filtering unrecognized labs flags
- Should allow maintainers to have a clear view of which labs flags are currently in use and manage them accordingly
2021-06-08 02:18:17 +12:00
Rishabh
4e01fe9d09 Updated price/products data for portal settings
refs https://github.com/TryGhost/Team/issues/718

The ids for default prices for a product is now stored directly on product model instead of on global settings. This change updates
- the products data sent to Portal to use list of products with their active monthly/yearly prices, as well as
- the prices data sent to Portal to use the prices of default(first) product
2021-06-04 13:03:33 +05:30
Kevin Ansfield
a87410ef28 🐛 Fixed logs showing an error when sending an email with no feature image
no issue

- we were attempting to read an image file to determine it's dimensions when no feature image was set. This wasn't a fatal error as it was handled gracefully and had no ill consequences but it was adding confusing errors to the logs
2021-06-01 17:07:33 +01:00
Daniel Lockyer
b98e0b0bbb v4.6.4
-----BEGIN PGP SIGNATURE-----
 
 iHUEABYIAB0WIQTqYa7kNs8D7Oo9dgLSEYbwtHKVrQUCYK/iogAKCRDSEYbwtHKV
 rcgCAP9qagl5CtrgE1mLx7fs+W+yy+UOslBqhR/o49aOQMTEbAD/ajcR5w2qv/mW
 GRK0qr9boWPcQm6YpiuEQ9j4SYMoCQU=
 =p+5B
 -----END PGP SIGNATURE-----
gpgsig -----BEGIN PGP SIGNATURE-----
 
 iHUEABYIAB0WIQTqYa7kNs8D7Oo9dgLSEYbwtHKVrQUCYK/jPgAKCRDSEYbwtHKV
 rcEgAQDh43UkCDj/rg3jxMeItspMXwnwrg+ZpscgoeCdm4nFKAD/UtsJCrma4+j8
 if2jOddYUy4xAAPUMwhpwsRJVMkWbgY=
 =gtbT
 -----END PGP SIGNATURE-----

Merged v4.6.4 into main

v4.6.4
2021-05-27 19:21:44 +01:00
Rishabh Garg
83c0cbb516
🐛 Fixed missing stripe connected check on boot (#12992)
refs https://github.com/TryGhost/Team/issues/598
refs https://github.com/TryGhost/Ghost/commit/5cdf910e

As part of the changes to disallow sites with starting up without https when they are connected to stripe, the conditional missed the check for stripe connection. As a result we were erroring in boot sequence for all sites starting without https irrespective if they are connected to Stripe or not which is incorrect. This fixes the `init` check for members service to only error for non-https sites if they are connected to Stripe.
2021-05-27 23:03:03 +05:30
Naz
a1556797b6 Refactored GhostMailer's send to current code standards
no issue

- While working on https://github.com/TryGhost/Team/issues/726 have questioned some of the options that were passed along to the `send` method. Documented findings and refactored the code slightly while touching it
2021-05-27 12:37:03 +04:00
Kevin Ansfield
5fe9036ecf Added sentry_dsn and sentry_env properties to /canary/site/
refs https://github.com/TryGhost/Team/issues/723

- enables Admin to report unhandled exceptions and API errors when configured
2021-05-26 16:54:04 +01:00
Kevin Ansfield
3b366dc55d
Fixed max-api-complexity linting warnings for site and config APIs (#12982)
no issue

- moved `config` and `site` API output generation to a `public-config` service allowing all API versions to use `publicConfig.config` or `publicConfig.site` in their query methods
- updated `config` and `site` API output serializers to use an allow-list that limits the data returned for each API version
2021-05-26 12:10:19 +01:00
Fabien O'Carroll
f4c40249ce Fixed members auth middleware
no-issue

The JWT library we used does not throw an error which can be used by
Ghost. So we need to catch and wrap it in our own errors from
@tryghost/errors.
2021-05-24 11:28:15 +01:00
Fabien O'Carroll
5cdf910e63 Checked boot preconditions in members service init
refs https://github.com/TryGhost/Team/issues/598

We now have several pre-conditions related to members which determine
whether or not Ghost is allowed to start. Rather than burying this
within the members-api module, we have now surfaced them to an init
method which can be called during the boot sequence of Ghost. This will
allow us to exit early and explicitly.
2021-05-20 12:08:45 +01:00
Fabien O'Carroll
3f6544bebc Renamed index.js -> service.js for lint reasons
no-issue

Our linter now requires that files named index.js have less than 50
lines, so this renames the index.js file to service.js and reexports
service.js from index.js so that linting will pass.
2021-05-20 12:08:45 +01:00
Rishabh
2e8db93ab6 Filtered selected prices in Portal settings
no refs

Filters active prices in Portal settings to only contain the selected prices by site owner in new monthly/yearly price id settings, ignoring all other prices for now.
2021-05-18 20:27:20 +05:30
Hannah Wolfe
d9367f5b20
Added debug to gscan checks for timings
- added a couple of extra debug calls to see how long gscan checks take in the boot process
2021-05-18 15:22:04 +01:00
Kevin Ansfield
84724537be 🐛 Fixed feature images in emails appearing very wide in Outlook
closes https://github.com/TryGhost/Team/issues/675

Outlook will display images at their native resolution if no `width` attribute is supplied. Content images were fixed a while ago but feature images would still render very wide and cause horizontal scroll and text size/alignment issues.

- modify `post.feature_image` and add a `post.feature_image_width` property before passing it through to the email template
  - for Unsplash images we assume all images are larger than 600px so we change the URL to reference a 1200px image and set the image width to 600 (to keep images on retina displays crisp)
  - for other images we probe the image to fetch the original dimensions and give set an image width of 600 if needed, if it's a locally-hosted image we update the URL to point at a max 1200px version
- updated email template to output a `width` attribute on the feature image `<img>` tag if it's set
2021-05-14 11:57:37 +01:00
Rishabh
196cdafe6b 💡 Removed 401 error for logged-out member on Portal
closes https://github.com/TryGhost/Team/issues/560
closes https://github.com/TryGhost/Ghost/issues/12870

The endpoint `/members/api/member/` is used by Portal for fetching member details on site load to setup different flows. The response from this endpoint for logged out member has now changed from 401 Unauthorized to 204 No Content.

Ghost API was previously returning 401 Unauthorized error for logged-out member as this seemed to be technically correct response for unauthorized access to membership features. This resulted in a lot of confusion for end users where visible 401 errors on console were perceived as errors in the script as well as caught by loggers as erroneous traffic. Also for an end user, in the context of visiting a website - the user themselves is not trying to gain access to anything so this becomes cause for more confusion.

After internal discussion, the endpoint - [SITE_URL]/members/api/member- now returns 204 No Content instead of 401 for logged out member, denoting server was able to process the request but did not find any associated member. This should avoid any unwanted error logging on Portal load on a site, as well as make Portal functioning more transparent for a site.
2021-05-13 15:26:07 +05:30
Fabien O'Carroll
41acc37865 Made complimentary_plan & stripe_customer_id exclusive
no-issue

When importing Members it is possible to have both the
complimentary_plan and the stripe_customer_id columns set, this can
result in unusual outcomes, for example when importing a customer with a
zero-amount subscription, they would end up with two "comped"
subscriptions, and there would be two "comped" prices in the database.

As we are deprecating the use of "comped" in favour of creating a
subscription with a specific price, we're updating the import to prefer
`stripe_customer_id` column, only using the `complimentary_plan` column
when it is the only of the two columns passed.
2021-05-11 12:33:28 +01:00
Kevin Ansfield
c36e749820
Added support for gating content by member labels and products (#12946)
refs https://github.com/TryGhost/Team/issues/581
closes https://github.com/TryGhost/Team/issues/582

Emails can now be sent to members with specific associated labels or products by specifying an NQL string. We want to bring the same members segment feature to content by allowing `visibility` to be an NQL filter string on top of the `public/members/paid` special-case strings.

As an example it's possible to set `posts.visibility` to `label:vip` to make a post available only to those members with the `vip` label.

- removed enum validations for `visibility` so it now accepts any string or `null`
    - bumped `@tryghost/admin-api-schema` for API-level validation changes
- added nql validation to API input validators by running the visibility query against the members model
- added transform of NQL to special-case visibility values when saving post model
    - ensures there's a single way of representing "members" and "paid" where NQL gives multiple ways of representing the same segment
    - useful for keeping theme-level checks such as `{{#has visibility="paid"}}` working as expected
- updated content-gating to parse nql from post's visibility and use it to query the currently logged in member to see if there's a match
    - bumped @tryghost/members-api to include label and product data when loading member
2021-05-10 19:32:11 +01:00
Rishabh
4d4286d255 Updated setup flow to rename default product
refs https://github.com/TryGhost/Team/issues/667

On clean and existing installs, the default product created should be named the same as the site title instead of the name in fixture. This change updates the default product's name to site title during the site setup. We use the Product name in Portal.
2021-05-10 13:59:00 +05:30
Rishabh
67ea6c44ef Added free price values to Portal site settings
refs https://github.com/TryGhost/Team/issues/637

With custom products it's possible to change the name and description of any price. This assumes that people would want to change the same properties of a Free membership, and wires up the values for free membership price settings to Portal site settings API for Portal UI
2021-05-07 22:56:16 +05:30
Fabien O'Carroll
a138586c83 Updated stripe_plans & portal_plans settings
refs https://github.com/TryGhost/Team/issues/586

We are no longer using the `stripe_plans` setting, instead we are using
the `stripe_prices` database table. However, we must keep the setting as
the migration from the setting to the database is not done as a standard
migration, but in code. This means our code has to still read and pass
the setting because we will never know if the migration in code has run
yet.

The `portal_plans` setting has been updated to only include 'free' by
default, because the setting must include id's now rather than names.
2021-05-07 16:31:08 +01:00
Naz
fc458efa90 Added periodic "emails" limit check when (re)sending a newsletter
refs https://github.com/TryGhost/Team/issues/588
refs d72ba77aba

- When limit is in place we don't want to allow sending out a new batch of emails if it would go over limit
- See referenced commit for example configuration
2021-05-07 18:45:14 +04:00
Naz
d72ba77aba Added support for max periodic limit check
refs https://github.com/TryGhost/Team/issues/588

- This is a new type of limit allowing to measure resource use (e.g. sent emails) per period (e.g. subscription, billing, cycle, etc)
- To enable periodical limit add  following values under `hostSettings.limits`:
```
"emails": {
    "maxPeriodic": 10,
    "error": "Your plan supports up to {{max}} emails. Please upgrade to reenable sending emails."
}
```
and following under `hostSettings.subscription`:
```
"subscription": {
    "start": "2020-04-02T15:53:55.000Z",
    "interval": "month"
}
```
- Above config would allow checking if 10 emails per month starting on the 2nd of every month has been reached untill now
2021-05-07 18:43:47 +04:00
Rishabh
fbd03525b0 Filtered active prices in portal settings
refs https://github.com/TryGhost/Team/issues/665

Portal only needs to work with active prices(not archived), this change filters prices sent to Portal to only include active prices
2021-05-07 19:12:22 +05:30
Kevin Ansfield
322664a145
Added ability to send a newsletter to members with a certain label or product (#12932)
refs https://github.com/TryGhost/Team/issues/581
refs https://github.com/TryGhost/Team/issues/582

When publishing a post via the API it was possible to send it using `?email_recipient_filter=all/free/paid` which allowed you to send to members only based on their payment status which is quite limiting for some sites.

This PR updates the `?email_recipient_filter` query param to support Ghost's `?filter` param syntax which enables more specific recipient lists, eg:

`?email_recipient_filter=status:free` = free members only
`?email_recipient_filter=status:paid` = paid members only
`?email_recipient_filter=label:vip` = members that have the `vip` label attached
`?email_recipient_filter=status:paid,label:vip` = paid members and members that have the `vip` label attached

The older `free/paid` values are still supported by the API for backwards compatibility.

- updates `Post` and `Email` models to transform legacy `free` and `paid` values to their NQL equivalents on read/write
  - lets us not worry about supporting legacy values elsewhere in the code
  - cleanup migration to transform all rows slated for 5.0
- removes schema and API `isIn` validations for recipient filters so allow free-form filters
- updates posts API input serializers to transform `free` and `paid` values in the `?email_recipient_filter` param to their NQL equivalents for backwards compatibility
- updates Post API controllers `edit` methods to run a query using the supplied filter to verify that it's valid
- updates `mega` service to use the filter directly when selecting recipients
2021-05-07 11:56:41 +01:00
Rishabh
6c2869db61 Updated products/prices data in member site settings
refs https://github.com/TryGhost/Team/issues/637
refs 75169b705b

With custom prices, Portal now needs to show all available custom prices in the UI as well as product's name and description in the Portal UI. This change adds product information to member site settings for Portal UI.
2021-05-07 15:15:10 +05:30
Daniel Lockyer
4e69f24964
Moved package-json wrapper outside implementation folder
no issue

- we're preparing the `package-json` lib to be extracted out of Ghost into
  its own package so moving the initialization wrapper outside of the
  folder makes the process a lot easier
2021-05-06 12:56:21 +01:00
Hannah Wolfe
01d9701096 Fixed i18n being called outside of a function
- calling i18n as a global const like this requires it to be loaded before anything else, when we have to manage this with the init() flow
- wrapping it inside the function where it's used ensures we don't call i18n til we need it
- also improved the i18n called without init error to include the key it was called with
2021-05-05 16:34:40 +01:00
Rishabh Garg
75169b705b
Added custom prices list to portal settings (#12912)
refs https://github.com/TryGhost/Team/issues/637

With custom prices, Portal now needs to show all available custom prices in the UI instead of just `monthly` and `yearly` prices. This change adds a list of all custom prices to Portal site settings for the default product which Portal will use to show the available prices in UI. 
Note: As part of cleanup, the stripe price ids will be removed from the prices list.

Also:
- Fixes product name in serialised subscriptions
- Adds `type` value in  serialised price object
2021-05-04 21:02:20 +05:30
Hannah Wolfe
273e220327 Moved i18n to shared
refs 829e8ed010

- i18n is used everywhere but only requires shared or external packages, therefore it's a good candidate for living in shared
- this reduces invalid requires across frontend and server, and lets us use it everywhere until we come up with a better option
2021-05-04 13:03:38 +01:00
Hannah Wolfe
829e8ed010 Expanded requires of lib/common i18n and events
- Having these as destructured from the same package is hindering refactoring now
- Events should really only ever be used server-side
- i18n should be a shared module for now so it can be used everywhere until we figure out something better
- Having them seperate also allows us to lint them properly
2021-05-03 17:14:52 +01:00
Naz
bc15f8c1bb Added "emails" limit check when (re)sending a newsletter
refs https://github.com/TryGhost/Team/issues/588

- This check allows for a on/off switch to be set up on the instance and control limits around sending emails
- An example configuration for such check would look like following in config's hostSettings section, e.g.:
```
"emails": {
    "disabled": true,
    "error": "Email sending has been temporarily disabled whilst your account is under review."
}
```
2021-05-03 17:51:33 +04:00
Fabien O'Carroll
fa327a7a85 Passed Settings model to @tryghost/members-api
refs https://github.com/TryGhost/Team/issues/637
refs https://github.com/TryGhost/Team/issues/591

We need to run migrations which will update the `portal_plans` setting
to use id's rather than names. This migration relies on the
`stripe_prices` table being complete populated. The migration to
populate the `stripe_prices` table was not added as a "normal"
migration because it needs to access the Stripe API over the network.
Any migrations that rely on this are unable to be run in a "normal"
migration as that cannot be sure that the database is in the correct
state.

The `portal_plans` setting migration is therefore run in code, and needs
access to the Settings model in order to modify the database.
2021-05-03 13:59:13 +01:00
Daniel Lockyer
15ecfd471b
Merged v4.3.3 into main
v4.3.3
2021-04-29 12:14:39 +01:00
Daniel Lockyer
95651b33a6
🐛 Fixed error when using staff access tokens
closes https://github.com/TryGhost/Team/issues/645

- we did some refactoring in [1] to turn promise chained code into
  async/await, but this removed an early `return` from the code
- therefore we'd continue on to further code, which breaks for obscure
  reasons that weren't apparent from the error
- this commit adds back a return at the end of the block where we handle
  staff API tokens to match the same functionality as before
- this is regression that landed in 4.3.0 and would break staff user
  tokens

[1]: b677927322 (diff-bc0bedcac8ec9646d0644c86a91e46f4759bc1b0c2aebac54a2b26ec474c3d15L148-L155)
2021-04-29 08:56:27 +01:00
Rishabh Garg
1d8b6ef1c3
Added members signup access setting to Portal (#12908)
refs https://github.com/TryGhost/Team/issues/579

The new signup access setting allows site owner to set the type of access level allowed for a member which Portal needs to handle
2021-04-27 21:02:22 +05:30
Kevin Ansfield
cf29ed8c30
Updated members allowSelfSignup() to take portal plans into account (#12909)
refs https://github.com/TryGhost/Team/issues/579

`members_signup_access = 'invite'` now forces invite-only mode so both free and paid setups both use the `'all'` setting. To ensure we're properly allowing/disabling free (self signup) signups in the members API we need to update `allowSelfSignup()` to take additional settings into account.

- `true` when Stripe is not connected. There are no paid plans available in this configuration so free signup is always enabled. To disable free signup on a site with no Stripe setup the members signup access should be set to `invite` or `none`.
- `true` when Stripe is configured and free plan is enabled in portal, without it Members API would not send magic link emails to signup requests
- `false` in all other situations such as invite-only and members-disabled signup access modes, or when the free plan has been disabled in portal configuration
2021-04-27 16:22:43 +01:00
Hannah Wolfe
bc75fab663 Moved theme service to core/server
refs: bf0823c9a2
refs: ae86254972

- continuing the work of splitting up the theme service into logical components

Themes Service
- The serverside theme service now serves just the API and boot
- It loads the theme and passes it to the theme-engine via the bridge

This achieves the bare minimum goal of removing all the cross requires between server and frontend around themes
There is still a lot more to do to achieve an ideal architecture here as laid out in ae86254972
2021-04-27 15:14:49 +01:00
Thibaut Patel
c471ae11d4 Added oauth login and invitation acceptance
issue https://github.com/TryGhost/Team/issues/614

- Users who have a password can directly sign-in via oauth
- User who are logged-in get their password disabled
- Users accepting an invitation get their password disabled
- The way we disable password is by setting it to a long random password
2021-04-21 19:36:27 +02:00
Daniel Lockyer
e9b21fdbd1 Updated bson-objectid calls to match API change
refs c873899e49

- as of `bson-objectid` v2.0.0, this library exports the function
  to generate an ObjectID directly, and then you need to use `.toHexString()`
  to get the 24 character hex string - 6696f27d82
- this commit removes all uses of `.generate()` and replaces with this
  change
2021-04-21 16:23:52 +01:00
Hannah Wolfe
bf0823c9a2 Moved hbs engine into new theme engine service
- This is the beginning of splitting up the theme service into:
   - Storage components used by the API (should be a server service)
   - Theme engine & rendering components used by the frontend (this new engine service)
   - The code to activate a theme which is shared code where the API & frontend need to communicate
- This is needed because currently the frontend theme service is required and used by the API, creating tight coupling.
- In my quest to truly separate the API and frontend, this is one of many battles that needs winning
2021-04-19 20:03:30 +01:00
Kevin Ansfield
5c41c67ffc Set @labs.members to false when members_signup_acess == 'none'
refs https://github.com/TryGhost/Team/issues/579

- setting `members_signup_access` to `'none'` effectively disables all built-in members functions on the front-end so setting `@labs.members` to `false` allows themes to react accordingly
- `@labs.members` keeps backwards compatibility with pre-4.0 versions where themes were using it to toggle member-related functionality
2021-04-19 18:28:51 +01:00
Kevin Ansfield
fff6a04c54
Migrated members_allow_free_signup setting to members_signup_access (#12886)
refs https://github.com/TryGhost/Team/issues/579

Currently the members signup setting is explicitly yes/no to allowing free members signup, with the implication that when set to "no" members is still active but members have to be created via Stripe or the admin API.

This change renames the setting and changes its type to allow more than a binary option.

- migration to create/update the new setting based on the old value
  - free signup = "all", no free signup = "invite"; matches the current UI for this setting
- rename setting everywhere it's used/tested against
- modify `getAllowSelfSignup()` used to configure members packages to only return `true` when the new setting is set to `'all'` to match behaviour to the older setting
- update importer to rename the setting when importing from an older Ghost version
2021-04-19 16:36:30 +01:00
Thibaut Patel
92b96a152c Fixed the previous commit
commit 375c71fc6a
2021-04-16 18:05:13 +02:00
Thibaut Patel
375c71fc6a 🔒 Added a way to hide the secret settings once they are set
issue https://github.com/TryGhost/Team/issues/621
2021-04-16 17:05:16 +02:00
Rish
2c0d3b9b37 Added price and product models to members service
refs https://github.com/TryGhost/Team/issues/586

- Passes new Product, Stripe Price and Stripe Product models to members API service
- Allows members service to populate the tables for existing plans and products
2021-04-12 21:53:59 +05:30
Naz
b10cc5f62d Blocked webhooks when integration limit is in place
https://github.com/TryGhost/Team/issues/599

- When custom integration limit is enabled all webhooks belonging to integrations have to be disabled as well. The result is the webhook would stop working and to discover that a user would need to navigate to Admin UI (this changes is yet to come, see refed issue)
2021-04-10 00:48:46 +12:00
Naz
cb0807d07a Blocked requests from integrations when integration limit is in place
https://github.com/TryGhost/Team/issues/599

- When custom integration limit is enabled all requests from existing integrations should not be accepted. With the exception of internal integrations like backup and scheduler
2021-04-10 00:45:26 +12:00
Naz
edd0e26a78 Fixed limits realoding during soft Ghost restart
refs https://github.com/TryGhost/Team/issues/599

- This is a precursor change to tests which verify the hostSettings limits are working correctly
- Bumped limits-service version which allows for multiple calls of loadLimits on the same service instance
2021-04-10 00:08:17 +12:00
Naz
b677927322 Refactored api key auth to use async/await syntax
https://github.com/TryGhost/Team/issues/599

- Before introducing limit checks into this codebase rewrote the code to use async/await for more clarity and less nesting
2021-04-07 16:52:26 +12:00
Naz
42a3197f6d Updated dependency @tryghost/limit-service to v0.4.0
refs https://github.com/TryGhost/Team/issues/510

- This version bump includes follwing interface improvements of the limit-service package: passing in errors as a parameter to "loadLimits" and allowing for custom "currentCountQuery" method implementations per limit
2021-04-07 15:31:52 +12:00
Rish
eb2b98a087 Fixed incorrect subdomain check
refs 2e81aa17fb

Incomplete push in the last commit with missing conditional check :/
2021-03-24 23:33:49 +05:30
Rish
2e81aa17fb 🏗 Removed www subdomain from default newsletter/support address
refs https://github.com/TryGhost/Team/issues/397

The default newsletter/support email address for a site is currently setup as noreply@DOMAIN , which means for a custom domain setup with www the email address becomes noreply@www.somesite.com which is not the expected behavior normally. This removes the `www` subdomain if present for those email addresses, but doesn't change any other subdomain
2021-03-24 23:31:00 +05:30
Rishabh Garg
82bb466316
Removed hardcoded accent color fallbacks (#12813)
refs https://github.com/TryGhost/Team/issues/536

From 4.0, we ensure and require that accent colour is always set. This change removes hardcoded accent color fallbacks to avoid confusion as well as cause accidental fallback that is undesired causing themes to look different
2021-03-24 18:25:26 +05:30
Peter Zimon
6d853ff43f Fixed email preheaders
- removed member import related preheader from invite, reset password and welcome emails
2021-03-23 09:49:33 +01:00
Rishabh Garg
738c8ac70d
🐛 Fixed Admin redirect for newsletter/support email update (#12810)
closes https://github.com/TryGhost/Team/issues/570

When site owner/admin updates their newsletter/support email address from settings, they receive an email with confirmation link which on success takes them to Ghost Admin on email settings screen with a toast about success. Since the path for email settings in Ghost Admin changed in v4, the fix updates the redirect link to new Admin settings URL.
2021-03-23 11:24:07 +05:30
Naz
81a54a70bc Fixed failing notification update for non-major version
refs 4dc413d6a1

- Fixed failing test cases which were designed to check non-major upgrade messages. There's no clear use cases for those, but still worth keeping such case in mind
2021-03-23 18:02:52 +13:00
Naz
4dc413d6a1 🐛 Fixed upgrade notification showing post-upgrade
closes https://github.com/TryGhost/Team/issues/564
refs https://github.com/TryGhost/Ghost/issues/10236

- The notification to upgrade to new 4.0 Ghost version was still visible to users after upgrading the instance to 4.0. This was caused by notification filtering not taking into account 3.x or 4.x versions.
- The fix filters out notifications that detect a major version notification using `x.0 is now available` pattern and compares current version to that major. This should future proof the issue from happening in Ghost 5.0 (but a proper holistic fix is preferable!)
2021-03-23 16:42:46 +13:00
Naz
265026d73d Added JSDoc to notifications service's constructor
refs https://github.com/TryGhost/Team/issues/564
refs https://github.com/TryGhost/Ghost/issues/10236
2021-03-23 15:22:14 +13:00
Peter Zimon
2d69c32481 Fixed member import email heading spacing 2021-03-22 18:15:11 +01:00
Peter Zimon
b3b6a603af
New logo in Admin (#12768)
refs https://github.com/TryGhost/Team/issues/547

- replaced link to static/squircle to orb
2021-03-18 17:27:27 +00:00
Kevin Ansfield
19d5448101
🐛 Fixed complimentary members' content gating (#12761)
no issue

Comped members were not able to view paid-member content because content gating was only looking for `member.status === 'paid'` which doesn't take into consideration members on a "complimentary" plan.

- added front-end acceptance tests for member access to posts
- updated content-gating check to take comped members into consideration
2021-03-15 19:13:48 +13:00
Peter Zimon
30bc751237 Updated 'Powered by' badge ref to 4.0 repo 2021-03-10 16:02:00 +01:00
Peter Zimon
acf6e235c7 Updated squircle ref to 4.0 repo 2021-03-10 15:53:59 +01:00
Hannah Wolfe
d51fcd072e Added new member limit & removed old
refs: https://github.com/TryGhost/Team/issues/510

- The current member limit was implemented as a member-specific concept
- The new limit service is much more generic, here we are swapping old for new
- The updated concept here is blocking all publishing, not just email sending, when a site is over its member limit
- To determine that we are publishing a post, we must be in the model layer. The code has been moved to the permissible function which makes sense as this is a permissions error that we are throwing
- I've left the extra check for email retries in, in case there is some loophole here (but we may wish to change it)
2021-03-04 18:16:34 +00:00
Hannah Wolfe
26f56626ce Updated host config to correctly use camelCase
refs https://github.com/TryGhost/Team/issues/510

- When the host config was introduced it was incorrectly introduced as host_settings instead of hostSettings
- All other Ghost config uses camelCase, so changing this now before it becomes a problem
- Note: Also removed some rogue return awaits that don't make sense. It's not possible to hit that case, but cleaning up anyway
2021-03-04 11:39:32 +00:00
Naz
41c93f982d Added 'locale' property to public settings
refs https://github.com/TryGhost/Team/issues/509

- During 3.x we standardised on "lang" instead of "default_locale" for the site setting, which was an assumption based on an earlier change to @site.lang to make <html lang="@site.lang"> read nicer. This was a mistake as the field represents more than "lang" its a "locale". With this changeset we introduce a transition to use "locale" name for the value instead of "lang"
- Adds `@site.locale" value in as well as new 'locale' property in Content API's response
- "lang" will be considered as deprecated starting with API v4 and will be dropped completely with API v5
2021-03-04 16:00:02 +13:00
Naz
c2a3297f5d Restructured public settings cache hash
refs https://github.com/TryGhost/Team/issues/509

- Flipping around key/newKey pair allows to map multiple keys to the same field in the settings
- This becomes handy when there's a need to deprecate a field. For example, we are about to introduce a 'locale' setting which would need to map to 'lang' db key, with current structure it's impossible to have many:1 mapping because it's impossible to have duplicate keys in the JS object ("hash")
2021-03-04 15:24:08 +13:00
Hannah Wolfe
edb626af18 Added custom integration feature limit
refs: https://github.com/TryGhost/Team/issues/510

- in the case that host config is provided, prevents the creation of custom integrations
2021-03-03 20:15:49 +00:00
Peter Zimon
a1aa18e5b2 Minor email copy updates 2021-03-03 17:42:28 +01:00
Peter Zimon
3064addcb5 Refined email template spacings 2021-03-03 17:34:35 +01:00
Peter Zimon
f71cc4e7b0 Updated email styles
- updated welcome, password reset and invitation email styles to match with 4.0 changes
2021-03-03 17:15:37 +01:00
Hannah Wolfe
c7b843471f Added limit service
refs: https://github.com/TryGhost/Team/issues/510

- added and wired up the new limit service, which is a lazy-loaded service
- this handles the case that there are host limits set in config, and wraps all the logic needed for detecting exceeded limits & throwing limit errors
- expects limits to be set in config under `host_settings.limits`
- supported limits are managed in the limit service, outside of core
2021-03-03 13:43:05 +00:00
Fabien 'egg' O'Carroll
b7a092a24a
🐛 Stopped Ghost crashing when sending bulk emails (#12718)
refs https://github.com/TryGhost/Ghost/issues/12610
refs https://github.com/mailgun/mailgun-js-boland/blob/v0.22.0/lib/request.js#L285-L333

The mailgun domain is used by the mailgun API to construct the URL for
the API. e.g for a domain of "mg.example.com" the URL for the API
messages would look like:

https://api.mailgun.net/v3/mg.example.com/messages

One weird thing about the mailgun API is that if the path does not map
to an API endpoint, then instead of a 404, we get a 200, with a body of
"Mailgun Magnificent API".

The `mailgun-js` library which we use, expects a JSON response, and will
return a body of undefined if it does not get one.

This all resulted in us trying to read the property `id` of an undefined
`body` variable. The fix here is to reject the containing Promise, if
there is no body. So that the default error handling will kick in.
2021-03-03 09:34:44 +00:00
Naz
b17a965c88 Switched members services to use v4 API
refs https://github.com/TryGhost/Team/issues/513

- Bumping version as this is now the latest stable API.
- The change might cause this particular side effect (acceptable for major version): if a member requests a login URL when the instance is on 3.x version and site owner upgrades do 4.x before member authenticates through login link, the login will fail and the member will have to request a new login URL.
2021-03-03 14:42:03 +13:00
Kevin Ansfield
e177e7e1b5 Fixed analytics require error
refs a06064b115

- fixed incorrect path in require for base email analytics service
2021-03-02 08:26:42 +00:00
Kevin Ansfield
a06064b115 Fixed email analytics require error
refs https://github.com/TryGhost/Ghost/pull/12541
refs https://github.com/TryGhost/Ghost/pull/12689

- the analytics job had been switched to create it's own instance of EmailAnalyticsService to avoid requiring logging but the analytics extraction branch was created before this change and wasn't picked up when merging
- pulled `queries` option object into a separate file for re-use
- updated `fetchLatest` job to conform to extracted library interface
2021-03-02 08:22:11 +00:00
Kevin Ansfield
11802ebee0
Extracted email analytics library code to external packages (#12541)
closes https://github.com/TryGhost/Team/issues/493

- all functionality except that directly related to Ghost's database and business logic now lives in external packages
  - @tryghost/email-analytics-service
  - @tryghost/email-analytics-provider-mailgun
2021-03-02 07:26:33 +00:00
Naz
9a6bfd0e71 Switched default API version for JWT
refs https://github.com/TryGhost/Ghost/issues/12716

- This change is made to avoid using now deprecated 'v2' API anywhere in the codebase.
- Switching to 'v4' should not cause sideeffects as this parameter is always present within the URL when Admin API is used
2021-03-02 15:19:33 +13:00
Naz
b3542e9176 Added JSDocs to MEGA service
refs d5cf0fc03e

- Makes sure required `apiVerson` parameter in MEGA is visible and easy to discover
2021-03-02 14:32:43 +13:00
Naz
d5cf0fc03e Bumped MEGA related serialization code to use v4 API
refs https://github.com/TryGhost/Ghost/issues/12716

- The code in serializePostModel was broken and always defaulted to 'v3'!  It refered to non-existent `model.get('api_version')` there's no such field in posts model! Changed the implementation so that the API version is passed in as a parameter to the method instead
- The style of providing "defaults" everywhere creates a need for future maintenance when we bump the version e.g in Ghost v5. Maybe reworking these methods to require a passed version and throwing an error instead would be more maintainable long-term?
2021-03-02 14:31:01 +13:00
Naz
72f25a7099 Bumped webhook API defaults to v4
refs https://github.com/TryGhost/Ghost/issues/12716

- The default API in Ghost v4 will be `'v4'`. When a new webhook is created or any data passes through serialization it should assume `v4` as a fallback if not specified.
2021-03-02 12:31:52 +13:00
Fabien O'Carroll
24afc5d4ff Fixed status code for unauthenticated requests
refs https://github.com/TryGhost/Team/issues/498

Requests to fetch the data of the logged in member made without a
session cookie were responsing with 400 Bad Request. This was incorrect
and always should have been a 401 Unauthorized.
2021-03-01 16:19:28 +00:00
Fabien O'Carroll
41d9610aef Removed warning logging from members 'middleware'
refs https://github.com/TryGhost/Team/issues/498

This was filling up logs with less than useful information - for every
single request made by a non-member to the frontend. Be gone!
2021-03-01 16:05:34 +00:00
Fabien O'Carroll
3eb8b91a6b Fixed paid flag on members
refs https://github.com/TryGhost/Ghost/issues/12602

As part of the member events, we added a third status of 'comped'.
Members with a status of 'comped' should still be considered paid, so
this fixes the definition of the paid flag to take that into account.
2021-02-25 14:26:41 +00:00
Fabien 'egg' O'Carroll
714e081efc
Removed hardcoded currency symbols (#12700)
refs https://github.com/TryGhost/Team/issues/473
refs https://github.com/TryGhost/Admin/commit/638e5027

We no longer want to maintain a list of currency symbols now that we support 100+ currencies,
so we will be relying on the `{{price}}` helper in themes and `Intl.NumberFormat` elsewhere.
2021-02-25 09:49:07 +00:00
Thibaut Patel
da7859c5e7 🐛 Fixed ENOTFOUND error handling on NodeJS >=13
no issue
2021-02-23 19:37:03 +01:00
Kevin Ansfield
08e1268aed
Added migration to remove surrounding <> in email_batches.provider_id (#12673)
refs https://github.com/TryGhost/Team/issues/221#issuecomment-759105424

- Mailgun responds to an email send with a provider id in the format `<x@y.com>` but everywhere else it's used in their API it uses the format `x@y.com`
- updates email batch save to strip the brackets, and migration removes brackets from existing records so we no longer have to add special handling for the stored id any time we use it
2021-02-23 08:48:21 +00:00
Kevin Ansfield
42e452b127
Removed models require from analytics job (#12689)
refs https://github.com/TryGhost/Ghost/issues/12496

By requiring the models layer the shared logging util was being required as a side-effect causing the open file descriptors problem to continue. Removing logging from the models layer isn't feasible due to deep require chains spreading across the codebase, it's much quicker to remove the need for models in the analytics job.

- models layer was only needed because it's used by the session service
- updated analytics job to create it's own instance of `EmailAnalyticsService` rather than the default instance in order to pass in custom dependencies
- pass in custom `logging` object that uses `parentPort.postMessage` as a way of writing log output
- pass in custom `settings` object that returns settings that have been manually fetched and cached during job instantiation
2021-02-22 12:10:19 +00:00
Naz
12a1c60424 Added custom worker message handler
refs https://github.com/TryGhost/Ghost/issues/12496

- Handling logging in the main thread avoids file handle leaks which happen due to leaky implementation of bunyan logger (see referenced issue for more context)
- Bumped job-manager version to allow for `workerMessageHandler` callback funciton
2021-02-22 20:02:00 +13:00
Naz
8a718ca99a Migrated jobs to use parentPort.postMessage
refs https://github.com/TryGhost/Ghost/issues/12496

- Using ghost-ignition logging caused file handle leaks. As there is no straight-forward way to handle write streams with bunyan (ghost-ignition's underlying logging library) this method of logging was chosen as an alternative to keep the amount of open file handles to minimum
- The follow up changes will include custom formatter for jobs service which should make logging match the same format  as has been used inside the jobs
2021-02-22 20:02:00 +13:00
Thibaut Patel
9e93ed7ef0 Adding sentry logging to all ping errors
issue TryGhost/Team#362
2021-02-19 14:00:50 +01:00
Thibaut Patel
6e58846c94 Added sentry logging to 429 ping errors
issue https://github.com/TryGhost/Team/issues/362
2021-02-18 19:21:15 +01:00
Naz
d0e0760dae Hardcoded labs to always return members:true
refs https://github.com/TryGhost/Ghost/issues/10318

- Because members is effectively "enabled" by default starting Ghost 4.0 have hardcoded labs setting to be such. The alternative of removing this key from labs would be equivalent to `labs.members === false` which is undesireable and would mean additional work on theme developer's side.
2021-02-17 19:55:16 +13:00
Thibaut Patel
58be6d23db Split the "Too many requests" error in ping service
issue https://github.com/TryGhost/Team/issues/362
2021-02-17 13:18:45 +13:00
Fabien 'egg' O'Carroll
f4cb5c57c6
Updated members_status_events table (#12647)
refs https://github.com/TryGhost/Ghost/issues/12602

* Updated members_status_events table

By replacing the `status` column with a `from_status` and `to_status`
column, we are able to track the changes between multiple statuses
easier, and accumulate the data. e.g. the delta of paid members in a
given time range is the sum of the `to_status` columns set to 'paid'
minus the sum of the `from_status` columns set to 'paid' within that
time range

* Updated MEGA to handle addition of 'comped' status

With the addition of the 'comped' status, we need to ensure that MEGA
will still send emails to the correct recipients. I've opted to use an
"inverse" filter, as that is the intention of the free/paid split in
MEGA - as far as MEGA is concerned, "free" is the opposite of "paid"

* Updated customQuery for MemberStatusEvent

With the `status` column replaced with `from_status` and `to_status`
this allows us to fix and update the customQuery to correctly accumulate
the data into deltas over time, broken down by day.

* Populated members_status_events table

As the table will be used to generate deltas, we need to backfill the
data so that existing sites will be able to sum up the deltas and
calculate correct data.

The assumptions used in backfilling is that a Member's current status,
is their only status.
2021-02-16 10:38:36 +00:00
Fabien O'Carroll
8880cc82c7 Refactored usage of the pre v4 "slack" setting
refs https://github.com/TryGhost/Ghost/issues/10318

- Change/compatibility table for this changeset:

Importer

+ "slack_username"  import
- "slack_url" ignore
+ "slack.username" import
- "slack.url" ignore

v3 & canary/v4 Admin APIs
GET /settings/ (browse)

+ "slack_username" present in response
+ "slack_url" present in response
+ "slack" present in response

GET /settings/:settingName (read)

+ "slack_username" present in response
+ "slack_url" present in response
+ "slack" present in response

PUT /settings/ (edit)

+ "slack_username" updates setting
+ "slack_url" updates setting
+ "slack" !NOTE! updates setting unless "slack_username" or "slack_ur"l are set (updated keys take priority)

v2 Admin API
GET /settings/ (browse)

- "slack_username" NOT present in response
- "slack_url" NOT present in response
+ "slack" present in response

GET /settings/:settingName (read)

- "slack_username" ignored (404)
- "slack_url" ignored (404)
- "slack" present in response

PUT /settings/ (edit)

- "slack_username" ignored (no error)
- "slack_url" ignored (no error)
+ "slack" updates setting
2021-02-15 15:25:54 +13:00
Fabien O'Carroll
7cf0252392 Added MemberPaidSubscriptionEvent model
refs https://github.com/TryGhost/Ghost/issues/12602

- Adds the aggregateMRRDeltas option to the `findAll` method,
allowing us to calculate MRR over time
2021-02-12 13:55:14 +00:00
Fabien O'Carroll
a68d037cef Added MemberEmailChangeEvent model
refs https://github.com/TryGhost/Ghost/issues/12602
2021-02-12 13:55:14 +00:00
Fabien O'Carroll
8e7d11ce40 Added MemberLoginEvent model
refs https://github.com/TryGhost/Ghost/issues/12602
2021-02-12 13:55:14 +00:00
Fabien O'Carroll
a8ee271336 Added MemberPaymentEvent model
refs https://github.com/TryGhost/Ghost/issues/12602

- Adds the aggregatePaymentVolume option to the `findAll` method,
allowing us to calculate volume over time
2021-02-12 13:55:14 +00:00
Fabien O'Carroll
1af6be63f0 Added MemberStatusEvent model
refs https://github.com/TryGhost/Ghost/issues/12602
2021-02-12 13:55:14 +00:00
Sanne de Vries
67b1745bcb Fixed bug for newsletter bookmarks without author displaying redundant separator 2021-02-09 09:35:45 +01:00
Sanne de Vries
118ceae29c Fixed gmail bug for newsletter bookmark cards not displaying correctly 2021-02-08 16:49:13 +01:00
Fabien O'Carroll
3dbc7ef5b4 Passed MemberSubscribeEvent to members-api
refs https://github.com/TryGhost/Ghost/issues/12602

This allows the Members module to create MemberSubscribeEvents when
appropriate
2021-02-05 16:02:08 +00:00
Fabien 'egg' O'Carroll
73f6fd8c51
Enabled Members for all sites (#12582)
no-issue

This removes all references to the members labs setting, any code that was run conditionally behind this flag now runs unconditionally. 

* Removed usage of Members labs flag
* Removed tests for Members disabled
* Added dynamic keypair generation for when setting is missing
2021-02-02 16:12:53 +00:00
Rishabh Garg
26ee648397
🏗 Flattened members subscriptions data in API (#12581)
no refs

- Updates member model serializer to directly set subscriptions on member object instead of `stripe.subscriptions`
- Updates all references to members subscriptions from nested `stripe.subscriptions` to `subscriptions`
- Updates v3 API serializer to still use `stripe.subscriptions`
- Updates tests
2021-02-02 16:12:53 +00:00
Fabien 'egg' O'Carroll
6af2706f10
Updated Admin API and Mega to use status flag (#12579)
no-issue

* Removed support for paid param from v3 & canary API
* Updated active subscription checks to use status flag
* Updated MEGA to use status filter over paid flag
* Removed support for paid option at model level
* Installed @tryghost/members-api@1.0.0-rc.0
* Updated members fixtures
2021-02-02 16:12:51 +00:00
Fabien 'egg' O'Carroll
229295d671
Exited process when members-api fatally errors (#12489)
closes https://github.com/TryGhost/Ghost/issues/12448

This adds handling to force the process to exit when we receive an error
from the members-api on initialisation.
2021-02-02 16:12:10 +00:00
Rish
c37c66609d
Removed portal-action and requestSrc
no issue

We added `portal-action` and `requestSrc` in 3.x to allow Portal to handle notifications only for auth actions trigged while using it directly, so that existing themes are not affected in any way. Going forward in 4.0, we don't want to have any special handling in backend for Portal but instead expect themes to handle any Portal specific behavior directly.

- Removes setting of `portal-action` for auth actions like signup
- Removes `requestSrc` being passed through to determine portal actions
2021-02-02 16:11:34 +00:00
Fabien O'Carroll
8631710b68 Fixed Members importer usage of linkStripeCustomer
no-issue

The method signature was updated in the refactor and this was missed
2021-01-26 21:26:19 +05:30
Aileen Nowak
c9d1fa0b96 Updated links to ghost.org sites
no issue

Follow-up task of the updated Ghost Docs structure. Updated links reflecting the new structure to prevent unnecessary 404s and redirects.
2021-01-20 09:59:45 +13:00
Thibaut Patel
6a49df36cb Moved the users service to use our DI pattern
refs #12537
2021-01-15 17:58:36 +01:00
Sanne de Vries
c8931a47b5 Updated newsletter blockquote font styling 2021-01-15 15:42:01 +01:00
Thibaut Patel
3576832c0c Moved the OEmbed service to use our DI pattern
refs #12537
2021-01-15 15:35:30 +01:00
Rish
a1663bea91 💄Updated newsletter design to use accent color
no issue

- Updates newsletter design to use accent color for blockquote border if present
2021-01-15 19:47:32 +05:30
Rish
417a4c7281 Added firstpromoter id to member site data
no issue

Adds FirstPromoter id to member site data from settings, allowing Portal to setup FirstPromoter integration if enabled
2021-01-15 19:09:59 +05:30
Thibaut Patel
0bea158832 Moved the service logic out of the users controller
refs #12537
2021-01-14 16:41:59 +01:00
Thibaut Patel
d8245216ec Moved the service logic out of the oembed controller
refs #12537
2021-01-14 16:41:33 +01:00
Naz
6d2dfff5b5 Refactored invites module into a class
refs #12537
2021-01-14 19:42:38 +13:00
Naz
01855389a5 Moved invites controller code into a service module
refs #12537

- Part of the controller code refactor into services to avoid code duplication during introduction of a new API.
2021-01-14 19:25:16 +13:00
Naz
1cec604318 Fixed update check tests
refs #12537

- Refactored overuse of rewire mocking blog-version.
- Fixed a bug introduced along the way when duplicate notifications errored instead of returning empty result
2021-01-14 18:30:09 +13:00
Naz
935ffdd0f6 Refactored notifications module into a class
refs #12537

- Classes with DI friendly constructors are the pattern that are followed across the codebase
2021-01-14 16:19:15 +13:00
Naz
d2f0f0d7bc Moved notifications controller code into a service module
refs #12537

- notifications controllers were overbloated with non controller related code and were identical. It is important to reduce unnecessary code ahead of v4 API introduction
- Follow up commit will transform newly created module into a class following DI pattern
2021-01-14 15:55:55 +13:00
Naz
aff4a7055e Extracted members controller's import method
refs #12537

- Moved code related to the importer into the MembersImporter class to  keep the controller code light
2021-01-14 14:05:22 +13:00
Naz
105bc6cff3 Added jsdoc to MembersStats class
no issue
2021-01-14 12:59:17 +13:00
Naz
10304c79a4 Refactored members stats module into class
refs #12537

- Class syntax is preferred over module functions because of constructor parameter injection (DI) which allows for easier module decoupling and testing
2021-01-13 22:43:05 +13:00
Naz
e62c4075f0 Moved members stats code out of members controller
refs #12537

- `stats` method in members controller is quite big and does much more then controller method code should - few calls to relevant modules
- Extracted code "as is" into members serivce
- Next step will be to refactor this module as a class pattern with DI parameters
2021-01-13 22:43:05 +13:00
Kevin Ansfield
63f7f9a827 🐛 Disabled auto-unsubscribe of members on permanent email failure events
refs https://github.com/TryGhost/Team/issues/446

Mailgun permanent failure events do not always correspond to unsubscribe-level events as originally thought, meaning some members could be unsubscribed unexpectedly due to delivery hiccups.

- disabled auto-unsubscribe on permanent failure events in the analytics event processor
- list maintenance will be added back in the future via alternative means
2021-01-12 18:40:31 +00:00
Naz
b2e7d2bf06 Bumped job-manager version to 0.7.0
closes https://github.com/TryGhost/Ghost-Utils/issues/122
2021-01-06 17:48:05 +13:00
Rish
d2543462fa 🐛 Fixed reply-to address not set for newsletters
closes https://github.com/TryGhost/Ghost/issues/12492

The changes to email processing models had set replyTo address for an email batch as `reply_to` instead of `replyTo` which was not picked by mailgun service for setting newsletter reply address
2021-01-05 17:58:55 +05:30
Naz
c1e3788570 Added central error handler to job manager
refs https://github.com/TryGhost/Ghost-Utils/issues/118

- Duplicating error handling across jobs is not best developer experience. Also, having custom error handling logic did not allow for recommended worker script behavior: allowing for unhandled exceptions to bubble up and be managed by parent process
2020-12-14 18:01:41 +13:00
Peter Zimon
c856d712ec
Import email refinements (#12473)
* Updated import email subject

* Updated import email title

* Fixed copy in import email to reflect if there was at least one member added
2020-12-10 12:19:26 +01:00
Kevin Ansfield
9586d1ce56 Fixed sqlite storing member email open rate as floats
no issue

- sqlite will store a float in an integer column due to it's type affinity resulting in long decimal numbers in the UI when we're expecting an integer
- use the `ROUND()` function to ensure we're storing integers in place of floats when performing open rate average calculations
2020-12-10 10:00:25 +00:00
Fabien O'Carroll
cfad396652 Exported MembersCSVImporter from Members service
no-issue

This will allow the controller to be updated to use the new importer
2020-12-09 19:25:57 +00:00
Fabien O'Carroll
ff7773d1cf Added new members importer module
no-issue

This module encapsulates the work around performing imports, it
currently uses the concept of a "Job" which at the moment is not
persisted to the database, however when we want to look at resuming
imports after a server restart, this should give us the flexibility to
do it.
2020-12-09 19:25:57 +00:00
Fabien O'Carroll
c7359a083f Added email template for completion of background jobs
no-issue

This will be used to generate email content used for notifying users
that their import has been completed.
2020-12-09 19:25:57 +00:00
Fabien O'Carroll
83c5270217 Removed support for batched CSV importer
no-issue

We are rewriting the Members CSV importer to use background jobs, the
batched importer will no longer be used locally.
2020-12-09 19:25:57 +00:00
Kevin Ansfield
baf635e27b Updated email stats aggregator for new member count columns
refs https://github.com/TryGhost/Ghost/issues/12461

- added two default aggregations for overall email count and opened email count
- when number of tracked emails is sufficient add the open rate aggregation to the update query
2020-12-09 13:14:39 +00:00
Kevin Ansfield
567eb6325f
Added members.email_open_rate aggregation to email analytics (#12458)
refs https://github.com/TryGhost/Ghost/issues/12421
requires https://github.com/TryGhost/Ghost/pull/12457

- updates stats aggregator to calculate and store an open rate for each member
  - uses two queries because I couldn't find a reasonable approach to perform the update in a single query as per the email aggregation
  - benchmarked locally at <1sec/1000members
  - will not store an open rate unless the number of tracked emails sent to a member is above a certain threshold (defaults to 5) to avoid new members being heavily weighted
- fixes typo in EmailAnalytics that was stopping member stats from being aggregated
2020-12-08 12:43:10 +00:00
Kevin Ansfield
84ae8585c6 Added guard for page.items existing in Mailgun response
no issue

- it's possible to get Mailgun responses where the `items` array doesn't exist so we need to guard against that so we don't error
2020-12-07 11:07:25 +00:00
Daniel Lockyer
47397ca7a6
Updated logging message for webhooks
- removes double space and tidies up the output
2020-12-03 17:50:53 +00:00
Kevin Ansfield
a945ae7c02 Added sentry error capturing to email analytics jobs
no issue

- jobs operate in their own process and so do not share any of the error capturing that is configured in the parent process
2020-12-02 14:57:15 +00:00
Kevin Ansfield
c40a8e978d Fixed incorrect knex syntax
refs 166d072cd4

- accidentally committed an earlier attempt 🤦🏻‍♂️ updated with the valid syntax
2020-12-02 14:53:10 +00:00
Kevin Ansfield
166d072cd4 Fixed email analytics job not being registered when creating an email
no issue

- job registration was checking for submitted emails in it's email count but the job registration method is called as soon as an email is created meaning the email has a status of 'pending' which prevented the analytics job from being started until a second email was sent
2020-12-02 14:47:18 +00:00
Kevin Ansfield
f802128cfc
Added emailAnalytics config feature flag (#12443)
no issue

- email analytics may be desirable to fully switch off in certain circumstances, when that happens we want to prevent related background jobs from running and expose the feature flag via the config endpoint in the Admin API so that clients can adjust accordingly
2020-12-02 13:22:12 +00:00
Kevin Ansfield
2951eb9eaf Added missing moment require
refs 92ef83c61a
2020-12-02 12:15:27 +00:00
Kevin Ansfield
92ef83c61a Adjusted email jobs registration to check for emails newer than 30 days
no issue

- if emails are older than 30 days we wouldn't be able to fetch any analytics for them and if a site used emails in the past but is no longer using them it doesn't make sense to keep potentially expensive background worker threads spinning up
2020-12-02 12:14:22 +00:00
Daniel Lockyer
947603d0e3
Added original error to thrown error in mega code
- the original error is not propagated up with the error we throw, so it
  can sometimes be difficult to see what's going on down the line
2020-12-02 09:04:20 +00:00
Kevin Ansfield
d675278b0b
Prevented scheduling of recurring analytics jobs when not using emails (#12441)
no issue

- recurring jobs spin up worker threads which can be quite CPU intensive even when not performing much processing, this can be problematic in environments where there are many Ghost instances running
- updated the email job scheduling to be skipped on bootup when there are no emails in the database and to be started when the first email is created as long as we're not in testing env
- increase analytics job schedule from every 2 minutes to every 5 minutes to help spread the load further across instances
2020-12-02 08:17:44 +00:00
Kevin Ansfield
249fd4f06a Fixed email count check in email-analytics service
no issue

- raw knex `.count()` does not return a straight number, we need to handle an array of rowDataPacket objects
2020-12-01 15:02:04 +00:00
Kevin Ansfield
db7fff0b2a Fixed misleading emailAnalytics.fetchLatest debug statement 2020-12-01 10:16:41 +00:00
Kevin Ansfield
d604f96a9c Added no-emails guard check when fetching latest email events
no issue

- replicates the same guard used in `fetchLatest()` to prevent contacting Mailgun when there are no emails in the database
2020-12-01 10:15:31 +00:00
Scott Cabot
da5586e5aa Fixed user import with no created_at date breaking graph
- users imported from CSV with no created_at date where having their created_at date being stored as an int rather than a datetime.
- this was causing parsing issues with the graph so this commit fixes the formatting
2020-11-30 11:21:22 +00:00
Kevin Ansfield
a29ac2691a
Fixed sqlite3 errors when email analytics jobs run (#12431)
no issue

- the 4.2.0 version of `sqlite3` that we're using is not compatible with `worker_threads`
- 5.0.0 should add support this but there are other errors
- 5.0.1 is released but not published (https://github.com/mapbox/node-sqlite3/issues/1386)
2020-11-26 15:12:12 +00:00
Kevin Ansfield
dcafebe379 Fixed linting 2020-11-26 13:11:18 +00:00
Kevin Ansfield
717543835c
Added email analytics service (#12393)
no issue

- added `EmailAnalyticsService`
  - `.fetchAll()` grabs and processes all available events
  - `.fetchLatest()` grabs and processes all events since the last seen event timestamp
  - `EventProcessor` passed event objects and updates `email_recipients` or `members` records depending on the event being analytics or list hygiene
    - always returns a `EventProcessingResult` instance so that progress can be tracked and merged across individual events, batches (pages of events), and total runs
    - adds email_id and member_id to the returned result where appropriate so that the stats aggregator can limit processing to data that has changed
    - sets `email_recipients.{delivered_at, opened_at, failed_at}` for analytics events
    - sets `members.subscribed = false` for permanent failure/unsubscribed/complained list hygiene events
  - `StatsAggregator` takes an `EventProcessingResult`-like object containing arrays of email ids and member ids on which to aggregate statistics.
  - jobs for `fetch-latest` and `fetch-all` ready for use with the JobsService
- added `initialiseRecurringJobs()` function to Ghost bootup procedure that schedules the email analytics "fetch latest" job to run every minute
2020-11-26 13:09:38 +00:00
Fabien 'egg' O'Carroll
18b87d9734
Included all subscriptions in stripeSubscriptions (#12414)
refs https://github.com/TryGhost/Ghost/issues/12256

We no longer want to filter out cancelled subscriptions, so we are able
to remove the whereIn clause of the relation.

* Fixed paid flag on member
* Fixed content gating for members

Now that the subscriptions for a member include all of them, we must
explicitly check that the member has an active subscription in order to
consider them "paid"
2020-11-25 10:39:07 +00:00
Kevin Ansfield
47e81e3ca1 Fixed error when creating emails when track opens is disabled
no issue

- ensure `email.track_opens` is a boolean rather than `null`
2020-11-25 08:08:15 +00:00
Kevin Ansfield
7bbbeeb5b0 Fixed linting 2020-11-23 19:58:34 +00:00
Kevin Ansfield
a8d8598003 Updated mega service to use open tracking from settings
refs https://github.com/TryGhost/Ghost/issues/12390

- switched from enabling open tracking based on `enableDeveloperExperiments` to using the track-opens setting
2020-11-23 18:53:55 +00:00
Kevin Ansfield
4c96aa5c95 Fixed bulk email only having 'bulk-email' tag in certain circumstances
no issue

- the `'bulk-email`' tag was only being added to bulk emails if another more specific tag was set up via config
- we always want the `'bulk-email'` tag to be present for better event filtering
2020-11-23 18:34:17 +00:00
Peter Zimon
bf9820f97d Fixed sans-serif link style
no refs.
- fixed regression that links in emails with sans-serif setting wasn't underlined
2020-11-23 15:50:02 +01:00
Fabien 'egg' O'Carroll
f8b617af64
Supported redirecting externally after signup (#12391)
refs #12391

Adds support for redirecting to external URL's after successful signup for members.
2020-11-23 09:36:45 +00:00
Thibaut Patel
5fefa9fe96 Disabled Personal Tokens of inactive/locked users
no issue
2020-11-20 09:53:18 +01:00
Thibaut Patel
be4146e324 Enabled authentication using an api key with a user_id
no issue

- Enables the Personal Tokens feature.
2020-11-20 09:53:18 +01:00
Fabien 'egg' O'Carroll
38d19c999d
Removed search params from custom signup redirects (#12386)
refs #12366

This means that custom redirects will not trigger the built in notifications handled by Portal.
2020-11-19 14:47:08 +00:00
Fabien 'egg' O'Carroll
9ba3e96790
Redirected to custom path on successful signup (#12372)
refs #12366 

This implements redirection based on the settings for successful member sign up!
- Removes support for redirecting to `req.path` afterwards, this was never used and
  we now have a more configurable implementation.
- Retains redirection to the homepage for unsuccessful sign up (invalid/expired token)
2020-11-19 09:58:32 +00:00
Peter Zimon
02d78a89aa Udpated email template footer spacings
no refs.
- updated padding between Ghost badge and rest of the footer
2020-11-16 12:31:37 +01:00
Peter Zimon
cfac18cb32 Newsletter email template refinements
no refs.
- updated color styles, links for footer text
- refined spacings for email footer paragraphs
2020-11-13 16:58:56 +01:00
Fabien 'egg' O'Carroll
3f9053ae94
Added setting for newsletter footer content (#12359)
no-issue

We want to give users to ability to customise the content of their newsletter, and the first step
toward that is a setting in which we can store text or html to embed in the template
2020-11-13 13:18:47 +00:00
Peter Zimon
215bfd0a7a
Added newsletter design settings (#12352)
refs https://github.com/TryGhost/Ghost/issues/12355

- Adds new default settings for newsletter customisations - `newsletter_show_badge`, `newsletter_show_header` and `newsletter_body_font_category`
- Adds migrations to update group for new settings
- Add migration to update settings based on existing config value for newsletter settings
- Passes new newsletter settings to newsletter template and updates design based on them
- Fix tests
2020-11-11 18:26:11 +05:30
Fabien 'egg' O'Carroll
6140a98351
Updated newsletter functionality to use email_recipient_filter (#12343)
no-issue

* Used email_recipient_filter in MEGA

This officially decouples the newsletter recipients from the post
visibility allowing us to send emails to free members only

* Supported enum for send_email_when_published in model

This allows us to migrate from the previously used boolean to an enum
when we eventually rename the email_recipient_filter column to
send_email_when_published

* Updated the posts API to handle email_recipient_filter

We now no longer rely on the send_email_when_published property to send
newsletters, meaning we can remove the column and start cleaning up the
new columns name

* Handled draft status changes when emails not sent

We want to reset any concept of sending an email when a post is
transition to the draft status, if and only if, and email has not
already been sent. If an email has been sent, we should leave the email
related fields as they were.

* Removed send_email_when_published from add method

This is not supported at the model layer

* Removed email_recipient_filter from v2&Content API

This should not be exposed on previous api versions, or publicly

* Removed reference to send_email_when_published

This allows us to move completely to the email_recipient_filter
property, keeping the code clean and allowing us to delete the
send_email_when_published column in the database. We plan to then
migrate _back_ to the send_email_when_published name at both the
database and api level.
2020-11-06 17:32:23 +00:00
Fabien O'Carroll
44b43a4755 Fixed error with removal of global.Promise override
refs #12182

These were missed in the cleanup and were causing errors.
2020-11-06 16:49:10 +00:00
Fabien O'Carroll
0ce8e3171b Bound SSOAdapter methods to their instance
no-issue

This allows custom SSO adapters to store instance config and refer to
in their methods
2020-11-05 13:08:11 +00:00
Kevin Ansfield
aface9ed4c Enabled Mailgun open tracking when dev experiments is enabled
no issue

- set `emails.track_opens` to `true` when the `enableDeveloperExperiments` flag is set
- update mailgun bulk-email provider to pass the open-tracking header to Mailgun when the email's `track_opens` flag is set
2020-11-05 12:55:08 +00:00
Kevin Ansfield
7a74b78940 Included email-id as user variable when sending bulk emails via mailgun
no issue

- adding user variables via the mailgun API when sending emails means that events related to email have those variables attached to them
- adding the `email.id` value to user variables means we can easily associate mailgun events to emails, otherwise we'd have look up the batch via `email_batches.provider_id` then use a join to get back to the associated email
2020-10-29 16:37:42 +00:00
Rish
c4c6bdf353 Updated mail template for member email change
no issue

The email change verification template was using the same as for `subscribe`, which did not have the right messaging that needs to be communicated about the action thats happening in this case. Updates the email template to same as what we use for email verification for support/newsletter address
2020-10-29 17:40:19 +05:30
Rish
746ccbd7ed Added custom action param for magic links
closes https://github.com/TryGhost/Ghost/issues/12253

- Allows using custom action param for requests from Portal by using a new `requestSrc` option that is passed down when a request for magic link is made via Portal
2020-10-29 13:42:52 +05:30
Peter Zimon
43b7c7b344
Updated members admin redirect to new page(#12296)
no refs.

- Members settings were moved to a new page [here](https://github.com/TryGhost/Ghost-Admin/pull/1736)
- updated members related redirect for from/newsletter email address change to point to new page
2020-10-22 16:10:25 +05:30
naz
8ddf83f3c5
Fixed "no-shadow" linting error in server modules (#12287)
refs 143921948d

- Continuation of changes started in referenced commit
2020-10-20 12:02:56 +13:00
Rish
54e638310c Added default support address for member site data
no issue

- In a case where support address is null, member's site data won't load as there was no fallback.
- The fix adds default `noreply` as the support address for site data
2020-10-15 14:44:16 +05:30