Commit Graph

11735 Commits

Author SHA1 Message Date
Nikolay A. G
e5f2fa650b
Removed unmatched closing span tag on AMP pages (#12183)
no issue

- The unmatched closing </span> tag is ignored by the browser anyway and is not semantically correct
- Checked the history if there was any possible purpose behind it in the past but doesn't seem like it
2020-09-21 12:53:27 +12:00
Renovate Bot
f8a918395d
Update dependency @tryghost/members-csv to v0.3.1 2020-09-18 16:48:12 +00:00
Fabien O'Carroll
e8f530db6c Updated @tryghost/members-api@0.30.1
no-issue

This includes a fix to the usage of getMagicLink which affected
impersonation of members via the memberSigninUrl controller.
2020-09-18 17:46:30 +01:00
Fabien 'egg' O'Carroll
7c5a3bb537
Updated magic links to use shorter, single us, longer lived tokens (#12218)
no-issue

* Added SingleUseTokenProvider to members service

This implements the TokenProvider interface required by members-api to
generate magic links. It handles checking if the token is expired and
pulls out any associated data.

Future improvments may include the email in the error for expired
tokens, which would make resending a token simpler.

* Passed SingleUseTokenProvider to members-api

This sets up the members-api module to use the new single use tokens

* Installed @tryghost/members-api@0.30.0

This includes the change to allow us to pass a token provider to the members-api
2020-09-18 17:32:18 +01:00
Renovate Bot
3c0f2d1595
Update dependency @sentry/node to v5.24.2 2020-09-18 14:12:08 +00:00
Fabien 'egg' O'Carroll
812e4b682f
Added SingleUseToken model (#12215)
no-issue

This is a model for the tokens table, which handles the single use
aspect by customising the `findOne` method to automatically destroy the
model after reading from it
2020-09-18 15:05:56 +01:00
Renovate Bot
f01f088066
Update dependency @tryghost/magic-link to v0.6.0 2020-09-18 12:26:10 +00:00
Renovate Bot
e0f2fa76ec
Update dependency @tryghost/magic-link to v0.5.0 2020-09-18 11:46:09 +00:00
Renovate Bot
eb0163bde6
Update dependency @sentry/node to v5.24.1 2020-09-18 10:35:51 +00:00
Fabien O'Carroll
fe1e480e2e Updated getMagicLink call to handle Promise return
no-issue

refs: https://github.com/TryGhost/Members/commit/63942f03

The above commit updated all magic-link methods to be async, this change
ensures that we will handle a Promise return when it's updated in Ghost.
`await` has no effect on non Promise return values so it's safe to add
this now.
2020-09-18 10:49:18 +01:00
Renovate Bot
d0844ee853
Update dependency @sentry/node to v5.24.0 2020-09-18 08:42:57 +00:00
Fabien O'Carroll
a862a58b5f Updated the tokens table to have 32 char tokens
no-issue

After discussion with Matt, we decided that 192 bits for the token is a
good number, as it has no padding when base64 encoded and is more secure
than 128 bits, whilst still a managable size.
2020-09-17 17:33:43 +01:00
Fabien O'Carroll
01e62e3d79 Added tokens table to the database
no-issue

This is a table to store single use tokens for use in magic links, the
columns are as simple as possible at the moment and are designed as:
id - standard ObjectID like all of our tables
token - 128bit base64 encoded string
data - arbitrary data to store against the token
created_at - timestamp to allow for expiry to be implemented for tokens
2020-09-17 16:08:35 +01:00
Fabien O'Carroll
ed7adf16a2 Added migration util for adding new tables
no-issue

This pulls out some commonly used logic into a util for future use
2020-09-17 16:08:35 +01:00
Rish
ac146f1e3b Updated ownership verification email flow to use different From address
no issue

- In a recent change to ownership verification email flow, we changed the FROM address of ownership verification mails to use the same email as the one we are verifying, aka TO address.
- Email clients like Gmail flags off such emails as possible spam
- Fix updates the `FROM` address to `noreply@domain.com` where domain.com is domain for TO address
-  In case the TO is already noreply@domain.com, we use no-reply@domain.com to bypass the same address restriction.
2020-09-17 19:52:51 +05:30
Rish
7a3839fb14 Added new portal config flag
no issue

- The new Portal config flag allows switching on Portal conditionally with config
- The dev experiment flag still works for enabling Portal
- The flag currently defaults to `false` as Portal is still a beta feature and switched off by default
- We expose it on the admin api config endpoint so that the Ghost-Admin client can use it to conditionally render Portal settings
2020-09-17 19:40:34 +05:30
Nazar Gargol
bd16c2216a Fixed acceptance tests for posts_meta refactor
refs d9dbcf6147
2020-09-17 21:19:26 +12:00
Nazar Gargol
d177852d1f Fixed posts_meta relation in test fixtures
refs d9dbcf6147

- Fixes broken commit
2020-09-17 20:46:45 +12:00
Nazar Gargol
d9dbcf6147 Corrected data generator fixture with posts_meta fields
refs https://github.com/TryGhost/Ghost/pull/11102

- When splitting out posts_meta table out of posts table, the fixtures were not corrected.
- There were no acceptance/regression tests which involved posts_meta fields which is why the error was not detected
2020-09-17 19:34:36 +12:00
Rish
49abeaf398 Added blog domain for default support address
no issue

- Members site data was not appending blog domain for default support address which is `noreply`
- The change allows Portal to use default support address correctly
2020-09-17 12:33:16 +05:30
Peter Zimon
ff2d9ebdfd Updated copy on default content CTA
no refs.
- updated button copy for logged in free members on default content CTA
2020-09-16 16:41:38 +02:00
Nazar Gargol
46932c6d75 Removed unused messages from i18n file
no issue

- These messages are never used throughout the system. Removed to keep things tidy.
2020-09-16 15:46:22 +12:00
Nazar Gargol
e9204c9168 Updated invitations test coverage with more specific checks
refs https://github.com/TryGhost/Ghost/issues/11878

- Updated test cases in preparation to improve messaging in referenced
issue.
2020-09-16 14:24:24 +12:00
Kevin Ansfield
80af56b530
Store email batch and recipient records when sending newsletters (#12195)
requires https://github.com/TryGhost/Ghost/pull/12192

- added initial `EmailBatch` and `EmailRecipient` model definitions with defaults and relationships
- added missing `post` relationship function to email model
- fetch member list without bookshelf
    - bookshelf can add around 3x overhead when fetching the members list for an email
    - we don't need full members at this point, only having the data is fine
    - if we need full models later on we can push the model hydration into background jobs where recipient batches are fetched ready for an email to be sent
    - bookshelf model instantiation of many models blocks the event loop, using knex directly keeps concurrent requests fast
- store recipient list before sending email
    - chunk already-fetched members list into batches and insert records into the `email_recipients` table via knex
    - chunked into batches of 1000 to match the number of emails that Mailgun accepts in a single API request but this may not be the absolute fastest batch size for recipient insertion:
        | Batch size | Batch time | Total time |
        | ---------- | ---------- | ---------- |
        |        500 |       20ms |     4142ms |
        |       1000 |       50ms |     4651ms |
        |       5000 |      170ms |     3540ms |
        |      10000 |      370ms |     3684ms |
    - create an email_batch record before inserting recipient rows so we can effeciently fetch recipients by batch and store the overall batch status
2020-09-14 15:40:00 +01:00
Daniel Lockyer
4e2d3e3505 v3.33.0 2020-09-14 12:39:47 +01:00
Daniel Lockyer
1221306212 Updated Ghost-Admin to v3.33.0 2020-09-14 12:39:47 +01:00
Kevin Ansfield
76c1b60a4d
Added schema+migration for email_{batches,recipients} tables (#12192)
no issue

We want to store a list of recipients for each bulk email so that we have a consistent set of data that background processing/sending jobs can work from without worrying about moving large data sets around or member data changing mid-send.

- `email_batches` table acts as a join table with status for email<->email_recipient
  - stores a provider-specific ID that we get back when submitting a batch for sending to the bulk email provider
  - `status` allows for batch-specific status updates and picking up where we left off when submitting batches if needed
  - explicitly tying a list of email recipients to a batch allows for partial retries
- `email_recipients` table acts as a join table for email<->member
  - `member_id` does not have a foreign key constraint because members can be deleted but does have an index so that we can efficiently query which emails a member has received
  - stores static copies of the member info present at the time of sending an email for consistency in background jobs and auditing/historical data
2020-09-14 12:21:58 +01:00
naz
cbdc91ce48
Added Location header to API's POST request responses (#12186)
refs #2635

- Adds 'Location' header to endpoints which create new resources and have corresponding `GET` endpoint as speced in JSON API - https://jsonapi.org/format/#crud-creating-responses-201. Specifically:
    /posts/
    /pages/
    /integrations/
    /tags/
    /members/
    /labels/
    /notifications/
    /invites/

- Adding the header should allow for better resource discoverability and improved logging readability
- Added `url` property to the frame constructor. Data in `url` should give enough information  to later build up the `Location` header URL for created resource.
- Added Location header to headers handler. The Location value is built up from a combination of request URL and the id that is present in the response for the resource. The header is automatically added to requests coming to `add` controller methods which return `id` property in the frame result
- Excluded Webhooks API  as there is no "GET" endpoint available to fetch the resource
2020-09-14 22:33:37 +12:00
Renovate Bot
50436656a7
Update dependency eslint to v7.9.0 2020-09-14 02:05:46 +00:00
Renovate Bot
cc67f0f3f5
Update dependency ajv to v6.12.5 2020-09-14 00:04:47 +00:00
Rish
7b5401b268 Fixed loading default template style for cta behind flag
refs e3a0bb535f

- The default style template was incorrectly loaded from the stripe config check instead of dev config flag
2020-09-11 11:10:24 +05:30
Daniel Lockyer
453faff41e Merged 3.32.2 into master
v3.32.2

* tag '3.32.2':
  v3.32.2
  Updated Ghost-Admin to v3.32.2
2020-09-10 17:02:50 +01:00
Daniel Lockyer
ae64d4eb2d v3.32.2 2020-09-10 16:55:18 +01:00
Daniel Lockyer
6e86bbf8e0 Updated Ghost-Admin to v3.32.2 2020-09-10 16:55:17 +01:00
Talha
afa976066a
🐛 Fixed table constraint error when updating member's email with an already existing email (#12178)
closes #12045

- When member's email is updated to an already existing email of different member it caused table's unique constraint error, which was not handled properly. 
- Added handling for this error similar to one in members `add` method.
2020-09-10 16:03:57 +12:00
Nazar Gargol
453bc2c491 Fixed regression test for integrity check
refs 5582d030e3

- When not touching this area for longer time always forge following:  `routes.yaml` configuration file in /content/settings comes as a copy of `default-routes.yaml` file from frontend/services/settings/
- Always remember to clean up junk "default" files in the content/settings folder to make things less confusing!
2020-09-10 12:57:03 +12:00
Nazar Gargol
a8202eb80d Fixed regression test for integrity check
no issue

- The file that is better suited for integrity check is the `*-default.yaml` config because it's the one that gets copied through when there is none or configuration is broken
2020-09-10 11:20:38 +12:00
Nazar Gargol
5582d030e3 Added routes.yaml content checksum storage to the db
closes #11999

- When the routes.yaml file changes (manually or through API) we need
to store a checksum to be able to optimize routes reloads in the future
- Added mechanism to detect differences between stored and current routes.yaml hash value
- Added routes.yaml sync on server boot
- Added routes.yaml handling in controllers
- Added routes hash synchronization method in core settings. It lives in core settings
as it needs access to model layer. To avoid coupling with the frontend settings it accepts
a function which has to resolve to a routes hash
- Added note about settings validation side-effect. It mutates input!
- Added async check for currently loaded routes hash
- Extended frontend settings loader with async loader. The default behavior of the loader is
to load settings syncronously for reasons spelled in 0ac19dcf84
To avoid blocking the eventloop added async loading method
- Refactored frontend setting loader for reusability of  settings file path
- Added integrity check test for routes.yaml file
2020-09-10 10:54:57 +12:00
Renovate Bot
74ec67ac1c
Update dependency @sentry/node to v5.23.0 2020-09-09 15:53:46 +00:00
Talha
c8a6939b2e
🐛 Fixed error caused by accepting invitation with existing email (#12172)
closes #12060

- A 500 error what happening when invited user provided an email that is associated with an existing user
- Additional validation for existing email address was added to prevent invalid data hitting db constraint error
2020-09-09 11:58:53 +12:00
Rish
3c7a95aefa Added members support address to site data
refs https://github.com/TryGhost/members.js/issues/90

- Includes new members_support_address in member site data endpoint for Portal
2020-09-08 20:48:57 +05:30
Rishabh Garg
e3a0bb535f
Added default CTA to content helper (#12157)
no issue

ATM users have to add logic to their themes in order to automatically hide restricted content. The {{content}} helper is updated to return a default CTA box instead of the post content for restricted posts with default static text using site's accent color and opening Portal for relevant action. This is currently behind the dev experiment flag.

- Adds new default content helper template in case of restricted content
- Updates content helper to trigger new CTA template in case of restricted content
2020-09-08 12:49:36 +05:30
Renovate Bot
5d47398d93
Update dependency gscan to v3.5.7 2020-09-07 16:29:13 +00:00
Daniel Lockyer
c175a48b85 v3.32.1 2020-09-07 12:06:06 +01:00
Daniel Lockyer
8eda575fac Updated Ghost-Admin to v3.32.1 2020-09-07 12:06:05 +01:00
Daniel Lockyer
3287e1c921 v3.32.0 2020-09-07 11:55:00 +01:00
Daniel Lockyer
a97c1e232f Updated Ghost-Admin to v3.32.0 2020-09-07 11:54:59 +01:00
naz
fcc6401445
Added routes_hash setting (#12171)
refs #11999

- The `routes_hash` setting will be used during the boot process to update the hash
of currently loaded routes.yaml file in case it's different from last restart
2020-09-07 21:54:55 +12:00
Nazar Gargol
4606c93e4f Refactored headers function to use async/await
no issue

- The async/await syntax makes it easier to reason about the code. Because adding 'Location' header is in the works it's a prep-work in a sense
2020-09-07 15:36:06 +12:00
Rish
2d2fa1a0ba Fixed missing subscription data in member update endpoint
closes https://github.com/TryGhost/members.js/issues/94

- The member-api package was recently updated to work directly with models and needs explicit `withRelated` options to attack relations
- Without options, the endpoint was returning the default member data without subscriptions attached, which in Portal showed paid member as free
- Fix updates the middleware for updating member data to correctly pass the relations needed to populate the member
2020-09-04 17:11:04 +05:30