Commit Graph

326 Commits

Author SHA1 Message Date
Simon Backx
7b3712a15b
Added visible theme errors in admin (#16081)
refs https://github.com/TryGhost/Team/issues/2393

- During boot and loading the active theme, we now cache the result of
the gscan validation. Cache configuration can happen in
`adapters.cache.gscan`
- We now also return non-fatal errors when activating or adding a theme.
- When the `themeErrorsNotification` feature flag is on, we fetch the
active theme (which includes the validation information) when loading
admin
- If the currently active theme has errors, we show an error
notification that can open the error modal
- Added a new endpoint: `/ghost/api/admin/themes/active/` that returns
the result of the last gscan validation of the active theme. If no cache
is available, it will run a new gscan validation.
- Added new permissions for the active action/endpoint (author, editor,
administrator)
2023-01-06 13:44:27 +01:00
Naz
3e209218d7
Made resource mismatch error more specific
refs https://github.com/TryGhost/Toolbox/issues/497

- During gscan fatal error downgrade to non-fatal some of the deprecated helpers were a bit vague to debug with no information on which exact "resource" was invalid
- Added resource name to the log for clarity. Should make life easier when debugging potential get helper misuses
2023-01-06 18:05:07 +07:00
Simon Backx
913ad18b71
Added DomainEvents.allSettled utility method (#16075)
no issue

With the increased usage of DomainEvents, it gets harder to build
reliable tests without having to resort to timeouts. This utility method
allows us to wait for all events to be processed before continuing with
the test.

This change should speed up tests and make them more reliable.

It only adds extra code when running tests and shouldn't impact
production.
2023-01-04 14:30:35 +01:00
Fabien 'egg' O'Carroll
50e99e013c
Added migrations to drop and recreate the suppressions table (#16070)
There are currently two issues with the suppressions table:
  - We have some incorrect rows
  - We have missing UNIQUE constraints

We want to completely wipe the tables and start fresh, as well as make
sure that the UNIQUE constraints are added, so we drop the table
completely, and then re-add it, which should result in an empty
suppressions table with all expected constraints.

We've also renamed the `email_address` column to `email` to match our
`users` & `members` tables
2023-01-04 17:26:57 +07:00
Simon Backx
819d0d884c
Improved email verification required checks (#16060)
fixes https://github.com/TryGhost/Team/issues/2366
refs https://ghost.slack.com/archives/C02G9E68C/p1670232405014209

Probem described in issue.

In the old MEGA flow:
- The `email_verification_required` check is now repeated inside the job

In the new email service flow:
- The `email_verification_required` is now checked (didn't happen
before)
- When generating the email batch recipients, we only include members
that were created before the email was created. That way it is
impossible to avoid limit checks by inserting new members between
creating an email and sending an email.
- We don't need to repeat the check inside the job because of the above
changes

Improved handling of large imports:
- When checking `email_verification_required`, we now also check if the
import threshold is reached (a new method is introduced in
vertificationTrigger specifically for this usage). If it is, we start
the verification progress. This is required for long running imports
that only check the verification threshold at the very end.
- This change increases the concurrency of fastq to 3 (refs
https://ghost.slack.com/archives/C02G9E68C/p1670232405014209). So when
running a long import, it is now possible to send emails without having
to wait for the import. Above change makes sure it is not possible to
get around the verification limits.

Refactoring:
- Removed the need to use `updateVerificationTrigger` by making
thresholds getters instead of fixed variables.
- Improved awaiting of members import job in regression test
2023-01-04 11:22:12 +01:00
Simon Backx
789e2c96c0
🐛 Fixed SingleUseTokens being cleared on boot (#15999)
fixes https://github.com/TryGhost/Team/issues/1996

**Issue**
Our Magic links are valid for 24 hours. After first usage, the token
lives for a further 10 minutes, so that in the case of email servers or
clients that "visit" links, the token can still be used.

The implementation of the 10 minute window uses setTimeout, meaning if
the process is interrupted, the 10 minute window is ignored completely,
and the token will continue to live for the remainder of it's 24 hour
validity period. To prevent that, the tokens are cleared on boot at the
moment.

**Solution**

To remove the boot clearing logic, we need to make sure the tokens are
only valid for 10 minutes after first use even during restarts.

This commit adds 3 new fields to the SingleUseToken model:
- updated_at: for storing the last time the token was changed/used). Not
really used atm.
- first_used_at: for storing the first time the token was used
- used_count: for storing the number of times the token has been used

Using these fields:
- A token can only be used 3 times
- A token is only valid for 10 minutes after first use, even if the
server restarts in between
- A token is only valid for 24 hours after creation (not changed)

We now also delete expired tokens in a separate job instead of on boot /
in a timeout.
2023-01-04 09:49:39 +01:00
Simon Backx
803bb18b8d Improved unit test coverage for the output mappers
no issue

Increases the unit test coverage to a bit above 60% to increase margin
2023-01-03 15:58:31 +01:00
Hannah Wolfe
5f90baf6fe
Added Revue Importer (#16012)
refs: https://www.getrevue.co/app/offboard

- Revue is stopping all paid subscriptions on 20th Dec, and shutting down on Jan 18th.
- This update allows Ghost to accept and handle the zip file Revue are providing as an export in Labs > Importer
- It will import posts (as best as we can with the data provided) and subscribers as free members
- At present it doesn't import paid subscribers, as we don't have that info, but you can disconnect Revue from your Stripe account to prevent all your subscriptions being cancelled & there's the option this can be fixed later
- There will be further updates to polish up this tooling - this is just a first pass to try to get something in people's hands

Co-authored-by: Paul Davis <PaulAdamDavis@users.noreply.github.com>
2022-12-15 17:22:54 +00:00
Simon Backx
94e85dc09e
Reduced webhook calls when updating last_seen_at for email opens (#16008)
refs https://ghost.slack.com/archives/C02G9E68C/p1670960248186789

This reverts a change that was made here:

f4fdb4fa6c (r93071549),
but it still moved the original code to a new location in the
LastSeenAtUpdater

It includes a new E2E test to make sure timezones are supported
correctly.

- By not using Bookshelf, we no longer fire webhook calls
- By not using the member repository, we don't fetch and update the
member model and the labels relation in a forUpdate transaction, which
caused deadlock issues on the labels/members_labels tables which were
hard to resolve. Until now I was unable to find the other conflicting
transaction that caused this deadlock. Moving to raw knex (instead of
Bookshelf) and only updating the last_updated_at column should remove
the deadlock issue.

This removed the test for the email service wrapper, since it started
failing for an unknown reason and the test didn't make much sense (was
added earlier only to bump test threshold).
2022-12-14 17:50:42 +01:00
Hannah Wolfe
270f288c48 Added a timeout to the get helper
- The get helper can sometimes take a long time, and in themes that have many get helpers, the request can take far too long to respond
- This adds a timeout to the get helper, so that the page render doesn't block forever
- This won't abort the request to the DB, but instead just means the page will render sooner, and without the get block
2022-12-14 15:35:07 +00:00
Rishabh
ce45571dc0 Updated email faqs in portal to show sender email address
refs https://github.com/TryGhost/Team/issues/2348

- updates email faqs to show the sender email from newsletter data
2022-12-12 15:09:32 +05:30
Daniel Lockyer
16f3ba573b
Updated ghost_head snapshots
refs de97d90cf9

- this should have been updated when I deleted some extraneous config
  that was lurking in the testing config files
2022-12-07 10:24:44 +07:00
Naz
e170f293e3
Extracted sleep method to e2e framework module
no issue

- The sleep method has been used in 8 modules reimplementing the same thing over and over again. It's usually a sign of async event processing outside of the request/response loop. It's good to have a single point of implementation for a "hack" like this, so we could track it easier and address the even processing delay in a more optimal way centrally if it ever becomes a bottleneck
2022-12-05 17:26:29 +07:00
Fabien "egg" O'Carroll
c0e91c73d3 Added unique constraint to email_spam_complaint_events table
We can fetch the same event multiple times from Mailgun so we need to
be able to protect against inserting duplicate events in the
database. This will allow us to catch duplicate errors on insert when
handling complaint events.
2022-12-01 17:27:02 +07:00
Kevin Ansfield
457c672c6a
Added URL transform for image cards in Lexical documents (#15890)
refs https://github.com/TryGhost/Team/issues/2225

- updated the `formatOnWrite` transform map for posts to include the new `nodes` and `transformMap` options used by `urlUtils` for transforming node payload data
- added `nodes` to the `lexicalLib` module that pulls in our default nodes to be passed in to the URL transform utilities
- added `urlTransformMap` to the `lexicalLib` module that maps transform type and data type to URL transform utility functions that accept a single URL argument
2022-11-29 16:57:01 +00:00
Fabien "egg" O'Carroll
ba5b8ea33d Added email_spam_complaint_events table and model
refs https://github.com/TryGhost/Team/issues/2318

As with our other events, we've disabled destroy and edit static methods
on the bookshelf model.
2022-11-29 18:13:12 +07:00
Fabien "egg" O'Carroll
83be54af42 Added suppressions table and model
refs https://github.com/TryGhost/Team/issues/2317

This table is used for persisting the email suppression list.
We don't have a member_id column because emails, not members are suppressed.
2022-11-29 18:12:24 +07:00
Simon Backx
f5045b9bf7
Added email renderer implementation draft (#15877)
fixes https://github.com/TryGhost/Team/issues/2308

- Still has some missing pieces, but mostly works.
- Uses new handlebars template for emails
- When sending emails with the new email stability flag enabled, one
test email is now sent via the default smtp ghost mailer.
2022-11-29 11:27:17 +01:00
Rishabh
64ac47f4ef Added table to store email recipient failures
refs https://github.com/TryGhost/Team/issues/2291

When sending out mails to individual recipients, its possible that recipient gets a temporary or permanent failure for receiving the mail. Temporary failures can generally get resolved after a bit when the recipient’s mail server accepts the email, unlike permanent failures. For both customer visibility and easier debugging on what went wrong while delivering to a particular recipient, we’ll store the permanent/temporary failure for a recipient.

- migration adds a new table that stores the failure information for the recipients
2022-11-29 15:19:36 +05:30
Hannah Wolfe
62cd52ff98 Improved Sentry server side error reporting
refs: https://github.com/TryGhost/Team/issues/1121
refs: 54574025e0

- The previous change to fall back to a generic error on the server side is resulting in lots of much less useful Sentry reports
- For unexpected errors, change what's sent to Sentry back to context
- This is done by adding a specific code, so we don't have to match on a string that might change
- Also add the error type, id, code & statusCode as tags to the events - these are searchable structured data
- Adding code as a tag also makes it possible to find all errors that showed the generic message
2022-11-23 12:37:24 +00:00
Hannah Wolfe
36b80f2dda Updated sentry to use @tryghost/version
- As demonstrated by my comments in the boot file, I thought sentry was already depending on the version package
- IMO it's undesirable to require package.json directly esp when we have a tool setup and ready for tis
- Added a bunch of tests to show that Sentry does roughly what we think
2022-11-23 12:37:24 +00:00
Elena Baidakova
8d9d22e5a7
Added member API for removing email from suppression list (#15867)
closes TryGhost/Team#2306
2022-11-23 14:41:00 +04:00
Simon Backx
4b4592630f
Added new email batch sending service (#15865)
fixes https://github.com/TryGhost/Team/issues/2284

New batch sending flow (still WIP). Logs the sent emails instead of actually sending them. Unit tests are coming in later commits.
2022-11-23 11:33:44 +01:00
Rishabh
5780fc2a93 Added new source and source type columns to emails table
refs https://github.com/TryGhost/Team/issues/2280

We are moving away from storing html and plaintext on email and instead will store the email data in source and source_type columns which allows us to store the email in other formats like mobiledoc and lexical. Storing in those formats allows greater flexibility for later html generation

- adds new `source` column that stores `mobiledoc`/`lexical`/`html` data for a newsletter
- adds new `source_type` column that stores one of `mobiledoc`/`lexical`/`html` to identify type of source
2022-11-23 15:04:11 +05:30
Rishabh Garg
d3267dd5b0
Added columns to store error information for email batches (#15859)
closes https://github.com/TryGhost/Team/issues/2290

Currently, if the whole batch of email fails to send we don’t capture
any errors directly tied to the batch. This makes it hard to debug which
and why a batch failed when debugging email errors. Going forward we'll
store the error information for a failing email batch directly that
allows easier debugging for batch.

- `error_status_code` : Captures statusCode returned by Mailgun,
available in error.status from the example batch error
- `error_message` : Captures short error message from Mailgun and
status, available in context object of batch error
- `error_data` : Captures while whole error json for a batch. As
mentioned in pitch, this will be huge data and we’ll figure out long
term how to best use this.
2022-11-23 13:13:49 +05:30
Sam Lord
fc291240d5 Updated importer test to use new object return format
no issue
2022-11-17 14:40:24 +00:00
Sodbileg Gansukh
78bff39c23
Update the cover image in default fixtures (#15817)
- updated the cover image to be simpler
- made the change in text fixtures as well, just to keep the fixtures in sync

Co-authored-by: Hannah Wolfe <github.erisds@gmail.com>
2022-11-15 21:19:50 +00:00
Rishabh
d4c3f86ce0 Cleaned up old member activity code usage
refs https://github.com/TryGhost/Team/issues/2216

The `membersActivity` flag was an alpha feature to test the first versions of member analytics, and is no longer active or in use. This change removes the remaining pieces of code that are setup behind that flag and are no longer in use or accessible.
2022-11-14 10:44:10 +05:30
Simon Backx
7cefd4f70b
Updated ghost head test to ignore Portal/Search version (#15777)
fixes https://github.com/TryGhost/Team/issues/2152

When updating the Portal of Search version, the snapshots will now
remain the same.
2022-11-07 15:09:26 +01:00
Kevin Ansfield
6a573d4511
Added lexical post support to email renderer (#15767)
closes https://github.com/TryGhost/Team/issues/2207

- adds conditional to the post email serializer to switch between
`mobiledocLib` and `lexicalLib` depending on which format the post
contains
2022-11-04 11:19:40 +00:00
Daniel Lockyer
c32a013087
Merged v5.22.5 into main
v5.22.5
2022-11-04 13:05:33 +07:00
Fabien "egg" O'Carroll
5a1364e46d 🐛 Fixed importer importing invalid Tier pricing data
closes https://github.com/TryGhost/Team/issues/2211

We were allowing paid Tiers to be imported with non-integer prices which was
causing the Admin to be bricked when attempting to load them. This adds some
validation to the price data of Tiers.
2022-11-04 11:27:15 +07:00
Naz
ac46c2f2e9
Fixed CORS vary header modification
refs https://github.com/TryGhost/Toolbox/issues/461

- The 'vary' header with 'Origin' value should only be set when an OPTIONS header is processed. Otherwise we are prone to leaking the vary header modification to further down in the request pipeline
2022-11-03 11:16:13 +08:00
Naz
9b2e36e4fb
Fixed CORS middleware unit test
refs https://github.com/TryGhost/Toolbox/issues/461

- The unit test was never using the "OPTIONS" request method, which did not actually trigger the full logic of the "cors" module used under the hood.
- Using the correct request method triggers all the right pathways and tests the state that's closer to the real world - for example the response does get "ended" instead of calling the "next" middleware.
2022-11-03 11:16:13 +08:00
Naz
f581e33400
Added Vary value for CORS in Admin API
refs https://github.com/TryGhost/Toolbox/issues/461

- Having a 'Origin' in vary header value present on each `OPTIONS` allows to correctly bucket "allowed CORS" and "disallowed CORS" responses in shared caches
2022-11-02 17:23:47 +08:00
Daniel Lockyer
92740e8967
Merged v5.22.0 into main
v5.22.0
2022-10-31 17:49:27 +07:00
Fabien "egg" O'Carroll
f878e84707 Fixed Tiers importer not correctly mapping price data
refs https://github.com/TryGhost/Toolbox/issues/464

Bceause the import does not use the API, any backwards compat code we put in the
API does not get run for imports, this means we need to update the importer to
map the stripe_prices data onto the products table so that we have valid data in
the database.
2022-10-31 17:30:16 +07:00
Hannah Wolfe
25d5839e96
Added shouldCompileToError test util
refs: a8b1676734

- Extended the newly created handlebars test utils with a shouldCompileToError method
- Updated the price helper tests tp use shouldCompileToExpected and shouldCompileToError
- This allows us to test our handlebars helpers in a much more conisstent way
2022-10-30 16:55:53 +00:00
Hannah Wolfe
a8b1676734
Refactored shouldCompileToExpected into shared util
no issue

- There are currently two patterns in our handlebars helper unit tests:
  1. Treating the helper as a function, and doing a function call
   - This is the original way the tests were done, and they're not great as they're approximating how the helpers are really used
  2. Using a template string, and rendering the string using a method called shouldCompileToExpected
   - These tests are more realistic and powerful and also easier to read

- The new method is only being used in a few places so far, and each place had re-created the `shouldCompileToExpected` method
- Therefore I've moved this method into a util that should make it easier to write unit tests for handlebars helpers
- I also renamed the method in the excerpt tests, because it doesn't do the same thing, it's just a wrapper around a function call rather than compiling a string

The aim is to refactor all of our handlebars helper tests to use `shouldCompileToExpected`
2022-10-30 15:26:29 +00:00
Hannah Wolfe
3ec0057567
Removed accidentally committed file
- This was accidentally included in a fixup commit
2022-10-30 14:38:59 +00:00
e.baidakova
d8fa21e43c Revert "Update portal version"
This reverts commit b5b7ce5f7a.
2022-10-27 23:36:58 +04:00
e.baidakova
b5b7ce5f7a Update portal version 2022-10-27 23:31:48 +04:00
Elena Baidakova
df51a852f5
Added new feedback flow to portal (#15716)
- Updated the way we're handling feedback from user, due to email
restrictions (email client opened all links in email).
2022-10-27 20:34:45 +04:00
Simon Backx
13fd64ebf7 Fixed tests for flag bumps and new setting
no issue
2022-10-27 18:26:46 +02:00
Rishabh Garg
6a619310f6
Removed attribution script if tracking sources is disabled (#15707)
refs https://github.com/TryGhost/Team/issues/2168

- site owners can now disable tracking sources from analytics settings.
- this change removes the loading of attribution script if tracking is
turned off so we don't capture any post/page or external source
attributions
2022-10-27 18:52:11 +05:30
Rishabh Garg
cca0f7d7dc
Added new setting to toggle tracking of member sources (#15705)
refs https://github.com/TryGhost/Team/issues/2168

- the new setting allows site owners to control if they want to track
the sources for new member signups and subscriptions
- its switched on by default, but can be toggled off from new analytics
settings page
2022-10-27 17:24:46 +05:30
Simon Backx
076e3c02b2
Added linking between member and subscription created events (#15693)
fixes https://github.com/TryGhost/Team/issues/2160

- Adds a `batch_id` to both events that contain the same ID if they were created at the same time.
- Removes duplicate signup/conversion events using the batch_id
- Requires an update in mongo-knex to work (refs https://ghost.slack.com/archives/C02G9E68C/p1666773313272409?thread_ts=1666767872.375009&cid=C02G9E68C)
- Some dependencies needed an update to load the latest mongo-knex
- Added tiers to membersUtils, loaded on startup (we can start to use this instead of fetching it every time)
2022-10-27 11:44:19 +02:00
Elena Baidakova
e3ab868b83
Added email feedback column (#15698)
closes TryGhost/Team#2159
- Added column to email table
- Hide the feedback tab on frontend depending on the column value

Co-authored-by: Daniel Lockyer <daniellockyer@fastmail.com>
2022-10-27 11:22:50 +04:00
Simon Backx
1f138893ff Fixed snapshot tests
refs 30ecaef329
2022-10-24 14:27:52 +02:00
Hannah Wolfe
3ab1c418bc
Updated webhook test snapshots
refs: 717a27c85c
refs: 6380b82793

- The snapshots just needed updating
2022-10-21 14:36:20 +01:00
Rishabh
d8bacf12d1 Added endpoint for fixing newsletter links
refs https://github.com/TryGhost/Team/issues/2104

- adds new bulk edit endpoint for links, updates all matching link with the current redirect url and update to new url
2022-10-20 17:50:02 +05:30
Daniel Lockyer
02c8690e87 Added ghost_subscription_id column to members_stripe_customers_subscriptions
refs https://github.com/TryGhost/Team/issues/2034

- this table will be used to link Stripe subscriptions to Ghost
  subscriptions via a foreign key that we add at a later point
- this also includes `constraintName` as the auto-generated one would be
  too long for MySQL 8
2022-10-20 10:59:36 +07:00
Rishabh Garg
318a5a809c
Added permissions for link edit endpoints (#15664)
refs https://github.com/TryGhost/Team/issues/2104

- adds edit permissions for links endpoints to fixtures
- new `bulkEdit` endpoint will use the permissions and allow fixing newsletter links via Admin
2022-10-20 09:11:26 +05:30
Rishabh Garg
60b10ad69a
Fixed permissions for links endpoint (#15656)
refs 5fcf5098a8

- links browse endpoint had permissions switched off unintentionally and was also missing the necessary permissions in fixtures.
- enables permissions for browse endpoint and adds migration insert permissions in DB
2022-10-20 08:18:29 +05:30
Daniel Lockyer
9b8c33484d
Merged v5.19.3 into main
v5.19.3
2022-10-19 06:22:38 +07:00
Simon Backx
a822c5a8c5 Added test to check if feedback buttons are hidden if alpha flag is disabled 2022-10-18 16:47:06 +02:00
Simon Backx
a01fb5f1aa
Added post_id filter and total to activity feed API (#15650)
fixes https://github.com/TryGhost/Team/issues/2091
fixes https://github.com/TryGhost/Team/issues/2089

- Added new fixtures to make testing easier for the activity feed
- Improved E2E test coverage of activity feed with separate test file
- Added data.post_id filter to enable filtering by events related to a
given post
- Fixed return types in JSDoc of test agents (TypeScript interprets
these as `typeof Agent` if we don't add `InstanceType<Agent>`)
- Added total pagination metadata to activity feed API (to allow a basic
type of pagination using filters)
2022-10-18 15:52:04 +02:00
Daniel Lockyer
2dcc4139b1
Merged v5.19.2 into main
v5.19.2
2022-10-18 17:04:23 +07:00
Simon Backx
d1e6870740
🐛 Fixed large mailgun recipient data (#15638)
fixes https://github.com/TryGhost/Team/issues/2096

When generating the recipient data for emails, the email clicks
implementation is resulting in a recipient variable being added called
replacement_xxx once for each link containing the same UUID.

This generates a lot of unnecessary data overhead for emails, and it
turns out that mailgun has a 25MB message limit. We wouldn't have come
close if we only included the uuid once.
2022-10-18 10:32:50 +02:00
Daniel Lockyer
3858f255b9 Dropped nullable status on subscriptions.tier_id
fixes https://github.com/TryGhost/Team/issues/2102

- this column was added with `nullable: true` but it should never be
  nullable, so we should drop the nullable status whilst it's easy to
2022-10-18 14:16:30 +07:00
Daniel Lockyer
c9d43b8fe1
Allowed constraintName in schema column spec
refs 0ba3d6df49

- this is used to indicate the name of the foreign key constraint and so
  we should let it through the schema checks
2022-10-18 10:29:55 +07:00
Elena Baidakova
46141efe05
Updated test snapshot after bumping Portal (#15623) 2022-10-14 20:29:46 +04:00
Elena Baidakova
e831be6bc2
Added the feedback buttons in the emails (#15619)
closes TryGhost/Team#2046
closes TryGhost/Team#2045
- Added feedback buttons markup.
- Added feedback links generation.
2022-10-14 18:12:17 +04:00
Fabien 'egg' O'Carroll
bd0f4b4b8c
Added Tier price and currency data to products table (#15366)
refs https://github.com/TryGhost/Team/issues/1765

In order to better handle deleted objects in Stripe we want to decouple
Members from Stripe.

These changes allow us to have the Tier concept completely independent
of the Stripe tables, such that the Stripe data can be generated as/when
it's needed - which will help to protect against missing data.
2022-10-14 06:40:17 +01:00
Rishabh Garg
e05889cd53
🐛 Fixed missing accent color for default content cta (#15611)
refs https://github.com/TryGhost/Casper/issues/901

- the site accent color property on default cta for upgrade link had a typo and was using wrong property
2022-10-13 13:42:21 +05:30
Naz
4db0be603f Added subscriptions table
refs https://github.com/TryGhost/Team/issues/2030

- adds `subscriptions` table to the DB schema
- this new table is aimed to support a native "subscription" primitive in Ghost
  that most resembles previously used `members_stripe_customers_subscriptions` table
2022-10-13 11:19:13 +07:00
Daniel Lockyer
74e1d52d6a Fixed incorrect maximum length definition in schema
refs https://github.com/TryGhost/Toolbox/issues/441

- whilst reviewing another PR, I noticed we were incorrectly using
  `maxLength` instead of `maxlength` in the schema column definition
- it turns out we've already been doing this wrong for a while with
  other columns
- this key is not acted upon, so the maximum column length was not applied
- fixing up the DB to the correct maximum length is something to fix in the
  future but right now, the schema does not reflect the size of the
  column that actually got created
- the fallback when `maxlength` is not provided is currently 191 [0], so
  this commit switches the schema and migrations to using the correct
  key name and column length that they are using when applied

[0]: 24670aa555/ghost/core/core/server/data/schema/commands.js (L27)
2022-10-13 09:58:19 +07:00
Daniel Lockyer
143ae857c9 Removed bool type from schema
refs https://github.com/TryGhost/Toolbox/issues/441

- we tend to have a mix of `bool` and `boolean` in the schema and
  migrations, which has become a real nit for me at this point
- we don't do any special handling between `bool` and `boolean`, it's
  just something we pass to Knex
- `bool` is an alias for `boolean` but `boolean` is actually documented - https://knexjs.org/guide/schema-builder.html#boolean
- this commit switches Ghost to only using `boolean` in the schema and
  migrations, and removes `bool` from the allowlist in tests to prevent
  us from adding it again in the future
- this should make absolutely no difference to the DB because both
  resulted in the same column
2022-10-13 09:37:38 +07:00
Daniel Lockyer
24670aa555
Fixed validating numbers as booleans in schema validator
refs https://github.com/TryGhost/Toolbox/issues/441

- I'm currently working on cleaning up our uses of `bool` and `boolean`
  in favor of `boolean`, and I've noticed we only handle converting
  numbers into booleans when the type is `bool`, so validation would
  otherwise fail
- given these can be used interchangeably, we should also support
  converting the numbers into booleans when the type is `boolean`
- this is going to get cleaned up again when I remove `bool` but this
  fixes the validation bug for now
2022-10-13 08:20:31 +07:00
jbenezech
3d44e37cbd
🐛 Fixed sitemaps with no content (#15571)
closes: https://github.com/TryGhost/Ghost/issues/14981

- Taxonomy-specific sitemaps were invalid xml when there was no data
- These invalid empty sitemaps were referenced in the index sitemap causing SEO tools to report errors
2022-10-12 14:11:19 +01:00
Daniel Lockyer
845f8d965e
Added test to validate schema structure
refs https://github.com/TryGhost/Toolbox/issues/441

- this is only v1 of the test I would like but it validates the keys on
  a column definition are part of an allowlist
- this has already uncovered a bug with `maxLength` (vs `maxlength`)
2022-10-12 19:19:33 +07:00
Daniel Lockyer
e2ba19b0df
Fixed import of bson-objectid in accordance to the typings
- there's a few different ways we can import it but I've chosen to
  append `.default` as we have done in several other places in the code
2022-10-12 14:54:35 +07:00
Simon Backx
68bdc1afea
Added post sentiment (#15592)
fixes https://github.com/TryGhost/Team/issues/2054

This change adds the sentiment and positive_feedback counts to the posts models. This change isn't really ideal because there are some problems here:
- sentiment isn't really a count
- we don't need to include the sentiment and positive_feedback as a default for posts (but the same is true for attribution)

It would make sense to move this to separate endpoints that only fetch the analytics for a given post when the analytics page is opened. But for our initial skateboard version of audience feedback this should be a good start to already see the data.
2022-10-11 17:52:14 +02:00
Elena Baidakova
1221ba5d1d
Added feedback_enabled to newsletters table (#15589)
closes TryGhost/Team#2042
- Added ability to enable audience feedback per newsletter (just on BE side).
2022-10-11 16:06:26 +04:00
Simon Backx
74d749fa63
Added members_feedback table (#15581)
fixes https://github.com/TryGhost/Team/issues/2041
2022-10-11 13:21:31 +02:00
Simon Backx
b96ff6ae4a Fixed snapshots for Portal update
refs e86e78fb6b
2022-10-05 12:52:50 +02:00
Fabien 'egg' O'Carroll
45d65663f4
Simplified link tracking related tables naming (#15480)
- Removes superfluous "link" from table names
- Fixes type definititon of dropTables util
- Updates & renames models
- Noop existing migrations to avoid unnecessary work
2022-09-29 22:08:45 +01:00
Simon Backx
0cd0fc838d
Added email track clicks column and cleaned up frontend checks (#15501)
fixes https://github.com/TryGhost/Team/issues/2008

- New column that stores email click tracking at the time it was created
- Improved frontend side checks for when to show analytics
2022-09-29 16:42:45 +02:00
Naz
8cbf913582 Increased Vary granularity for versioned requests
refs https://github.com/TryGhost/Toolbox/issues/425
refs https://github.com/TryGhost/Toolbox/issues/280

- The versioned API responses vary based on requested version (passed in request's 'accept-version' header). shared caches that sit between Ghost's origin server and the browser would be putting responses with same Vary into the same caching bucket, which is incorrect.
- This change makes response's Vary more granular and tells caching mechanisms to take 'Accept-Version' request header into account when caching.
- Informative read on the topic - https://www.fastly.com/blog/getting-most-out-vary-fastly
2022-09-28 14:48:43 +08:00
Rishabh
d886bc4b0d Fixed tests for portal bump
refs 8848fd0f59

- last commit didn't update test snapshots for new portal version bump
2022-09-28 00:04:32 +05:30
Rishabh Garg
90034577b8
Added member attribution history frontend script (#15482)
- bumps member attribution script from alpha feature to now load for all sites. The script captures recent url history in localstorage to capture correct attribution for members.
- script is only loaded on the site if members is enabled
2022-09-27 23:45:41 +05:30
Rishabh
0349acb7e3 Updated content cta to use global accent color property
refs https://github.com/TryGhost/Ghost/pull/15471#discussion_r979902374

- the accent color value used by default content cta was copying the global site property which is redundant, and can be directly used
- originally, the accentColor property was extended to allow a fallback value for content ctas, but was later removed as we added default value to global site property directly
- the accentColor property is now deprecated and will be removed in next version, as existing themes might be relying on it for custom cta helpers
2022-09-27 09:41:54 +05:30
Rishabh
1410a4237e 🐛 Fixed default content CTA message to reflect page vs post
closes https://github.com/TryGhost/Team/issues/1898

- the default content cta always used the terminology as `post` when showing message that users don't have access to some content
- this caused confusion when users were looking at a page and message showed "This post is for subscribers only"
- updates the message to correctly reflect `page` vs `post` on the default cta
2022-09-27 09:41:54 +05:30
Rishabh Garg
b99c5428d0
Added referrer attribution columns to events table (#15436)
refs TryGhost/Team#1931

- referrer source, medium and url will be stored in the events table along with rest of attribution data
- stores referrer information on two tables
  - `members_created_events` for signups
  - `members_subscription_created_events` for paid conversions
2022-09-21 19:01:36 +05:30
Hannah Wolfe
f31a50270d Removed search labs flag
- the search helper was promoted to GA and we no longer need the feature flag
2022-09-20 15:22:06 +01:00
Daniel Lockyer
74f1f1bba6
Fixed CDN URLs in tests
refs de416629e6

- I forgot to update them when making the referenced commit
2022-09-19 14:51:31 +01:00
Daniel Lockyer
1ba6d2e5a7
Updated test snapshot
refs de416629e6

- this was forgotten after I'd changed the endpoint in the referenced
  commit
2022-09-19 14:43:07 +01:00
Simon Backx
901a28f3c6
Added link_redirects and members_link_click_events tables (#15421)
closes https://github.com/TryGhost/Team/issues/1912
closes https://github.com/TryGhost/Team/issues/1913
2022-09-19 11:20:36 +02:00
Aileen Nowak
be45d4ebcf Added post stats service to return total posts in Explore endpoint
no issue

- The explore endpoint needs to expose the total amount of published posts
- To be more consistent, this PR creates a PostStats class which is exposed as `stats` method within the PostService; just like it's done with the MemberService
- Moved existing method to return the date of the most recently published post into the stats service
- Updated the explore service test to reflect the new return property
2022-09-16 13:56:14 +01:00
Kevin Ansfield
3b21d26be7
Wired up creation of post_revisions entries when saving posts with lexical (#15422)
no issue

- added `PostRevsion` model
- duplicated `mobiledoc_revision` creation routine in Post model's onSaving hook to create `post_revision` when model's `lexical` field has changed
- updated `mobiledoc_revision` creation to skip when `lexical` field is populated
2022-09-16 11:59:35 +01:00
Kevin Ansfield
1581f439e9
Added post_revisions table (#15420)
no issue

- initially this will perform the same function as `mobiledoc_revisions` but storing `lexical` instead of `mobiledoc`
- naming is intentionally generic ready for later expansions
2022-09-16 10:19:05 +01:00
Simon Backx
a7b583050c
Added link tracking to paywall (#15414)
closes https://github.com/TryGhost/Team/issues/1908

### Problem:
- We need tracking on the paywall links in each email. (we cannot ignore them because those buttons are probably gonna have a higher paid conversion attribution than others).
- Currently we only add the paywall HTML to an email when processing each batch. So if we batch an email to 1.000 recipients per 100, we'll generate the paywall HTML 10 times. 
- We cannot replace links in `renderEmailForSegment` because that methods will get called multiple times. We don't want to have multiple redirect instances created for the same link in the same email.

###  Solution:
- Move the generation of the paywall to the `serialize` method of the post email serializer.
- Surround the generated paywall with HTML-comments so we can remove it if required in `renderEmailForSegment` depending on the member segment we are sending the email to.

---

### Before:

**Serialize output:**
```html
<html>
    <body>
        <h1>Generated email header</h1>
        <p>Generated text</p>

        <div>
            <!-- POST CONTENT START -->
                <h1>Post title</h1>
                <p>Content visible for all members</p>
            <!--members-only-->
                <p>Content visible for paid members only</p>
            <!-- POST CONTENT END -->
        </div>
    </body>
</html>
```

To be modified later by  `renderEmailForSegment`:

**Paid members (nothing changed):**
```html
<html>
    <body>
        <h1>Generated email header</h1>
        <p>Generated text</p>

        <div>
            <!-- POST CONTENT START -->
                <h1>Post title</h1>
                <p>Content visible for all members</p>
            <!--members-only-->
                <p>Content visible for paid members only</p>
            <!-- POST CONTENT END -->
        </div>
    </body>
</html>
```

**Free members (paywall _added_):**
```html
<html>
    <body>
        <h1>Generated email header</h1>
        <p>Generated text</p>

        <div>
            <!-- POST CONTENT START -->
                <h1>Post title</h1>
                <p>Content visible for all members</p>
                <h2>Generated paywall here</h2>
                <a href="https://subscribe.com">Subscribe to read the full post</a>
            <!-- POST CONTENT END -->
        </div>
    </body>
</html>
```

### After this change:

**Serialize output:**
```html
<html>
    <body>
        <h1>Generated email header</h1>
        <p>Generated text</p>

        <div>
            <!-- POST CONTENT START -->
                <h1>Post title</h1>
                <p>Content visible for all members</p>
            <!--members-only-->
                <p>Content visible for paid members only</p>
             <!-- PAYWALL -->
                <h2>Generated paywall here</h2>
                <a href="https://subscribe.com/?tracked">Subscribe to read the full post</a>
            <!-- POST CONTENT END -->
        </div>
    </body>
</html>
```

To be modified later by  `renderEmailForSegment`:

**Paid members (paywall removed):**
```html
<html>
    <body>
        <h1>Generated email header</h1>
        <p>Generated text</p>

        <div>
            <!-- POST CONTENT START -->
                <h1>Post title</h1>
                <p>Content visible for all members</p>
            <!--members-only-->
                <p>Content visible for paid members only</p>
            <!-- POST CONTENT END -->
        </div>
    </body>
</html>
```

**Free members (members-only content removed):**
```html
<html>
    <body>
        <h1>Generated email header</h1>
        <p>Generated text</p>

        <div>
            <!-- POST CONTENT START -->
                <h1>Post title</h1>
                <p>Content visible for all members</p>
            <!-- PAYWALL -->
                <h2>Generated paywall here</h2>
                <a href="https://subscribe.com/?tracked">Subscribe to read the full post</a>
            <!-- POST CONTENT END -->
        </div>
    </body>
</html>
```
2022-09-16 10:08:12 +02:00
Kevin Ansfield
c240f7afa4
Added rendering of posts.lexical to posts.html when saving (#15416)
no issue

- added `@tryghost/kg-lexical-html-renderer` dependency
- added `lexical` lib following the same pattern as our `mobiledoc` lib
- updated the Post model's `onSaving` hook to generate the `html` value from `lexical` when present
2022-09-15 16:49:14 +01:00
Simon Backx
e9974d8cc0
🐛 Fixed feature image caption escaped twice in newsletters (#15417)
fixes https://github.com/TryGhost/Team/issues/1909

- The feature image caption is already escaped on the frontend
- Doing it again in the backend breaks the possibility to add links to the caption
- I checked and the `feature_image_alt` is not escaped in the frontend.
2022-09-15 17:07:10 +02:00
Simon Backx
699e67f4e4
Added email_track_clicks setting (#15409)
fixes https://github.com/TryGhost/Team/issues/1900
refs https://github.com/TryGhost/Team/issues/1901

- Defaults to the same value as the current email_track_opens setting for existing installations, otherwise defaults to true
- Had to use a custom migration because the `addSetting` helper doesn't support using an existing setting as current value
- Added a minimal UI to change the setting, but this still needs some design magic 🪄
- Link replacement is disabled if `email_track_clicks` is disabled. In the future we might consider to still do parial additions, such as source attribution and maybe redirects (to discuss).
2022-09-15 15:48:22 +02:00
Fabien "egg" O'Carroll
4726742673 Renamed redirects to custom-redirects
We're going to be adding more redirection logic into Ghost and it's
going to get confusing if we have names this generic. This makes it
clear which feature this service is related to.

Ideally in the future we can combine all of these into one redirects
service, but for now we will be running a specific service per feature
2022-09-14 13:24:17 -04:00
Kevin Ansfield
8cdd2e10b7
Added posts.lexical database field (#15397)
closes https://github.com/TryGhost/Team/issues/1884

- adds `post.lexical` ready for use by the lexical-powered editor re-write
- fulfils the same purpose as `posts.mobiledoc` so uses the same field properties
- added `lexical` to allowed formats in Post model so it won't be included by default in API responses meaning tests/snapshots don't need updating at present
2022-09-13 11:21:47 +01:00
Rishabh
4187f0da54 Updated staff service to trigger alerts via events
refs https://github.com/TryGhost/Team/issues/1865

- refactors staff service to listen to member and subscription events
- triggers email alerts based on events instead of directly calling the service
- removes staff service dependency for members api
2022-09-10 11:06:34 +05:30
Hannah Wolfe
04f3ac37d3 Removed defunct applyPublicRules code
closes: https://github.com/TryGhost/Toolbox/issues/369
refs: https://github.com/TryGhost/Toolbox/issues/229

- this code is defunct as we no longer accept a status parameter via the API for posts, pages, users or authors
2022-09-09 17:14:15 +01:00
Hannah Wolfe
b0234dd58e
Removed apiVersions from test urlUtils
refs: https://github.com/TryGhost/Toolbox/issues/229

- These properties were removed from urlUtils in 5.0 and no longer do anything
2022-09-09 13:32:30 +01:00
Ronald Langeveld
c9e6f42ca8 Bumped Portal to 2.12.0
ref https://github.com/TryGhost/Team/issues/1800

- Adds v 2.12.0 of Portal
- Updates snapshots for tests
2022-09-09 13:54:43 +02:00
Ronald Langeveld
eb6534bd7f
Replaced all 'bio' references with 'expertise' for member comments. (#15359)
closes https://github.com/TryGhost/Team/issues/1772

- The user facing side of comments recently replaced `bio` with `expertise`.
- To remain consistent we replaced all the references of `bio` with `expertise` throughout the codebase.
- This includes a database column name changing migration, within the `members` table.
- Bumped up the comments-ui version to a new minor (0.10.x) as its a breaking change.
2022-09-09 10:14:49 +02:00
Hannah Wolfe
7084217d3d
Added same-origin referrer rule to post previews
- this prevents the referrer/referer header being sent for requests that go to external domains
- this in turn prevents preview URLs from appearing in the analytics of sites that are linked to and clicked on from previews
- otherwise, preview URLs can be leaked to the owners of the linked and clicked sites
2022-09-08 12:39:13 +01:00
rw4nn
dc84983550
🐛 Fixed square brackets being % encoded in URLs (#14977)
fixes: #14863
refs: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/encodeURI#encoding_for_ipv6

- added a simple Regex replace for the percent-encoded square brackets to get them back to non-encoded
- a preferred solution might be using new URL(), but that causes other issues. The regex solves the immediate need.
2022-09-08 12:09:40 +01:00
Simon Backx
6bffa893b1
Added snapshot tests to ghost_head helper (#15327)
refs https://github.com/TryGhost/Team/issues/1795

- Snapshots help us detect unexpected changes in the `<head>` of all sites (e.g., newly introduced script tags)
- Added ghost_head tests for comment count helper
2022-09-08 13:04:34 +02:00
Simon Backx
4534b693e4
Added test that validates output HTML of email template (#15365)
refs https://github.com/TryGhost/Team/issues/1871

This commit adds a test to the serialize method of `post-emaiserializer`. It checks whether the generated email HTML is valid and standard HTML5 and that all properties are escaped.

To do this validation, I depend on the new `html-validate` dev dependency. Just parsing the HTML with a HTML parser is not enough to guarantee that the HTML is okay.

Apart from that this fixes:
- Removed the sanitizeHTML method and replaced it with normal HTML escaping. We don't want to allow any HTML in the escaped fields. Whereas `sanitizeHTML` still allows valid HTML, but we don't want that and want the same behaviour as on the site. E.g., a post with a title `All your need to know about the <br /> tag` should actually render the same title and non-html content, being `All your need to know about the &lt;br /&gt; tag`
- The file, nft and audio card didn't (always) escape the injected HTML fields (new version @tryghost/kg-default-cards) 
- `@tryghost/string` is bumped because it contains the new escapeHtml method
2022-09-08 10:11:01 +02:00
Daniel Lockyer
04e3ee9f10 Added property cleaning to tag relations on pages + posts
refs https://github.com/TryGhost/Ghost/pull/15375

- we currently pass all properties for the `tags` property of a
  `page`/`post` body down further into Ghost, which is causing issues
  because it's handling properties it doesn't expect
- this is showing up because it's triggering save history events for
  tags when a post is edited
- this commit introduces a clean util which has an allowlist of
  properties allows on tag relations
- this list was taken from the schema: 128f8fb006/packages/admin-api-schema/lib/schemas/posts.json (L214-L227)
2022-09-07 22:28:56 +01:00
Daniel Lockyer
4a6f57b105
Merged v5.13.2 into main
v5.13.2
2022-09-06 16:45:52 +01:00
Fabien 'egg' O'Carroll
f7a58ecafc
🐛 Fixed OpenSea NFT OEmbeds (#15372)
refs https://github.com/TryGhost/Team/issues/1879

OpenSea updated their URL format for NFTs after adding support for Solana
which broke our regex, this updates to support the new format.
2022-09-06 11:29:35 -04:00
Naz
920a3aeb4c
Fixed adapter-related unit test
refs 37dd187fe6
refs c36575627d/ghost/core/core/server/data/importer/handlers/image.js (L16)

- The tests were failing because they were stubbing a "generic" adapter, instead of the one which the module under test was using (see referenced code to see what I mean)
2022-09-06 18:46:32 +08:00
Naz
c36575627d
Fixed unit test
refs 37dd187fe6

- The referenced commit lacked cleanup after module removal
2022-09-06 18:11:22 +08:00
Naz
a96a7340c0
Added JSDoc to adapter options resolver
refs https://github.com/TryGhost/Toolbox/issues/384

- Added jsdoc for intellisence/typechecking
- Cleaned up the naming of returned values to resemble the usecases a bit better
2022-09-06 17:51:57 +08:00
Naz
67df9a6105
Removed unused adapterType variable
refs https://github.com/TryGhost/Toolbox/issues/384

- The adapter manager can parse the adapter type internally from the "type:feature" syntax, so there's no need to pass it around.
2022-09-06 17:51:57 +08:00
Naz
1fc8c8d671
Added more explicit adapter config syntax
refs https://github.com/TryGhost/Toolbox/issues/384

- Existing adapter config was based on the notion there can only be one configuration per one adapter class. With adapter cache now allowing instantiating multiple adapter instances with the same base class it opened up a possibility to have shared configuration for a base class and then extend/override it in "feature" configurations (see tests in this commit for specific examples)
2022-09-06 17:51:57 +08:00
Simon Backx
2e85ae98be
🐛 Fixed sending emails from email domain that includes www subdomain (#15348)
fixes https://github.com/TryGhost/Team/issues/1855
fixes https://github.com/TryGhost/Team/issues/1866

This commit moves all duplicate methods to get the support email address to a single location. Also methods to get the default email domain are moved.

For the location, I initially wanted to put it at the settings service. But that service doesn't feel like the right place. Instead I created a new settings helpers service. This service takes the settingsCache, urlUtils and config and calculates some special 'calculated' settings based on those:

- Support email methods
- Stripe (active) keys / stripe connected (also removed some duplicate code that calculated the keys in a couple of places)
- All the calculated settings are moved to the settings helpers

I'm not 100% confident in whether this is the right place to put the helpers. Suggestions are welcome.
2022-09-02 16:57:59 +02:00
Daniel Lockyer
dd2bfb8c0e
Merged v5.12.4 into main
v5.12.4
2022-09-02 15:13:37 +01:00
Simon Backx
999b111fce
🐛 Fixed paid email preview stopped working in emails (#15356)
fixes https://github.com/TryGhost/Team/issues/1870

Disables email sanitization that was enabled earlier because this bug is more important and urgent.

The recently introduced email sanitzation removes HTML comments from the post html.
- This breaks the email paid preview, because it depends on the `<!--members-only-->` comment.
- Breaks the Outlook comments `<!--[if !mso !vml]-->`

This commit reverts this change.
2022-09-02 15:49:39 +02:00
Fabien 'egg' O'Carroll
e4cbb3d24d
Reset magic link rate limiting upon successful login (#15345)
refs https://github.com/TryGhost/Team/issues/1771

We don't have access to `req.brute.reset` due to the way the flow
works, we have one endpoint which sends an email with a magic link,
and another route which handles the login. We don't want to apply
brute force protection to both because our rate limiting is designed
for API requests not web page visits (which is how login is handled).

Because of this we require access to the underlying ExpressBrute
instance exposed by the spam-protection module, so that we can
perform the reset.
2022-09-01 08:54:14 -04:00
Naz
945ebd4806
Fixed ERR_NOCK_NO_MATCH warning during test runs
refs https://github.com/TryGhost/Toolbox/issues/389

- The e2e test suite log was full of ERR_NOCK_NO_MATCH warnings when the logging level was set to "warn". The cause of this warning was legit duplicated webhook trigger processing on test environment. Gah!
- The source of duplicate webhook processing was duplication of event handlers. Event handlers were registered multiple times for same event because of the singleton nature of the "common/events" module - it remains the same instance and is not cleaned up between reboots. The deeper issue of events module initialization should be solved separately, this slightly hacky approach fixes the problem now and highlights it to be tackled in the future.
2022-09-01 12:25:47 +08:00
Fabien 'egg' O'Carroll
21e473ff78
🐛 Fixed newsletters not rendering with non-HTML safe chars (#15331)
Co-authored-by: Kevin Ansfield <kevin@lookingsideways.co.uk>
2022-08-30 11:26:01 +01:00
Simon Backx
a8239bfa97
Added ENUM validation for member/subscription created events (#15312)
closes https://github.com/TryGhost/Team/issues/1842

- members_created_events: source + attribution_type
- members_subscription_created_events: attribution_type
- members_subscribe_events: source
2022-08-25 15:39:37 +02:00
Rishabh
effd5af615 Handled fetching staff users for email alerts
refs TryGhost/Team#1826

- adds a method on user model which fetches all eligible users for a type of email alert
- restricts users to active `Owner` and `Administrators` with setting turned on
2022-08-25 18:01:52 +05:30
Hannah Wolfe
c9864ee63f Added {{search}} theme helper
closes: https://github.com/TryGhost/Team/issues/1732

- adds a theme helper which outputs a working search button with a standard icon
- the icon adopts whatever the current color is from css, and has a set of default styles
- styles can be overridden with !important or the data attribute
- alternatively, any element in a theme may be turned into a search button by adding data-ghost-search
- this is meant to be a simple tool for non-theme-developers to easily add a search icon to their themes in a way that doesn't require css or html knowledge
2022-08-24 21:34:20 +01:00
Rishabh
326bb97d2c Enabled free trials via tiers and offers
refs https://github.com/TryGhost/Team/issues/1724
refs https://github.com/TryGhost/Team/issues/1726

- allows site owners to add a default free trial period to their tiers
- allows site owners to create custom offers that allow free trials on tiers
2022-08-24 19:24:31 +05:30
Rishabh Garg
9abfae2ddb
Switched off email alerts for subscription cancellation by default (#15304)
refs TryGhost/Team#1825

- all sites are expected to have cancellation alerts off by default
2022-08-24 18:54:00 +05:30
Rishabh Garg
c48c65cc88
Added member email alert notification columns for staff (#15276)
refs https://github.com/TryGhost/Team/issues/1825

- adds 3 new columns to users table for storing email alert preferences for member signups/cancellation
- adds column for new member signup alert
- adds column for paid subscription started alert
- adds column for paid subscription canceled alert
- Updated default fixtures and tests for new columns
2022-08-23 22:11:38 +05:30
Hannah Wolfe
809c1a6e08 🐛 Fixed error deleting post with comment replies
closes: https://github.com/TryGhost/Ghost/issues/15252

- comments are deleted when posts are deleted. Without cascade delete on parent_id, replies cannot be deleted
- this change means that deleting a post will delete all comments and replies without error
2022-08-19 15:49:58 +01:00
Hannah Wolfe
d2acf3aada Fixed attribution table missing on cascade delete
refs: https://github.com/TryGhost/Ghost/issues/15252

- all columns with a foreign key (references prop) must have a deletion strategy
- we just found a bug with this in the comments table - see referenced issue
- this fix adjusts the schema and migration for this change before its released so we don't have to write a horrible migration later
2022-08-19 15:28:45 +01:00
Rishabh
1258156c38 Handled storing complimentary subscription expiry
refs https://github.com/TryGhost/Team/issues/1727

- if feature flag is enabled, handles storing expiry date on complimentary subscriptions in `expiry_at` column of `members_products`
- updates the expiry value on both member edit or add with tiers
- expiry is passed as `expiry_at` in `tiers` list of a member
- includes `expiry_at` on tiers data of a member when flag is enabled
2022-08-19 18:20:52 +05:30
Hannah Wolfe
704f17ff96
Added beta search helper implementation (#15236)
refs: TryGhost/Team#1732

- We're testing out the feasibility of having a {{search}} helper that outputs an pre-styled icon to trigger search.
2022-08-19 12:27:38 +01:00
Rishabh Garg
43b8ad5069
Added expiry column for complimentary subscriptions (#15241)
refs https://github.com/TryGhost/Team/issues/1727

- allows adding expiry date to complimentary subscriptions, effectively allowing cardless trials.
- expiry is stored in members<>products mapping table
2022-08-19 08:41:40 +05:30
Fabien "egg" O'Carroll
37ef0582e6 Added members_subscription_created_events table & model
refs https://github.com/TryGhost/Team/issues/1803
2022-08-18 10:45:53 -04:00
Fabien "egg" O'Carroll
03155a61ff Added members_created_events table & model
refs https://github.com/TryGhost/Team/issues/1802
2022-08-18 10:45:53 -04:00
Naz
6cf49d8f89 Fixed inconsistent format of image sizes cache
refs https://github.com/TryGhost/Toolbox/issues/364
refs 147ec91162

- This looks like a subtle bug that has gone unnoticed for years. Have checked if we rely on the logic anywhere (mostly used in image-dimensions frontend helper) - we don't access the "url" directly.
- There is no reasoning attached behind why the cached size was stored as a url (see refed commit)

- WHY is this even being fixed? Caches can store anything... does not mean we should! Inconsistent data becomes a real PITA  if the cache is persisted and is hard to repopulate (e.g. to migrate the cached data format).
2022-08-16 15:20:49 +02:00
Naz
e4d02657ee Swapped image size "Map" cache for adapter cache
refs https://github.com/TryGhost/Toolbox/issues/364

- The "new Map()" cache was a "hidden cache" that did not follow any specific pattern. Following the cache adapter pattern here makes it possible swapping out the cache for alternative implementations - e.g. Redis storage
2022-08-16 12:35:35 +02:00
Naz
05eb719d0a Swapped image size cache with InMemoryCache
refs https://github.com/TryGhost/Toolbox/issues/364

- The InMemoryCache is an implementation of the cache adapter interface and allows to test cache in the works which is "close to the real world". Being able to do so in tests for image sizes cache manager proves we can use other cache adapters such as Redis based ones.
2022-08-16 12:35:35 +02:00
Naz
e549528985 Simplified image size cache module constructor
refs https://github.com/TryGhost/Toolbox/issues/364

- As little as possible should be passed in in the parameters for any method/constructor/whatever. Specific Function > vague Object
2022-08-16 12:35:35 +02:00
Naz
92d6c998b3 Fixed error handling bug in image size cache
refs https://github.com/TryGhost/Toolbox/issues/364

- Doing the `.catch(errors.NotFoundError...` was throwing another error as this syntax did not work with native promises. Checking `instanceof` works 100% and is way more explicit/readable way to handle this type of error differently
2022-08-16 12:35:35 +02:00
Naz
38c76847e9 Refactored cache image size test suite
refs https://github.com/TryGhost/Toolbox/issues/364

- It was using an outdated syntax and relied on Bluebird depencency. Updated the syntax to async/await and dropped the Bluebird dependency.
2022-08-16 12:35:35 +02:00
Naz
7d3b678d4e Refactored cached image size to have cache as DI
refs https://github.com/TryGhost/Toolbox/issues/364

- Groundwork before swapping out existing cache for in-memory cache from the adapter
2022-08-16 12:35:35 +02:00
Fabien 'egg' O'Carroll
96cd6df6fa
🐛 Fixed Comments administration for self hosters (#15239)
refs https://github.com/TryGhost/Team/issues/1799

Rather than using the `adminAuthAssets` config which is not updated to
be aware of running in a different directory to the cwd, we use the
getContentPath method which handles all of the directory checking.

Without this, we were unable to serve the admin-auth iframe, as the
directory was incorrect for self hosters.
2022-08-15 15:38:19 -04:00
Simon Backx
a666b846e6
🐛 Fixed unsubscribe flow for comment reply emails (#15232)
refs https://github.com/TryGhost/Team/issues/1790

- Currently we go to the account settings when you click 'Manage your email preferences' in the footer of an email that informs a comment author that they received a reply.
- Related Portal changes are here: https://github.com/TryGhost/Portal/pull/255
2022-08-15 11:36:08 +02:00
Simon Backx
27df4e536f
Added native comments (#15223)
no issue

- Removes the comments lab flag (bumping it to GA).
- Bumps comments-ui to v0.8
2022-08-12 16:06:06 +02:00
Aileen Nowak
5ac779f86b Added core type integrations to API serializer 2022-08-12 14:18:44 +01:00
Aileen Nowak
0e284edbad Allowed core integrations to work with Admin API 2022-08-12 14:18:44 +01:00
Daniel Lockyer
687e68d5de Extracted shared API framework to separate package
refs https://github.com/TryGhost/Toolbox/issues/363

- this API framework is standalone and should be pulled out into a
  separate package so we can define its boundaries more clearly, and
  promote better testing of smaller parts
2022-08-11 17:44:59 +02:00
Daniel Lockyer
dc0cec56ea Moved settings mappers from shared frame to endpoints
- I'm still not sure where these are meant to be but they don't really
  belong in shared and I'm about to pull that folder out into a separate
  package
2022-08-11 17:44:59 +02:00
Daniel Lockyer
7d52ae52cf Deleted concept of external user
refs https://github.com/TryGhost/Toolbox/issues/363

- AFAICT, this is an outdated and unused concept from when subscribers
  were a thing, but members are now how we do things, and this is causing code
  to linger around that we don't need
2022-08-11 16:05:00 +02:00
Daniel Lockyer
42e722d627
Moved Cache-Control middleware to separate package
refs https://github.com/TryGhost/Toolbox/issues/363

- this middleware is standalone and I suspect we're going to be touching
  it further when we work on Ghost's caching in the near future
2022-08-11 14:14:20 +02:00
Fabien 'egg' O'Carroll
f34740d6d0
Added support for autowrap and class to the comment_count helper (#15203)
refs https://github.com/TryGhost/Team/issues/1760

This allows theme developers to wrap the output of the comment_count
helper in an element, which will only be shown when there is content
to output.

This makes styling a lot easier, as the default output for no comments
is nothing, meaning that separators defined with CSS will not be rendered.
2022-08-09 13:08:36 +01:00
Rishabh Garg
098f40bbe3
Added trial info to member subscription detail (#15193)
refs https://github.com/TryGhost/Team/issues/1757

- exposes trial start and end dates in member's subscription object
- allows portal and admin to show member's trial information in UI
2022-08-09 17:28:00 +05:30
Rishabh Garg
5704ac061e
Handled storing of trial start/end info for subscription (#15161)
refs https://github.com/TryGhost/Team/issues/1724

With free trials, members can start subscriptions with a trial period. This change stores the information about trial start and end date for every subscription so it can be shown on Admin/Portal for member.

- adds new `trial_start_at` column for storing trial start date on Stripe subscription. Will in most cases match the start of subscription date.
- adds new `trial_end_at` column for storing trial end date on Stripe subscription.
- wires storing trial start and end values on stripe subscription
2022-08-05 17:50:40 +05:30
Naz
fa8d94fce2 Fixed the typo
refs e9bfc4ef01

- Did a typo in the find and replace... and now correcting a typo of a typo  -_-
2022-08-04 15:38:32 +01:00
Naz
2de7688eaf Used in memory cache adapter in settigs cache manager
refs https://github.com/TryGhost/Toolbox/issues/364

- Settings Manager used to store all of it's settings values in a hash - an in memory cache in disguise. Having a hidden cache made it hard to reason about it's impact of memory usage and did not allow to swap it out for an alternative storage metchanism easily. Having a cache storage abstraction in Settings Manager allows to get rid of long lasting memory problems + decouples storage mechanism from the logic around transforming stored values.
2022-08-05 02:28:33 +12:00
Naz
ed79d3e9b3 Added basic memory cache storage implementation
refs https://github.com/TryGhost/Toolbox/issues/364

- This is groundwork to substitute in memory caches we use across the codebase. The first candidate would be settings cache. The interface of the memory cache was kept to the minimum. The "keys" method is a somewhat acceptable alternative to the "getAll" method used in the codebase right now.
- The next iteration over this would be adding async methods are alternative key/value storage methanisms like Redis.
2022-08-05 02:28:33 +12:00
Naz
af0014917b Moved cache storage initialization
refs https://github.com/TryGhost/Toolbox/issues/364

- Passing "cache" through constructor did not work out because cache setting is still dependent upon on the model layer (gets called before it has a chance to initialize during db migrations)
- To remove the initialization dependency blockers were:
    "defaults" method in the post model - the value resolved to "undefined" anyway during the fixture insertion
    validate-password module - checks the password against "undefined" during fixture initialization
- Passing the cache through "init" method works too, but is not as clear as with constructor DI pattern.
2022-08-05 02:28:33 +12:00
Naz
e9bfc4ef01 Changed the lingo to US of A variation
refs 16728a3ef1

- It's 'merica time!
2022-08-05 02:28:33 +12:00
Simon Backx
bac8f4b8db
Added bio to members api (#15168)
refs https://github.com/TryGhost/Team/issues/1716

- Adds the bio field to the API output
- Allow setting bio when updating the member
- Includes new E2E tests for the members API that were missing
2022-08-04 15:51:23 +02:00
Rishabh Garg
c931d80cc0
Added migration to store trial days on tier (#15129)
refs https://github.com/TryGhost/Team/issues/1724

- storing trial period days allows site owners to give free trial for N days to all members signing up on a tier
- by default, all tiers have 0 trial period days(same as no trial)
2022-08-04 11:35:57 +05:30
Daniel Lockyer
464b5ca426 Extracted html-to-plaintext shared lib into package
refs https://github.com/TryGhost/Toolbox/issues/363

- this shared library is standalone, and it used in various places of
  Ghost core, so we can pull it out to keep it easier to reason about
- we also use the `html-to-text` dependency in another package but it's
  outdated and could now switch to this new package
2022-08-03 16:51:56 +02:00
Fabien "egg" O'Carroll
8dc8ab88ca Added default values for comment_count helper
refs https://github.com/TryGhost/Team/issues/1695

We add english defaults for the singular and plural params leaving the
empty param blank so as to not draw attention to low engagement.
2022-08-03 12:23:37 +01:00
Fabien "egg" O'Carroll
023d85d561 Updated comment_count helper and frontend script
refs https://github.com/TryGhost/Team/issues/1695

This updates the comment_count helper from a block to inline, and the
frontend script to replace the entire element with the comment count
text. This means that theme designers will have the most flexibility
as they can choose whether or not to wrap the text in an element, as
well as which element.
2022-08-03 11:52:08 +01:00
Naz
452132600b Fixed trailing whitespace in test file
no issue

- The linter does not seem to pickup the trailing whitespace problem in test files. This would be good to fix project-wise with the `--fix` options or something similar
- This change is for example purpuses :)
2022-08-03 11:48:02 +01:00
Naz
492960b9a8 Refactored settings cache to use class/DI pattern
refs https://github.com/TryGhost/Toolbox/issues/364

- This is a groundwork which moves the "cache" property in settings cache to be injectable parameter, so we can swap it out with different implementations.
- The module will be broken downn into two concepts - an injectable cache  and a cache manager (the update system)
2022-08-02 17:18:07 +01:00
Kevin Ansfield
0a34be4012
Updated admin asset serving for ember-auto-import@2 compatibility (#15128)
refs https://github.com/TryGhost/Admin/pull/2252
closes https://github.com/TryGhost/Team/issues/1182

- Admin now copies it's build output to a single env-specific directory rather than splitting html and assets
  - `core/built/admin/{development|production}/*`
- updated the admin app's `serveStatic` definition for assets and controller's html serving to reflect the new asset paths
2022-08-02 13:43:45 +01:00
Simon Backx
b7f3892be0
Added format option to img-url helper (#14962)
fixes https://github.com/TryGhost/Ghost/issues/14323

- Fixed support for resizing images from Unsplash using the `img-url` helper (previously the size property was ignored for images from Unsplash)
- Added support for `avif` file formats (supported by sharp out of the box)
- Added support for setting the format of images, with a new  `format` option:

E.g. to convert an image to webp (only works in combination with size for now, except for Unsplash where you can use it without size):
```
{{img_url @site.cover_image size="s" format="webp"}}
```

This can help improve the performance of a theme, by serving assets in `<picture>` elements with webp and fallback image formats.

Usage example:
```html
<picture>
    <source 
        srcset="{{img_url feature_image size="s" format="avif"}} 300w,
                {{img_url feature_image size="m" format="avif"}} 600w,
                {{img_url feature_image size="l" format="avif"}} 1000w,
                {{img_url feature_image size="xl" format="avif"}} 2000w"
        sizes="(min-width: 1400px) 1400px, 92vw" 
        type="image/avif"
    >
    <source 
        srcset="{{img_url feature_image size="s" format="webp"}} 300w,
                {{img_url feature_image size="m" format="webp"}} 600w,
                {{img_url feature_image size="l" format="webp"}} 1000w,
                {{img_url feature_image size="xl" format="webp"}} 2000w"
        sizes="(min-width: 1400px) 1400px, 92vw" 
        type="image/webp"
    >
    <img
        srcset="{{img_url feature_image size="s"}} 300w,
                {{img_url feature_image size="m"}} 600w,
                {{img_url feature_image size="l"}} 1000w,
                {{img_url feature_image size="xl"}} 2000w"
        sizes="(min-width: 1400px) 1400px, 92vw"
        src="{{img_url feature_image size="xl"}}"
        alt="{{#if feature_image_alt}}{{feature_image_alt}}{{else}}{{title}}{{/if}}"
    >
</picture>
```
2022-08-01 14:45:54 +02:00
Aileen Booker
304b660c9d
Added migrations to change Ghost Explore integration type 2022-08-01 09:08:06 +02:00
Rishabh Garg
be3a8db828
Fixed failing preview test email (#15105)
refs https://github.com/TryGhost/Team/issues/1680

- using test emails via email preview in admin were failing due to missing post data attached to them
- adds test to make sure email segment rendering doesn't crash even with missing data
2022-07-28 21:05:47 +05:30
Fabien "egg" O'Carroll
93694d837e Updated {{comments}} helper with new options
refs https://github.com/TryGhost/Team/issues/1695

This updates the comments helper based on the design document

Changes include:
 - `color_scheme` renamed to `mode`
 - `avatar_saturation` renamed to `saturation`
 - `saturation` default changed from 50 to 60
 - `count` option added
 - `title` option added

The count and title options allow theme developers to better customise
the output of comments, so that they can either pass in their own
title, or pass in no title, and instead provide HTML in the them to
handle it. The same is the case for the count option, which is used to
toggle whether or not the comment count is shown.
2022-07-28 14:53:33 +01:00
Djordje Vlaisavljevic
f0d1cf2d66
Updated newsletter paywall cta design (#15111)
refs https://github.com/TryGhost/Team/issues/1705

- Made `max-width` smaller to avoid super-long lines
- Added `span` elements with `nowrap` to avoid one or two-word orphans

Co-authored-by: Rishabh <zrishabhgarg@gmail.com>
2022-07-28 16:10:09 +05:30
Rishabh Garg
9c616fe067
Added content paywall for newsletters (#15048)
refs TryGhost/Team#1680

- extends the public preview card so that the paywall is shown in newsletters for paid-only posts based on member's access
- adds CTA for paywalled content in newsletters
- the segmentation for paywall only considers free and non-free members, so post with specific `tiers` and `paid-only` access settings are sent to all paid members

Co-authored-by: Djordje Vlaisavljevic <dzvlais@gmail.com>
2022-07-26 19:16:08 +05:30
Rishabh
eb75888b48 Updated frontend apps to use new config
refs https://github.com/TryGhost/Team/issues/1719

- creates scripts injected for portal/search/comments using new config format
- adds `data-styles` to search/comments script for adding external styles URL
2022-07-25 23:00:37 +05:30
Daniel Lockyer
ee5753a6b7
Fixed minor linting issue in member-count tests
- we don't need to return this value, and eslint complains about it if
  we do
2022-07-25 11:25:14 +02:00
Naz
c667620d8f 🏗 Added jobs table creation migration
refs https://github.com/TryGhost/Toolbox/issues/357

- One time jobs need a storage mechanism to be run only ever once.
- Field notes:

- `id`, `created_at`, `updated_at` - standard Ghost fields
- `name` - unique name of the job, could also be used with prefixing to identify certain type of job (e.g.: backup-bob-2022-10-16, backup-sam-2023-01-13 identifying backup jobs run by users)
- `status` - 'started' | 'finished' | 'failed' | 'queued'  (need to identify when the job is in progress, done, added to the execution queue, or errored)
- `started_at` - when the job started execution
- `finished_at` - when the job successfully finished execution
2022-07-22 23:32:58 +12:00
Simon Backx
30c4f11e27
Added report API for comments (#15043)
closes https://github.com/TryGhost/Team/issues/1684

**Migrations:**
- Added report permissions (fixtures + migrations)
- Dropped reason field in reports (no textarea in reports in V1)
- Dropped nullable from comment_likes.member_id (can't be null)
- Added SET NULL/CASCADE foreign keys for comments related tables(*)

(*):
fixes https://github.com/TryGhost/Team/issues/1687
refs https://ghost.slack.com/archives/C02G9E68C/p1658217288591369

This commit adds support for `SET NULL` foreign keys in schema and migration helpers + also fixes the foreign keys for the comment_reports, comment_likes and comments tables.

- When a member is deleted, we **do** want to keep their reports (SET NULL)
- When a member is deleted, we **do not** want to keep their likes (CASCADE)
- When a member is deleted, we **do** want to keep the comments (SET NULL)

**Changes:**
- Added report API: `POST /members/api/comments/{id}}/report/`
- Sends an email to the owner when a comment is reported
- Saves a report to the database (not used for now, but might be useful later)
2022-07-22 12:03:05 +02:00
Daniel Lockyer
3d989eba23 Converted Ghost repo into a monorepo
refs https://github.com/TryGhost/Toolbox/issues/354

- this commit turns the Ghost repo into a monorepo so we can bring our
  internal packages back in, which makes life easier when working on
  Ghost
2022-07-20 16:41:05 +02:00