- I recently added a bunch of strict rules to our eslint plugin around returns: ca9af37866
- These mostly are issues that occur whilst writing code, that you spot and fix as you're developing, but they're annoying to notice/find and eslint can be used to flag them quickly
- There are of course, edge cases where you don't need to return from array fns, but this rule also suggests better patterns might be available
- For our excert helper and new assertEvent helper, I've updated the code to use simpler patterns that are easier to read, so as to avoid the warnings
- For our old API I've simply disabled the rule as we're about to delete this code
refs https://github.com/TryGhost/Toolbox/issues/202
- during DB init, we have to create all the tables
- right now we loop over all tables and call the `createTable` command
- this command checks if the table exists and if not, creates the table
- this works fine but it means we query the database for every table
- in MySQL, we query the information_schema table, which we've seen
issues with before because it doesn't have indexes
- the smarter thing to do here is to get all the tables that already exist,
remove them from the list, and just straight up create them without
further checks
- this entire thing should be protected by the migration lock so we
shouldn't encounter issues from multiple processes initializing the DB
and tables existing after the initial check
- this commit also removes the check from `createTable` because this isn't
really needed. We should be using the migration utils, which do
check for existing tables. I've added a note to the function and
audited anywhere we still call the function
- this commit removes (- 49 tables + 1 initial check) 48 queries from
the initial DB init
refs https://github.com/TryGhost/Toolbox/issues/214
- The values configuration for the settings table need to become configurable to be able to run our test environment with a pre-defined set of configurations (e.g Stripe-related values).
- This change makes it possible to define the default settings file location (currently a JSON)
- A new key is now exposed through the "paths.defaultSettings" key in settings, which can be overloaded for the needs of the environment
refs https://github.com/TryGhost/Toolbox/issues/214
- The `defaultSettings` path name in the config (one pointing to routes yaml file) creates confusion with the `defaultSettings` which populate defaults for in the database settings table.
- Furthermore, the name collision creates a problem when trying to make database default settings dynamic - being able to load them from configurable file path.
- Rename makes "routing" explicit to avoid ambiguity and free up the name for the database defaults
- The value seems to be safe to be renamed as all keys used in `overrides.json` are taking priority - the name "defaultRouteSettings" hasn't surfaced at any point in the git history
- Fixed test fixtures so that members with subscriptions also have products/tiers
- Fixed test fixtures so that default&free tiers can be updated for tests
- Added tests for the signin functionality and welcome page redirects
- Extended `setupStripe` to setup other Members settings - this needs some more
thought around how we proceed
refs https://github.com/TryGhost/Team/issues/1322
Since the Members service is no longer started when Stripe is
reconfigured the url config used for checkout sessions was not updated.
This moves all of the default Stripe urls into the Stripe service, where
they will be updated as/when Stripe is connected/disconnected.
refs: https://github.com/Ghost/Ghost/commit/b5ee17b25
- When moving the site tests into e2e-api they broke because they were getting the changed settings value from the settings test
- The solution is to ensure the settingsCache is reset between tests, which is what this commit does
- This commit also renames the shutdown method to reset, because this is not a permanent operation
- It also renames the resetDb method to resetData, because the concept is we want the internal data to be reset, not just the DB
no-issue
The way we choose which email to send is too tightly coupled with the
magic link generation, which meant that when we added the new paid
signup email, the magic link looked like <url>?action=signup-paid rather
than <url>?action=signup - our welcome page logic was working off of
only using welcome pages for action=signup and so this broke.
- When starting ghost for e2e tests we create a content folder in the os tmp dir
- This means that the folder can change between suites as ghost is started and restarted
- For the most part this is fine, but theme storage caches the path to config (which makes sense, it's not meant to change whilst Ghost is in-memory)
- This is a quick-n-dirty fix that just makes it possible to update that path in the tests, so we know it's in sync
- Ideally we'd not cache the path, use a function to fetch it etc, or fully reset the theme storage layer, but this is the fix I have working today
and so it's going in to unblock things for now
refs https://github.com/TryGhost/Team/issues/1071
We switched to using tiers pivot table that stores list of tiers with access to post when visibility is set to `tiers`. For themes using v3 API while having posts restricted to specific tiers visibility, the post data will not include the list of tiers, which caused an unexpected error while trying to determine post access from tiers list. This change blocks access to post if specific tiers visibility is enabled without data available for list of tiers on post.
refs daeb06e835
- This is an additive change, and the parameter is optional so it's better to ship it right away (testing was hard as it's a chicken and an egg problem to have a feature flag in the setup route).
refs https://github.com/TryGhost/Team/issues/1321
- when updating the Email record after submitting all email batches we have a `catch` call but it was only logging the error
- added a call to Sentry so there's more visibility if those saves fail
refs https://github.com/TryGhost/Toolbox/issues/202
- this code suffers from two problems:
- when we don't have any new settings to insert, we still end up
fetching the columnInfo and owner info, even though we only need
them if we're inserting data. This results in 3 extra queries upon
boot
- secondly, we insert every setting with a separate query - MySQL and
SQLite both support batch inserts and knex has a utility to help us
that I've [used
before](38821c5242).
With 95 settings at the time of writing, this adds 94 extra queries
during the DB init
- this commit refactors the code so that we only fetch the columnInfo and
owner data if we've got new settings to insert, and batches the
inserts using knex's batchInsert util
- this query results in ~95 less queries during DB init and saves a
couple of queries during boot
refs https://github.com/TryGhost/Team/issues/1296
- The `theme` must be a github `org/repo` string
- This uses the internal API instead of the services because the API has extra implementation details not present in the services.
refs https://github.com/TryGhost/Team/issues/1071
Default visibility for a post when set to specific tiers needs special handling as data for specific tiers is stored as an array of tiers on a pivot table. This change handles the default visibility for a new post when set to specific tiers to generate the right default values in model.
closes https://github.com/TryGhost/Team/issues/1311
For some sites, the `portal_products` array was created without any value and due to a possible bug in older version of Ghost, it also didn't get filled on Stripe connect with default product. This causes a side-effect of sites not showing the prices in Portal when tiers beta is enabled or is out as GA. This change populates the missing product data in `portal_product` for sites that have a single tier (haven't enabled tiers beta), as they right now don't have an option to hide the tier.
refs https://github.com/TryGhost/Team/issues/1071
Default content visiblity for specific tiers is now stored split between `default_content_visiblity` and `default_content_visibility_tiers` setting, with former storing the value as `tiers` and the latter stores the list of tiers that the visibility is restricted to. This migration transforms all existing sites that have default visibility stored as an NQL string from previous versions to follow the new model and store correctly on the new setting.
refs https://github.com/TryGhost/Team/issues/1071
Default content visibility for a post can be one of `public|members|paid|tiers`, where `tiers` denotes visibility restricted to specific tiers. This change adds a new setting to store the tier ids when default content visibility is set to `tiers`. This closely matches how the visibility is stored on `posts` table as well, with `visibility` stored as `tiers` and tiers data is stored on tiers pivot table.
refs https://github.com/TryGhost/Team/issues/1287
Currently we have a hard limit of how large an email filter can be,
which is very restrictive once a site starts using Tiers - by moving
toward a TEXT column, we essentially give the filters unlimited size.
This currently doesn't handle SQLite as there are no limits on VARCHARS
in SQLite.
The down migration is a loop so we don't have to handle values larger than
50 characters
no issue
- we check the presence of `members_free_signup_redirect` here but the
log line said `members_paid_signup_redirect`
- this must have been missed in review but it's simple enough to fix
no issue
- `this` isn't referring to the correct `this` in this scope, and the
library is an import of the file anyway, so this fixes the undefined
function call as a result
fixes: https://github.com/TryGhost/Ghost/issues/12871
- This Ghost recommended email scanner, mail-tester.com, reports not having this alt as having an impact of -0.5 out of 10 on your email score.
refs https://github.com/TryGhost/Team/issues/1168
This migrates the existing settings onto the Tier objects, so that users
with Tiers enabled can seamless move from global settings to Tier level
settings - without losing/modifying data/functionality.
refs https://github.com/TryGhost/Team/issues/1071
Going forward, if the visibility of a page/post is set for specific tiers, we send a `tiers` array in API response that contains list of tiers with access. This change -
- updates post/page mapper to transform existing data where `visibility` is a custom nql string to tiers array
- updates default include for post/pages to include `products`, which allows attaching relevant tiers from the pivot table
- cleans up usage of `visibility_filter` in serialization
refs https://github.com/TryGhost/Team/issues/1071
We used `posts.visibility` originally to store visibility as `free|paid` with a character limit of 50. This same field was repurposed to store an NQL filter when member tiers is enabled. The NQL filter uses the slug of the tier name, which can easily create a filter longer than 50 characters, adding an unwanted limitation on number of tiers that can be added to post's visibility.
Going forward, we'd like to store the visibility of posts for tiers in a separate pivot table and instead store the value of `visibility` as `tiers` when restricting post access to specific tiers. This change -
- adds a new pivot table fixture for storing relation between posts and tiers
- adds a migration for creating the new table
- updates tests
refs https://github.com/TryGhost/Team/issues/1168
Because Tiers is still behind a flag - any users which are not using
Tiers will still be editing their settings, and when they switch to
Tiers - the redirects will not necessarily be correct, unless we keep
the Tier columns up to date with any settings changes.
refs https://github.com/TryGhost/Team/issues/1168
This updates the JSON API Schema to allow for the welcome_page_url
property, as well as including the new column in API responses, so that
we can both read and write the value via the API.
refs: TryGhost/Toolbox#166
The new VerificationTrigger listens to events form the members repository, and will cause the verification workflow to be triggered if the number of events is greater than the configured threshold in a rolling 30 day window.
The importer also no longer depends on the import threshold, so the threshold testing is now done in the processImport method in Ghost - seems sensible since we already had this wrapper and the logic is now tiny, since it's just relying on @tryghost/verification-trigger to handle the real stuff.
refs https://github.com/TryGhost/Team/issues/1168
Rather than using a single url for paid signup redirects, we want to
support setting a welcome page on a tier by tier basis. This column will
be used to store the URL. A text column of length 2000 is how we have
stored URL's elsewhere in the schema.
no-issue
The refactor of Stripe boot logic missed catching any errors from the
migrations running or the webhooks initialising. This adds try/catches
to the services so that we can log the errors.
refs https://github.com/TryGhost/Team/issues/1079
These were missing when benefits were added to products, and will allow
theme developers to make more useful custom subscribe pages.
refs https://github.com/TryGhost/Team/issues/1277
- Enabled the `filter` attribute on the route.
- The events are now filtered in-memory instead of in the database.
- This fixes a wrong logic where the API user would have to know the internal event structure.
no-issue
Knex.js aliases "bool" -> "boolean" - this means that you can use either
one in our schema.json file and it'll correctly create the column. Our
model plugin however would only handle columns which used the "bool"
alias. This fixes the plugin to handle both strings.
refs https://github.com/TryGhost/Team/issues/1252
We need a way to signal whether or not a Tier is active or archived, and
we'll be using the active flag in the same way we do for Offers.
refs https://github.com/TryGhost/Team/issues/1277
- The new events types are: `email_delivered_event`, `email_opened_event` and `email_failed_event`.
- This makes existing data accessible to the admin dashboard
refs https://github.com/TryGhost/Team/issues/1257
As well as fixing the code so that we do not create duplicate offer
redemptions going forward, we need to clean up the existing database.
We loop in code because the query to find and delete duplicates is complicated,
and will introduce more risk
refs https://github.com/TryGhost/Team/issues/1257
Offer Redemptions were being overcounted due to the way we were updating
Stripe configuration for the Members service. We would create a new
instance of the members-api, which would have event handlers for
creating Offer Redemptions - by creating a new instance each time Stripe
config changed, we would overcount them.
Here we've pulled out Stripe related logic into the Stripe service, and
updated it internally - rather than creating a new instance. This means
that we've been able to remove all of the logic for re-instantiating the
members-api.
- Bumped members-api & stripe-service
- Removed reinstantiation of members-api
- Used stripe service to execute migrations
- Updated Stripe Service to handle webhooks & migrations
- Used webhook controller from stripe service
- Used disconnect method from stripe service
- Removed unused stripe dependency
- Removed Stripe webhook config from members-api
refs https://github.com/TryGhost/Toolbox/issues/175
- we're going to be making some changes in v5 wrt supported databases
- we needed a way of detecting the difference between MySQL 5 + 8,
MariaDB etc
- I've created `@tryghost/database-info`, which is a small wrapper
around `knex`, which returns this information
- this commit:
- adds the library to Ghost
- initializes the DB info library upon boot
- updates the Admin API /config/ endpoint and UpdateCheck to return
the new string - `mysql5`, `mysql8` etc
Since we now have 2 products by default for all ghost sites, free and default paid, the usage of default product which so far was using first product needs to be updated to use the first paid product.
- updates default product usage to use first paid tier
- updates tests
refs https://github.com/TryGhost/Team/issues/1037
Tiers now have a `type` column to differentiate between `free` and `paid` tiers. This change allows `type` data for a tier in Admin and Portal API output.
refs https://github.com/TryGhost/Team/issues/1037
Free tier is now setup the same way as other tiers, to allow custom description/benefits. This change:
- adds a migration to add a default free tier for all sites
- adds a default fixture to insert a free tier for all new sites
refs https://github.com/TryGhost/Team/issues/1037
- adds new `type` column for tiers to differentiate between free and paid tiers
- all existing tiers are updated to be `paid` tiers
refs https://github.com/TryGhost/Team/issues/1189
Support for AMP is slowly in decline, and makes developing new cards trickier,
since AMP no longer has an effect of SEO we're going to disable it by default
as a first step toward moving away from it.
Co-authored-by: Thibaut Patel <thibaut@ghost.org>
closes https://github.com/TryGhost/Zapier/issues/56
- fixes tag creation when creating posts with `tags: [{slug: 'new'}]` which should be supported
- assigning tags with only `{slug: 'new'}` was triggering our validation for the required `name` property then bubbling up to the `bookshelf-relations` library resulting in a 500 error
- the fix applied here is to set the `name` field to the same as the `slug` field if a name is not provided
refs https://github.com/TryGhost/Team/issues/1240
As this endpoint is hit on every page load when Portal is enabled, it
can cause a lot of traffic to Ghost. The data does not change very
frequently so we've added a 30s cache policy to alleviate load on servers.
refs https://github.com/TryGhost/Team/issues/1257
refs https://github.com/TryGhost/Team/issues/1261
Certain event listens are being added twice due to the way we "reload"
the MembersAPI which can cause duplicate counts of Offer Redemptions.
Rather than creating multiple instances of the MembersAPI we're moving
toward being able to reload the config in place or passing getters for
the config which will allows us to use the MembersAPI as a singleton,
and remove any bugs which come from creating multiple instances.
This also fixes a bug with the allowSelfSignup config not refreshing when
the portal_plans setting was changed.
refs https://github.com/TryGhost/Team/issues/1262
- Flickr embeds have fixed width/height attributes on the `img` elements but the displayed width is always constrained to the width of the email content container which is less than the attribute width, however the fixed height is observed by email clients resulting in images that are stretched vertically
- added a `height: auto` CSS override so email clients will correctly recalculate the image size when resized to fit the container width
refs https://github.com/TryGhost/Team/issues/1216
Since we are increasing the grace period to 10 minutes, we want to make
sure that server crashes during those 10 minutes do not cause single use
tokens to remain in the system. The quickest was to ensure that without
restartable background jobs is to delete all tokens upon boot.
closes https://github.com/TryGhost/Team/issues/1164
- `customThemeSettings` feature is GA so any conditionals can be cleaned up
- removed conditional loading of custom theme settings and associated API routes
- removed event trigger for reloading custom theme settings when the feature flag is toggled
- removed flag from labs GA list
refs https://github.com/TryGhost/Team/issues/1216
Some email security clients are scanning links at delivery, rather than
at the point the user clicks on them. This is causing magic links to
expire. To get around this we're increasing the grace period in which a
link can be used multiple times to 10 minutes.
no issue
@tryghost/errors no longer exports GhostError, as we should only be using subclasses. Replace with InternalServerError as a new default, but should be replaced with a relevant error when one exists.
refs: https://github.com/TryGhost/Toolbox/issues/146
Switched to @tryghost/logging instead of passing around the library. The main sticking points of this change are jobs. When jobs are launched we don't want them to use a separate @tryghost/logging instance because they would start parallel rotation jobs. @tryghost/logging v2.x passes all logs to the parent process if run in a child process, so that we can use the same patterns in jobs and the rest of the codebase.
refs https://github.com/TryGhost/Toolbox/issues/152
- This stops the mounting of the admin and frontend from being buried deep in express initialisation
- Instead it's explicit, which makes two things almost possible:
1. we can potentially boot the frontend or backend independently
2. we can pass services and settings loaded during boot into the frontend
- This needs more work, but we can start to group all the frontend code together
- Meanwhile we also need to rip apart the routing and url services to decouple the frontend from the backend fully
- BABY STEPS!
Co-authored-by: Hannah Wolfe <erisds@gmail.com>
refs https://github.com/TryGhost/Ghost/issues/13837
- a [refactor](9fa8800b9d) in `@tryghost/bootstrap-socket` changed the signature of the method it exprots, but the use of this
library wasn't updated in Ghost
- therefore, the library was trying to send the `@tryghost/logging` library, which caused
errors with Ghost-CLI down the line, as mentioned in the issue
- this commit updates the method signature to match what is expected
