refs https://github.com/TryGhost/Ghost/issues/12646
- when a blank mobiledoc document is created, ensure the correct Ghost version is added so content output is pinned to the version of Ghost that created it
- Currently the new boot process breaks the loading of inactive themes because the theme loader event isn't wired until after the server.start event is fired
- This is a WIP fix, there needs to be some bigger refactoring of the boot process once the old process is gone
refs https://github.com/TryGhost/Ghost/issues/12646
- `card-markdown` support is being removed from the default mobiledoc cards so we need to make sure imported content doesn't fail
- removed long-unused `cardName` payload property from tests
refs https://github.com/TryGhost/Ghost/issues/10318
- Object format used in previous "slack" setting was considered an
anti-pattern. Flag structure of separate slack_url and slack_username
values was extracted out of the "slack" JSON.
no refs
The Portal URL that is shipped with every Ghost version is so far hardcoded in `ghost_head` and updated every-time we ship a new Portal minor/major change. This change brings the Portal URL inside the default Ghost config, which has few advantages -
- Allows easier access/managing of active Portal url/version
- Allows override for Portal URL for development/other purposes, where `config.*.json` allows using a Portal URL pointing to locally built copy for testing
refs https://github.com/TryGhost/Ghost/issues/12602
As we only want to add events for when an email is changed, we need to store the previous email if we want a complete log of all emails for a member
refs https://github.com/TryGhost/Ghost/issues/12608
- adds `admin/canary/themes/install` endpoint to the Admin API
- requires two query params. `source` must be set to "github". `ref` should refer to a GitHub repo in the format "{org}/{repo}"
- downloads zip archive for the repo from github
- runs downloaded zip through the same process as uploaded zips
refs https://github.com/TryGhost/Ghost/issues/12602
* Added members_payment_events table
This table will store successful and unsuccessful payment attempts, and
can be used to calculate gross volume over time.
* Added members_login_events table
This table can be used to audit member logins
* Added members_email_change_events table
This table will allow us to store a history of email addresses associated with a member
* Added members_status_events table
This table will allow us to track the change in status over time for members, as well
as calculate aggregates over time, e.g. paid members over time
* Added members_paid_subscription_events
This table will allow us to track subscriptions changes for members, as well as
calculating MRR over time
refs 0b79abf5b2
- in the referenced commit, we refactored the boot code to simplify it
- one of the required files was not updated with the change, so the path
was wrong
- this wouldn't have been easily visible from local testing because it's
based upon a config flag that isn't on by default
- this commit updates the path to the file
no issue
refs e3a0bb535f
Previously, {{content}} helper was updated use default or custom template to show a CTA when trying to use helper without access to content.
While the change was expected to not affect any existing themes as `{{content}}` helper is not supposed to be used on member sites without `access` check, we found quite a few themes use a pattern of using `content.hbs` with `{{content}}` helper inside them as default, which causes infinite loop in how content helper works in certain cases.
Whilst this followed the pattern of the other helpers using a partial with the same name, there are 2 key differences:
- this partial template pattern is being introduced way after the inception of themes with a fairly generic name “content”
- this partial template isn’t used to render the helper all the time - just the CTA part under certain circumstances.
This change updates the template name to `content-cta.hbs` to which makes it less likely to clash, and makes more sense as to what it is.
refs https://github.com/TryGhost/Ghost/issues/12567
- Changing unique constraint from slug to slug+type should allow for posts and pages to be created with the same slug
- The constraint will be present on application layer for API v4 while we figure out how to deal with it in API v5
Background:
- Ghosts existing boot process is split across multiple files, has affordances for outdated ways of running Ghost and is generally non-linear making it nigh-impossible to follow
- The web of dependencies that are loaded on boot are also impossible to unpick, which makes it really hard to decouple Ghost
- With 4.0 we want to introduce a new, linear, simpler, clearer way to boot up Ghost to unlock decoupling Ghost into much smaller pieces
This commit:
- adds a new ghost.js file which switches between boot mode with `node index` or `node index old` so that if we find bugs we can work around them this week
- Note: the old boot process will go away very soon, but ghost.js will remain as the interface between the command to start Ghost and the application code
- reworks the database migration process into a standalone utility, so that the DB is handled as one simple step of the boot process, decoupled from everything else
- is missing tests for this new db utility
- leaves a lot of work to do around loading core code, services, express apps in a sensible order, as work to fix this would start to break the old boot process
- doesn't use the new maintenance app because we aren't restarting the server here, instead we have the concept of a "core app" that starts in maintenance mode - need to think about how apps will be decoupled in the near future
closes https://github.com/TryGhost/Ghost/issues/12600
The bookshelf-relations plugin which we use will **remove** all
relations when they are set to an empty array, but will leave them alone
if it's set to undefined.
Our logic to deduplicate uppercase & lowercase version of the same label
was in advertently always setting the labels to an array, but when the
model was saved without passing the labels, this array would be empty.
Here we've added a check which will skip all label handling, if there
are no labels set.
refs https://github.com/TryGhost/Ghost/issues/12602
In order to build up a list of un/subscribes over time we have to use
the customQuery functionality to run SQL aggregates - this is "hidden"
behind an option, so that we can find{All,Page} as usual.
refs https://github.com/TryGhost/Ghost/issues/12602
Member Events in general are read-only after they've been created, so
we've explicitly disallowed destroying and editing via the model
refs https://github.com/TryGhost/Ghost/issues/12602
This allows us to backfill the data for testing purposes as well as for
use by existing sites. The mechanism for determining the date of an
unsubscribe event is crude, we use the date of the last update, which we
might expect to be _sometimes_ correct, but not always. We store the
`source` as NULL for unsubscribes as we cannot determine that with any
certainty whatsoever.
refs https://github.com/TryGhost/Ghost/issues/10921
refs https://github.com/TryGhost/Ghost/pull/11068
- When there is no twitter or og image specified for a post or a page the new default falls back to appropriate site-wide twitter/og image or site's cover image.
- New rules of follback follow this logic:
post/page/amp = post.twitter_image || post.feature_image || settings.twitter_image || settings.cover_image;
post/page/amp = post.og_image || post.feature_image || settings.og_image || settings.cover_image;
refs https://github.com/TryGhost/Ghost/issues/12567
- The method was using unneeded lodash dependency and was too complex for what it was doing
- Reshuffled internal code to use native JS filter/forEach iterators
refs https://github.com/TryGhost/Ghost/issues/12567
- Introduced here @@UNIQUE_CONSTRAINTS@@ notation allows to create unique contraints over multiple database fields. This will be needed to change posts' table unique constraint from `slug` to `slug+type`.
- The notation is equivalent to SQL's: UNIQUE(column_name1,column_name2)
- Example use in schema:
posts: {
slug: {type: 'string', maxlength: 191, nullable: false},
type: {type: 'string', maxlength: 50, nullable: false, defaultTo: 'post', validations: {isIn: [['post', 'page']]}},
'@@UNIQUES@@': [
['slug', 'type']
]
}
no issue
- when saving Stripe keys with `stripeDirect: true` config, if either key didn't match the key format the returned validation error always contained `stripe_secret_key`
- updated to output `stripe_publishable_key` if it was the publishable key that was invalid
refs https://github.com/TryGhost/Ghost/issues/12565
- Dropping tables happens sporadicaly, usually during major version releases. It made sense to create a utility based on previous migrations of this type (e.g.: 3.0/03-drop-client-auth-tables.js migration) to avoid code duplication in the future
no-issue
This removes all references to the members labs setting, any code that was run conditionally behind this flag now runs unconditionally.
* Removed usage of Members labs flag
* Removed tests for Members disabled
* Added dynamic keypair generation for when setting is missing
no refs
- Updates member model serializer to directly set subscriptions on member object instead of `stripe.subscriptions`
- Updates all references to members subscriptions from nested `stripe.subscriptions` to `subscriptions`
- Updates v3 API serializer to still use `stripe.subscriptions`
- Updates tests
no-issue
* Removed support for paid param from v3 & canary API
* Updated active subscription checks to use status flag
* Updated MEGA to use status filter over paid flag
* Removed support for paid option at model level
* Installed @tryghost/members-api@1.0.0-rc.0
* Updated members fixtures
no-issue
This value is nullable from the Stripe API so we're making sure that we
can store exact values locally.
SQLite3 does not supports altering columns so instead we have to:
1. Create a temp table and copy the data to it
2. Delete the original table
3. Recreate the original table with the necessary modifications
4. Copy the data from the temp table
5. Drop the temp table
refs #12568
- DI pattern through constructor allows for very clear separation of concerns and makes it easy to extract the class into a separate module
refs #12568
- When the server is in maintenance mode (e.g. during migration) it needs to serve a 503 page. Previously this role was delegated to the "frontend" which tightly coupled server bootup and frontend.
- With a dedicated HTTP application serving up 503 server is no longer coupled to the frontend during maintenance
refs #12568
- Allowing express app (frontend) to be booted after the migrations
decouples server maintenance mode from a need to have the "frontend"
loaded beforehand
refs #12160
This flag will allow us easier filtering of members via the API
* Added status column to members table
This flag will be used to determine if a member is free or paid, rather
than relying on joins with the customers and subscriptions tables.
* Added migration to populate members.status
As we add the column with a default value of "free" we only need to care
about the paid members here. We also preemptively handle migrations for
SQLite where there are > 998 paid members.
refs #12568
- The readivility of multiple nested promises is very poor
- Refactored this piece of code before doing a bigger reshuffle of frontend/server boot process
no issue
We added `portal-action` and `requestSrc` in 3.x to allow Portal to handle notifications only for auth actions trigged while using it directly, so that existing themes are not affected in any way. Going forward in 4.0, we don't want to have any special handling in backend for Portal but instead expect themes to handle any Portal specific behavior directly.
- Removes setting of `portal-action` for auth actions like signup
- Removes `requestSrc` being passed through to determine portal actions
closes https://github.com/TryGhost/Ghost/issues/12449
We’re starting to bump into errors with our current exporter due to the size of some of the tables in the db and hitting an issue with Ghost running out of memory during export. The intention for the export/import is not to be backup/restore functionality, but for exporting content and authors.
In addition, exporting and re-importing Stripe secret/publishable keys can cause unexpected side-effects for sites that can has major side-effects. This change -
- Removes `email_batches` and `email_recipients` tables from export data to reduce export size due to large amount of analytics data
- Removes stripe's secret/publishable/webhook keys to avoid unexpected issues with re-import
closes https://github.com/TryGhost/Ghost/issues/12347
This change allows a token to be used multiple times for the first 10
seconds after its initial use, this will stop dynamic link checking
software from invaliding magic links.
no refs
Adds new FirstPromoter settings similar to amp, which allows sites to take advantage of FirstPromoter to launch their own member referral program natively.
- Adds new firstpromoter settings group
- Adds `firstpromoter` setting to group
- Adds `firstpromoter_id` setting to group for FirstPromoter referral tracking id
- Updated tests
refs #12537
- Refactored overuse of rewire mocking blog-version.
- Fixed a bug introduced along the way when duplicate notifications errored instead of returning empty result
refs #12537
- notifications controllers were overbloated with non controller related code and were identical. It is important to reduce unnecessary code ahead of v4 API introduction
- Follow up commit will transform newly created module into a class following DI pattern
refs #12537
- Class syntax is preferred over module functions because of constructor parameter injection (DI) which allows for easier module decoupling and testing
refs #12537
- `stats` method in members controller is quite big and does much more then controller method code should - few calls to relevant modules
- Extracted code "as is" into members serivce
- Next step will be to refactor this module as a class pattern with DI parameters
refs https://github.com/TryGhost/Team/issues/446
Mailgun permanent failure events do not always correspond to unsubscribe-level events as originally thought, meaning some members could be unsubscribed unexpectedly due to delivery hiccups.
- disabled auto-unsubscribe on permanent failure events in the analytics event processor
- list maintenance will be added back in the future via alternative means
closes https://github.com/TryGhost/Ghost/issues/12493
- bumps `bookshelf-relations` to new version that allows for `hasMany` children to be kept when a parent record is destroyed
- adds necessary config to the `Member.email_recipients` relationship to keep related records when a member is deleted
closes https://github.com/TryGhost/Ghost/issues/12492
The changes to email processing models had set replyTo address for an email batch as `reply_to` instead of `replyTo` which was not picked by mailgun service for setting newsletter reply address
closes#12503
Since we include _all_ subscriptions in the `stripeSubscriptions`
relation, we must check that for an existing active complimentary
subscription to decide whether or not we should skip creation of a
complimentary subscription
ref #10898
- The redirects configuration's `to` & `from` URL parameters used to ignore it's query string parameters, which resulted in unexpected behavior
- Current changeset only partially fixes the issue. Now `to` URL's query parameters always take precedence over incoming query parameters and the rest of query parameters are passed through.
closes#12271
- When previous active theme did not have locale data for certain language, loading a theme which has such data did not result in correct locale being loaded
- Underlying issue was in settings cache being outdated during theme change related i18n initialization
- Fix focuses on removing settings cache dependency and and rely on most up to date data about currently active theme
- The benefit of this approach is reduced coupling with settings cache
closes#12181
refs https://github.com/TryGhost/Ghost/pull/12265
- The 'core' value is invalid for settings 'type' column
- The 'core' default value for 'type' column in model is also invalid
- Both need to be removed as they are never used and only introduce confusion into the codebase
no issue
- Previously used notation was not a valid JSDoc one. With this correction IDEs will provide more accurate information about all furnction parameters
refs https://github.com/TryGhost/Ghost-Utils/issues/118
- Duplicating error handling across jobs is not best developer experience. Also, having custom error handling logic did not allow for recommended worker script behavior: allowing for unhandled exceptions to bubble up and be managed by parent process
refs c1d66f0b01
- fixed base model allowing '@@INDEXES@@' as a permitted attribute/order
- fixed base model automatically setting `@@INDEXES@@` to null on the model when creating
- added `doAuth('members:emails')`
- creates an `email_batch` record attached to the first email in the fixtures
- creates an `email_recipients` record for each member
- runs analytics aggregation so the email and member counts are as expected
- added acceptance test for `/member/:id/?include=email_recipients`
no-issue
We now include all subscriptions as part of the member, so we need to
ensure the comped flag is only true if the member has an **active**
complimentary plan
refs https://github.com/TryGhost/Ghost-Admin/pull/1796
We want to be able to display an email activity timeline in Ghost-Admin for each member. The quickest way to achieve that right now is to provide access to the `email_recipient` data for the member when fetching, this will allow clients to build up a timeline based on the event timestamps included with each email_recipient/email pair.
- sets up `email_recipients` relationship in `Member` model
- updates members API read endpoint to accept an `email_recipients` include parameter
- appends `email_recipients.email` to the `withRelated` array when `email_recipients` is included so that we have data available for email subject and html/plaintext for previews
- updates members API output serializer to include the email_recipients object in the output
no issue
- sqlite will store a float in an integer column due to it's type affinity resulting in long decimal numbers in the UI when we're expecting an integer
- use the `ROUND()` function to ensure we're storing integers in place of floats when performing open rate average calculations
no-issue
This module encapsulates the work around performing imports, it
currently uses the concept of a "Job" which at the moment is not
persisted to the database, however when we want to look at resuming
imports after a server restart, this should give us the flexibility to
do it.
refs https://github.com/TryGhost/Ghost/issues/12461
- added two default aggregations for overall email count and opened email count
- when number of tracked emails is sufficient add the open rate aggregation to the update query
refs https://github.com/TryGhost/Ghost/issues/12461
- adds `members.email_count` and `members.email_opened_count` columns to contain cached counts for faster queries when outputting member data via API
- adds migration to populate cached counts with existing data
- tested locally on ~50k members which took ~4sec on mysql
- updates members output serializer to include the new fields in API output
refs https://github.com/TryGhost/Ghost/issues/12421
- add `orderRawQuery` function to members model so that we can ensure members with an open rate are ordered before members without an open rate no matter the order direction chosen
- added `email_open_rate` to members in the test fixtures to allow testing of order
refs https://github.com/TryGhost/Ghost/issues/12421
requires https://github.com/TryGhost/Ghost/pull/12457
- updates stats aggregator to calculate and store an open rate for each member
- uses two queries because I couldn't find a reasonable approach to perform the update in a single query as per the email aggregation
- benchmarked locally at <1sec/1000members
- will not store an open rate unless the number of tracked emails sent to a member is above a certain threshold (defaults to 5) to avoid new members being heavily weighted
- fixes typo in EmailAnalytics that was stopping member stats from being aggregated
refs https://github.com/TryGhost/Ghost/issues/12421
- nullable so we can distinguish between members that have and haven't received any trackable emails
- indexed because we'll be using this column for sorting
