refs https://github.com/TryGhost/Team/issues/1497
- This enables us to keep track of which newsletter an email was sent to even if the related post is deleted.
- Adds the `newsletter_id` property to the email API
refs https://github.com/TryGhost/Team/issues/1500
The newsletter table schema has bunch of changes to go through for new and existing columns, this consolidates the schema changes into a single re-create table migration that drops and adds the newsletter table with correct schema. The table re-create migration needs to run before any of the tables using newsletter as foreign key. The changes include -
- new columns for design related fields
- new slug column for filtering
- unique constraint to `name` column
- remove `default` column (noops the existing default column migration)
- `sender_reply_to` has a default of newsletter and a validation of ['newsletter', 'support']
- updated default values for `subscribe_on_signup` and `recipient_filter`
* Added subscription_id to MRR events
refs https://github.com/TryGhost/Team/issues/1453
As part of 5.0 we want to have cancelled evetns affect MRR. We are going
to backfill and begin populating cancelled events _before_ 5.0 however,
adding a migration to set the MRR deltas as part of 5.0
This migration will need to associate the cancelled events with a
subscription, so we are adding the subscription_id now. This will allow
us to guarantee that all cancelled events will have a subscription_id.
The column is missing a NOT NULL and REFERENCES constraint because
we have not populated the missing values. These will be added in the future
once we have populated the column for all rows
refs https://github.com/TryGhost/Team/issues/1248
This is the underlying cause of the problems we've seen whilst handling
Stripe webhooks. A transaction ensures that the operations are atomic,
but not that they can run concurrently.
If you have some code which does this, concurrently:
1. Starts a transaction
2. Reads a value
3. Changes the values
4. Ends the transaction
Without applying the `FOR UPDATE` lock - you will have both sequenes
read the same value at step 2.
With the `FOR UPDATE` lock - one of the sequences will hang at step 2,
waiting for the other transaction to end, at which point it will resume
and read the _changed_ value.
Because the `edit` method explicitly does a read followed by a write, we
have also add the `FOR UPDATE` lock to this by default, to avoid any race
conditions. This does however require that `edit` is called within a
transaction. An issue here https://github.com/TryGhost/Team/issues/1503
considers running in a transaction by default.
refs https://github.com/TryGhost/Toolbox/issues/280
- In response to 'Accept-Version' header in the request headers, Ghost will always respond with a content-version header indicating the version of the Ghost install that is responding. This should signal to the client the content version that is bein g served
- This is a bare bones implementation and more logic with edge cases where `content-version` is served with a version value of "best format API could respond with" will be added later.
refs https://github.com/TryGhost/Team/issues/1470
Instead of counting the MRR by resolving all the deltas from the past until now, we should start with the current calculated MRR and resolve it until the first event. That would give a more accurate recent MRR (in exchange for a less accurate MRR for older data) and allows us to limit the amount of returned days in the future.
- Includes MRR stats service that can fetch the current MRR per currency
- The service can return a history of the MRR for every day and currency
- New admin API endpoint /stats/mrr that returns the MRR history
- Includes tests for these new service and endpoint
refs https://github.com/TryGhost/Team/issues/1474
- The `default` concept will be replaced by the first newsletter based on the `sort_order`
- This removes the `default` value from the newsletter API
- This simplifies the design to make the api and datastructure more maintainable
refs https://github.com/TryGhost/Team/issues/1471
- This is a many-to-one relation so that many posts can be linked to a specific newsletter
- The `newsletters` table had to come first in the schema file so that it's initialized before the `posts` table (because of the foreign key)
- Updated the model to make sure the new field doesn't leak in the API for now
- This migration isn't using the `createAddColumnMigration` util because of a performance issue. In MySQL, adding/dropping a column without `algorithm=copy` uses the INPLACE algorithm which was too slow on big posts tables (~3 minutes for 10k posts). Switching to the COPY algorithm fixed the issue (~3 seconds for 10k posts).
- SQLite isn't using the codepath where we run a raw SQL query because `knex` is doing multiple queries to add/remove a column
refs https://github.com/TryGhost/Team/issues/1469
Currently, all new members get auto subscribed to the default newsletter. This change adds same behavior with multiple newsletters by auto subscribing all available newsletters on site for new members(If flag is enabled).
Note: In future, this will also take into consideration the `subscribe_on_signup` flag for a newsletter to filter which newsletters should a member be auto-subscribed.
- adds newsletters service for working with newsletter data
- bumps `@tryghost/members-api` package which handles default subscription
- adds new test fixture/data for newsletters
- The url service was moved from frontend to server some time ago but the tests were forgotten
- This is only being done now because in 5.0 major changes are happening and it'll be annoying if the
files move on that branch
- This fits more closely, as this service is to so with rendering helpers and small parts
- Whereas we want to use "rendering" for things concerned with rendering pages
refs: https://github.com/TryGhost/Ghost/commit/11867ab43
- These checks live in the wrong place. They are mostly a frontend thing
- The only server place they were used was slack and that was fixed in 11867ab43
- Moving these to the frontend they fit neatly into the frontend data service
- This is the only piece of server code that relies on the schema.checks, the rest are all frontend
- IMO this code should actually check for the post properties that the slack message needs
- OR it should switch based on the event type
- either way there's no need to have a shared util for this simple use case
- especially becaue it's confusing the use case for it and creating cross-coupling between server and frontend
- Some of the helpers inside the routing service would be better suited to their own service
- These two helpers fetchData and entryLookup talk to the API to get data & so make a decent start for a data service
- The data service would be the single point of contact with the API for the frontend
- Doing this now cos I'm moving some files around ahead of deleting things for 5.0
refs https://github.com/TryGhost/Team/issues/1469
With multiple newsletters, members will now be able to subscribe to one or more newsletters on the site. Previously, the subscription to default newsletter for a member was controlled via a single boolean `subscribed` column on the member table.
This change allows mapping multiple newsletters to a member via new pivot table that stores relation between a member and newsletter.
- adds new `members_newsletters` pivot table
- update tests
refs https://github.com/TryGhost/Team/issues/1463
- Allow admins to perform all newsletter operations
- We can adjust and be more permissive in the future if needed
- Added the tests back as permissions are configured correctly now
refs TryGhost/Team#1458
refs TryGhost/Team#1459
refs TryGhost/Team#1372
- Added a new stats service, which is divided into several categories. Currently only the 'members' category for member related stats.
- When there are missing or corrupt members status events in the DB, the totals returned by the old member stats endpoint (`/members/stats/count`) were wrong. This is fixed in the new service by counting in reverse order and starting with the actual totals.
- New Stats API, with the new `/stats/members/count-history` endpoint.
- This new endpoint also returns the paid deltas -> dashboard 5.0 will show subscribed and canceled paid members for each day
- Includes tests for the new stats service and endpoint
refs: https://github.com/TryGhost/Toolbox/issues/245
- The default behaviour of a serializer is to call a mapper for each object
- Instead of all the boilerplate code we had in the snippets serializer, all we need is a single mapper function
- Added tests for the mapper function as well
refs https://github.com/TryGhost/Team/issues/1433
- The `default` property stores whether a newsletter is set as default by the admin
- The `status` property stores whether a newsletter is archived or not
- The `recipient_filter` property is only storing whether a newsletter is "paid-only" or not for now, although it can be expanded to more specific filters in the future
- The `subscribe_on_signup` property stores whether a new member should be automatically signed up to the newsletter
- The `sort_order` property enables displaying the newsletter list in an order chosen by the admins
refs: https://github.com/TryGhost/Toolbox/issues/245
- .all methods are fallback serializers not to be run as well as a custom serializer
- The default serializer is also a fallback
- The "All" file with before and after are global hooks that _always_ get run as well as other serializers
- There's a lot of room for further improvement here especially with naming but this logic makes more sense
for the usecases AND doesn't affect v2 & v3 etc. We can do another pass after 5.0
refs https://github.com/TryGhost/Toolbox/issues/213
- `better-sqlite3` doesn't like multiple queries in the same statement
so we can make the change here to split them up ahead of the switch
refs: https://github.com/TryGhost/Ghost/commit/e68cb8b31
- I found that some of the places we use rewire are totally unnecessary
- Rewire seems to mess with coverage sometimes
- It's also a code smell in general so I've ripped it out where possible
refs: https://github.com/TryGhost/Toolbox/issues/245
- we don't need this serializer because the default serializer will call the tags mapper
- for now I've changed, rather than removed the tag serializer test as this shows default works the same!
- settings cache was appearing as untested because we use rewire!
- rewire was totally unnecessary in this case, so I removed it
- updated to 100% test coverage whilst there, including removing one unreachable branch
- commented some undesirable behaviour I found whilst trying to reach all branches
- I don't want to change the behaviour to return false correctly without having a reason beyond improving coverage
refs: https://github.com/TryGhost/Toolbox/issues/245
- Added a serializer called default to the canary API
- Ideally, this would be part of the shared framework, but this would change v2/v3 and we're about to get rid of them
- Therefore, we change just canary for now, and we can refactor again later.
- Added wiring to handler that uses the default serializer, if there is a default, and isn't an explicit serializer for the endpoint
- Removed the invites serializer, so that one endpoint now uses the default
Note: previous commits have added explicit serializers to every endpoint, this is the first step towards paring
that back so that we have less serializers overall, not more!
refs: https://github.com/TryGhost/Toolbox/issues/245
- Ghost's API framework has a mixed up concept of what a serializer is. Mappers are true serializers! What we call serializers are little more than a small formatting step.
- This PR splits mappers into individual files and uses the endpoint's docname as the mapper name. This will help us to automate the calling of a mapper for an endpoint later.
- This is one tiny step in reworking the framework to need less code to make it work, and to have clearer concepts for how to do things.
As multiple tiers is now GA, we want to allow devs to be able to work with Tiers and offers via content/Admin API. This change -
- updates fixtures to add permissions to admin integration role for new sites
- adds migration to update existing sites to have correct permissions for role
- whitelists add/edit/read/browse on tiers and offers API for integrations
refs https://github.com/TryGhost/Toolbox/issues/169
- There was not much clarity around how the tokens created for the versioned API audience would behave when non-versioned API is introduced. The tests added here illustrate the tokens being forward compabible (created for versioned -> verified at non-versioned API) and not backwards combatible (created for non-versioned -> verified at versioned)
refs https://github.com/TryGhost/Toolbox/issues/169
- Before releasing Ghost v5 we would like to move all canary-related URLs to a non-versioned format, which will become a default in v5.
- 'canary' is by definition unstable, so breaking any unprepared client explicitly using the canary is expected
- Removed the aliased /content/ and /admin/ apps from app.js because with updated configuration they become duplicates of 'canary' endpoints
refs https://github.com/TryGhost/Team/issues/1363
- When uploading a zip of images in Settings > Labs > [Import], it will skip images that have an uppercase extension, citing an 'unsupported file type' error.
- Cause: Glob ignored those files when matching extensions in ImportManager
- Fix: Added nocase option where needed
- Extended tests to also test the processZip method of ImportManager with getFilesFromZip
- Added isValidZip for zip with uppercase image
- Cleaned up JSDoc in ImportManager, and replaced some older JS syntax
Fixed zipContainsMultipleDataFormats error never thrown:
When a zip combines two data formats, no error was thrown.
- The promise error was only returned in an _.each loop, but never thrown
- Previously when combining multiple data types in a zip file, no error got thrown
- Added a test for this error
- Also added a test for noContentToImport error
Other errors and fixes:
- Added missing length in getBaseDirectory check
- getContentTypes fixed (returned duplicate values). Type error came up after adding all JSDocs
- updated tests to match real types from JSDoc and pass type validations
- Rewrote some methods in the async await syntax
- Added tests for ImportManager clean up
- Our old fixtures were designed as a guide to getting started to Ghost, but they got in the way
- The old fixtures now live as part of ghost.org/resources - a living guide to starting with Ghost
- These new fixtures mean the site is ready to go as soon as it's setup
Co-authored-by: Hannah Wolfe <github.erisds@gmail.com>
no issue
- The support for the misformated roon imports was temporary anyway, based on the comments in code. It's also unecessary to keep around any code related to Roon as it's been _ages_ since anybody needed this kind of migration
refs https://github.com/nodeca/probe-image-size/blob/master/CHANGELOG.md#600---2020-11-04
- `probe-image-size` v6 now supports `.ico` files so we can
allow probing of dimensions via this library rather than falling back
to downloading the entire image via `image-size`
- also updates a test because .ico files no longer use the internal
request lib, which simplifies things a little bit
- Have ensured we have 100% coverage of core/server/api/shared/serializers/handle.js
- This meant I had to swap around two validation clauses as one was unreachable
- I have done this as I want to make some changes in this area of the codebase, and want to ensure we have tests
and a clear understanding of what this code does before I change it
Welcome pages are now moved as a property of individual tiers instead of global property on settings. Since we removed the alpha flag for new welcome page behavior, tests had to be updated to test the flow against new behavior.
- we catch error arising from creating webhooks and check for specific codes
- if our error does not match one of those codes, we don't propagate the
error up
- this becomes a problem if saving a webhook fails for some other reason
because upstream code assumes we return an error or model
- this commit re-throws the error and adds a test that would have caught
this
refs https://github.com/TryGhost/Team/issues/1004
Replaces {{products}} helper usage with updated {{tiers}} helper. Default output for {{tiers}} helper is the same as {{products}} helper.
refs https://github.com/TryGhost/Team/issues/1004
- adds new `{{tiers}}` helper behind `multipleProducts` flag
- `{{tiers}}` outputs a string with list of tiers that have access to specific post when used in a post context in theme
- outputs empty string when used out of a post context and without access to `visibility` property
- uses tiers attached to post column for data
refs https://github.com/TryGhost/Team/issues/1004
The `tiers` column for a post/page only contained data if its visibility is set to `tiers`, otherwise its empty. This is because originally the purpose of `tiers` column on `post` was to capture specific tiers with access to post.
The best way to ensure a consistent behavior for `tiers` column data on post is to update it to always contain list of all `tiers` that have access to post, and not just when the visibility is `tiers`. This means the value is set to all tiers when visibility is one of public|members, and only paid tiers when visibility is `paid`. This change also allows on frontend to get all relevant `tiers` information for a post locally within post context instead of relying on additional information from outside.
This change -
- updates the output serializer for post/page to add all desired tiers manually in case of visibility is not `tiers`
- updates tests
- We have an existing pattern for using `visibility: public` instead of `visible: true|false`
- We no-op the existing migration and roll forward so that we don't have to manually revert db changes
refs https://github.com/TryGhost/Team/issues/1367
- The {{access}} property of a post always returned false for non-public content
- Added the member context to the get, next-post and prev-post helpers
- The get, next-post and prev-post helpers didn't add the member context to the internal API calls
- Added the members context to these calls
- Added tests that check if the member context is passed to the API calls
- Transformed next_post helper tests to async await syntax
- Transformed prev_post helper tests to async await syntax
refs https://github.com/TryGhost/Team/issues/1386
- The current match handler supports normal (in)equality operators, but no numeric comparisons (<, >, <=, >=)
- A use case for these new operators is to show the latest post in a separate way from other posts
Includes unit tests to check the new behaviour.
Run via `yarn test test/unit/frontend/helpers/match.test.js`
