refs https://github.com/TryGhost/Team/issues/1504
- The permissions were missing in the fixture file
- This caused some Ghost installs to not have the right permissions
- This is fixed by adding the missing permissions to the fixture file and creating a migration to resolve the missing permissions
refs https://github.com/TryGhost/Team/issues/1484
While sending a post to a specific newsletter, we'll need to get list/count of members eligible to receive the post. This change enables members admin API to filter list of members on specific newsletter by their slug.
closes https://github.com/TryGhost/Team/issues/1491
With multiple newsletters feature, a site should always have at-least one newsletter by default. Also, as with the default product, the default newsletter also needs to be renamed to the site title during the setup flow.
- adds default newsletter to main and test fixtures
- updates setup flow to rename newsletter name and sender name to site title
- updates model to extend default value for fields
- updates test
refs https://github.com/TryGhost/Team/issues/1457
We want to save the MRR with a subscription to simplify the calculation of the total MRR once, in 5.0, we also take 'forever' offers into account into the MRR (so we can just SUM the MRR of all subscriptions).
- Sets the MRR to 0 for now.
- Separate commit will fill in all the values in a data migration, but this needs to get merged first because we need this new column in order to update the members-api package (so we already save the MRR before doing the data migration).
- Updated `test/e2e-api/admin/legacy-members.test.js` with improved body assertions.
refs https://github.com/TryGhost/Team/issues/1302
We do not want to update MRR calculations until 5.0 - so will be adding
a separate migration to populate the mrr_delta column for 5.0
We've only added events for non-expired subscriptions as this is simpler
and won't impact the mrr events when mrr_delta is updated
refs https://github.com/TryGhost/Team/issues/1302
Currently we only have three event types: created, updated & expired.
A created event always has a `from_plan` of null
An updated event will always have a different `to_plan` and `from_plan`
as the subscription has changed tier/cadence.
An expired event _should_ have a `to_plan` of null, but due to a bug we
have events with the same `from_plan` and `to_plan`.
refs https://github.com/TryGhost/Team/issues/1500
- newsletter schema columns were using incorrect syntax for `isIn` validations
- adds schema test to make sure schema `isIn` validations are in correct format
refs https://github.com/TryGhost/Team/issues/1497
- This enables us to keep track of which newsletter an email was sent to even if the related post is deleted.
- Adds the `newsletter_id` property to the email API
refs https://github.com/TryGhost/Team/issues/1500
The newsletter table schema has bunch of changes to go through for new and existing columns, this consolidates the schema changes into a single re-create table migration that drops and adds the newsletter table with correct schema. The table re-create migration needs to run before any of the tables using newsletter as foreign key. The changes include -
- new columns for design related fields
- new slug column for filtering
- unique constraint to `name` column
- remove `default` column (noops the existing default column migration)
- `sender_reply_to` has a default of newsletter and a validation of ['newsletter', 'support']
- updated default values for `subscribe_on_signup` and `recipient_filter`
* Added subscription_id to MRR events
refs https://github.com/TryGhost/Team/issues/1453
As part of 5.0 we want to have cancelled evetns affect MRR. We are going
to backfill and begin populating cancelled events _before_ 5.0 however,
adding a migration to set the MRR deltas as part of 5.0
This migration will need to associate the cancelled events with a
subscription, so we are adding the subscription_id now. This will allow
us to guarantee that all cancelled events will have a subscription_id.
The column is missing a NOT NULL and REFERENCES constraint because
we have not populated the missing values. These will be added in the future
once we have populated the column for all rows
refs https://github.com/TryGhost/Team/issues/1248
This is the underlying cause of the problems we've seen whilst handling
Stripe webhooks. A transaction ensures that the operations are atomic,
but not that they can run concurrently.
If you have some code which does this, concurrently:
1. Starts a transaction
2. Reads a value
3. Changes the values
4. Ends the transaction
Without applying the `FOR UPDATE` lock - you will have both sequenes
read the same value at step 2.
With the `FOR UPDATE` lock - one of the sequences will hang at step 2,
waiting for the other transaction to end, at which point it will resume
and read the _changed_ value.
Because the `edit` method explicitly does a read followed by a write, we
have also add the `FOR UPDATE` lock to this by default, to avoid any race
conditions. This does however require that `edit` is called within a
transaction. An issue here https://github.com/TryGhost/Team/issues/1503
considers running in a transaction by default.
refs https://github.com/TryGhost/Toolbox/issues/280
- In response to 'Accept-Version' header in the request headers, Ghost will always respond with a content-version header indicating the version of the Ghost install that is responding. This should signal to the client the content version that is bein g served
- This is a bare bones implementation and more logic with edge cases where `content-version` is served with a version value of "best format API could respond with" will be added later.
refs https://github.com/TryGhost/Team/issues/1470
Instead of counting the MRR by resolving all the deltas from the past until now, we should start with the current calculated MRR and resolve it until the first event. That would give a more accurate recent MRR (in exchange for a less accurate MRR for older data) and allows us to limit the amount of returned days in the future.
- Includes MRR stats service that can fetch the current MRR per currency
- The service can return a history of the MRR for every day and currency
- New admin API endpoint /stats/mrr that returns the MRR history
- Includes tests for these new service and endpoint
refs https://github.com/TryGhost/Team/issues/1474
- The `default` concept will be replaced by the first newsletter based on the `sort_order`
- This removes the `default` value from the newsletter API
- This simplifies the design to make the api and datastructure more maintainable
refs https://github.com/TryGhost/Team/issues/1471
- This is a many-to-one relation so that many posts can be linked to a specific newsletter
- The `newsletters` table had to come first in the schema file so that it's initialized before the `posts` table (because of the foreign key)
- Updated the model to make sure the new field doesn't leak in the API for now
- This migration isn't using the `createAddColumnMigration` util because of a performance issue. In MySQL, adding/dropping a column without `algorithm=copy` uses the INPLACE algorithm which was too slow on big posts tables (~3 minutes for 10k posts). Switching to the COPY algorithm fixed the issue (~3 seconds for 10k posts).
- SQLite isn't using the codepath where we run a raw SQL query because `knex` is doing multiple queries to add/remove a column
refs https://github.com/TryGhost/Team/issues/1469
Currently, all new members get auto subscribed to the default newsletter. This change adds same behavior with multiple newsletters by auto subscribing all available newsletters on site for new members(If flag is enabled).
Note: In future, this will also take into consideration the `subscribe_on_signup` flag for a newsletter to filter which newsletters should a member be auto-subscribed.
- adds newsletters service for working with newsletter data
- bumps `@tryghost/members-api` package which handles default subscription
- adds new test fixture/data for newsletters
no issue
- It was hard to plug in with additional code into current `.then` based chain. Refactoring to use a more modern syntax helps with readability and allows for easier edits
- This fits more closely, as this service is to so with rendering helpers and small parts
- Whereas we want to use "rendering" for things concerned with rendering pages
refs https://github.com/TryGhost/Team/issues/1469
- wires newsletter preferences for a member in Portal to view/edit real data
- allows members to control their subscription to multiple newsletters
refs https://github.com/TryGhost/Team/issues/1469
We have an after hook to update `sort_order` for pivot tables after update as base bookshelf plugin. Since new `members_newsletters` table doesn't has a `sort_order` column on it, this change disables the after hook for it to avoid errors.
refs https://github.com/TryGhost/Team/issues/1469
- allows Portal access to multiple newsletters on a site
- allows Portal to update member's newsletter subscription preference
refs https://github.com/TryGhost/Team/issues/1469
- updates member model to add relation to newsletter via pivot table
- updates member api serializer to include newsletter data
- updates tests
refs: https://github.com/TryGhost/Ghost/commit/11867ab43
- These checks live in the wrong place. They are mostly a frontend thing
- The only server place they were used was slack and that was fixed in 11867ab43
- Moving these to the frontend they fit neatly into the frontend data service
- This is the only piece of server code that relies on the schema.checks, the rest are all frontend
- IMO this code should actually check for the post properties that the slack message needs
- OR it should switch based on the event type
- either way there's no need to have a shared util for this simple use case
- especially becaue it's confusing the use case for it and creating cross-coupling between server and frontend
- Some of the helpers inside the routing service would be better suited to their own service
- These two helpers fetchData and entryLookup talk to the API to get data & so make a decent start for a data service
- The data service would be the single point of contact with the API for the frontend
- Doing this now cos I'm moving some files around ahead of deleting things for 5.0
- We had an incident where a migration file was misnamed.
- This is caused by using slimer with spaces rather than hyphens
- We didn't imagine this case when writing the regex for matching filenames
- However, now we know it, it's easy to tweak the regex to match this case
- This requires adding an override to the one badly named file
- Keeping overrides inside the file means the context of why a rule isn't firing is present inline when modifying a file
- This makes it easier to maintain files, at the cost of needing to search to find all overrides
refs 2fa3985d42
- Running tests with error logging set to "error" lever, produced a massive amounts of errors related to failed Stripe keys. Making it hard to look through the output.
- When Ghost is running in teste environment by default it is configured with an invalid Stripe key that looks like `sk_test***`. In this case the Members migrations runs creating requiest to Stripe, which fail.
refs https://github.com/TryGhost/Team/issues/1469
With multiple newsletters, members will now be able to subscribe to one or more newsletters on the site. Previously, the subscription to default newsletter for a member was controlled via a single boolean `subscribed` column on the member table.
This change allows mapping multiple newsletters to a member via new pivot table that stores relation between a member and newsletter.
- adds new `members_newsletters` pivot table
- update tests
- if we add a column with a foreign key reference, the `down` migration
will try to remove that column
- you can't remove a column without deleting the foreign key reference
first
- our migration utils didn't take that into account and there's nothing
in Knex to do this for us
- this commit deletes the foreign key before removing the column if we
have one referenced in the column spec
- also updates the code to pass the column spec into the util
refs 7172db74b0 (r70370252)
- The changes referenced above changed fixtures without taking "test fixtures" in "test/utils/fixtures/fixtures.json" into account. This fix is only cosmetical to unlutter the test output with logging level set to "error". Once the referenced comment is answered a proper fix should be applied here
- this commit removes the colons from the log messages to make each
line flow easier when you're reading it
- also removes a couple of superfluous words that aren't needed
refs https://github.com/TryGhost/Team/issues/449
- Once we deprecate site.lang use in favor of site.locale this helper will need inspection for potentially breaking change. Left a note to come back to it once it's time
refs https://github.com/TryGhost/Team/issues/1463
- Allow admins to perform all newsletter operations
- We can adjust and be more permissive in the future if needed
- Added the tests back as permissions are configured correctly now
refs TryGhost/Team#1458
refs TryGhost/Team#1459
refs TryGhost/Team#1372
- Added a new stats service, which is divided into several categories. Currently only the 'members' category for member related stats.
- When there are missing or corrupt members status events in the DB, the totals returned by the old member stats endpoint (`/members/stats/count`) were wrong. This is fixed in the new service by counting in reverse order and starting with the actual totals.
- New Stats API, with the new `/stats/members/count-history` endpoint.
- This new endpoint also returns the paid deltas -> dashboard 5.0 will show subscribed and canceled paid members for each day
- Includes tests for the new stats service and endpoint
refs https://github.com/TryGhost/Team/issues/1463
- This enables listing, creating and editing newsletters
- The tests are commented out as the permissions will be added in a follow-up commit
refs https://github.com/TryGhost/Toolbox/issues/213
- our `knex` dependency has been out of date for a while so my aim was
to bring it up to date
- this required also switching `sqlite3` to `@vscode/sqlite3` because
knex switched the default sqlite driver
- this commit also bumps knex-migrator and switches to a mock-knex fork
until Knex 1.0 support has been merged
- also updates an error message to handle a new code in SQLite
refs: https://github.com/TryGhost/Toolbox/issues/245
- The default behaviour of a serializer is to call a mapper for each object
- Instead of all the boilerplate code we had in the snippets serializer, all we need is a single mapper function
- Added tests for the mapper function as well
refs https://github.com/TryGhost/Team/issues/1433
- The `default` property stores whether a newsletter is set as default by the admin
- The `status` property stores whether a newsletter is archived or not
- The `recipient_filter` property is only storing whether a newsletter is "paid-only" or not for now, although it can be expanded to more specific filters in the future
- The `subscribe_on_signup` property stores whether a new member should be automatically signed up to the newsletter
- The `sort_order` property enables displaying the newsletter list in an order chosen by the admins
refs: https://github.com/TryGhost/Team/issues/1446
- These endpoints are unused, so they are safe to remove
- We're starting to remove as much unused & unnecessary code as possible to try to reduce the codebase and increase test coverage
refs: https://github.com/TryGhost/Toolbox/issues/245
refs: https://github.com/TryGhost/Ghost/commit/73f91a524
- we don't need this serializer because the default serializer will do the same thing
- commit 73f91a524 fixes the logic so that the default serializer is called as a fallback
even though the email_preview serialzier exists, as there's no matching method name
- sadly the route name here is wrong, it should be email_previews plural, but the response format is correct
to make this work we have to fix the docName and rename the serializer
refs: https://github.com/TryGhost/Toolbox/issues/245
- .all methods are fallback serializers not to be run as well as a custom serializer
- The default serializer is also a fallback
- The "All" file with before and after are global hooks that _always_ get run as well as other serializers
- There's a lot of room for further improvement here especially with naming but this logic makes more sense
for the usecases AND doesn't affect v2 & v3 etc. We can do another pass after 5.0
refs: https://github.com/TryGhost/Toolbox/issues/245
- There are several serializers which are "passthroughs" that return the response from the query
function as-is.
- The intention here is to make them all look consistent so they're easy to spot and understand what they do
refs: 0ef5a5c97a
- As per the previous commit, our mixed filename casing inadvertently resulted in a bug
- The casing in the codebase is meant to be kebab-case always, so fixing this everywhere that's relevant to the API whilst there's a good reason
refs: https://github.com/TryGhost/Toolbox/issues/245
refs: https://github.com/TryGhost/Team/issues/1360
- As a result of my changes in https://github.com/TryGhost/Ghost/commit/3bd4d098 the members connect endpoint had started returning JSON
- This is because the members connect endpoint relied on the old default behaviour of the serializer being to return no response, whereas now it does our default JSON response format
- I had written a tool to iterate over all endpoints and ensure that they all had explicit serializers before changing the default behaviour, but it missed this endpoint due to the snake case naming
- I have double checked and this was the only missed endpoint, the only other one was member_signin_urls.permissions but that was not a true endpoint and was removed in https://github.com/TryGhost/Ghost/commit/202696382
- Note: the snapshot file for this test was generated from running the test against https://github.com/TryGhost/Ghost/commit/e6b92aed9 - one commit before I added the new default behaviour.
- Without the new serializer this test fails on main
- With the new serialzier, this test passes again, showing the response format has gone back to what we expect
- This doesn't affect permissions, only permissions inside an endpoint config block does that
- Rather it creates an extra unused "endpoint" called member_signin_urls.permissions
- same as https://github.com/TryGhost/Ghost/commit/b53296c4d
refs https://github.com/TryGhost/Toolbox/issues/254
- it's useful for us to know how long it takes the theme service to
init, especially as it is dealing with a lot of user-provided files
outside of the Ghost codebase
refs https://ghost.slack.com/archives/C02G9E68C/p1647599592576139
refs https://ghost.slack.com/archives/C02G9E68C/p1647620250625909
Issue: `Cannot destructure property 'fromRegex' of 'this.redirectsredirectId]' as it is undefined.` is being thrown, only when running all tests.
Cause: duplicate redirects are added to a redirectManager, and not cleared correctly in the redirectManager, which throws an error when removing one of the duplicate redirects.
Because the same redirectManager is used, multiple event listeners are connected to the same redirectManager. So when the offer service has been initialised multiple times, multiple listeners are added, which create a redirect for every newly created offer... to the same redirectManager.
So there are three possible fixes for the same problem (would be best to fix them all):
- Create a new redirectManager every time the offer service is initialised
- Figure out a way to remove DomainEvents subscribers between tests
- Don't add the same redirect id multiple times to redirectIds in addRedirect from the express-dynamic-redirects package
This commit contains a fix for the first solution.
It also moved the offers service initialising before the frontend (to `initServicesForFrontend`)
With tiers going GA, this change brings a massive visual overhaul to Portal for almost all pages and flows, along with adding consistency between different multiple tier flows. It also overhauls the tests to match our new UI/UX for Portal.
refs: https://github.com/TryGhost/Toolbox/issues/245
- we don't need this serializer because the default serializer will call the authors mapper
- added an author mapper which is just an alias for users. I did this in a named file, not in index.js
- because we want to change the api framework to load files automatically without needing index files.
- makes sense to hold off that deeper change until we only have one api version else we have to change old APIs
refs: https://github.com/TryGhost/Toolbox/issues/245
- we don't need this serializer because the default serializer will call the tags mapper
- for now I've changed, rather than removed the tag serializer test as this shows default works the same!
refs: https://github.com/TryGhost/Toolbox/issues/245
- There's no need for a mapper or serializer as labels uses the default behaviour
- Added a full suite of tests, consolidating from regression and using the new framework to prove nothing is broken
- settings cache was appearing as untested because we use rewire!
- rewire was totally unnecessary in this case, so I removed it
- updated to 100% test coverage whilst there, including removing one unreachable branch
- commented some undesirable behaviour I found whilst trying to reach all branches
- I don't want to change the behaviour to return false correctly without having a reason beyond improving coverage
no issue
Prevents errors from being uploaded to Sentry when a 404 happens in Ghost Admin. At the moment, 404s in Ghost Admin create an ENOENT error in express' static library. Our generic 404 handler at the end will only intercept requests that don't have any errors in the context, so a simple middleware can strip out 404 errors just before we add in our own.
The Ghost-specific error that we attach to requests does not get uploaded to Sentry :)
refs: https://github.com/TryGhost/Toolbox/issues/245
- Added a serializer called default to the canary API
- Ideally, this would be part of the shared framework, but this would change v2/v3 and we're about to get rid of them
- Therefore, we change just canary for now, and we can refactor again later.
- Added wiring to handler that uses the default serializer, if there is a default, and isn't an explicit serializer for the endpoint
- Removed the invites serializer, so that one endpoint now uses the default
Note: previous commits have added explicit serializers to every endpoint, this is the first step towards paring
that back so that we have less serializers overall, not more!
refs: https://github.com/TryGhost/Toolbox/issues/245
- Offers didn't have a serializer because it's deliberately set at the API level
- With the upcoming refactor we want to have all the serializers defined explicitly
- This will allow us to change the default behaviour
- Tests were added to cover this endpoint in ...
refs: https://github.com/TryGhost/Toolbox/issues/245
- Upload, updateMembersEmail, validateMembersEmailUpdate & disconnectStripeConnectIntegration were all missing serializers
- Upload is an as-is response, same as download
- updateMembersEmail, validateMembersEmailUpdate & disconnectStripeConnectIntegration are all passthroughs with no response
- With the upcoming refactor we want to have all the serializers defined explicitly
- This will allow us to change the default behaviour
- Updated the file based tests to prove the body doesn't change
- Tests were added to cover updateMembersEmail, validateMembersEmailUpdate & disconnectStripeConnectIntegration in 68c1bc0285
refs: https://github.com/TryGhost/Toolbox/issues/245
- Session didn't have a serializer because it's deliberately a passthrough
- With the upcoming refactor we want to have all the serializers defined explicitly
- This will allow us to change the default behaviour
- Tests were added to cover this endpoint in 2cf7e00493
refs: https://github.com/TryGhost/Toolbox/issues/245
- sendTestEmail was missing a serializer because it's deliberately a passthrough
- With the upcoming refactor we want to have all the serializers defined explicitly
- This will allow us to change the default behaviour
- Updated the tests to prove the body doesn't change
refs: https://github.com/TryGhost/Toolbox/issues/245
- Slack didn't have a serializer because it's deliberately a passthrough
- With the upcoming refactor we want to have all the serializers defined explicitly
- This will allow us to change the default behaviour
- Tests already cover this endpoint
refs: https://github.com/TryGhost/Toolbox/issues/245
- The destroy endpoint was missing a serializer
- Instead of adding one, I've refactored to use an all method that's a passthrough
- Updated the tests to use the same pattern as others to make it clearer this is tested
refs: https://github.com/TryGhost/Toolbox/issues/245
- The destroy endpoint was missing a serializer
- As this serializer uses the same createSerializer pattern as members, I've copied the passthrough from members into here
- This ensures the behaviour will stay the same when the default behaviour changes
- Updated the tests to prove the body doesn't change
refs: https://github.com/TryGhost/Toolbox/issues/245
- This was the only endpoint in the members API that didn't have a serializer
- It just needs to be a simple passthrough, but we define it so we can change the default
refs: https://github.com/TryGhost/Toolbox/issues/245
- The destroy endpoint was missing a serializer
- Instead of adding one, we've refactored to use the standard structure for this serializer
- Updated the tests to prove the body doesn't change
refs: https://github.com/TryGhost/Toolbox/issues/245
- There was only a serializer in place for redirects.download.
- Upload was falling through, which means nothing happens by default atm
- We want to change this default, so I'm making sure all our routes have serializers declared and tests
- Updated the tests and checked the behaviour was the same before and after:
- We can't use our new framework here yet because it doesn't support uploads or downloads
- Instead, just add simple matching for the body of the responses
no issue
- Just a small refactor as it was becoming painful to change the version in three places while experimenting with gscan
- Also follows the "rule of 3"
refs https://github.com/TryGhost/Toolbox/issues/169
refs 7becf0a2b2
- The referenced commit has dropped existance of Content and Admin APIs under `/canary/` prefix, which made a breaking change and clients that are still relying on "canary" started to break.
- The `/canary/` prefix should be used up untill the introduction of Ghost v5, otherwise we run the risk of accidentally breaking API clients.
