* Added subscription_id to MRR events
refs https://github.com/TryGhost/Team/issues/1453
As part of 5.0 we want to have cancelled evetns affect MRR. We are going
to backfill and begin populating cancelled events _before_ 5.0 however,
adding a migration to set the MRR deltas as part of 5.0
This migration will need to associate the cancelled events with a
subscription, so we are adding the subscription_id now. This will allow
us to guarantee that all cancelled events will have a subscription_id.
The column is missing a NOT NULL and REFERENCES constraint because
we have not populated the missing values. These will be added in the future
once we have populated the column for all rows
refs https://github.com/TryGhost/Team/issues/1248
This is the underlying cause of the problems we've seen whilst handling
Stripe webhooks. A transaction ensures that the operations are atomic,
but not that they can run concurrently.
If you have some code which does this, concurrently:
1. Starts a transaction
2. Reads a value
3. Changes the values
4. Ends the transaction
Without applying the `FOR UPDATE` lock - you will have both sequenes
read the same value at step 2.
With the `FOR UPDATE` lock - one of the sequences will hang at step 2,
waiting for the other transaction to end, at which point it will resume
and read the _changed_ value.
Because the `edit` method explicitly does a read followed by a write, we
have also add the `FOR UPDATE` lock to this by default, to avoid any race
conditions. This does however require that `edit` is called within a
transaction. An issue here https://github.com/TryGhost/Team/issues/1503
considers running in a transaction by default.
refs https://github.com/TryGhost/Toolbox/issues/280
- In response to 'Accept-Version' header in the request headers, Ghost will always respond with a content-version header indicating the version of the Ghost install that is responding. This should signal to the client the content version that is bein g served
- This is a bare bones implementation and more logic with edge cases where `content-version` is served with a version value of "best format API could respond with" will be added later.
refs https://github.com/TryGhost/Team/issues/1470
Instead of counting the MRR by resolving all the deltas from the past until now, we should start with the current calculated MRR and resolve it until the first event. That would give a more accurate recent MRR (in exchange for a less accurate MRR for older data) and allows us to limit the amount of returned days in the future.
- Includes MRR stats service that can fetch the current MRR per currency
- The service can return a history of the MRR for every day and currency
- New admin API endpoint /stats/mrr that returns the MRR history
- Includes tests for these new service and endpoint
refs https://github.com/TryGhost/Team/issues/1474
- The `default` concept will be replaced by the first newsletter based on the `sort_order`
- This removes the `default` value from the newsletter API
- This simplifies the design to make the api and datastructure more maintainable
refs https://github.com/TryGhost/Team/issues/1471
- This is a many-to-one relation so that many posts can be linked to a specific newsletter
- The `newsletters` table had to come first in the schema file so that it's initialized before the `posts` table (because of the foreign key)
- Updated the model to make sure the new field doesn't leak in the API for now
- This migration isn't using the `createAddColumnMigration` util because of a performance issue. In MySQL, adding/dropping a column without `algorithm=copy` uses the INPLACE algorithm which was too slow on big posts tables (~3 minutes for 10k posts). Switching to the COPY algorithm fixed the issue (~3 seconds for 10k posts).
- SQLite isn't using the codepath where we run a raw SQL query because `knex` is doing multiple queries to add/remove a column
refs https://github.com/TryGhost/Team/issues/1469
Currently, all new members get auto subscribed to the default newsletter. This change adds same behavior with multiple newsletters by auto subscribing all available newsletters on site for new members(If flag is enabled).
Note: In future, this will also take into consideration the `subscribe_on_signup` flag for a newsletter to filter which newsletters should a member be auto-subscribed.
- adds newsletters service for working with newsletter data
- bumps `@tryghost/members-api` package which handles default subscription
- adds new test fixture/data for newsletters
no issue
- It was hard to plug in with additional code into current `.then` based chain. Refactoring to use a more modern syntax helps with readability and allows for easier edits
- This fits more closely, as this service is to so with rendering helpers and small parts
- Whereas we want to use "rendering" for things concerned with rendering pages
refs https://github.com/TryGhost/Team/issues/1469
- wires newsletter preferences for a member in Portal to view/edit real data
- allows members to control their subscription to multiple newsletters
refs https://github.com/TryGhost/Team/issues/1469
We have an after hook to update `sort_order` for pivot tables after update as base bookshelf plugin. Since new `members_newsletters` table doesn't has a `sort_order` column on it, this change disables the after hook for it to avoid errors.
refs https://github.com/TryGhost/Team/issues/1469
- allows Portal access to multiple newsletters on a site
- allows Portal to update member's newsletter subscription preference
refs https://github.com/TryGhost/Team/issues/1469
- updates member model to add relation to newsletter via pivot table
- updates member api serializer to include newsletter data
- updates tests
refs: https://github.com/TryGhost/Ghost/commit/11867ab43
- These checks live in the wrong place. They are mostly a frontend thing
- The only server place they were used was slack and that was fixed in 11867ab43
- Moving these to the frontend they fit neatly into the frontend data service
- This is the only piece of server code that relies on the schema.checks, the rest are all frontend
- IMO this code should actually check for the post properties that the slack message needs
- OR it should switch based on the event type
- either way there's no need to have a shared util for this simple use case
- especially becaue it's confusing the use case for it and creating cross-coupling between server and frontend
- Some of the helpers inside the routing service would be better suited to their own service
- These two helpers fetchData and entryLookup talk to the API to get data & so make a decent start for a data service
- The data service would be the single point of contact with the API for the frontend
- Doing this now cos I'm moving some files around ahead of deleting things for 5.0
- We had an incident where a migration file was misnamed.
- This is caused by using slimer with spaces rather than hyphens
- We didn't imagine this case when writing the regex for matching filenames
- However, now we know it, it's easy to tweak the regex to match this case
- This requires adding an override to the one badly named file
- Keeping overrides inside the file means the context of why a rule isn't firing is present inline when modifying a file
- This makes it easier to maintain files, at the cost of needing to search to find all overrides
refs 2fa3985d42
- Running tests with error logging set to "error" lever, produced a massive amounts of errors related to failed Stripe keys. Making it hard to look through the output.
- When Ghost is running in teste environment by default it is configured with an invalid Stripe key that looks like `sk_test***`. In this case the Members migrations runs creating requiest to Stripe, which fail.
refs https://github.com/TryGhost/Team/issues/1469
With multiple newsletters, members will now be able to subscribe to one or more newsletters on the site. Previously, the subscription to default newsletter for a member was controlled via a single boolean `subscribed` column on the member table.
This change allows mapping multiple newsletters to a member via new pivot table that stores relation between a member and newsletter.
- adds new `members_newsletters` pivot table
- update tests
- if we add a column with a foreign key reference, the `down` migration
will try to remove that column
- you can't remove a column without deleting the foreign key reference
first
- our migration utils didn't take that into account and there's nothing
in Knex to do this for us
- this commit deletes the foreign key before removing the column if we
have one referenced in the column spec
- also updates the code to pass the column spec into the util
refs 7172db74b0 (r70370252)
- The changes referenced above changed fixtures without taking "test fixtures" in "test/utils/fixtures/fixtures.json" into account. This fix is only cosmetical to unlutter the test output with logging level set to "error". Once the referenced comment is answered a proper fix should be applied here
- this commit removes the colons from the log messages to make each
line flow easier when you're reading it
- also removes a couple of superfluous words that aren't needed
refs https://github.com/TryGhost/Team/issues/449
- Once we deprecate site.lang use in favor of site.locale this helper will need inspection for potentially breaking change. Left a note to come back to it once it's time
refs https://github.com/TryGhost/Team/issues/1463
- Allow admins to perform all newsletter operations
- We can adjust and be more permissive in the future if needed
- Added the tests back as permissions are configured correctly now
refs TryGhost/Team#1458
refs TryGhost/Team#1459
refs TryGhost/Team#1372
- Added a new stats service, which is divided into several categories. Currently only the 'members' category for member related stats.
- When there are missing or corrupt members status events in the DB, the totals returned by the old member stats endpoint (`/members/stats/count`) were wrong. This is fixed in the new service by counting in reverse order and starting with the actual totals.
- New Stats API, with the new `/stats/members/count-history` endpoint.
- This new endpoint also returns the paid deltas -> dashboard 5.0 will show subscribed and canceled paid members for each day
- Includes tests for the new stats service and endpoint
refs https://github.com/TryGhost/Team/issues/1463
- This enables listing, creating and editing newsletters
- The tests are commented out as the permissions will be added in a follow-up commit
refs https://github.com/TryGhost/Toolbox/issues/213
- our `knex` dependency has been out of date for a while so my aim was
to bring it up to date
- this required also switching `sqlite3` to `@vscode/sqlite3` because
knex switched the default sqlite driver
- this commit also bumps knex-migrator and switches to a mock-knex fork
until Knex 1.0 support has been merged
- also updates an error message to handle a new code in SQLite
refs: https://github.com/TryGhost/Toolbox/issues/245
- The default behaviour of a serializer is to call a mapper for each object
- Instead of all the boilerplate code we had in the snippets serializer, all we need is a single mapper function
- Added tests for the mapper function as well
refs https://github.com/TryGhost/Team/issues/1433
- The `default` property stores whether a newsletter is set as default by the admin
- The `status` property stores whether a newsletter is archived or not
- The `recipient_filter` property is only storing whether a newsletter is "paid-only" or not for now, although it can be expanded to more specific filters in the future
- The `subscribe_on_signup` property stores whether a new member should be automatically signed up to the newsletter
- The `sort_order` property enables displaying the newsletter list in an order chosen by the admins
refs: https://github.com/TryGhost/Team/issues/1446
- These endpoints are unused, so they are safe to remove
- We're starting to remove as much unused & unnecessary code as possible to try to reduce the codebase and increase test coverage
refs: https://github.com/TryGhost/Toolbox/issues/245
refs: https://github.com/TryGhost/Ghost/commit/73f91a524
- we don't need this serializer because the default serializer will do the same thing
- commit 73f91a524 fixes the logic so that the default serializer is called as a fallback
even though the email_preview serialzier exists, as there's no matching method name
- sadly the route name here is wrong, it should be email_previews plural, but the response format is correct
to make this work we have to fix the docName and rename the serializer
refs: https://github.com/TryGhost/Toolbox/issues/245
- .all methods are fallback serializers not to be run as well as a custom serializer
- The default serializer is also a fallback
- The "All" file with before and after are global hooks that _always_ get run as well as other serializers
- There's a lot of room for further improvement here especially with naming but this logic makes more sense
for the usecases AND doesn't affect v2 & v3 etc. We can do another pass after 5.0
refs: https://github.com/TryGhost/Toolbox/issues/245
- There are several serializers which are "passthroughs" that return the response from the query
function as-is.
- The intention here is to make them all look consistent so they're easy to spot and understand what they do
refs: 0ef5a5c97a
- As per the previous commit, our mixed filename casing inadvertently resulted in a bug
- The casing in the codebase is meant to be kebab-case always, so fixing this everywhere that's relevant to the API whilst there's a good reason
refs: https://github.com/TryGhost/Toolbox/issues/245
refs: https://github.com/TryGhost/Team/issues/1360
- As a result of my changes in https://github.com/TryGhost/Ghost/commit/3bd4d098 the members connect endpoint had started returning JSON
- This is because the members connect endpoint relied on the old default behaviour of the serializer being to return no response, whereas now it does our default JSON response format
- I had written a tool to iterate over all endpoints and ensure that they all had explicit serializers before changing the default behaviour, but it missed this endpoint due to the snake case naming
- I have double checked and this was the only missed endpoint, the only other one was member_signin_urls.permissions but that was not a true endpoint and was removed in https://github.com/TryGhost/Ghost/commit/202696382
- Note: the snapshot file for this test was generated from running the test against https://github.com/TryGhost/Ghost/commit/e6b92aed9 - one commit before I added the new default behaviour.
- Without the new serializer this test fails on main
- With the new serialzier, this test passes again, showing the response format has gone back to what we expect
- This doesn't affect permissions, only permissions inside an endpoint config block does that
- Rather it creates an extra unused "endpoint" called member_signin_urls.permissions
- same as https://github.com/TryGhost/Ghost/commit/b53296c4d
refs https://github.com/TryGhost/Toolbox/issues/254
- it's useful for us to know how long it takes the theme service to
init, especially as it is dealing with a lot of user-provided files
outside of the Ghost codebase
refs https://ghost.slack.com/archives/C02G9E68C/p1647599592576139
refs https://ghost.slack.com/archives/C02G9E68C/p1647620250625909
Issue: `Cannot destructure property 'fromRegex' of 'this.redirectsredirectId]' as it is undefined.` is being thrown, only when running all tests.
Cause: duplicate redirects are added to a redirectManager, and not cleared correctly in the redirectManager, which throws an error when removing one of the duplicate redirects.
Because the same redirectManager is used, multiple event listeners are connected to the same redirectManager. So when the offer service has been initialised multiple times, multiple listeners are added, which create a redirect for every newly created offer... to the same redirectManager.
So there are three possible fixes for the same problem (would be best to fix them all):
- Create a new redirectManager every time the offer service is initialised
- Figure out a way to remove DomainEvents subscribers between tests
- Don't add the same redirect id multiple times to redirectIds in addRedirect from the express-dynamic-redirects package
This commit contains a fix for the first solution.
It also moved the offers service initialising before the frontend (to `initServicesForFrontend`)
With tiers going GA, this change brings a massive visual overhaul to Portal for almost all pages and flows, along with adding consistency between different multiple tier flows. It also overhauls the tests to match our new UI/UX for Portal.
refs: https://github.com/TryGhost/Toolbox/issues/245
- we don't need this serializer because the default serializer will call the authors mapper
- added an author mapper which is just an alias for users. I did this in a named file, not in index.js
- because we want to change the api framework to load files automatically without needing index files.
- makes sense to hold off that deeper change until we only have one api version else we have to change old APIs
refs: https://github.com/TryGhost/Toolbox/issues/245
- we don't need this serializer because the default serializer will call the tags mapper
- for now I've changed, rather than removed the tag serializer test as this shows default works the same!
refs: https://github.com/TryGhost/Toolbox/issues/245
- There's no need for a mapper or serializer as labels uses the default behaviour
- Added a full suite of tests, consolidating from regression and using the new framework to prove nothing is broken
- settings cache was appearing as untested because we use rewire!
- rewire was totally unnecessary in this case, so I removed it
- updated to 100% test coverage whilst there, including removing one unreachable branch
- commented some undesirable behaviour I found whilst trying to reach all branches
- I don't want to change the behaviour to return false correctly without having a reason beyond improving coverage
no issue
Prevents errors from being uploaded to Sentry when a 404 happens in Ghost Admin. At the moment, 404s in Ghost Admin create an ENOENT error in express' static library. Our generic 404 handler at the end will only intercept requests that don't have any errors in the context, so a simple middleware can strip out 404 errors just before we add in our own.
The Ghost-specific error that we attach to requests does not get uploaded to Sentry :)
refs: https://github.com/TryGhost/Toolbox/issues/245
- Added a serializer called default to the canary API
- Ideally, this would be part of the shared framework, but this would change v2/v3 and we're about to get rid of them
- Therefore, we change just canary for now, and we can refactor again later.
- Added wiring to handler that uses the default serializer, if there is a default, and isn't an explicit serializer for the endpoint
- Removed the invites serializer, so that one endpoint now uses the default
Note: previous commits have added explicit serializers to every endpoint, this is the first step towards paring
that back so that we have less serializers overall, not more!
refs: https://github.com/TryGhost/Toolbox/issues/245
- Offers didn't have a serializer because it's deliberately set at the API level
- With the upcoming refactor we want to have all the serializers defined explicitly
- This will allow us to change the default behaviour
- Tests were added to cover this endpoint in ...
refs: https://github.com/TryGhost/Toolbox/issues/245
- Upload, updateMembersEmail, validateMembersEmailUpdate & disconnectStripeConnectIntegration were all missing serializers
- Upload is an as-is response, same as download
- updateMembersEmail, validateMembersEmailUpdate & disconnectStripeConnectIntegration are all passthroughs with no response
- With the upcoming refactor we want to have all the serializers defined explicitly
- This will allow us to change the default behaviour
- Updated the file based tests to prove the body doesn't change
- Tests were added to cover updateMembersEmail, validateMembersEmailUpdate & disconnectStripeConnectIntegration in 68c1bc0285
refs: https://github.com/TryGhost/Toolbox/issues/245
- Session didn't have a serializer because it's deliberately a passthrough
- With the upcoming refactor we want to have all the serializers defined explicitly
- This will allow us to change the default behaviour
- Tests were added to cover this endpoint in 2cf7e00493
refs: https://github.com/TryGhost/Toolbox/issues/245
- sendTestEmail was missing a serializer because it's deliberately a passthrough
- With the upcoming refactor we want to have all the serializers defined explicitly
- This will allow us to change the default behaviour
- Updated the tests to prove the body doesn't change
refs: https://github.com/TryGhost/Toolbox/issues/245
- Slack didn't have a serializer because it's deliberately a passthrough
- With the upcoming refactor we want to have all the serializers defined explicitly
- This will allow us to change the default behaviour
- Tests already cover this endpoint
refs: https://github.com/TryGhost/Toolbox/issues/245
- The destroy endpoint was missing a serializer
- Instead of adding one, I've refactored to use an all method that's a passthrough
- Updated the tests to use the same pattern as others to make it clearer this is tested
refs: https://github.com/TryGhost/Toolbox/issues/245
- The destroy endpoint was missing a serializer
- As this serializer uses the same createSerializer pattern as members, I've copied the passthrough from members into here
- This ensures the behaviour will stay the same when the default behaviour changes
- Updated the tests to prove the body doesn't change
refs: https://github.com/TryGhost/Toolbox/issues/245
- This was the only endpoint in the members API that didn't have a serializer
- It just needs to be a simple passthrough, but we define it so we can change the default
refs: https://github.com/TryGhost/Toolbox/issues/245
- The destroy endpoint was missing a serializer
- Instead of adding one, we've refactored to use the standard structure for this serializer
- Updated the tests to prove the body doesn't change
refs: https://github.com/TryGhost/Toolbox/issues/245
- There was only a serializer in place for redirects.download.
- Upload was falling through, which means nothing happens by default atm
- We want to change this default, so I'm making sure all our routes have serializers declared and tests
- Updated the tests and checked the behaviour was the same before and after:
- We can't use our new framework here yet because it doesn't support uploads or downloads
- Instead, just add simple matching for the body of the responses
no issue
- Just a small refactor as it was becoming painful to change the version in three places while experimenting with gscan
- Also follows the "rule of 3"
refs https://github.com/TryGhost/Toolbox/issues/169
refs 7becf0a2b2
- The referenced commit has dropped existance of Content and Admin APIs under `/canary/` prefix, which made a breaking change and clients that are still relying on "canary" started to break.
- The `/canary/` prefix should be used up untill the introduction of Ghost v5, otherwise we run the risk of accidentally breaking API clients.
- we use this property in the code but it was missing from the messages
object
- this was causing an error in the `tpl` library so this commit adds the
correct message
no issue
- adds a separate activity feed screen with all member events
- can be filtered to show specific event types
- can be filtered by a specific member to show their full event feed
- updated member details screen to show all events instead of just email events
refs https://github.com/TryGhost/Team/issues/1141
- switched to the same member fetch method as used in `GET /member/:id/` so there's consistent data available when rendering the API responses
refs https://github.com/TryGhost/Team/issues/1302
- Added the `MemberCancelEvent` model to MembersApi
- Triggered when a subscription is canceled by an admin or by the member
- Updated shared dependencies
Co-authored-by: Simon Backx <simon@ghost.org>
- CSS was updated previously, but the minified version wasn't
- as a result the private site landing page was visually broken
- minified CSS is built as part of `grunt prod`, called during `grunt release`
refs c4470ff732
- labs flag was removed under the false assumption it was a client-side only flag but the `last_seen_at` property in API responses was also gated meaning the member details screen showed "Not seen yet" and the members list did not show the last seen date of all members when filtering
refs https://github.com/TryGhost/Team/issues/1367
Because we are passing through a different member object as the context
in the get helper, the content gating was not working correctly, as the
member was missing a status property, this adds the property which fixes
content gating.
- Added extra tests for get helper {{access}} property
- Added extra test for {{access}} property in next_post helper
- In the future we might want to update the tests so they test the whole request -> HBS context flow. Currently the has context is still stubbed manually.
closes https://github.com/TryGhost/Team/issues/1426
When fetching tiers using the content API, we incorrectly returned all tiers including archived ones unless the active:true filter is passed. Correct behaviour is to always hide archived tiers, so this filter should not be required.
- forces `active:true` filter for tiers content api browse
- updates test to check for archived test removal in tiers content api
refs: https://github.com/TryGhost/Toolbox/issues/245
- Ghost's API framework has a mixed up concept of what a serializer is. Mappers are true serializers! What we call serializers are little more than a small formatting step.
- This PR splits mappers into individual files and uses the endpoint's docname as the mapper name. This will help us to automate the calling of a mapper for an endpoint later.
- This is one tiny step in reworking the framework to need less code to make it work, and to have clearer concepts for how to do things.
As multiple tiers is now GA, we want to allow devs to be able to work with Tiers and offers via content/Admin API. This change -
- updates fixtures to add permissions to admin integration role for new sites
- adds migration to update existing sites to have correct permissions for role
- whitelists add/edit/read/browse on tiers and offers API for integrations
refs https://github.com/TryGhost/Toolbox/issues/240
- As gscan's canary is about to be aliased to upcoming v5 version current version of Ghost should be using an explicit v4 checks.
- These will change to v5 once Ghost is doing final preparations for v5 release (on the branch most likely)
- This doesn't affect permissions, only permissions inside an endpoint config block does that
- Rather it creates an extra unused "endpoint" called identities.permissions
refs https://github.com/TryGhost/Toolbox/issues/169
- Before releasing Ghost v5 we would like to move all canary-related URLs to a non-versioned format, which will become a default in v5.
- 'canary' is by definition unstable, so breaking any unprepared client explicitly using the canary is expected
- Removed the aliased /content/ and /admin/ apps from app.js because with updated configuration they become duplicates of 'canary' endpoints
refs https://github.com/TryGhost/Team/issues/1267
Multiple tiers is coming out of beta -
- allows site owners to create multiple tiers
- allows setting individual welcome page for each tier
- allows setting visibility for individual tiers for portal and themes
refs: https://github.com/TryGhost/Team/issues/1369
Change already made for JSON responses in @tryghost/mw-error-handler, but this change also fixes the order of operations for displaying theme errors.
refs https://github.com/TryGhost/Team/issues/1363
- When uploading a zip of images in Settings > Labs > [Import], it will skip images that have an uppercase extension, citing an 'unsupported file type' error.
- Cause: Glob ignored those files when matching extensions in ImportManager
- Fix: Added nocase option where needed
- Extended tests to also test the processZip method of ImportManager with getFilesFromZip
- Added isValidZip for zip with uppercase image
- Cleaned up JSDoc in ImportManager, and replaced some older JS syntax
Fixed zipContainsMultipleDataFormats error never thrown:
When a zip combines two data formats, no error was thrown.
- The promise error was only returned in an _.each loop, but never thrown
- Previously when combining multiple data types in a zip file, no error got thrown
- Added a test for this error
- Also added a test for noContentToImport error
Other errors and fixes:
- Added missing length in getBaseDirectory check
- getContentTypes fixed (returned duplicate values). Type error came up after adding all JSDocs
- updated tests to match real types from JSDoc and pass type validations
- Rewrote some methods in the async await syntax
- Added tests for ImportManager clean up
- Our old fixtures were designed as a guide to getting started to Ghost, but they got in the way
- The old fixtures now live as part of ghost.org/resources - a living guide to starting with Ghost
- These new fixtures mean the site is ready to go as soon as it's setup
Co-authored-by: Hannah Wolfe <github.erisds@gmail.com>
- we send the roles data array in when we're changing the role of the
user
- if we send an empty array, we don't want to edit the user's role
- the code _thought_ that's what it was doing, but we only check the
falsiness of the array, which is truthy for `[]`
- it also needs to check the length of the array
- this commit includes a test which would fail with a 500 error without
the fix
refs https://github.com/TryGhost/Team/issues/1284
When you create a new post with a tag slug that contains spaces, those spaces will get replaced by dashes. But instead of reusing an existing tag, a new tag is always created.
- New tag slugs are cleaned up before matching with existing tags in the Post model onSaving method
- Cleaned up multiple loops in onSaving of Post model
- Cleaned up syntax when cleaning up tag slug
- Added tests for slugs with spaces
- Added test for too long tag slug causing duplication
no issue
- The support for the misformated roon imports was temporary anyway, based on the comments in code. It's also unecessary to keep around any code related to Roon as it's been _ages_ since anybody needed this kind of migration
refs https://github.com/nodeca/probe-image-size/blob/master/CHANGELOG.md#changed-1
- version 6 of `probe-image-size` switched from using `request` to
`needle`
- this means we need to update our options to reflect the changes
- we still use request in this file so I've duplicated the options for now
- also adds a few extra error codes to the catches because needle
reports different codes to request
refs https://github.com/nodeca/probe-image-size/blob/master/CHANGELOG.md#600---2020-11-04
- `probe-image-size` v6 now supports `.ico` files so we can
allow probing of dimensions via this library rather than falling back
to downloading the entire image via `image-size`
- also updates a test because .ico files no longer use the internal
request lib, which simplifies things a little bit
- Have ensured we have 100% coverage of core/server/api/shared/serializers/handle.js
- This meant I had to swap around two validation clauses as one was unreachable
- I have done this as I want to make some changes in this area of the codebase, and want to ensure we have tests
and a clear understanding of what this code does before I change it
refs https://github.com/TryGhost/Team/issues/1408
- switched from `@nexes/nql` to `@tryghost/nql` and bumped `@tryghost/bookshelf-plugins` to get access to the latest NQL version across the app
- adds "contains" operator support
- `:~'string'` - contains
- `:-~'string'` - does not contain
- `:~^'string'` - starts with
- `:-~^'string'` - does not start with
- `:~$'string'` - ends with
- `:-~$'string'` - does not end with
- enables `'` escaping in strings, eg `'O\'Nolan'`
Tiers will soon go to GA, and these small features that were added as part of tiers beta are now ready to go live as well along with tiers GA, so we are removing their flags and bumping them as part of tiers beta.
refs https://github.com/TryGhost/Team/issues/1420
- This changeset makes the "/ghost/api" base path for the APIs centralized in one place and reused by dependent modules. There are couple benefits this refactor brings: easy way to spot where the API base path is used (was hard to find it in regexp) and makes it easy to change the hardcoded path to a configurable one in the future (e.g. host all APIs under `domain.tld/custom-path/awesome-apis/posts`)
- I hear that scream from the back of your head: "But hey! This introduced coupling to url-utils!". To that my unswer is: "No. This change only makes the coupling explicit, it's been there already and now can be addressed if we need to!".
- A neat thing about his change, making the API work on a custom path is one line away, by moving the hardcoded `/ghost/api` to a config ;)
refs https://github.com/TryGhost/Team/issues/1417
- we no longer want to display a notification if theme install fails
- the notification has been removed so we fail silently, but log the warning
refs https://github.com/TryGhost/Team/issues/1387
Bumps portal with changes to handle new tier visibility property instead of portal plans/products setting. Also adds new UI refinements for tiers GA.
refs https://github.com/TryGhost/Team/issues/1387
This will allow us to move from the portal_products and portal_plans
settings to using the visibility property on tiers to determine whether
or not a tier should be visible in Portal.
This also fixes a bug with the Tiers Admin API read method permissions.
- we catch error arising from creating webhooks and check for specific codes
- if our error does not match one of those codes, we don't propagate the
error up
- this becomes a problem if saving a webhook fails for some other reason
because upstream code assumes we return an error or model
- this commit re-throws the error and adds a test that would have caught
this
https://github.com/TryGhost/Team/issues/1387
This is split into two migrations, one for the portal_products setting
and one for the portal_plans setting, as dealing with both of them in a
single migration led to too many branches.
refs https://github.com/TryGhost/Team/issues/1240
We were selectively invalidating cache on tier/product edit which was consistent with pattern for other APIs, but in case of tier/product, the model changed method always returns false due to how its setup. This change updates the edit to always invalidate cache, similar to tier add, to ensure sites don't see old tier values.
refs https://github.com/TryGhost/Team/issues/1004
Replaces {{products}} helper usage with updated {{tiers}} helper. Default output for {{tiers}} helper is the same as {{products}} helper.
refs https://github.com/TryGhost/Team/issues/1004
- adds new `{{tiers}}` helper behind `multipleProducts` flag
- `{{tiers}}` outputs a string with list of tiers that have access to specific post when used in a post context in theme
- outputs empty string when used out of a post context and without access to `visibility` property
- uses tiers attached to post column for data
refs https://github.com/TryGhost/Team/issues/1004
The output serializer for product was not handling `include` option value as comma separated string, which is passed via `#get` helper to fetch related prices for tiers. This change fixes the handling to work with both array as well as comma-separated include values.
refs https://github.com/TryGhost/Team/issues/1004
The `tiers` column for a post/page only contained data if its visibility is set to `tiers`, otherwise its empty. This is because originally the purpose of `tiers` column on `post` was to capture specific tiers with access to post.
The best way to ensure a consistent behavior for `tiers` column data on post is to update it to always contain list of all `tiers` that have access to post, and not just when the visibility is `tiers`. This means the value is set to all tiers when visibility is one of public|members, and only paid tiers when visibility is `paid`. This change also allows on frontend to get all relevant `tiers` information for a post locally within post context instead of relying on additional information from outside.
This change -
- updates the output serializer for post/page to add all desired tiers manually in case of visibility is not `tiers`
- updates tests
- We have an existing pattern for using `visibility: public` instead of `visible: true|false`
- We no-op the existing migration and roll forward so that we don't have to manually revert db changes
refs https://github.com/TryGhost/Team/issues/1404
The archived tiers are hidden in Portal normally as they are automatically not included in the `portal_products` list. This change also removes the archived tiers from being sent in list of tiers sent as part of site data to Portal. This ensures Portal doesn't use any archived tier.
- Caching is causing issues with Portal preview
- Changes to tiers and Portal settings are taking too long to display on the site, causing confusion for users
- This reverts commit 5d8f491823.
refs https://github.com/TryGhost/Team/issues/1367
- The {{access}} property of a post always returned false for non-public content
- Added the member context to the get, next-post and prev-post helpers
- The get, next-post and prev-post helpers didn't add the member context to the internal API calls
- Added the members context to these calls
- Added tests that check if the member context is passed to the API calls
- Transformed next_post helper tests to async await syntax
- Transformed prev_post helper tests to async await syntax
refs https://github.com/TryGhost/Team/issues/1386
- The current match handler supports normal (in)equality operators, but no numeric comparisons (<, >, <=, >=)
- A use case for these new operators is to show the latest post in a separate way from other posts
Includes unit tests to check the new behaviour.
Run via `yarn test test/unit/frontend/helpers/match.test.js`
refs https://github.com/TryGhost/Team/issues/1313
When adding the tiers endpoint the Content API was missed, this is
needed so that themes can access Tiers via the `{{#get}}` helper.
refs https://github.com/TryGhost/Toolbox/issues/174
- this commit switches Ghost from using the `mysql` library to the
`mysql2` one
- we've done this for several reasons:
- `mysql2` is more actively maintained
- `mysql2` natively supports the default auth plugin on MySQL 8
- `mysql2` is fasterrrr
- there have been various other commits refactoring the groundwork for
this commit but this commit should be short and sweet:
- alias `mysql` to `mysql2` client so we maintain backwards
compatibility with all configs who use `"client": "mysql"`
- enabled `decimalNumbers` so we maintain the same functionality as
`mysql`
- replaced the dependencies and updated `knex-migrator`
- hardcoded the newer authentication plugin in MySQL 8 CI. Before
switching to `mysql2`, this would break because it didn't support
this
refs https://github.com/TryGhost/Toolbox/issues/169
- Before next major version release we need to prepare for removal of API versioning.
- This change allows unversioned API requests to work under following endpoints:
- /ghost/api/admin
- /ghost/api/content
- This change should allow further preparation of the API clients (SDKs, Integrations, etc.) to non-versioned APIs in Ghost instances in ^5.0.0
- Changed default e2e test targets to non-versioned API. It's a trial, to have working examples. In the future all tests should switch to use only non-versioned endpoints.
refs https://github.com/TryGhost/Toolbox/issues/169
- As Ghost prepares to drop API versioning in future major release it the authentication mechanism should take into account non-versioned token audience support. The audience for non-versioned api requests would be limited to "admin" rather than "canary/admin"
refs https://github.com/TryGhost/Team/issues/1362
- Casper is already installed, so the installation from github always fail.
- There is no need to display an error message in that case.
Added regression tests for blog setup with the default theme:
- Check whether there are no notifications after completing the setup
- Also test the setup with the default theme
refs https://github.com/TryGhost/Toolbox/issues/174
- this just uses the same SQL queries for certain queries when using the
`mysql2` library as when using the `mysql` one
- we can remove the `mysql` line when we fully switch to `mysql2`
refs https://github.com/TryGhost/Toolbox/issues/174
- right now, our migrations manually check the client of the knex
instance to see whether we're running on MySQL or SQLite
- that's been working fine, but the problem is that we're due to switch
to the mysql2 driver soon, so all these checks will be faulty
- i've altered the functionality of `@tryghost/database-info` to accept
a knex instance, and it'll return if the DB is MySQL or SQLite in some
helper functions
- this commit bumps the package and switches to that format
- originally I used a shared instance of the class within
`@tryghost/database-info` but there's a chance that the knex instance
inside migrations actually comes from knex-migrator, and not Ghost, so
that wouldn't work
refs https://github.com/TryGhost/Team/issues/1306
- This adds a `MemberPageViewEvent` event when a page is viewed by a member (post/page/tag/author/...)
- Integrates the `LastSeenAtUpdater` service that listens to the `MemberPageViewEvent` events to update `member.last_seen_at`
- Follows the latest testing recommendation (end to end test + testing for side-effects)
refs https://github.com/TryGhost/Team/issues/1306
- Updates once every day (in the publication timezone)
- Also updates when the value is `NULL`
- This is implemented as a SQL query as the code is on the job and doesn't have visibility to the members code
- throughout the migration utils we use the passed in DB connection or
fallback to the `db.knex` instance
- this works, but it means we have places we need to make sure to
implement `(transaction || db.knex)` everywhere
- as I'm working on refactoring the utils, this was also causing
problems because I'd be checking the `transaction` instance but that may
be null/undefined
- this commit pulls the fallback into the function parameters where it's
evaluated at runtime
- this also has the added benefit that we get jsdoc typing now because
the types are easy to figure out
- note: `transaction` should probably be renamed to `connection` because
it's not necessary a transaction... but baby steps 🤓
- this section of code handles the errors that arise when we add a
foreign key to a table
- locally, I get different errors than the one listed - `ER_FK_DUP_KEY`
and `ER_FK_DUP_NAME`
- I've been trying to find a good source for what each code is but it
looks highly likely to be differences in DB engines
- we should probably handle these errors anyway because we don't want
migrations to error out
refs https://github.com/TryGhost/Toolbox/issues/219
- These two APIs similarly to Images API should be accessible by the Admin-API SDK. Opens up a way to write custom scripts uploading files and media
refs https://github.com/TryGhost/Team/issues/1387
We are moving away from the portal_products setting to instead store
each tiers visiblity on the tier itself. This column will be used for
that data.
Both of the default Tiers should be visible, but newly created tiers
should not be.
refs https://github.com/TryGhost/Team/issues/1029
- members browse endpoint didn't have `include` in its option list, so `?include...` was ignored in the api
- endpoint always reverted to using default relations in output, so `product` was never attached even if added in `include`
refs https://github.com/TryGhost/Team/issues/1382
- PR feedback
- Self hosters don't have the option to add a site description when setting up
- Update the logic to continue with the settings setup, even when no description is given
refs https://github.com/TryGhost/Team/issues/1382
- Added two possible new setup values: `accentColor` and `description` to define the brand colour and site description on initial setup
- Updated tests to reflect those changes
- Only the params when passed and fall back to default site description
refs https://github.com/TryGhost/Team/issues/1313
Rather than removing the /products API we're adding a /tiers API as
a first step towards renaming "products" to "tiers". The initial idea was
to alias the URL's but out API framework doesn't easily allow for this so
we've duplicated it instead.
- I recently added a bunch of strict rules to our eslint plugin around returns: ca9af37866
- These mostly are issues that occur whilst writing code, that you spot and fix as you're developing, but they're annoying to notice/find and eslint can be used to flag them quickly
- There are of course, edge cases where you don't need to return from array fns, but this rule also suggests better patterns might be available
- For our excert helper and new assertEvent helper, I've updated the code to use simpler patterns that are easier to read, so as to avoid the warnings
- For our old API I've simply disabled the rule as we're about to delete this code
refs https://github.com/TryGhost/Toolbox/issues/202
- during DB init, we have to create all the tables
- right now we loop over all tables and call the `createTable` command
- this command checks if the table exists and if not, creates the table
- this works fine but it means we query the database for every table
- in MySQL, we query the information_schema table, which we've seen
issues with before because it doesn't have indexes
- the smarter thing to do here is to get all the tables that already exist,
remove them from the list, and just straight up create them without
further checks
- this entire thing should be protected by the migration lock so we
shouldn't encounter issues from multiple processes initializing the DB
and tables existing after the initial check
- this commit also removes the check from `createTable` because this isn't
really needed. We should be using the migration utils, which do
check for existing tables. I've added a note to the function and
audited anywhere we still call the function
- this commit removes (- 49 tables + 1 initial check) 48 queries from
the initial DB init
refs https://github.com/TryGhost/Toolbox/issues/214
- Having this config for test environment allows to pre-populate default settings values in the settings table
- Right now the default-settings.json is an exact copy of the original "/data/schema/default-settings/default-settings.json". Having a starter file as an exact copy, allows to track the differences between environments as they are introduced easier
refs https://github.com/TryGhost/Toolbox/issues/214
- The values configuration for the settings table need to become configurable to be able to run our test environment with a pre-defined set of configurations (e.g Stripe-related values).
- This change makes it possible to define the default settings file location (currently a JSON)
- A new key is now exposed through the "paths.defaultSettings" key in settings, which can be overloaded for the needs of the environment
refs https://github.com/TryGhost/Toolbox/issues/214
- The `defaultSettings` path name in the config (one pointing to routes yaml file) creates confusion with the `defaultSettings` which populate defaults for in the database settings table.
- Furthermore, the name collision creates a problem when trying to make database default settings dynamic - being able to load them from configurable file path.
- Rename makes "routing" explicit to avoid ambiguity and free up the name for the database defaults
- The value seems to be safe to be renamed as all keys used in `overrides.json` are taking priority - the name "defaultRouteSettings" hasn't surfaced at any point in the git history
- Fixed test fixtures so that members with subscriptions also have products/tiers
- Fixed test fixtures so that default&free tiers can be updated for tests
- Added tests for the signin functionality and welcome page redirects
- Extended `setupStripe` to setup other Members settings - this needs some more
thought around how we proceed
refs https://github.com/TryGhost/Toolbox/issues/163
- as of Node 15, unhandled rejections will exit the process so if
Ghost is running on Node 15+ and encounters one, it will kill Ghost
- if Sentry is enabled, it will add a handler for the event that will
send it to Sentry but the logging is sent to stdout/stderr, which means
we lose it in Ghost logs
- this commit adds a process handler for the `unhandledRejection` event
which will log the reason to Ghost logs and prevent Ghost from
exiting
- Ghost themes are allowed to define the API version they want to use, but this concept has never really worked correctly
- We have a theory that it doesn't ever do quite what people want, and so always loading latest would not really break anything - this commit tests that theory
- We're pinning to canary, as we're aiming to get rid of the concept of versions altogether
- I could have done return config.get('api:versions:default');, but this actually returns v4 (although the same as canary, conceptually different)
and it also seemed like an unnecessary level of indirection. This change should be easy to understand and to revert if we are wrong
refs https://github.com/TryGhost/Team/issues/1322
Since the Members service is no longer started when Stripe is
reconfigured the url config used for checkout sessions was not updated.
This moves all of the default Stripe urls into the Stripe service, where
they will be updated as/when Stripe is connected/disconnected.
refs: https://github.com/Ghost/Ghost/commit/b5ee17b25
- When moving the site tests into e2e-api they broke because they were getting the changed settings value from the settings test
- The solution is to ensure the settingsCache is reset between tests, which is what this commit does
- This commit also renames the shutdown method to reset, because this is not a permanent operation
- It also renames the resetDb method to resetData, because the concept is we want the internal data to be reset, not just the DB
no-issue
The way we choose which email to send is too tightly coupled with the
magic link generation, which meant that when we added the new paid
signup email, the magic link looked like <url>?action=signup-paid rather
than <url>?action=signup - our welcome page logic was working off of
only using welcome pages for action=signup and so this broke.
- When starting ghost for e2e tests we create a content folder in the os tmp dir
- This means that the folder can change between suites as ghost is started and restarted
- For the most part this is fine, but theme storage caches the path to config (which makes sense, it's not meant to change whilst Ghost is in-memory)
- This is a quick-n-dirty fix that just makes it possible to update that path in the tests, so we know it's in sync
- Ideally we'd not cache the path, use a function to fetch it etc, or fully reset the theme storage layer, but this is the fix I have working today
and so it's going in to unblock things for now
refs https://github.com/TryGhost/Team/issues/1071
We switched to using tiers pivot table that stores list of tiers with access to post when visibility is set to `tiers`. For themes using v3 API while having posts restricted to specific tiers visibility, the post data will not include the list of tiers, which caused an unexpected error while trying to determine post access from tiers list. This change blocks access to post if specific tiers visibility is enabled without data available for list of tiers on post.
refs daeb06e835
- This is an additive change, and the parameter is optional so it's better to ship it right away (testing was hard as it's a chicken and an egg problem to have a feature flag in the setup route).
refs https://github.com/TryGhost/Team/issues/1321
- when updating the Email record after submitting all email batches we have a `catch` call but it was only logging the error
- added a call to Sentry so there's more visibility if those saves fail
refs https://github.com/TryGhost/Toolbox/issues/202
- this code suffers from two problems:
- when we don't have any new settings to insert, we still end up
fetching the columnInfo and owner info, even though we only need
them if we're inserting data. This results in 3 extra queries upon
boot
- secondly, we insert every setting with a separate query - MySQL and
SQLite both support batch inserts and knex has a utility to help us
that I've [used
before](38821c5242).
With 95 settings at the time of writing, this adds 94 extra queries
during the DB init
- this commit refactors the code so that we only fetch the columnInfo and
owner data if we've got new settings to insert, and batches the
inserts using knex's batchInsert util
- this query results in ~95 less queries during DB init and saves a
couple of queries during boot
loses TryGhost#13971
This fixes an issue with links containing # anchor. It makes sure the # part is at the end of the url like url?v=hash#anhor
Co-authored-by: Hannah Wolfe <github.erisds@gmail.com>
no issue
- the new eslint bump seems to flag this line up for using camelcase
- i'm not sure why this hasn't flagged before but it seems we want to
keep the naming because the function name has the same linting exemption
refs https://github.com/TryGhost/Team/issues/1296
- The `theme` must be a github `org/repo` string
- This uses the internal API instead of the services because the API has extra implementation details not present in the services.
refs https://github.com/TryGhost/Team/issues/1071
Default visibility for a post when set to specific tiers needs special handling as data for specific tiers is stored as an array of tiers on a pivot table. This change handles the default visibility for a new post when set to specific tiers to generate the right default values in model.
closes https://github.com/TryGhost/Team/issues/1311
For some sites, the `portal_products` array was created without any value and due to a possible bug in older version of Ghost, it also didn't get filled on Stripe connect with default product. This causes a side-effect of sites not showing the prices in Portal when tiers beta is enabled or is out as GA. This change populates the missing product data in `portal_product` for sites that have a single tier (haven't enabled tiers beta), as they right now don't have an option to hide the tier.
refs https://github.com/TryGhost/Team/issues/1071
Default content visiblity for specific tiers is now stored split between `default_content_visiblity` and `default_content_visibility_tiers` setting, with former storing the value as `tiers` and the latter stores the list of tiers that the visibility is restricted to. This migration transforms all existing sites that have default visibility stored as an NQL string from previous versions to follow the new model and store correctly on the new setting.
refs https://github.com/TryGhost/Team/issues/1071
Default content visibility for a post can be one of `public|members|paid|tiers`, where `tiers` denotes visibility restricted to specific tiers. This change adds a new setting to store the tier ids when default content visibility is set to `tiers`. This closely matches how the visibility is stored on `posts` table as well, with `visibility` stored as `tiers` and tiers data is stored on tiers pivot table.
refs https://github.com/TryGhost/Team/issues/1287
Currently we have a hard limit of how large an email filter can be,
which is very restrictive once a site starts using Tiers - by moving
toward a TEXT column, we essentially give the filters unlimited size.
This currently doesn't handle SQLite as there are no limits on VARCHARS
in SQLite.
The down migration is a loop so we don't have to handle values larger than
50 characters
no issue
- we check the presence of `members_free_signup_redirect` here but the
log line said `members_paid_signup_redirect`
- this must have been missed in review but it's simple enough to fix
refs https://github.com/TryGhost/Team/issues/1228
- We were displaying a message referencing paid subscriptions to all users, which was causing confusion
- We don't have the `member.paid` flag in this context, so we use the match helper on member.status not free
no issue
- `this` isn't referring to the correct `this` in this scope, and the
library is an import of the file anyway, so this fixes the undefined
function call as a result
fixes: https://github.com/TryGhost/Ghost/issues/12871
- This Ghost recommended email scanner, mail-tester.com, reports not having this alt as having an impact of -0.5 out of 10 on your email score.
refs https://github.com/TryGhost/Team/issues/1168
This migrates the existing settings onto the Tier objects, so that users
with Tiers enabled can seamless move from global settings to Tier level
settings - without losing/modifying data/functionality.
refs https://github.com/TryGhost/Team/issues/1071
- `products` relation is needed on post/page to tackle custom tiers visibility, this change attaches `products` as default include for frontend controllers
refs https://github.com/TryGhost/Team/issues/1071
Going forward, if the visibility of a page/post is set for specific tiers, we send a `tiers` array in API response that contains list of tiers with access. This change -
- updates post/page mapper to transform existing data where `visibility` is a custom nql string to tiers array
- updates default include for post/pages to include `products`, which allows attaching relevant tiers from the pivot table
- cleans up usage of `visibility_filter` in serialization
