Commit Graph

109 Commits

Author SHA1 Message Date
Thibaut Patel
88877312b5
Removed the default column from the newsletters table (#14428)
refs https://github.com/TryGhost/Team/issues/1474

- The `default` concept will be replaced by the first newsletter based on the `sort_order`
- This removes the `default` value from the newsletter API
- This simplifies the design to make the api and datastructure more maintainable
2022-04-07 11:50:20 +02:00
Thibaut Patel
2bfd8f8b7e
Added the post-newsletter relation (#14411)
refs https://github.com/TryGhost/Team/issues/1471

- This is a many-to-one relation so that many posts can be linked to a specific newsletter
- The `newsletters` table had to come first in the schema file so that it's initialized before the `posts` table (because of the foreign key)
- Updated the model to make sure the new field doesn't leak in the API for now
- This migration isn't using the `createAddColumnMigration` util because of a performance issue. In MySQL, adding/dropping a column without `algorithm=copy` uses the INPLACE algorithm which was too slow on big posts tables (~3 minutes for 10k posts). Switching to the COPY algorithm fixed the issue (~3 seconds for 10k posts).
- SQLite isn't using the codepath where we run a raw SQL query because `knex` is doing multiple queries to add/remove a column
2022-04-07 10:26:37 +02:00
Rishabh Garg
90e7887007
Added default newsletter subscription for new members (#14431)
refs https://github.com/TryGhost/Team/issues/1469

Currently, all new members get auto subscribed to the default newsletter. This change adds same behavior with multiple newsletters by auto subscribing all available newsletters on site for new members(If flag is enabled). 
Note: In future, this will also take into consideration the `subscribe_on_signup` flag for a newsletter to filter which newsletters should a member be auto-subscribed.

- adds newsletters service for working with newsletter data
- bumps `@tryghost/members-api` package which handles default subscription
- adds new test fixture/data for newsletters
2022-04-07 08:30:00 +05:30
Simon Backx
1957b5b789
Removed pagination from members stats endpoint and added extra day to output (#14429) 2022-04-06 17:10:47 +02:00
Hannah Wolfe
0581314796
Fixed location of url service unit tests
- The url service was moved from frontend to server some time ago but the tests were forgotten
- This is only being done now because in 5.0 major changes are happening and it'll be annoying if the
  files move on that branch
2022-04-06 13:05:41 +01:00
Hannah Wolfe
11867ab43a
Replaced schema.isPost in slack service /w custom fn
- This is the only piece of server code that relies on the schema.checks, the rest are all frontend
- IMO this code should actually check for the post properties that the slack message needs
- OR it should switch based on the event type
- either way there's no need to have a shared util for this simple use case
- especially becaue it's confusing the use case for it and creating cross-coupling between server and frontend
2022-04-05 14:24:42 +01:00
Rishabh Garg
ee0a6910ce
Added table to store multiple newsletters for member (#14408)
refs https://github.com/TryGhost/Team/issues/1469

With multiple newsletters, members will now be able to subscribe to one or more newsletters on the site. Previously, the subscription to default newsletter for a member was controlled via a single boolean `subscribed` column on the member table.
This change allows mapping multiple newsletters to a member via new pivot table that stores relation between a member and newsletter.

- adds new `members_newsletters` pivot table
- update tests
2022-04-04 18:11:36 +05:30
Simon Backx
31c1d4f513 Renamed stats endpoint to /stats/member-count and added totals metadata
refs https://ghost.slack.com/archives/C02G9E68C/p1648737467414789?thread_ts=1648644801.253699&cid=C02G9E68C

- Anything in the API should use snakeCase
- Reduce amount of nesting in endpoint name
- Added totals metadata
2022-03-31 17:05:08 +02:00
Thibaut Patel
fdb0e3d44d Added the newsletter API permissions
refs https://github.com/TryGhost/Team/issues/1463

- Allow admins to perform all newsletter operations
- We can adjust and be more permissive in the future if needed
- Added the tests back as permissions are configured correctly now
2022-03-31 16:08:23 +02:00
Simon Backx
ae54352a29
Created new stats service and stats API to get member count history (#14391)
refs TryGhost/Team#1458
refs TryGhost/Team#1459
refs TryGhost/Team#1372

- Added a new stats service, which is divided into several categories. Currently only the 'members' category for member related stats.
- When there are missing or corrupt members status events in the DB, the totals returned by the old member stats endpoint (`/members/stats/count`) were wrong. This is fixed in the new service by counting in reverse order and starting with the actual totals.
- New Stats API, with the new `/stats/members/count-history` endpoint.
- This new endpoint also returns the paid deltas -> dashboard 5.0 will show subscribed and canceled paid members for each day
- Includes tests for the new stats service and endpoint
2022-03-31 16:01:11 +02:00
Thibaut Patel
f096e29922 Added the newsletters table
refs https://github.com/TryGhost/Team/issues/1433

- The `default` property stores whether a newsletter is set as default by the admin
- The `status` property stores whether a newsletter is archived or not
- The `recipient_filter` property is only storing whether a newsletter is "paid-only" or not for now, although it can be expanded to more specific filters in the future
- The `subscribe_on_signup` property stores whether a new member should be automatically signed up to the newsletter
- The `sort_order` property enables displaying the newsletter list in an order chosen by the admins
2022-03-29 15:23:26 +02:00
Daniel Lockyer
32e2d15880
Split apart DB init queries in tests
refs https://github.com/TryGhost/Toolbox/issues/213

- `better-sqlite3` doesn't like multiple queries in the same statement
  so we can make the change here to split them up ahead of the switch
2022-03-28 16:17:17 +01:00
Hannah Wolfe
a31069d49d
Removed unnecessary use of rewire
refs: https://github.com/TryGhost/Ghost/commit/e68cb8b31

- I found that some of the places we use rewire are totally unnecessary
- Rewire seems to mess with coverage sometimes
- It's also a code smell in general so I've ripped it out where possible
2022-03-24 19:32:06 +00:00
Rishabh Garg
f9aa18a534
Enabled Admin integration for tiers and offers API (#14325)
As multiple tiers is now GA, we want to allow devs to be able to work with Tiers and offers via content/Admin API. This change -

- updates fixtures to add permissions to admin integration role for new sites
- adds migration to update existing sites to have correct permissions for role
- whitelists add/edit/read/browse on tiers and offers API for integrations
2022-03-16 16:46:26 +05:30
Naz
cbba5471a2 Renamed versioned API URL variable
refs https://github.com/TryGhost/Toolbox/issues/169

- Improved code readability, otherwise it could be confusing what the default API URL means
2022-03-14 21:22:54 +13:00
Naz
1dd47c4191 Added test coverage proving token back compatibility
refs https://github.com/TryGhost/Toolbox/issues/169

- There was not much clarity around how the tokens created for the versioned API audience would behave when non-versioned API is introduced. The tests added here illustrate the tokens being forward compabible (created for versioned -> verified at non-versioned API) and not backwards combatible (created for non-versioned -> verified at versioned)
2022-03-14 21:22:54 +13:00
Naz
7becf0a2b2 Aliased canary endpoints to point to non-versioned URLs
refs https://github.com/TryGhost/Toolbox/issues/169

- Before releasing Ghost v5 we would like to move all canary-related URLs to a non-versioned format, which will become a default in v5.
- 'canary' is by definition unstable, so breaking any unprepared client explicitly using the canary is expected
- Removed the aliased /content/ and /admin/ apps from app.js because with updated configuration they become duplicates of 'canary' endpoints
2022-03-14 21:22:54 +13:00
Thibaut Patel
4ce9a5a167 Added the MemberCancelEvent model
refs https://github.com/TryGhost/Team/issues/1302

- This event stores the members' subscription cancelations.
2022-03-11 21:50:06 +01:00
Simon Backx
42ac8c41e4
🐛 Fixed uppercase file extensions ignored in content import (#14268)
refs https://github.com/TryGhost/Team/issues/1363

- When uploading a zip of images in Settings > Labs > [Import], it will skip images that have an uppercase extension, citing an 'unsupported file type' error.
- Cause: Glob ignored those files when matching extensions in ImportManager
- Fix: Added nocase option where needed
- Extended tests to also test the processZip method of ImportManager with getFilesFromZip
- Added isValidZip for zip with uppercase image
- Cleaned up JSDoc in ImportManager, and replaced some older JS syntax

Fixed zipContainsMultipleDataFormats error never thrown:

When a zip combines two data formats, no error was thrown.

- The promise error was only returned in an _.each loop, but never thrown
- Previously when combining multiple data types in a zip file, no error got thrown
- Added a test for this error
- Also added a test for noContentToImport error

Other errors and fixes:

- Added missing length in getBaseDirectory check
- getContentTypes fixed (returned duplicate values). Type error came up after adding all JSDocs
- updated tests to match real types from JSDoc and pass type validations
- Rewrote some methods in the async await syntax
- Added tests for ImportManager clean up
2022-03-11 09:17:58 +01:00
Matt Hanley
7172db74b0
💡 Updated fixtures to make it easier to get started (#14299)
- Our old fixtures were designed as a guide to getting started to Ghost, but they got in the way
- The old fixtures now live as part of ghost.org/resources - a living guide to starting with Ghost
- These new fixtures mean the site is ready to go as soon as it's setup

Co-authored-by: Hannah Wolfe <github.erisds@gmail.com>
2022-03-10 17:41:46 +00:00
Naz
16e0736379 Removed Roon related importer code
no issue

- The support for the misformated roon imports was temporary anyway, based on the comments in code. It's also unecessary to keep around any code related to Roon as it's been _ages_ since anybody needed this kind of migration
2022-03-10 16:29:54 +08:00
Daniel Lockyer
8a7c7f08e9 Enabled fetching .ico dimension size via probe-image-size
refs https://github.com/nodeca/probe-image-size/blob/master/CHANGELOG.md#600---2020-11-04

- `probe-image-size` v6 now supports `.ico` files so we can
  allow probing of dimensions via this library rather than falling back
  to downloading the entire image via `image-size`
- also updates a test because .ico files no longer use the internal
  request lib, which simplifies things a little bit
2022-03-09 20:27:38 +00:00
Rishabh Garg
9794945549
Updated tests for custom welcome page redirects (#14288)
Welcome pages are now moved as a property of individual tiers instead of global property on settings. Since we removed the alpha flag for new welcome page behavior, tests had to be updated to test the flow against new behavior.
2022-03-09 17:49:25 +05:30
Naz
da9f018c70 🐛 Fixed theme activation with capitalized names
closes https://github.com/TryGhost/Team/issues/1420
refs da0dee548c
refs https://github.com/TryGhost/Toolbox/issues/169

- After introducing non-versioned API urls the "isAPI" regex failed to pass the test as it was expecting a `canary/vX` in the API URL. This caused "uncapitalization" to stop working for API requests.
- Regex visualizer for quick reference: https://jex.im/regulex/#!flags=&re=%5E(.*%5C%2Fghost%5C%2Fapi(%5C%2F(v%5B%5Cd.%5D%2B%7Ccanary))%3F%5C%2F.*%3F%5C%2F)
2022-03-09 20:07:04 +13:00
Daniel Lockyer
1a3aa69c68 Re-throw error when attempting to create webhooks
- we catch error arising from creating webhooks and check for specific codes
- if our error does not match one of those codes, we don't propagate the
  error up
- this becomes a problem if saving a webhook fails for some other reason
  because upstream code assumes we return an error or model
- this commit re-throws the error and adds a test that would have caught
  this
2022-03-07 13:54:00 +00:00
Matt Hanley
c6617459a5
Switched products.visible for products.visibility (#14264)
- We have an existing pattern for using `visibility: public` instead of `visible: true|false`
- We no-op the existing migration and roll forward so that we don't have to manually revert db changes
2022-03-04 11:07:38 +00:00
Naz
cd1183c9d8 Added JWT authentication for non-versioned API
refs https://github.com/TryGhost/Toolbox/issues/169

- As Ghost prepares to drop API versioning in future major release it the authentication mechanism should take into account non-versioned token audience support. The audience for non-versioned api requests would be limited to "admin" rather than "canary/admin"
2022-03-03 20:21:22 +13:00
Fabien 'egg' O'Carroll
db202fb162
Added visible column to products table (#14221)
refs https://github.com/TryGhost/Team/issues/1387

We are moving away from the portal_products setting to instead store
each tiers visiblity on the tier itself. This column will be used for
that data.

Both of the default Tiers should be visible, but newly created tiers
should not be.
2022-03-01 18:11:59 +02:00
Naz
4700b14d93 Made default settings file location configurable
refs https://github.com/TryGhost/Toolbox/issues/214

- The values configuration for the settings table need to become configurable to be able to run our test environment with a pre-defined set of configurations (e.g Stripe-related values).
- This change makes it possible to define the default settings file location (currently a JSON)
- A new key is now exposed through the "paths.defaultSettings" key in settings, which can be overloaded for the needs of the environment
2022-02-21 20:06:20 +13:00
Naz
7f0bf62ec9 Renamed defaultSettings to defaultRouteSettings
refs https://github.com/TryGhost/Toolbox/issues/214

- The `defaultSettings` path name in the config (one pointing to routes yaml file) creates confusion with the `defaultSettings` which populate defaults for  in the database settings table.
- Furthermore, the name collision creates a problem when trying to make database default settings dynamic - being able to load them from configurable file path.
- Rename makes "routing" explicit to avoid ambiguity and free up the name for the database defaults
- The value seems to be safe to be renamed as all keys used in `overrides.json` are taking priority - the name "defaultRouteSettings" hasn't surfaced at any point in the git history
2022-02-21 20:06:20 +13:00
Fabien 'egg' O'Carroll
9c5c41b927
🐛 Fixed welcome pages not working for "subscribe" links (#14176)
- Fixed test fixtures so that members with subscriptions also have products/tiers
- Fixed test fixtures so that default&free tiers can be updated for tests
- Added tests for the signin functionality and welcome page redirects
- Extended `setupStripe` to setup other Members settings - this needs some more
  thought around how we proceed
2022-02-20 16:02:42 +02:00
Daniel Lockyer
55204bf725 Fixed admin views path mocking override in tests
- for some reason, this test seems to be failing now we've pulled it out
  of the general CI
- it makes sense when the repo is clean, because the html files don't
  exist, but I don't understand how they were working before... 🤔
- anyway, we should be overriding the path to the test fixtures admin
  view files here
- this fixes the unit tests
2022-02-16 19:22:20 +01:00
Rishabh
56a8d21263 Added unit tests for content gating post access
refs https://github.com/TryGhost/Team/issues/1343
refs 3ccd3601b3
2022-02-11 14:26:04 +05:30
Thibaut Patel
479df36a48 Added the last_seen_at column to members
refs https://github.com/TryGhost/Team/issues/1304

- This migration adds a column to store when a members was last seen
- The utils.js e2e test file was refactored according to the commit 06dd9bac59
2022-02-09 11:57:45 +01:00
Rishabh
6f89ccf901 Added default_content_visibility_tiers setting for default tiers visibility
refs https://github.com/TryGhost/Team/issues/1071

Default content visibility for a post can be one of `public|members|paid|tiers`, where `tiers` denotes visibility restricted to specific tiers. This change adds a new setting to store the tier ids when default content visibility is set to `tiers`. This closely matches how the visibility is stored on `posts` table as well, with `visibility` stored as `tiers` and tiers data is stored on tiers pivot table.
2022-02-04 19:56:11 +05:30
Fabien 'egg' O'Carroll
d60d24e744
Migrated email filter columns from VARCHAR to TEXT (#14091)
refs https://github.com/TryGhost/Team/issues/1287

Currently we have a hard limit of how large an email filter can be,
which is very restrictive once a site starts using Tiers - by moving
toward a TEXT column, we essentially give the filters unlimited size.

This currently doesn't handle SQLite as there are no limits on VARCHARS
in SQLite.

The down migration is a loop so we don't have to handle values larger than
50 characters
2022-02-04 16:09:35 +02:00
Rishabh Garg
eeafc8603b
Added pivot table to store tiers on post (#14038)
refs https://github.com/TryGhost/Team/issues/1071

We used `posts.visibility` originally to store visibility as `free|paid` with a character limit of 50. This same field was repurposed to store an NQL filter when member tiers is enabled. The NQL filter uses the slug of the tier name, which can easily create a filter longer than 50 characters, adding an unwanted limitation on number of tiers that can be added to post's visibility.
Going forward, we'd like to store the visibility of posts for tiers in a separate pivot table and instead store the value of `visibility` as `tiers` when restricting post access to specific tiers. This change -

- adds a new pivot table fixture for storing relation between posts and tiers
- adds a migration for creating the new table
- updates tests
2022-01-31 15:33:58 +05:30
Fabien 'egg' O'Carroll
8393c1e90a
Added welcome_page_url column to products table
refs https://github.com/TryGhost/Team/issues/1168

Rather than using a single url for paid signup redirects, we want to
support setting a welcome page on a tier by tier basis. This column will
be used to store the URL. A text column of length 2000 is how we have
stored URL's elsewhere in the schema.
2022-01-26 11:44:40 +02:00
Hannah Wolfe
4c8ff38a44
Updated misc unversioned tests to run on canary
refs: https://github.com/TryGhost/Toolbox/issues/168

- All of our unversioned tests should be running against canary already
- These tests are erroneously running on the wrong version

We're going to be dropping the idea of having multiple versions of the API in each Ghost version.
Because this has not achieved the goal of making it easier to make breaking changes, but it has
created an ordinate amount of technical debt and maintenance overhead.

As we know this is going away in the next major, there is no benefit to us constantly running tests
that check if those versions still work, especially given how long they take.

Instead we're starting work to ensure that all of our test work on canary, and that canary has
excellent test coverage so that we can be sure that our one API version works really well and that
any changes, no matter how subtle are deliberate, tracked and understood.
2022-01-21 15:11:48 +00:00
Hannah Wolfe
12e8c974a1
Cleaned up weird refs to versions in tests
refs: https://github.com/TryGhost/Toolbox/issues/168

- These are all places where we reference an API version like v2 or v3 but it's not actually
used or relevant.
- The aim is to get rid of all mentions of these old versions to make it clearer that we're only running tests on canary
2022-01-21 15:10:03 +00:00
Hannah Wolfe
e47b55b466
Removed some hidden v2 and v3 tests
refs: https://github.com/TryGhost/Toolbox/issues/168

- These are all test files I missed in ffcd3fbe313b4a413833da9a7473376cb21246fd

We're going to be dropping the idea of having multiple versions of the API in each Ghost version.
Because this has not achieved the goal of making it easier to make breaking changes, but it has
created an ordinate amount of technical debt and maintenance overhead.

As we know this is going away in the next major, there is no benefit to us constantly running tests
that check if those versions still work, especially given how long they take.

Instead we're starting work to ensure that all of our test work on canary, and that canary has
excellent test coverage so that we can be sure that our one API version works really well and that
any changes, no matter how subtle are deliberate, tracked and understood.
2022-01-21 15:07:17 +00:00
Fabien 'egg' O'Carroll
534ebecc19
Added active column to products table (#14016)
refs https://github.com/TryGhost/Team/issues/1252

We need a way to signal whether or not a Tier is active or archived, and
we'll be using the active flag in the same way we do for Offers.
2022-01-20 15:50:18 +02:00
Fabien 'egg' O'Carroll
a565da06b2
🐛 Fixed Offer Redemptions being over counted (#13988)
refs https://github.com/TryGhost/Team/issues/1257

Offer Redemptions were being overcounted due to the way we were updating
Stripe configuration for the Members service. We would create a new
instance of the members-api, which would have event handlers for
creating Offer Redemptions - by creating a new instance each time Stripe
config changed, we would overcount them.

Here we've pulled out Stripe related logic into the Stripe service, and
updated it internally - rather than creating a new instance. This means
that we've been able to remove all of the logic for re-instantiating the
members-api.

- Bumped members-api & stripe-service
- Removed reinstantiation of members-api
- Used stripe service to execute migrations
- Updated Stripe Service to handle webhooks & migrations
- Used webhook controller from stripe service
- Used disconnect method from stripe service
- Removed unused stripe dependency
- Removed Stripe webhook config from members-api
2022-01-18 17:56:47 +02:00
Rishabh
62777d7f26 Handled default product to use first paid product
Since we now have 2 products by default for all ghost sites, free and default paid, the usage of default product which so far was using first product needs to be updated to use the first paid product.

- updates default product usage to use first paid tier
- updates tests
2022-01-17 22:59:34 +05:30
Fabien 'egg' O'Carroll
10c214c148
Switched AMP to be 'off' by default in all new Ghost instances (#13907)
refs https://github.com/TryGhost/Team/issues/1189

Support for AMP is slowly in decline, and makes developing new cards trickier,
since AMP no longer has an effect of SEO we're going to disable it by default
as a first step toward moving away from it.

Co-authored-by: Thibaut Patel <thibaut@ghost.org>
2022-01-14 18:55:48 +02:00
Fabien 'egg' O'Carroll
bc75d20cef
Updated SingleUseToken grace period to 10 minutes (#13926)
refs https://github.com/TryGhost/Team/issues/1216

Some email security clients are scanning links at delivery, rather than
at the point the user clicks on them. This is causing magic links to
expire. To get around this we're increasing the grace period in which a
link can be used multiple times to 10 minutes.
2022-01-03 17:55:53 +02:00
Sam Lord
97c68dd388
Replace error handler middleware with @tryghost/mw-error-handler (#13879)
refs: https://github.com/TryGhost/Toolbox/issues/137

Extract error handling middleware and replace with a package.
2021-12-14 15:18:46 +00:00
Naz
42cd78e05e Added coverage for route settings class
refs 3150c87935

- Adds basic coverage for a bug that was fixed in referenced commit.
Next time it should be easier to add more tests to the suite as there's
already an example starter to work your way from.
2021-12-13 12:21:47 +04:00
Daniel Lockyer
2d090b8b2a Revert "Remove unnecessary references to ghostVersion"
This reverts commit 443ee369d2.
2021-12-08 14:28:41 +00:00
Sam Lord
443ee369d2
Remove unnecessary references to ghostVersion
refs: https://github.com/TryGhost/Toolbox/issues/146
GhostVersion was passed in here but not used at all. This just removes those unnecessary usages.
2021-12-07 13:48:22 +00:00