no-issue
These utils are the first steps toward getting the models out of our
migrations! The utils here interact directly with the database and where
possible don't reinvent the wheel, by using smaller building blocks to
build more comples ones.
refs https://github.com/TryGhost/Ghost/issues/10318
- Added additional check for correct "type" fields in settings responses
- The list is based on pre-migration response from API v2, so makes it very clear there is no breaking change after the migration
refs TryGhost/Ghost#10318
refs 8fc526ff6
- This is symetric change to one done for v3 API (commited as 8fc526ff6)
- Added 'core' filtering for v2 API controller
refs https://github.com/TryGhost/Ghost/issues/10318
refs 1dc0405803
- Adds 1:1 mapping for filtering options to renamed settings "type" to "gorup"
- Ignores the name changes and any old types
- Detailsed type -> group mappings can be checked in the refereneced migration commit
refs https://github.com/TryGhost/Ghost/issues/10318
- `group`
- to replace the `type` column, provides a more descriptive name for the columns use
- for existing sites it will be populated by migrating data from the `type` column in a later migration
- for new sites a minimal update has been added to `parseDefaultSettings()` to populate the `group` field when settings are created during startup - fixes the NOT NULL constraint on `settings.group`
- `flags`
- signifies special handling that is different to other settings in a group
- eg, `PUBLIC,RO` would indicate that the setting is available via unauthenticated endpoints and is read-only
refs https://github.com/TryGhost/Ghost/issues/10318
- At the moment there were no test checking correct behavior of `?type` parameter. Given ongoing settings refactoring work expanded tests with some essential coverage
- Moved one of the test cases to be consistent with order in other suites
refs 8a817050c5
- This change reflects field reshuffle done in referenced commit
- NOTE: removed `members_email_auth_secret` key from respnse body validation as it seems like it should never have been exposed in the first place! This needs verification as it's a breaking change theretically
refs #10318
refs 2614565d5a
- Adds importer mapping for fields as in referenced migration
- The intention is to allow exports from v2/v3 to still be compatilbe with current version
- Adds a mapper for deprecated fields and imports them with new settings "keys"
refs #10318
refs 2614565d5a
- Renamed ghost_head/ghost_foot in settings to match the new names
introduced in migrations
- Above change lead to reshufling in the mappings in input/output
serializers
- Makes sure change is compatible with v2 API
refs https://github.com/TryGhost/Ghost/issues/10318
refs 2614565d5a
- Renames to match referenced migration renames
- Fixed API responses so they are consistent with newly renamed fields
- Not returning lang and timezone keys from settings in API v2 ther rest should be returned in API v3/canary
refs https://github.com/TryGhost/Ghost/issues/10318
- precursor to migrating from `settings.type` to `settings.group`
- renames `blog` type to `site`
- renames `bulk_email` type to `email`
- moves settings out of `site` (previously `blog`) into more appropriate groups such as `core` or individual feature groups
refs 19c5e94259
- deleting `urlUtils` from the require cache caused problems in later unit tests that were stubbing `urlUtils` because they had required the `urlUtils` instance that existed before it got deleted/re-initialised in this test, so their modules under test ended up using non-stubbed instances
- fix by stubbing the `urlUtils` methods in this test too (we should probably fix this properly at some point so that config changes correctly propagate to `urlUtils` and we don't need to stub internals)
no issue
- there is a test interdependency issue between the `populateImageSizes` tests and routing tests when we clear the cached `url-utils` require
- commented the cache clear and subsequently broken test for now pending a deeper investigation
no issue
- when using subdirectories, images can be stored in the database both with and without the subdirectory prefix. We weren't taking that into account and so images without the subdirectory were not having the `/content/images/` prefix removed when passed to the storage adapter resulting in the storage adapter not finding the image
no issue
- 3.21.0 added `width/height` attributes to `<img>` elements output by image cards which was a potentially breaking change for any sites whose CSS does not expect those attributes, resulting in squashed and distorted images
- updated to `@tryghost/kg-default-cards` which modifies the image card renderer to not output `width/height` attributes
refs #10318
refs https://github.com/TryGhost/Ghost/pull/11942
- Removes force_i18n, permalinks, and members_session_secret usage from the codebase
- We deprecated these flasgs and have not used since Ghost v2. It's good time to remove them before we introduce bigger changes to how `settings` table opeartes.
- Fixed importer test. The test was meant to check if string values were converted properly, the check agains boolean didn't make much sense in this context, so removed it.
- Following this change are going to come ralated migrations to fix existing data (see ref)
no issue
- by including `sizes` with media-queries that match typical content widths we can give the browser better hints as to which responsive image size to fetch resulting in more performant pages
- also fixes `srcset` output where "unknown" sizes were used instead of linking to the original image src which was causing 302 redirects and slowing image loading in some cases
no issue
- Moves out CSV parsing and serialization related code into separate package as a part of push to modularize Ghost repo.
- Next up is to remove `csv-parser` dependency from this new package
no issue
- fixed incorrect method name when calling
- fixed problem with setting post.mobiledoc to a promise rather than waiting for the size population to finish and setting it to a mobiledoc string
no issue
- adds `populateImageSizes()` to our mobiledoc lib module
- uses `image-size` lib to speed up reading of image dimensions
- for local images, use storage adapter with same guards as used by `handle-image-sizes` middleware so that we don't insert srcsets for images that aren't transformable
- for unsplash images, remove any width and crop params from the url so it points to the full-size image
- use `populateImageSizes(mobiledoc)` to modify post model's mobiledoc when re-rendering
- There were various cases where it was possible to trigger a private site to display a 404 instead of redirecting to /private/
- Private mode was also not always displaying the correct robots.txt
- This PR includes tests for all cases in test/frontend-acceptance/default_routes_spec.js & where possible the unit tests have also been updated for completeness
- Fixing the 404 issues required
- Better handling of paths using req.path instead of req.url in filterPrivateRoutes
- Additional error handling, to cover the case that a tag/author RSS feed does not exist
- Fixing the robots.txt required the order of middleware to be changed, so that private blogging gets a chance to render first
- NOTE private blogging is the only app with a setupMiddleware function so nothing else is affected
no-issue
These tests were not testing the functionality correctly, but trying to
test it based on metadata, which can become out of date. This tests the
expected responses and is resilient for future changes to fixtures.
no-issue
This issue only occurs when using custom redirects with a subdirectory
setup, and the path to be redirected from is expressed as a regex, and
the url that is being redirected to is not an external url.
The issue has a few components:
- Redirect paths as a regex generally use the ^ to ensure that they
match the beginning of the path.
- The path that the regex is matched against conditionally excludes the
subdirectory, specifically, the subdirectory is excluded for external
urls
These combined means you end up with a regex like /^\/custom-redirect/
and a path like /subdir/custom-redirect, these will not match/replace
correctly, and you'll end in an infinite redirect loop.
The fix here is to *always* remove the subdirectory when testing regex's
and then conditionally adding it back *only* for the redirect, and only
if it is an internal redirect
- the existing tests were extremely confusing as tests for logged in and out states were mushed together
- tests for the special private rss feeds only existed for one state, but shouldexist for both
- hopefully it's clearer what's being tested!
This is a bunch of small changes, that simplifies working with the private blogging module:
- remove reference to really old paginated RSS behaviour
- remove handling for /rss and allow our standard redirects to redirect to /rss/ and then execute
- readd should to tests so that they can be run standalone
- fix eslint warning
no issue
- pass `srcsets` value through to mobiledoc renderer
- it will stop adding `srcset` attributes to images when explicitly set to `false`
- adds `reload()` method to `mobiledocLib` so that memoized instances of the renderer can be reset during tests that change config values
closes https://github.com/TryGhost/Ghost/issues/11913
- updated `@tryghost/kg-default-cards` with fixed `srcset` behaviour
- does not output `srcset` unless we know the image's original size
- does not output `w` values for sizes that are bigger than the original image
no issue
- screen readers often resort to reading out the filename when alt text is not provided which is not useful. By adding `alt=""` (or just `alt`) to `<img>` elements the image is skipped by screen readers providing a smoother experience
- updated `@tryghost/kg-default-cards` which contains above change for `image` and `gallery` cards
refs https://github.com/TryGhost/Ghost/issues/11414
- Importing data currently overwrites the existing "from address" with new value
- "from address" needs to go through email validation flow before update which was bypassed
- Updates importer to not allow overwrite for "from address" and use existing
- Adds test for "from address" overwrite
no issue
- The intention is to move away from using file type names in URLs. This endpoint is meant to correspond to current `POST /members/csv` endpoint, that is planned to be renamed into `/members/upload`. And the `GET /members/csv` to be renamed to `/members/download` respectively.
no issue
- NQL does not support the relationship setup that members->stripe customer<->stripe subscriptions uses so it wasn't possible to use the `filter` param to query against having an active subscription
- adds `customQuery` bookshelf plugin that allows customisation of SQL query used in `findPage` method by individual models
- use `customQuery` in Member model to set up joins and conditionals to select free/paid members when `options.paid` is present
- allow `?paid` param through API and permitted options for member model
no issue
- This new format allows to return additional metadata with failed import records. The data for invalid records is returned in following format:
```
{
count: {count_of_invalid_records},
errors: [{
message: "Members not imported. Members with duplicate Stripe customer ids are not allowed." // message field of the error
context: "Attempting to import members with duplicate Stripe customer ids." // context field of the error
help: "Remove duplicate Stripe customer ids from the import file, and re-run the import." // help field of the error
count: 2 // count of this specific error
}]
};
- Errors are grouped by their context fields because message fields sometimes can contain unique information like Stripe customer id, which would produce too many errors in case of bigger datasets.
no issue
- This endpoint is meant to be used for validation of imported members
- Main function at the moment is to validate if stripe_customer_id present in the dataset exists in connected Stripe account
no issue
- adds a set of hardcoded "content image sizes" to the base config
- adjusts `handle-image-sizes` middleware to always allow the hardcoded content image sizes to be genreated
- updates `@tryghost/kg-card-factory` to allow passthrough of options to card renderers
- updates `@tryghost/kg-default-cards` to add `srcset` output for image and gallery cards
no issue
- When imported member contains stripe_customer_id data but there is no Stripe configured on the Ghost instance such import should faiil. The logic is consistent with one where import fails after not being able to find customer in linked Stripe account
- Fixed import stats to show import failures instead of "duplicate" when the validation error is of "Stripe" origin
no issue
- whilst updating moment, a test that was doing timezone calculation was
failing. We were only importing moment before, and not moment-timezone
- this commit fixes that
- Currently theme uploads delete the existing theme before copying the new files into place
- If something goes wrong with the delete action, you will end up in a bad state
- Some or all of the files may be deleted, but now Ghost won't try to put the new theme in place, instead returning an error
- This leaves you with an invalid active theme and a broken site
- Unlike delete, move is a one-hit operation that succeeds or fails, there moving a theme is safer than deleting
- This updated code moves the old theme to a folder with the name [theme-name]-[uuid] before copying the new theme into place
- Even if this fails, the files should not be gone
- There's a cleanup operation to remove the theme backup at the end, but we don't care too much if this fails
refs https://github.com/TryGhost/Ghost/issues/11212
- if a bookmark card fetch is performed (either directly or from fallback) and the page does not have an extractable title, return a more specific error message than "No provider found for supplied URL."
no issue
- There is a need to be able to label certain import group of members
with custom labels. This will allow to distinguish/filter these newly
imported members.
- Allowed `POST /members/csv/` endpoint to accept `labels`
field parameter which assigns labels to every member from imported csv.
no issue
- Similarly to other additive api methods (e.g. members.add) returned more specific ValidationError with contex filled in with the reason why adding did not succed.
- This change is needed for more graceful label handling when adding new members through import
no issue
- Adding labels doesn't cause any content to invalidate, similarly to adding members. Unlike it's caunterpart - tags, there is no dependent "frontend" content that would become invalid
no issue
- added an `externalRequest` lib
- uses same underlying `got` module as our `request` lib
- uses `got`'s `beforeRequest` and `beforeRedirect` hooks to perform it's own dns resolution for each url that's encountered and aborts with an error if it resolves to a private IP address block
- includes a bypass for Ghost's configured url so that requests to it's own hostname+port are not blocked
- updated v2 and canary oembed controllers to use the `externalRequest` lib
no-issue
If the stripeDirect config value is NOT set / false, then connect token
should be used over the API keys if both are present, or else use
whichever is present. If the stripeDirect config value is set /
true,then API keys should be used and stripe connect token should be
completely ignored.
The reason for this is that we want to use Stripe Connect as the primary
auth method going forward, but we want to keep the direct version
availiable should the Ghost Foundation ever dismantle. This will allow
people to use all the same features with no dependency on an external
service.
no-issue
This module handles the creation of a url used for authorization of
Stripe Connect, and also the parsing of the data eventually received
from the authorization flow.
closes https://github.com/TryGhost/Ghost/issues/11841
- Migration adds mapping between permissions and roles for email_preview send test mail
- Only owner previously had correct permission to send test emails
- Fixture existed to allow Admin/Editor/Integrations to send test mails but had missing migration
- Adds tests for roles to send test email
* 🔥 removed duplicate error tests
* add lint:shared package script, and updated lint to run it as well
* moved test/unit/config to test/unit/shared/config
- Ghost Admin has a redirect system built in, so not using the # does work
- However, the usage of # or no # was inconsistent, which is always wrong
- Using the correct Ghost Admin URLs makes it easier to reason about the behaviour
closes#11762
- iframe embeds which include a line break were undetected by the regex which checks for the existence and injects the required amp script
- this fix updates the regex to include any non-word character, as line-breaks are not included in the general `.` token
- Represents that logging is shared across all parts of Ghost at present
* moved core/server/lib/common/logging to core/shared/logging
* updated logging path for generic imports
* updated migration and schema imports of logging
* updated tests and index logging import
* 🔥 removed logging from common module
* fixed tests
no issue
- adds `search` bookshelf plugin that calls out to an optional `searchQuery()` method on individual models to apply model-specific SQL conditions to queries
- updated the base model's `findPage()` method to use the search plugin within `findPage` calls
- added a `searchQuery` method to the `member` model that performs a basic `LIKE %query%` for both `name` and `email` columns
- allowed the `?search=` parameter to pass through in the `options` object for member browse requests
* moved `server/config` to `shared/config`
* updated config import paths in server to use shared
* updated config import paths in frontend to use shared
* updated config import paths in test to use shared
* updated config import paths in root to use shared
* trigger regression tests
* of course the rebase broke tests
* refactored core/frontend/services/proxy to import common dependency like a normal person
* removed all imports of `common/errors`
* 🔥 removed common/errors module
Co-authored-by: Vikas Potluri <vikaspotluri123.github@gmail.com>
no issue
- moves members stats generation for the admin graph from the client to the server
- outputs a basic totals count across a requested date range of 30, 90, 365 days, or all time. See below for the response shape
- leaves heavy lifting of the counts to the SQL engines - tested on a dataset of 100k members and query performance is <100ms
```
GET /ghost/api/canary/members/stats/?days=30
{
total: 100000,
total_in_range: 20000,
total_on_date: {
'2020-04-25': 19000,
'2020-04-26': 19500,
// continues until today's date
},
new_today: 200
}
```
* refactored `core/frontend/apps` to destructure common imports
* refactored `core/frontend/services/{apps, redirects, routing}` to destructure common imports
* refactored `core/frontend/services/settings` to destructure common imports
* refactored remaining `core/frontend/services` to destructure common imports
* refactored `core/server/adapters` to destructure common imports
* refactored `core/server/data/{db, exporter, schema, validation}` to destructure common imports
* refactored `core/server/data/importer` to destructure common imports
* refactored `core/server/models/{base, plugins, relations}` to destructure common imports
* refactored remaining `core/server/models` to destructure common imports
* refactored `core/server/api/canary/utils/serializers/output` to destructure common imports
* refactored remaining `core/server/api/canary/utils` to destructure common imports
* refactored remaining `core/server/api/canary` to destructure common imports
* refactored `core/server/api/shared` to destructure common imports
* refactored `core/server/api/v2/utils` to destructure common imports
* refactored remaining `core/server/api/v2` to destructure common imports
* refactored `core/frontend/meta` to destructure common imports
* fixed some tests referencing `common.errors` instead of `@tryghost/errors`
- Not all of them need to be updated; only updating the ones that are
causing failures
* fixed errors import being shadowed by local scope
no issue
- updated `filter` plugin with appropriate label relationship and replacement config for NQL
- fleshed out member and label fixture data and tools to facilitate tests
no-issue
The flag currently defaults to `true` as we are still using stripe
direct. We expose it on the admin api config endpoint so that the
Ghost-Admin client can use it to conditionally render
closes#11304
- Google requires an `ImageObject` to be always returned for `publisher.logo` (https://developers.google.com/search/docs/data-types/article)
- The previous fix 3f5daa60c8 added a second nested `url` error and got therefore reverted with 7ac614030d
- This commit updates the image object generation fn to **always** return an `ImageObject` with minimum of `url` and `@type` properties. If dimensions are available, we'll pass those in there as well
closes#11825
- The initial implementation had a typo in a role name which didn't allow "Administrator" to edit post's "visibility" attribute
- Added unit tests to check administrator specific role and visibility attribute permission
closes#11772
- Ensures that isSecondary is a boolean true or false
- Added tests that cover the bug, switching to using compile because the helpers have to be run together
- TODO: all tests for helpers should be switched to compile, it's SO MUCH easier
- Add a query param that indicates whether signin/up succeeded or failed
- Add unit tests for all 3 possible cases for the createSessionFromMagicLink middleware
- Added an acceptance test to show the behaviour works in principle
- Fixed session invalidation for "locked" user
- Currently Ghost API was returning 404 for users having status set to "locked". This lead the user to be stuck in Ghost-Admin with "Rousource Not Found" error message.
- By returning 401 for non-"active" users it allows for the Ghost-Admin to redirect the user to "signin" screen where they would be instructed to reset their password
- Fixed error message returned by session API
- Instead of returning generic 'access' denied message when error happens during `User.check` we want to return more specific error thrown inside of the method, e.g.: 'accountLocked' or 'accountSuspended'
- Fixed messaging for 'accountLocked' i18n, which not corresponds to the
actual UI available to the end user
- Added automatic password reset email to locked users on sign-in
- uses alternative email for required password reset so it's clear that this is a security related reset and not a user-requested reset
- Backported the auto sending of required password reset email to v2 sign-in route
- used by 3rd party clients where the email is necessary for users to know why login is failing
Co-authored-by: Kevin Ansfield <kevin@lookingsideways.co.uk>
no issue
- Adds new endpoint on integration to refresh admin/content api key secret
- Allows owner/admin to refresh their content or admin API keys for an integration via Ghost Admin
- Adds a new `refreshed` event to actions table for anytime an api_key secret is refreshed
- Added a wrapper around express.Router to our shared/express util
- Also export static and _express
- Use this shared util everywhre, meaning express is only used directly in this one file
- ATM this file is mostly an experiment / debug helper, it might be removed again later
- The aim is to have a minimal framework wrapping express that allows us to:
- reduce our usage of express() in favour of Router()
- unify some of our duplicated logic
- fix some structural issues e.g. Sentry
- make it easier to understand the codebase
- create a new app for the /members/ endpoint
- moved all /members/ routes and middleware onto this app
- helps to separate members and frontend/site logic so we can start to decouple things more
- this redirect is the only one in our tests which is permanent
- also I verified that this new test works the same before and after the recent changes :)
refs #10898
- Execute string replacement on external paths
- Take non-top-level base URLs into consideration (to avoid #10776 dups)
- Added tests for all of the above cases
- All var declarations are now const or let as per ES6
- All comma-separated lists / chained declarations are now one declaration per line
- This is for clarity/readability but also made running the var-to-const/let switch smoother
- ESLint rules updated to match
How this was done:
- npm install -g jscodeshift
- git clone https://github.com/cpojer/js-codemod.git
- git clone git@github.com:TryGhost/Ghost.git shallow-ghost
- cd shallow-ghost
- jscodeshift -t ../js-codemod/transforms/unchain-variables.js . -v=2
- jscodeshift -t ../js-codemod/transforms/no-vars.js . -v=2
- yarn
- yarn test
- yarn lint / fix various lint errors (almost all indent) by opening files and saving in vscode
- grunt test-regression
- sorted!
- added core/shared to watched folders in grunt
- moved sentry to shared
- moved express initialisation to a shared file
- always set trust proxy + sentry error handler
- use this new express init everywhere, and remove duplicate trust proxy and sentry error handler code
- updated various files I noticed were outdated on my travels around the codebase
- doesn't make any more advanced ES6 changes, this is mostly in the persuit of getting rid of var x = y, z = a; lists at the top of files
closes#11766, refs 7284227f1
- when we changed from host to hostname, more changed than just using the x-forwarded-host if trusted because express req.hostname does not return the port
- this causes issues with an infinite redirect if you try to set a different admin host with a port
- added a test to demonstrate the case, that didn't fail due to an error in the test logic
- switched from redirecting based on req.hostname to using req.vhost.host which has the correct trusted, requested value that we should rely on
- simplified the comparison logic to explicitly compare host with host
no issue
- eslint-plugin-ghost will warn for console.* calls
- we use a mini-hack in the test suite to print out the start time for
Ghost
- we want to keep these for the mean time, so explicitly allow them for
now
- This code was a little verbose, which made it hard to see what was happening (it still is a bit)
- Used destructuring to reduce the code
- Renamed a few variables
- we had urlRedirects, urlRedirects.adminRedirect and adminRedirects
- all do kinda similar things, but for different contexts so for now I've done a minimal renaming for clarity
- and updated some comments!!
- also removed totally unnecessary if res.isAdmin clause, as we don't use that, and it was never true
