More typical in TOTP setups for each token to last 1 minute, and to
allow some older tokens.
Also moved the options setting out of the generate scope in case
verify is called first (unlikely but possible).
closes https://linear.app/tryghost/issue/ENG-1617/
closes https://linear.app/tryghost/issue/ENG-1619/
- updated cookie authenticator's `authenticate` method to accept an `{identification, pasword, token}` object
- if `token` is provided, hit our `PUT /session/verify/` endpoint passing through the token instead of hitting the `POST /session/` endpoint
- added `signin/verify` route
- displays a 2fa code input field, including required attributes for macOS auto-fill from email/messages to work
- uses `session.authenticate({token})` when submitted
- updated signin routine to detect token-required state
- detects a `403` response with a `2FA_TOKEN_REQUIRED` code property when authenticating
- if detected transitions to the `signin/verify` route
refs ENG-1622
Currently unused by the API, this session variable will be used to
confirm whether the user has authenticated their session with an email
OTP. The verified status is not removed on logout, so sessions are now
retained instead of being destroyed.
ref 324211f
- this includes changes to improve package size
Package size was found to be bloated due to expanding i18n strings. We
were packing all i18n strings instead of just the ones relevant to the
package. Thanks to @cathysarisky for identifying this!
ref https://linear.app/tryghost/issue/ENG-1653
- we were always setting a `style="background-color: #123456"` attribute on the buttons but that didn't allow for different button states such as the red failure state to correctly override meaning there was some odd behaviour when hovering
- removed the fixed `style` attribute and adjusted `<GhTaskButton>`
- added `@useAccentColor` prop
- when `@useAccentColor` is true, add the necessary `style` attribute except when showing the failure state
no issue
- The `test:*` commands in `ghost/core` are all implicitly dependent on
the TS packages in the whole monorepo being built, but we hadn't
explicitly declared this dependency to NX.
- Now if you run `yarn nx run ghost:test:e2e` (or any other `test:*`
commands in ghost), NX will know that it needs to rebuild the TS
packages, unless they are cached and haven't changed.
- With this, you should be able to directly clone the repo and run `yarn
nx run ghost:test:e2e` to run e2e tests, without running `yarn dev` or
`yarn nx run-many -t build:ts` first.
- This is especially useful for getting tests to run properly in docker
no issue
- I apparently never added @tryghost/metrics-server as a dependency to
ghost/core/package.json. It worked in most cases as a 'phantom
dependency' — yarn installs all node_modules in a flat structure, so
even though it wasn't a dependency in package.json, it still resolved to
the correct package, as long as the typescript packages were all built
first.
- This passed CI because we explicitly run ts:build on all packages
before running tests, and it worked in production because we build the
TS packages as part of the docker build. However, when trying to run
tests locally, it would sometimes fail unless you explicitly ran nx
run-many -t build:ts at the top level before running the tests.
- Adding it as a dependency in package.json fixes this problem.
ref https://linear.app/tryghost/issue/ONC-387/
With some recent changes, we added validation to unsubscribe URLs to verify the source, allowing us to cut down on spam and improving security, as the underlying key could be re-generated should the need arise. This had the side effect of making unsubscribe URLs difficult to reconstruct when using third-party/downstream integrations, such as ActiveCampaign, which fills a gap in the current Ghost feature set.
Now any authenticated query to `/api/members` will return an `unsubscribe_url` field that can be used directly.
no ref
When running tests, occasionally we'll see some varying sort in the
members api response because members are generally all created with the
same timestamp. While `ObjectId` should be progressive, and our defalut
sort is `ORDER BY created_at desc, id desc`, we still would sometimes
see issues. This ought to remove any flakiness.
no ref
Stubbed expected test errors. In general, we should be expecting these
errors in the tests as we write them as that is the expected behavior
(or that behavior should change).
closes#16748
The members/:member_id/signin_urls endpoint currently only does
cookie-based authentication. When #21249 is merged, turning on 2FA is
going to break any 3rd party processes that use it (including my social
sign-in offering).
This patch gives admin API keys 'read' permission on this endpoint, and
enables 3rd party processes to handle user logins the right way, instead
of via a staff member's email/password.
Migration included. Feedback appreciated.
I have the wrong name on my migration. I can see it doesn't follow the
naming convention, but I'm not sure how the names are generated.
---------
Co-authored-by: Michael Barrett <mike182uk@gmail.com>
ref https://linear.app/tryghost/issue/ONC-433
- due to a regression introduced in commit 871d21a, incoming
recommendations were not rendering in Admin Settings anymore, as they
were marked as deleted
- this commit updates the refresh logic of incoming recommendations on
boot: previously deleted incoming recommendations are refetched, and if
now available, restored
- when a recommendation is restored, we don't send a staff email
notification
ref https://linear.app/tryghost/issue/ONC-433
- due to a regression introduced in commit 871d21a, incoming
recommendations were not rendering in Admin Settings anymore, as they
were marked as deleted
- this commit updates the refresh logic of incoming recommendations on
boot: previously deleted incoming recommendations are refetched, and if
now available, restored
- when a recommendation is restored, we don't send a staff email
notification
- concat is too heavy of a function to call on the hotpath, so we can
just replace it with a native spread, which is much faster
- this cuts ~1.5% from boot time for sites with a lot of posts
No issue
Wrapped a missing /month and /year string. Should work with and without
trials now.
Added a few additional translations for Japanese and French.
Adjusted German - some strings were too long to fit in the layout.
Changed (the German equivalents of) "Start a X day free trial" to
"Select", because the German is just not going to fit on the button.
ref https://linear.app/tryghost/issue/DEV-32/remove-migratejs-script
- we want to switch to using this code path instead of our separate
migrate.js script on Pro
- the main things we're missing are metrics + monitoring for when things
go wrong, so this adds that to the DatabaseStateManager
- this allows us to eventually delete the script without losing
functionality
- the code kept an array of IDs, and would check new entries against the
values of this array
- this algorithm is O(n^2) and became quite slow when the site had a lot
of redirects
- we can do away with this entirely, and just compute the keys of the
redirects to get the IDs
- this speeds up loading redirects by 3x or so
- this will prevent the `ready` variable from being set to true if there
is an error with minification, as we have not correctly generated the
assets yet
- we don't need any of these fields to do URL service calculations, so
we can exclude them from being fetched, which improves performance of
URL service init
- ultimately, we should switch this to an include list to make this more
explicit
- this code is on the hotpath for the URL service and has shown to be
slow for sites with a lot of posts
- this is due to the overhead of the lodash functions we use here
- we can take advantage of how JS executes if-statements and move the
variable into the if-statement, which lazy evaluates it (for the URL
service, this branch is not hit, so it's a big win)
- this cuts about 2% from CPU time
- this code is a hotpath for the URL service and has shown to be slow
for sites with a lot of posts
- this is because of the overhead of lodash
- we can just do away with lodash and use native JS, which has
a negligible performance cost
- this cuts about 5% CPU time during boot of large sites
- moment calls are unbelievably heavy and we should do away with it
where possible
- this code doesn't need moment and we can just use native JS Date here
- this saves about 5% CPU time when booting sites with a lot of posts
- this code has shown to be chronically slow, due to the `Object.assign`
- we don't really need this, as we can just use a normal assign in this
case
- this cuts 15% CPU from boot time for sites with a lot of resources
(posts)
- it turns out we it the false case of this if-statement quite a lot,
and _generateUrl is heavy enough that we should try and do it less
- by moving it into the if-statement, we cut 4% CPU time from boot on
heavy sites
- lodash adds non-negligible runtime to this loop, so we can just
replace it with native JS and cut 3% CPU time from boot for sites
with a large number of posts
- we have to deserialize the values from the DB to turn them into moment + boolean values
- the use of lodash adds unnecessary overhead to the function, and writing it in native JS
is a low faster
- also fixes the naming of the functions to make it clearer in flamegraphs
no issue
- config helpers are required early during boot and it requiring lodash added some unnecessary require+compile time
- switched to using an inlined escapeRegExp function in place of requiring lodash
no issue
- config utils are required early during boot and it requiring lodash added some unnecessary require+compile time
- switched to using native JS for the few lodash methods we used
no issue
- bumped gscan version to provide `skipChecks` flag to `check` function
- added `optimization:themes:skipBootChecks` config flag defaulting to `false` to maintain current behaviour
- updated theme service initialization to use `skipChecks: true` when the config flag is set
- we only want to skip the checks during boot in specific cases to improve performance, they are still useful for general development and any production use-cases where themes get edited directly on the server
- updated our theme validate module to accept and pass through `skipChecks` option
- switched the `isZip` positional argument of `validate.check()` to an options object property to make usage cleaner
- right now, we minify the assets on boot. This is wasteful because they're not even needed
- this commit implements a change which lazy-minifies these assets and
allows for cache invalidation when the theme changes
- it also introduces some middleware that each asset calls to ensure
that the assets are minified before serving
no issue
- avoid accessing `mobiledocLib.mobiledocHtmlRenderer` directly to access the `render()` method in favor of using `mobiledocLib.render()` instead
- standardising on this pattern should help avoid accidentally requiring the full library before we have a need to render
no issue
- the lazy-loading `mobiledocHtmlRenderer` getter was being accessed by the email renderer that gets initialized during boot
- switched the pattern to match our lexical lib where we have a `render()` method that doesn't load the renderer until it's actually needed
- we don't need to require the entire package and this costs 5% of our
boot time
- this commit bumps NQL to the latest version, which fixes the requires
to help with treeshaking and loading less code
- the return type of `getLastModifiedForDatum` is a moment object, and
we're just wrapping it again in another moment call
- moment is very heavy so we shouldn't do it unnecessarily
- this makes boot time 1% quicker of heavy sites
no issue
- Following commit #21190, the following updates were made:
- **Filled missing translations and made style corrections to existing
translations to improve clarity and consistency.**
- Total new translations added: 29 entries filled.
- Existing translations improved: 7 entries updated for better
consistency and clarity.
no issue
- Fixed syntax error from "Comenzar" to "Empezar", in the context given it
is "Empezar", both are synonyms, but it depends on the use of the whole
sentence
no ref
Re-translate the correct words in Vietnamese and adapted new commits:
*Improved comments UI styles (#20957)
*Updated comment report modal (#21084)
no ref
Added Locale for Swahili Language🎊!
This pull request adds support for the Swahili language locale to our
project. As a fluent Swahili speaker (Swahili is my native language) and
i am from Tanzania, I have ensured that all translations are accurate
and culturally appropriate.
Changes
-Added sw locale files for Swahili
-Included translations for all the json files for comments.json ,
ghost.json , portal.json ,signup-form.json
Why
Swahili is a widely spoken language in East Africa, with over 80 million
native speakers and an estimated 200 million speakers in total. Adding
Swahili support will make our project more accessible to this large and
growing user base
---------
Co-authored-by: Steve Larson <9larsons@gmail.com>
no ref
- Added some translations for Croatian
- Provided to me by a customer of ours
- added this for them as they mentioned they were struggling with
Github.
no ref
We've shipped quite a few updates to our i18n package these past couple
weeks and are due for an update to the consuming packages. Special
thanks to our community contributors for the translations, and in
particular to @cathysarisky for their continued effort!
This PR contains the following updates:
| Package | Type | Update | Change |
|---|---|---|---|
|
[parse-prometheus-text-format](https://redirect.github.com/yunyu/parse-prometheus-text-format)
| devDependencies | pin | [`^1.1.1` ->
`1.1.1`](https://renovatebot.com/diffs/npm/parse-prometheus-text-format/1.1.1/1.1.1)
|
Add the preset `:preserveSemverRanges` to your config if you don't want
to pin your dependencies.
---
### Configuration
📅 **Schedule**: Branch creation - "every weekday" (UTC), Automerge - At
any time (no schedule defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.
♻ **Rebasing**: Whenever PR is behind base branch, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box
---
This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/TryGhost/Ghost).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOC45Ny4wIiwidXBkYXRlZEluVmVyIjoiMzguOTcuMCIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOltdfQ==-->
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
ref
https://linear.app/tryghost/issue/ENG-1505/add-prometheus-metrics-server-to-allow-monitoring-ghost-metrics
# Summary
This commit includes two main components: a prometheus client class to
collect metrics from Ghost, and a standalone metrics server that exposes
a /metrics endpoint at a separate port (9416 by default) from the main
Ghost app.
The prometheus client is a very thin wrapper around
[prom-client](https://github.com/siimon/prom-client). We could use
prom-client directly, but this approach should make it easier to switch
to a different prometheus client package (or make our own) if we ever
need to down the line.
The list of default metrics this enables is specified in an e2e test
[here](https://github.com/TryGhost/Ghost/pull/21192/files#diff-ebc52236be2cd14b40be89220ae961f48d3f837693f7d1da76db292348915941R66-R92).
This also gives us the ability to create and collect custom metrics,
although none are included in this commit yet.
# Configuration
The prometheus client and the metrics server are both disabled by
default, but can be enabled by setting the metrics_server:enabled flag
to true.
You can also define a custom host and port using `metrics_server:host`
and `metrics_server:port`.
## Why not expose the /metrics endpoint in one of the existing express
apps?
The standalone express app exists for two main reasons:
1. We don't want these metrics to be public, and the easiest way to
accomplish that is to expose the /metrics endpoint at a different port
that won't be exposed to the internet.
2. Creating a standalone express instance decouples the metrics endpoint
from the Ghost server, so if Ghost is not responding for whatever
reason, we should still be able to scrape metrics to understand what's
going on internally.
## Impact on Boot & Shut down time
The prometheus client is initialized early in the boot process so we can
collect metrics during the boot sequence. Testing locally has shown that
this increases boot time by ~20ms. The metrics server which exposes the
/metrics endpoint is not initialized until after the background
services, and it is not awaited, to avoid impacting boot time. None of
this code, including the requires, will run if the
metrics_server:enabled flag is set to false (or not set).
Shutting down the metrics server is added as a cleanup task for the main
Ghost server instance, and is setup to shut down with 0 grace period to
avoid impacting shut down time.
no issue
- These two tests were relying on the ordering of the posts in the posts
list, which can vary when running the test suite all together (as we do
in CI), since multiple tests can be running and creating/deleting posts
in parallel.
- This commit navigates directly to the post's URL to avoid this
problem.
no ref
Expose (some) Portal error strings for translations
💩This is a somewhat hacky (but test-passing and individual inspection
passing) solution to the way Portal handles errors. Or rather, the
half-dozen ways Portal handles errors.
Passing 't' around with context and state, and occasionally recreating
it from the site object. Yes, I am also somewhat horrified, but a better
implementation will need a major rewrite of Portal.
Addresses errors in both the popover React app and in the
data-attributes.
There are probably more. Since Portal exposes raw API responses in some
places, it's hard to enumerate everything that /might/ end up being
client-facing, but at least I've gotten the ones that I've commonly
seen.
Improvements very welcome.
no issue
Initial pass at refactoring the tests and making each independent so they aren't relying on data from previous tests.
- DRYed up some repeated API requests by extracting `testPostComment()`, `testGetComments()`, and `assertAuthorEmailSent()`
- removed data inter-dependency across tests
- truncated the tables under test before each test is run
- added suite of db fns to populate the database without having to do so via API requests
- updated all tests to include their own data setup calls rather than relying on fixtures and modifications from earlier tests
no ref
We had an instance where this was a ms off and I should've used mock
timers when I first wrote this. This should prevent any rare clock
mishaps.
no ref
Translated/updated all relevant strings in the search & portal feature
to Bahasa Indonesia.
---------
Co-authored-by: Cathy Sarisky <42299862+cathysarisky@users.noreply.github.com>
ref https://linear.app/tryghost/issue/ANAL-96/data-discrepancy-between-charts-when-filtering
- This adds a set of tests to describe what the data should look like when we filter on various values
- We have tests for source and browser which are pulled from different MVs
- The result files are generated using ./scripts/gen_test_results.sh, and then manually verified
- We know they are not yet fully correct
- Added yarn command to update TB CLI, as that needs doing frequently and I can never remember the command
- Improved safety & usability of tinybird test script by ensuring branches are correctly created before running & adding optional delete
- Updated tinybird test to warn only for sanity check as that's not always a valid check (Will prob remove soon)
- Improved output of tinybird test script on failure, so that the diff is readable and closer to what git shows you
- Added tool to convert tinybird ndjson to csv to make it easier to bring the data into google sheets for verifying numbers
no issue
- This test file starts a Ghost server, but doesn't close it, which can
cause other tests to fail when they try to start an instance of Ghost,
with an `EADDRINUSE` error.
- This change closes the server in the `after` hook
no ref
This PR adds the translations for `search.js`. It's a bit opinionated in
that ...
- it uses the Anglicism "Posts" to ... "translate" ... "Posts" 😆;
- it uses an asterisk to make sure we address all genders when
translating "authors".
Happy to discuss. ☺️
no ref
This is a merge & tidy up of [[ayangizzat](https://github.com/ayangizzat)] 's original (https://github.com/TryGhost/Ghost/pull/20698), which was missing the changes to locale, causing folders & files not to generate correctly. All credit goes to @ayangizzat , and thank you for your work!
Kazakh-speaking help still needed for search.json, which is untranslated, and the addition of a few other new strings, and refinements anywhere needed. Please add comments if not yet merged, or open a new PR if already merged.
---------
Co-authored-by: ayangizzt
no ref
We've had several PRs for new locales that forgot to update i18n.js,
which causes them not to get regenerated by yarn translate. This PR
checks for that error, and also the reverse one where they update i18n
but we're missing folders. (Never expected to error, because 'yarn test'
is defined to include 'yarn translate', which generates the folders.)
ref 86d61304b1
ref https://linear.app/tryghost/issue/ONC-323
- added `tracked()` to our proxy model object properties
- fixes default data always showing when opening the modal
- fixed data push after completing modal
- `post.tiers` is set up as an attribute in Admin rather than a relationship
- fixes incorrect tiers list showing when the change access modal is opened again after changing access before the post is re-fetched from the API
- fixed flash of failure button state when saving modal changes
- expanded tests to cover tiers selection
[ANAL-95](https://linear.app/tryghost/issue/ANAL-95/internal-beta-qa)
Various design refinements and fixes for the Stats page:
- Updated scroll area in detail modals so that the Close button and the footer is never outside the viewport
- The detail modal didn't close after clicking on the filter values
- "Show all" button was displayed also when there were no new items in the detail modal
- Dropdown styles needed a visual update: the toggles were way too huge and inconsistent with other dropdowns
- If no audience was selected we still showed stats. Now it's displaying the default empty screen in this case
- Click through filter indicators had low discoverability
- Technical data styles needed some love: changed the alignment and color scheme
- Mobile size viewports were not handled
- The google favicon API returned 404 many times for sources. Swapped the service for another one that returns favicons more reliably
- Default favicon was not handled. Now it comes from static.ghost.org
no ref
This very small helper adds {{content_api_key}} to the collection of handlebars helpers available to theme creators. This will make it easier for themes to access the content API key, without either requiring the user to get it from the integrations page and input it on the theme setting page or resorting to JavaScript to read it from one of the built-in script tag attributes -- both ugly workarounds.
ref https://linear.app/tryghost/issue/ONC-323
After changing a post's access via the posts list context menu, creating new posts or members would not work correctly.
- the issue stemmed from `this.post.set('currentState.parentState.isNew', false);` that was called when changing a post's access level, after that all Ember Data models created from the store would have `isNew: false` causing Ember Data to attempt a PUT request to update the not-yet-created model rather than a POST request to create it
- we were only using a real post model instance in order to run validations against the post access level settings but we can do that just as easily by creating a new object and injecting our validation mixin
no ref
This PR adds and adjusts various German translations in `portal.json`.
I'm a native speaker and have made adjustments to the best of my
knowledge. As in #21163, the gender variations are a bit opinionated and
I'm happy to discuss.
fixes#21104
This PR corrects a single translation. The German translations for "sign
up" and "sign in" _can_ both be "anmelden", but are confusing when used
together. So here we're switch to "register now" and "sign in" to make
the distinction clearer. IMHO even in English "register now" and "sign
in" might be a better (more distinguishable) combination, but it's much
harsher in German.
no ref
This PR restores a few strings that were renamed during the time the
Greek strings were not being correctly processed.
---------
Co-authored-by: Vikas Potluri <vikaspotluri123.github@gmail.com>
ref https://linear.app/tryghost/issue/ONC-323
- we're sometimes seeing our force-refresh failing when Ember Data gets into a bad state but we're not sure why so this log should tell us if it's the browser's native "leave site" modal that is preventing the refresh
- updated the `onbeforeunload` event handler to match modern JS approach
- modern browsers use `event.preventDefault()` to show their dialog
- older browsers use `event.returnValue = true` (this is what our old string return was triggering)
- no browser supports a custom message in the native dialog
- TODO: make these run in CI
- Right now you run them by running `yarn tb` and then `./script/branch_and_test.sh`
- These are snapshot tests that check we get the desired result
Co-authored-by: alejandromav <hi@alejandromav.com>
- Apparently several PRs for new languages were merged without the
corresponding additions to i18n.js, which has caused new strings not to
be generated for those languages, and missing translations in the
frontend, even though we had them in the .json file.
- I've added language codes for all languages that had at least some
strings translated in i18n/locales, and have run yarn translate to
create missing strings and files for these languages.
New language codes for:
* Arabic
* Swiss German
* Greek
* Estonian
* Macedonian
* Serbian (Cyrillic)
* Thai
