More typical in TOTP setups for each token to last 1 minute, and to
allow some older tokens.
Also moved the options setting out of the generate scope in case
verify is called first (unlikely but possible).
closes https://linear.app/tryghost/issue/ENG-1617/
closes https://linear.app/tryghost/issue/ENG-1619/
- updated cookie authenticator's `authenticate` method to accept an `{identification, pasword, token}` object
- if `token` is provided, hit our `PUT /session/verify/` endpoint passing through the token instead of hitting the `POST /session/` endpoint
- added `signin/verify` route
- displays a 2fa code input field, including required attributes for macOS auto-fill from email/messages to work
- uses `session.authenticate({token})` when submitted
- updated signin routine to detect token-required state
- detects a `403` response with a `2FA_TOKEN_REQUIRED` code property when authenticating
- if detected transitions to the `signin/verify` route
refs ENG-1622
Currently unused by the API, this session variable will be used to
confirm whether the user has authenticated their session with an email
OTP. The verified status is not removed on logout, so sessions are now
retained instead of being destroyed.
ref 324211f
- this includes changes to improve package size
Package size was found to be bloated due to expanding i18n strings. We
were packing all i18n strings instead of just the ones relevant to the
package. Thanks to @cathysarisky for identifying this!
ref https://linear.app/tryghost/issue/ENG-1653
- we were always setting a `style="background-color: #123456"` attribute on the buttons but that didn't allow for different button states such as the red failure state to correctly override meaning there was some odd behaviour when hovering
- removed the fixed `style` attribute and adjusted `<GhTaskButton>`
- added `@useAccentColor` prop
- when `@useAccentColor` is true, add the necessary `style` attribute except when showing the failure state
no issue
- The `test:*` commands in `ghost/core` are all implicitly dependent on
the TS packages in the whole monorepo being built, but we hadn't
explicitly declared this dependency to NX.
- Now if you run `yarn nx run ghost:test:e2e` (or any other `test:*`
commands in ghost), NX will know that it needs to rebuild the TS
packages, unless they are cached and haven't changed.
- With this, you should be able to directly clone the repo and run `yarn
nx run ghost:test:e2e` to run e2e tests, without running `yarn dev` or
`yarn nx run-many -t build:ts` first.
- This is especially useful for getting tests to run properly in docker
no issue
- I apparently never added @tryghost/metrics-server as a dependency to
ghost/core/package.json. It worked in most cases as a 'phantom
dependency' — yarn installs all node_modules in a flat structure, so
even though it wasn't a dependency in package.json, it still resolved to
the correct package, as long as the typescript packages were all built
first.
- This passed CI because we explicitly run ts:build on all packages
before running tests, and it worked in production because we build the
TS packages as part of the docker build. However, when trying to run
tests locally, it would sometimes fail unless you explicitly ran nx
run-many -t build:ts at the top level before running the tests.
- Adding it as a dependency in package.json fixes this problem.
ref https://linear.app/tryghost/issue/ONC-387/
With some recent changes, we added validation to unsubscribe URLs to verify the source, allowing us to cut down on spam and improving security, as the underlying key could be re-generated should the need arise. This had the side effect of making unsubscribe URLs difficult to reconstruct when using third-party/downstream integrations, such as ActiveCampaign, which fills a gap in the current Ghost feature set.
Now any authenticated query to `/api/members` will return an `unsubscribe_url` field that can be used directly.
no ref
When running tests, occasionally we'll see some varying sort in the
members api response because members are generally all created with the
same timestamp. While `ObjectId` should be progressive, and our defalut
sort is `ORDER BY created_at desc, id desc`, we still would sometimes
see issues. This ought to remove any flakiness.
no ref
Stubbed expected test errors. In general, we should be expecting these
errors in the tests as we write them as that is the expected behavior
(or that behavior should change).
closes#16748
The members/:member_id/signin_urls endpoint currently only does
cookie-based authentication. When #21249 is merged, turning on 2FA is
going to break any 3rd party processes that use it (including my social
sign-in offering).
This patch gives admin API keys 'read' permission on this endpoint, and
enables 3rd party processes to handle user logins the right way, instead
of via a staff member's email/password.
Migration included. Feedback appreciated.
I have the wrong name on my migration. I can see it doesn't follow the
naming convention, but I'm not sure how the names are generated.
---------
Co-authored-by: Michael Barrett <mike182uk@gmail.com>
ref https://linear.app/tryghost/issue/ONC-433
- due to a regression introduced in commit 871d21a, incoming
recommendations were not rendering in Admin Settings anymore, as they
were marked as deleted
- this commit updates the refresh logic of incoming recommendations on
boot: previously deleted incoming recommendations are refetched, and if
now available, restored
- when a recommendation is restored, we don't send a staff email
notification
ref https://linear.app/tryghost/issue/ONC-433
- due to a regression introduced in commit 871d21a, incoming
recommendations were not rendering in Admin Settings anymore, as they
were marked as deleted
- this commit updates the refresh logic of incoming recommendations on
boot: previously deleted incoming recommendations are refetched, and if
now available, restored
- when a recommendation is restored, we don't send a staff email
notification
- concat is too heavy of a function to call on the hotpath, so we can
just replace it with a native spread, which is much faster
- this cuts ~1.5% from boot time for sites with a lot of posts
No issue
Wrapped a missing /month and /year string. Should work with and without
trials now.
Added a few additional translations for Japanese and French.
Adjusted German - some strings were too long to fit in the layout.
Changed (the German equivalents of) "Start a X day free trial" to
"Select", because the German is just not going to fit on the button.
ref https://linear.app/tryghost/issue/DEV-32/remove-migratejs-script
- we want to switch to using this code path instead of our separate
migrate.js script on Pro
- the main things we're missing are metrics + monitoring for when things
go wrong, so this adds that to the DatabaseStateManager
- this allows us to eventually delete the script without losing
functionality
- the code kept an array of IDs, and would check new entries against the
values of this array
- this algorithm is O(n^2) and became quite slow when the site had a lot
of redirects
- we can do away with this entirely, and just compute the keys of the
redirects to get the IDs
- this speeds up loading redirects by 3x or so
- this will prevent the `ready` variable from being set to true if there
is an error with minification, as we have not correctly generated the
assets yet
- we don't need any of these fields to do URL service calculations, so
we can exclude them from being fetched, which improves performance of
URL service init
- ultimately, we should switch this to an include list to make this more
explicit
- this code is on the hotpath for the URL service and has shown to be
slow for sites with a lot of posts
- this is due to the overhead of the lodash functions we use here
- we can take advantage of how JS executes if-statements and move the
variable into the if-statement, which lazy evaluates it (for the URL
service, this branch is not hit, so it's a big win)
- this cuts about 2% from CPU time
- this code is a hotpath for the URL service and has shown to be slow
for sites with a lot of posts
- this is because of the overhead of lodash
- we can just do away with lodash and use native JS, which has
a negligible performance cost
- this cuts about 5% CPU time during boot of large sites
- moment calls are unbelievably heavy and we should do away with it
where possible
- this code doesn't need moment and we can just use native JS Date here
- this saves about 5% CPU time when booting sites with a lot of posts
- this code has shown to be chronically slow, due to the `Object.assign`
- we don't really need this, as we can just use a normal assign in this
case
- this cuts 15% CPU from boot time for sites with a lot of resources
(posts)
- it turns out we it the false case of this if-statement quite a lot,
and _generateUrl is heavy enough that we should try and do it less
- by moving it into the if-statement, we cut 4% CPU time from boot on
heavy sites
- lodash adds non-negligible runtime to this loop, so we can just
replace it with native JS and cut 3% CPU time from boot for sites
with a large number of posts
- we have to deserialize the values from the DB to turn them into moment + boolean values
- the use of lodash adds unnecessary overhead to the function, and writing it in native JS
is a low faster
- also fixes the naming of the functions to make it clearer in flamegraphs
no issue
- config helpers are required early during boot and it requiring lodash added some unnecessary require+compile time
- switched to using an inlined escapeRegExp function in place of requiring lodash
no issue
- config utils are required early during boot and it requiring lodash added some unnecessary require+compile time
- switched to using native JS for the few lodash methods we used
no issue
- bumped gscan version to provide `skipChecks` flag to `check` function
- added `optimization:themes:skipBootChecks` config flag defaulting to `false` to maintain current behaviour
- updated theme service initialization to use `skipChecks: true` when the config flag is set
- we only want to skip the checks during boot in specific cases to improve performance, they are still useful for general development and any production use-cases where themes get edited directly on the server
- updated our theme validate module to accept and pass through `skipChecks` option
- switched the `isZip` positional argument of `validate.check()` to an options object property to make usage cleaner
- right now, we minify the assets on boot. This is wasteful because they're not even needed
- this commit implements a change which lazy-minifies these assets and
allows for cache invalidation when the theme changes
- it also introduces some middleware that each asset calls to ensure
that the assets are minified before serving
no issue
- avoid accessing `mobiledocLib.mobiledocHtmlRenderer` directly to access the `render()` method in favor of using `mobiledocLib.render()` instead
- standardising on this pattern should help avoid accidentally requiring the full library before we have a need to render
no issue
- the lazy-loading `mobiledocHtmlRenderer` getter was being accessed by the email renderer that gets initialized during boot
- switched the pattern to match our lexical lib where we have a `render()` method that doesn't load the renderer until it's actually needed
- we don't need to require the entire package and this costs 5% of our
boot time
- this commit bumps NQL to the latest version, which fixes the requires
to help with treeshaking and loading less code
- the return type of `getLastModifiedForDatum` is a moment object, and
we're just wrapping it again in another moment call
- moment is very heavy so we shouldn't do it unnecessarily
- this makes boot time 1% quicker of heavy sites
no issue
- Following commit #21190, the following updates were made:
- **Filled missing translations and made style corrections to existing
translations to improve clarity and consistency.**
- Total new translations added: 29 entries filled.
- Existing translations improved: 7 entries updated for better
consistency and clarity.
no issue
- Fixed syntax error from "Comenzar" to "Empezar", in the context given it
is "Empezar", both are synonyms, but it depends on the use of the whole
sentence
no ref
Re-translate the correct words in Vietnamese and adapted new commits:
*Improved comments UI styles (#20957)
*Updated comment report modal (#21084)
no ref
Added Locale for Swahili Language🎊!
This pull request adds support for the Swahili language locale to our
project. As a fluent Swahili speaker (Swahili is my native language) and
i am from Tanzania, I have ensured that all translations are accurate
and culturally appropriate.
Changes
-Added sw locale files for Swahili
-Included translations for all the json files for comments.json ,
ghost.json , portal.json ,signup-form.json
Why
Swahili is a widely spoken language in East Africa, with over 80 million
native speakers and an estimated 200 million speakers in total. Adding
Swahili support will make our project more accessible to this large and
growing user base
---------
Co-authored-by: Steve Larson <9larsons@gmail.com>
no ref
- Added some translations for Croatian
- Provided to me by a customer of ours
- added this for them as they mentioned they were struggling with
Github.
no ref
We've shipped quite a few updates to our i18n package these past couple
weeks and are due for an update to the consuming packages. Special
thanks to our community contributors for the translations, and in
particular to @cathysarisky for their continued effort!
This PR contains the following updates:
| Package | Type | Update | Change |
|---|---|---|---|
|
[parse-prometheus-text-format](https://redirect.github.com/yunyu/parse-prometheus-text-format)
| devDependencies | pin | [`^1.1.1` ->
`1.1.1`](https://renovatebot.com/diffs/npm/parse-prometheus-text-format/1.1.1/1.1.1)
|
Add the preset `:preserveSemverRanges` to your config if you don't want
to pin your dependencies.
---
### Configuration
📅 **Schedule**: Branch creation - "every weekday" (UTC), Automerge - At
any time (no schedule defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.
♻ **Rebasing**: Whenever PR is behind base branch, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box
---
This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/TryGhost/Ghost).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOC45Ny4wIiwidXBkYXRlZEluVmVyIjoiMzguOTcuMCIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOltdfQ==-->
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
ref
https://linear.app/tryghost/issue/ENG-1505/add-prometheus-metrics-server-to-allow-monitoring-ghost-metrics
# Summary
This commit includes two main components: a prometheus client class to
collect metrics from Ghost, and a standalone metrics server that exposes
a /metrics endpoint at a separate port (9416 by default) from the main
Ghost app.
The prometheus client is a very thin wrapper around
[prom-client](https://github.com/siimon/prom-client). We could use
prom-client directly, but this approach should make it easier to switch
to a different prometheus client package (or make our own) if we ever
need to down the line.
The list of default metrics this enables is specified in an e2e test
[here](https://github.com/TryGhost/Ghost/pull/21192/files#diff-ebc52236be2cd14b40be89220ae961f48d3f837693f7d1da76db292348915941R66-R92).
This also gives us the ability to create and collect custom metrics,
although none are included in this commit yet.
# Configuration
The prometheus client and the metrics server are both disabled by
default, but can be enabled by setting the metrics_server:enabled flag
to true.
You can also define a custom host and port using `metrics_server:host`
and `metrics_server:port`.
## Why not expose the /metrics endpoint in one of the existing express
apps?
The standalone express app exists for two main reasons:
1. We don't want these metrics to be public, and the easiest way to
accomplish that is to expose the /metrics endpoint at a different port
that won't be exposed to the internet.
2. Creating a standalone express instance decouples the metrics endpoint
from the Ghost server, so if Ghost is not responding for whatever
reason, we should still be able to scrape metrics to understand what's
going on internally.
## Impact on Boot & Shut down time
The prometheus client is initialized early in the boot process so we can
collect metrics during the boot sequence. Testing locally has shown that
this increases boot time by ~20ms. The metrics server which exposes the
/metrics endpoint is not initialized until after the background
services, and it is not awaited, to avoid impacting boot time. None of
this code, including the requires, will run if the
metrics_server:enabled flag is set to false (or not set).
Shutting down the metrics server is added as a cleanup task for the main
Ghost server instance, and is setup to shut down with 0 grace period to
avoid impacting shut down time.
no issue
- These two tests were relying on the ordering of the posts in the posts
list, which can vary when running the test suite all together (as we do
in CI), since multiple tests can be running and creating/deleting posts
in parallel.
- This commit navigates directly to the post's URL to avoid this
problem.
no ref
Expose (some) Portal error strings for translations
💩This is a somewhat hacky (but test-passing and individual inspection
passing) solution to the way Portal handles errors. Or rather, the
half-dozen ways Portal handles errors.
Passing 't' around with context and state, and occasionally recreating
it from the site object. Yes, I am also somewhat horrified, but a better
implementation will need a major rewrite of Portal.
Addresses errors in both the popover React app and in the
data-attributes.
There are probably more. Since Portal exposes raw API responses in some
places, it's hard to enumerate everything that /might/ end up being
client-facing, but at least I've gotten the ones that I've commonly
seen.
Improvements very welcome.
no issue
Initial pass at refactoring the tests and making each independent so they aren't relying on data from previous tests.
- DRYed up some repeated API requests by extracting `testPostComment()`, `testGetComments()`, and `assertAuthorEmailSent()`
- removed data inter-dependency across tests
- truncated the tables under test before each test is run
- added suite of db fns to populate the database without having to do so via API requests
- updated all tests to include their own data setup calls rather than relying on fixtures and modifications from earlier tests
no ref
We had an instance where this was a ms off and I should've used mock
timers when I first wrote this. This should prevent any rare clock
mishaps.
no ref
Translated/updated all relevant strings in the search & portal feature
to Bahasa Indonesia.
---------
Co-authored-by: Cathy Sarisky <42299862+cathysarisky@users.noreply.github.com>
ref https://linear.app/tryghost/issue/ANAL-96/data-discrepancy-between-charts-when-filtering
- This adds a set of tests to describe what the data should look like when we filter on various values
- We have tests for source and browser which are pulled from different MVs
- The result files are generated using ./scripts/gen_test_results.sh, and then manually verified
- We know they are not yet fully correct
- Added yarn command to update TB CLI, as that needs doing frequently and I can never remember the command
- Improved safety & usability of tinybird test script by ensuring branches are correctly created before running & adding optional delete
- Updated tinybird test to warn only for sanity check as that's not always a valid check (Will prob remove soon)
- Improved output of tinybird test script on failure, so that the diff is readable and closer to what git shows you
- Added tool to convert tinybird ndjson to csv to make it easier to bring the data into google sheets for verifying numbers
no issue
- This test file starts a Ghost server, but doesn't close it, which can
cause other tests to fail when they try to start an instance of Ghost,
with an `EADDRINUSE` error.
- This change closes the server in the `after` hook
no ref
This PR adds the translations for `search.js`. It's a bit opinionated in
that ...
- it uses the Anglicism "Posts" to ... "translate" ... "Posts" 😆;
- it uses an asterisk to make sure we address all genders when
translating "authors".
Happy to discuss. ☺️
no ref
This is a merge & tidy up of [[ayangizzat](https://github.com/ayangizzat)] 's original (https://github.com/TryGhost/Ghost/pull/20698), which was missing the changes to locale, causing folders & files not to generate correctly. All credit goes to @ayangizzat , and thank you for your work!
Kazakh-speaking help still needed for search.json, which is untranslated, and the addition of a few other new strings, and refinements anywhere needed. Please add comments if not yet merged, or open a new PR if already merged.
---------
Co-authored-by: ayangizzt
no ref
We've had several PRs for new locales that forgot to update i18n.js,
which causes them not to get regenerated by yarn translate. This PR
checks for that error, and also the reverse one where they update i18n
but we're missing folders. (Never expected to error, because 'yarn test'
is defined to include 'yarn translate', which generates the folders.)
ref 86d61304b1
ref https://linear.app/tryghost/issue/ONC-323
- added `tracked()` to our proxy model object properties
- fixes default data always showing when opening the modal
- fixed data push after completing modal
- `post.tiers` is set up as an attribute in Admin rather than a relationship
- fixes incorrect tiers list showing when the change access modal is opened again after changing access before the post is re-fetched from the API
- fixed flash of failure button state when saving modal changes
- expanded tests to cover tiers selection
[ANAL-95](https://linear.app/tryghost/issue/ANAL-95/internal-beta-qa)
Various design refinements and fixes for the Stats page:
- Updated scroll area in detail modals so that the Close button and the footer is never outside the viewport
- The detail modal didn't close after clicking on the filter values
- "Show all" button was displayed also when there were no new items in the detail modal
- Dropdown styles needed a visual update: the toggles were way too huge and inconsistent with other dropdowns
- If no audience was selected we still showed stats. Now it's displaying the default empty screen in this case
- Click through filter indicators had low discoverability
- Technical data styles needed some love: changed the alignment and color scheme
- Mobile size viewports were not handled
- The google favicon API returned 404 many times for sources. Swapped the service for another one that returns favicons more reliably
- Default favicon was not handled. Now it comes from static.ghost.org
no ref
This very small helper adds {{content_api_key}} to the collection of handlebars helpers available to theme creators. This will make it easier for themes to access the content API key, without either requiring the user to get it from the integrations page and input it on the theme setting page or resorting to JavaScript to read it from one of the built-in script tag attributes -- both ugly workarounds.
ref https://linear.app/tryghost/issue/ONC-323
After changing a post's access via the posts list context menu, creating new posts or members would not work correctly.
- the issue stemmed from `this.post.set('currentState.parentState.isNew', false);` that was called when changing a post's access level, after that all Ember Data models created from the store would have `isNew: false` causing Ember Data to attempt a PUT request to update the not-yet-created model rather than a POST request to create it
- we were only using a real post model instance in order to run validations against the post access level settings but we can do that just as easily by creating a new object and injecting our validation mixin
no ref
This PR adds and adjusts various German translations in `portal.json`.
I'm a native speaker and have made adjustments to the best of my
knowledge. As in #21163, the gender variations are a bit opinionated and
I'm happy to discuss.
fixes#21104
This PR corrects a single translation. The German translations for "sign
up" and "sign in" _can_ both be "anmelden", but are confusing when used
together. So here we're switch to "register now" and "sign in" to make
the distinction clearer. IMHO even in English "register now" and "sign
in" might be a better (more distinguishable) combination, but it's much
harsher in German.
no ref
This PR restores a few strings that were renamed during the time the
Greek strings were not being correctly processed.
---------
Co-authored-by: Vikas Potluri <vikaspotluri123.github@gmail.com>
ref https://linear.app/tryghost/issue/ONC-323
- we're sometimes seeing our force-refresh failing when Ember Data gets into a bad state but we're not sure why so this log should tell us if it's the browser's native "leave site" modal that is preventing the refresh
- updated the `onbeforeunload` event handler to match modern JS approach
- modern browsers use `event.preventDefault()` to show their dialog
- older browsers use `event.returnValue = true` (this is what our old string return was triggering)
- no browser supports a custom message in the native dialog
- TODO: make these run in CI
- Right now you run them by running `yarn tb` and then `./script/branch_and_test.sh`
- These are snapshot tests that check we get the desired result
Co-authored-by: alejandromav <hi@alejandromav.com>
- Apparently several PRs for new languages were merged without the
corresponding additions to i18n.js, which has caused new strings not to
be generated for those languages, and missing translations in the
frontend, even though we had them in the .json file.
- I've added language codes for all languages that had at least some
strings translated in i18n/locales, and have run yarn translate to
create missing strings and files for these languages.
New language codes for:
* Arabic
* Swiss German
* Greek
* Estonian
* Macedonian
* Serbian (Cyrillic)
* Thai
no issue
- This commit removes all OpenTelemetry related code and dependencies
from Ghost.
- The initial implementation was done as a POC but it raised some
performance concerns with boot time, so we never actually enabled it
widely.
- We can revisit this in the future, but in the meantime it's just
adding unnecessary dependencies and bloating the codebase.
ref https://ghost-foundation.sentry.io/issues/5908152800/
- In the current state, we are maintaining an 'index' key for all
revisions in localStorage. This gives us quick and easy access to all
the revisions in localStorage, but it requires additional "bookkeeping"
to update the index each time we add/remove a key.
- In some obscure edge cases, this results in the `remove()` method
throwing a `QuotaExceededError` (since removing a revision also requires
updating the index with `localStorage.setItem()`). If the `remove()`
call fails, we are sort of stuck — the only way to reduce our storage
usage is to remove items, but if the `remove()` method throws errors, we
can't do that.
- This change removes the whole index concept, and instead loops over
all the keys in localStorage, filtering by the prefix to find all our
revisions. This makes the `keys()` method slightly more complex, as it
has to filter out keys in localStorage that aren't related to revisions,
but it simplifies saving and removing revisions.
- Critically, this also means that `remove()` should never throw a
`QuotaExceededError`, since it no longer needs to call
`localStorage.setItem()` — it now simply calls
`localStorage.removeItem()` for the revision, which should never fail.
ref https://linear.app/tryghost/issue/AP-438
This is going to allow us to load the activitypub package from the jsdelivr
cdn, which means we can release new versions without releasing the admin.
no issue
Either a node or macOS update resulted in our broken image upload tests causing native stack traces:
```
# /Users/kevin/.nvm/versions/node/v20.16.0/bin/node[8841]: static void node::Blob::ToSlice(const FunctionCallbackInfo<v8::Value> &) at ../src/node_blob.cc:248
# Assertion failed: args[1]->IsUint32()
```
- updated our `blob.slice()` calls to ensure the second argument is always an integer rather than possibly a float
This pull request adds Estonian (et) language translations to Ghost.
Translated files:
- comments.json
- ghost.json
- portal.json
- search.json
- signup-form.json
Changes:
- Created new 'et' folder in the locales directory
- Translated all strings from English to Estonian in the above files
I'm a native Estonian speaker, but if there are any questions about
specific translations, please let me know.
Thank you for considering this contribution to make Ghost more
accessible to Estonian-speaking users!
---------
Co-authored-by: Ronald Langeveld <hi@ronaldlangeveld.com>
A needed update of the Bulgarian version. Fixing some errors. Made some
improvements. And some new strings were translated.
---------
Co-authored-by: Ronald Langeveld <hi@ronaldlangeveld.com>
ref ONC-364
- Adds a condition to check whether the record is deleted or if deleting
is in progress before firing the `setFeatureImageCaption`.
- Adds tests. Managed to reproduce the issue using tests.
no issue
- Added Sentry logs to capture how often we are running into
`QuotaExceededErrors` when saving local revisions to localStorage, to
help in deciding if localStorage is sufficient, or if we need to expand
to e.g. IndexedDB.
- Also adds some handling to ignore errors when calling
`localStorage.setItem()` elsewhere in the admin app to avoid crashing if
localStorage isn't supported or the quota is exceeded.
ref https://app.incident.io/ghost/incidents/107
ref cc88757e2a
- added new path in admin `/restore`
- added basic ui for restoring posts from local storage
- added limits for # of revisions for posts with an `id` (5 revisions)
This commit adds a simple UI for restoring posts in case of data loss.
This is a backstop for very rare situations in which it seems Ember gets
into a conflicted state. See ref'd commit for more info. Clicking
'Restore' will create a new post with the saved off content.
ref
https://linear.app/tryghost/issue/ANAL-60/click-through-filtering-for-sources
- In our stats page we use the referrer without a protocol or www, that
is the pure domain as our source that we output
- Meanwhile all the data pipelines had the full url as the referrer
passed through
- When we come to add clickthroughs/filtering, we'll need to use this
value to filter the data. If we have a different value locally in the UI
to what is in the DB, we won't be able to make the filters match
- Also, we pay for everything we store, and this removes all the
https:// and www. data
- Whilst we are in development, we can safely make changes to all
aspects of our pipeline without worrying
- This is because currently, it's safe to delete all data and start over
- This script removes everything excepts the analytics_events
datasource, and then recreates everything fresh, repopulating from the
datasource where possible
- This shouldn't be used after tinybird is in production, we need a
better change process
ref https://github.com/TryGhost/Ghost/issues/16628
This adds translation support to search, which should be the last missing piece of i18n support for Ghost's frontend 🎉
- Translation (t) helper added to sodo-search.
- Ghost head tweaked to include data-locale.
- All (I hope) strings in sodo-search wrapped in the t helper.
- Possibly poor-quality French translation strings added.
---------
Co-authored-by: Vikas Potluri <vikaspotluri123.github@gmail.com>
ref
https://linear.app/tryghost/issue/ANAL-53/10-stats-page-engineering-stuff
- All of these changes are intended to improve developer experience
going forward, to make it easier to implement further changes to the
stats page.
- Moved the modal into the stats components, as it has a lot of shared
code, and it makes it easier to update them all
- Removed various comment blocks that were outdated or didn't really add
value
- Fixed all imports to use the same pattern starting `ghost-admin/`
- Ensured all the components had the correct name
- Dried up the generation of params for Tinybird charts into a utility
function as we'll need to add several more in the near future
- Tried to use a consistent pattern everywhere for the order of
operations
- Dried up the implementation of technical.js which handles the
device/browser charts
REF PLG-226
- Changed title copy from "You want to report this comment?" to "Report
this comment?"
- Changed button copy from "Report this comment" to "Report"
- Updated styles to be more responsive
DES-774
- The buttons in the Admin got very diverse over the last couple of
years. This PR updates the styles to use outline buttons, white
background for better contrast and slightly more rounded corners.
Additionally the right click and dropdown menu typography and spacing
has also been updated.
ref https://linear.app/tryghost/issue/ONC-323
When the store gets into a bad state for new posts that causes saves to fail we can detect that by looking at the `model.isNew` property. Currently our best approach to fix this state is to restart the app.
- added a `didTransition()` hook to our `lexical-edit.new` route
- detects the bad state, logs the error, and triggers a browser refresh
- logs with a `recreatedPostIsGood` property that will let us know if we could instead just try recreating the post and avoiding a full refresh (so far we have no reproduction case so we need to learn what we can)
- added `sinon-chai` dependency for better assertions on spies/stubs
- added `sentry-testkit` dependency so we can test our Sentry integration calls
- we can't use sinon for these calls because of the way Sentry's es6 imports work
- extracted our full Sentry config object generation to a util function so it can be re-used in unit tests
- updated our integrations list to disable the default `dedupe` integration because it can cause very unexpected/difficult to debug test failures when you're asserting using `sentry-testkit`
REF DES-770
- In certain email clients such as Protonmail, the newsletter title
line-height was inherited from the `body` rather than the parent `td`.
This commit adds line-height to the title link explicitly.
ref https://linear.app/tryghost/issue/ONC-323
- sometimes posts can be deleted by another user or in a different tab but then edited in an old tab that had the post loaded in the editor
- in this situation we were displaying our "Editor crashed" error put in place for the rarer situation where the editor is genuinely in a bad state
- added an extra conditional for the bad state and a custom error message for the deleted post state
ref https://app.incident.io/ghost/incidents/107
- We have a rare bug that causes the initial `POST` request to create a
new post from the editor to be skipped or fail. Subsequent `PUT`
requests then fail because there is no post ID, potentially resulting in
data loss. The aim of this commit is to start saving revisions of posts
in the editor to the browser's localStorage, as a last-ditch option to
restore lost work.
- Since we don't know where the bug is yet, and to protect against
future bugs, we've deliberately avoided depending too heavily on the
`lexical-editor` controller or the ember store. We've aimed to create a
direct route to the state in the editor, by hooking into the
`updateScratch` method (effectively the `onChange` handler for the
editor).
- The `scheduleSave` function on the new `local-revisions` service is
called immediately upon any changes to the state of the lexical editor,
which is effectively every keystroke. The service has some logic and
timeouts, so it doesn't actually save a revision on every change to the
editor.
- The "schema" of the datastore is a simple key-value store, where the
key is of the format: `post-revision-${postId}-${timestamp}` if the post
has an ID, or `post-revision-draft-${timestamp}` for an unsaved draft.
There is also an array of all the revisions' keys, which allows us to
clear all the revisions without having to loop over every key in
localStorage (along with some other conveniences, like filtering).
- There is currently no UI for viewing/restoring revisions. In the event
that you need to restore a revision, you can access the service in the
browser console. You can access all the saved revisions using the
`list()` method, which logs all the revisions to the console by title &
timestamp. You can then choose a revision to restore, and call
`restore(revision_key)`, which will `POST` the revision's data to the
server to create a new post.
- Since localStorage data is limited to a 5mb quota in most browsers,
the service has a mechanism for evicting the oldest revisions once it
meets the quota. If a save fails because it would exceed the quota, the
`performSave` method will evict the oldest revision, then recursively
try to save again.
---------
Co-authored-by: Steve Larson <9larsons@gmail.com>
ref https://linear.app/tryghost/issue/ONC-323
- added debug logs to print to console each time the post state changes and include a full list of post state changes within the editor session in the error reports when we hit the 404 error caused by a bad editor state
closes https://linear.app/tryghost/issue/ENG-1533
- the code to switch to "from analytics" state for the editor was applying when clicking the create post button in the nav menu whilst on the analytics screen which was confusing because you not only lost the `< Posts` link in the editor but you couldn't see the post's saving status
closes https://linear.app/tryghost/issue/ANAL-77/na-data-should-be-zero
ref https://www.tinybird.co/blog-posts/tips-9-filling-gaps-in-time-series-on-clickhouse
- Sometimes we have no matching data for a particular date/date range, which makes our charts look super janky
- Clickhouse has a feature to fill these in called WITH FILL, which makes it really easy to fix this!
- WITH FILL works except for on bounce rate. That seems to be due to the column being marked as nullable and so WITH FILL fills missing data with NULL instead of 0
- To fix that, I've updated the code that generates the bounce rate so that it doesn't generate nulls, and that seems to result in a not-nullable column, which then works with WITH FILL
- When I went off, I quickly recreated all our endpoints with some new functionality
- However, I forgot that I was manually managing tokens, this meant the UI for stats broke with a token error
- Adding the tokens to the endpoint definitions should prevent this happening again, by automating the management of the token scopes
ref https://linear.app/tryghost/issue/ONC-323
- the post model state appears to be in an odd situation when this issue occurs, the extra log context should help us determine if the bad state is occurring at the route level or inside the editor controller
Ref: https://linear.app/tryghost/issue/SLO-188/set-a-maximum-limit-for-get-members-api
Endpoint: ghost/api/admin/members/?limit=all
Change Overview: We are updating the GET ghost/api/admin/members/ endpoint to remove support for the limit=all parameter. Previously, a request like GET ghost/api/admin/members/?limit=all would return a list of all members. Going forward, any request with limit=all or a limit greater than 100 will only return up to 100 members per request.
This change aims to improve the performance and scalability of the API.
What changes for users? - They will have to implement pagination to retrieve the list of all members.
refs https://linear.app/tryghost/issue/DEV-23/workaround-for-yarn-caching-issues
- in our build pipeline, we add some more dependencies for our internal
adapters
- recently we've been seeing caching issues with these dependencies, not
sure why
- to workaround that, we'll just include them here and eventually bring
the adapters into the OSS repo
ref https://linear.app/tryghost/issue/ENG-661
Logging the full lexical objects to Sentry for the "showing leave editor modal" event isn't very useful because they almost always truncated due to size or stripped due to potentially sensitive data which makes the reports difficult to debug and action.
- added `code` to the context for each report to make identification easier
- updated `reason` for the lexical diverged reason to better match what's happened
- stripped `lexical`, `scratch`, and `secondaryLexical` from the context that is logged to Sentry because they aren't actionable there and just add noise
- added a diff to the logged context for the lexical change reason to more easily identify the changes that triggered an unexpected modal display
- previously we didn't get full objects in Sentry so couldn't do a comparison and the local workflow was to grab the logged scratch and secondaryLexical values and run a manual diff - this should help in both cases
ref https://linear.app/tryghost/issue/ENG-1466
ref https://linear.app/tryghost/issue/ENG-1484
- Previously, filtering members with multiple "Unsubscribed from
newsletter x" led to no filtering at all, all members were returned
- This was caused by a bug in NQL, that is fixed in version 0.12.5, cf.
[commit](dd18d1d6ca)
- We're also removing the safeguard in the product around bulk deletion
when multiple newsletter filters are in use, as the root problem has
been fixed
ref https://linear.app/tryghost/issue/ONC-323
- our fallback 404 error handler assumed we always had a transition along with the error
- this wasn't a bad assumption, it should be very unlikely that we see a 404 outside of navigating to a non-existent/deleted resource
- unfortunately we weren't handling the error thrown by our error handler which meant the error was silent as far as the user was concerned
- having a silent error meant that in very rare circumstances the editor could get into a state where saving was failing but there was no indication of that
- updated the fallback 404 error handler to only do something when navigation was occurring in which case it transitions to the 404 screen, otherwise let the error continue to our generic API error handling which will stay on the current screen but show an error alert
- adjusted the editor saving to actually trigger autosave-after-change when testing (albeit with 100ms wait compared to 3s) so the tests better reflect actual behaviour
ref
https://linear.app/tryghost/issue/ENG-1543/debounce-the-members-lastseenatupdater
- The `LastSeenAtUpdater.updateLastSeenAt` function is called in
response to a `MemberClickEvent` — when a member clicks a link in an
email with tracking enabled. This function can be called many times for
the same member in a short period of time if e.g. a link checker is
clicking all the links in an email they received.
- This function should only update a member's `last_seen_at` timestamp
once per day. To accomplish this, `updateLastSeenAt` runs a
`select...for update` query to find the member's current `last_seen_at`
timestamp, and only updates the timestamp if the current `last_seen_at`
is before the start of the current day. The `for update` is required to
avoid a race condition, which previously caused this function to update
the `last_seen_at` timestamp more frequently than needed, which results
in many unnecessary database queries. However, we still run the initial
`select...for update` query for each event, which seems to be resulting
in contention for locks on the member's row in the `members` table.
- This commit introduces a simple in-memory cache so that we avoid
calling `updateLastSeenAt` if the member's `last_seen_at` timestamp has
already been updated in the current day, which should avoid running so
many `select...for update` queries and locking the `members` table up.
ref https://linear.app/tryghost/issue/ONC-323
- we're seeing a rare situation where the editor can get into a bad state when creating a new post that means saves silently fail
- added logging to help debug
- Added activity icon for Replies
- Updated Replies design
- Updated hard-coded Profile values to more realistic ones
- Renamed ActivityPub nav item and moved it to the top of the navbar
- Added a check for post attachments
[ANAL-79](https://linear.app/tryghost/issue/ANAL-79/stats-page-v10-design-refinements)
- typography is inconsistent in table headings
- chart section headings are not unified
- spacings are off between section and data headings
- inactive KPI tab colors are too light
- tech data numbers are not formatted
no issue
- These tests were all in a single `describe` block, making it difficult
to navigate and add more tests.
- This commit adds a nested `describe` block for each method on the
`lastSeenAtUpdater`, making it easier to understand the test cases, and
to add more. None of the tests themselves have changed, just the
organization of them.
Ref:
https://linear.app/tryghost/issue/ENG-1526/errors-from-members-api-routercontroller-are-being-lost
The try/catch/re-throw pattern, that hides errors, is used throughout
the RouterController.js file.
I have not changed the try/catch/re-throw pattern because it helps in
sending clean messages to the users. We may not want to return internal
errors as API responses.
I have added logs and Sentry messages that will help us debug without
losing error messages.
