ref https://app.incident.io/ghost/incidents/107
- We have a rare bug that causes the initial `POST` request to create a
new post from the editor to be skipped or fail. Subsequent `PUT`
requests then fail because there is no post ID, potentially resulting in
data loss. The aim of this commit is to start saving revisions of posts
in the editor to the browser's localStorage, as a last-ditch option to
restore lost work.
- Since we don't know where the bug is yet, and to protect against
future bugs, we've deliberately avoided depending too heavily on the
`lexical-editor` controller or the ember store. We've aimed to create a
direct route to the state in the editor, by hooking into the
`updateScratch` method (effectively the `onChange` handler for the
editor).
- The `scheduleSave` function on the new `local-revisions` service is
called immediately upon any changes to the state of the lexical editor,
which is effectively every keystroke. The service has some logic and
timeouts, so it doesn't actually save a revision on every change to the
editor.
- The "schema" of the datastore is a simple key-value store, where the
key is of the format: `post-revision-${postId}-${timestamp}` if the post
has an ID, or `post-revision-draft-${timestamp}` for an unsaved draft.
There is also an array of all the revisions' keys, which allows us to
clear all the revisions without having to loop over every key in
localStorage (along with some other conveniences, like filtering).
- There is currently no UI for viewing/restoring revisions. In the event
that you need to restore a revision, you can access the service in the
browser console. You can access all the saved revisions using the
`list()` method, which logs all the revisions to the console by title &
timestamp. You can then choose a revision to restore, and call
`restore(revision_key)`, which will `POST` the revision's data to the
server to create a new post.
- Since localStorage data is limited to a 5mb quota in most browsers,
the service has a mechanism for evicting the oldest revisions once it
meets the quota. If a save fails because it would exceed the quota, the
`performSave` method will evict the oldest revision, then recursively
try to save again.
---------
Co-authored-by: Steve Larson <9larsons@gmail.com>
ref https://linear.app/tryghost/issue/ONC-323
- added debug logs to print to console each time the post state changes and include a full list of post state changes within the editor session in the error reports when we hit the 404 error caused by a bad editor state
closes https://linear.app/tryghost/issue/ENG-1533
- the code to switch to "from analytics" state for the editor was applying when clicking the create post button in the nav menu whilst on the analytics screen which was confusing because you not only lost the `< Posts` link in the editor but you couldn't see the post's saving status
closes https://linear.app/tryghost/issue/ANAL-77/na-data-should-be-zero
ref https://www.tinybird.co/blog-posts/tips-9-filling-gaps-in-time-series-on-clickhouse
- Sometimes we have no matching data for a particular date/date range, which makes our charts look super janky
- Clickhouse has a feature to fill these in called WITH FILL, which makes it really easy to fix this!
- WITH FILL works except for on bounce rate. That seems to be due to the column being marked as nullable and so WITH FILL fills missing data with NULL instead of 0
- To fix that, I've updated the code that generates the bounce rate so that it doesn't generate nulls, and that seems to result in a not-nullable column, which then works with WITH FILL
- When I went off, I quickly recreated all our endpoints with some new functionality
- However, I forgot that I was manually managing tokens, this meant the UI for stats broke with a token error
- Adding the tokens to the endpoint definitions should prevent this happening again, by automating the management of the token scopes
ref https://linear.app/tryghost/issue/ONC-323
- the post model state appears to be in an odd situation when this issue occurs, the extra log context should help us determine if the bad state is occurring at the route level or inside the editor controller
Ref: https://linear.app/tryghost/issue/SLO-188/set-a-maximum-limit-for-get-members-api
Endpoint: ghost/api/admin/members/?limit=all
Change Overview: We are updating the GET ghost/api/admin/members/ endpoint to remove support for the limit=all parameter. Previously, a request like GET ghost/api/admin/members/?limit=all would return a list of all members. Going forward, any request with limit=all or a limit greater than 100 will only return up to 100 members per request.
This change aims to improve the performance and scalability of the API.
What changes for users? - They will have to implement pagination to retrieve the list of all members.
refs https://linear.app/tryghost/issue/DEV-23/workaround-for-yarn-caching-issues
- in our build pipeline, we add some more dependencies for our internal
adapters
- recently we've been seeing caching issues with these dependencies, not
sure why
- to workaround that, we'll just include them here and eventually bring
the adapters into the OSS repo
ref https://linear.app/tryghost/issue/ENG-661
Logging the full lexical objects to Sentry for the "showing leave editor modal" event isn't very useful because they almost always truncated due to size or stripped due to potentially sensitive data which makes the reports difficult to debug and action.
- added `code` to the context for each report to make identification easier
- updated `reason` for the lexical diverged reason to better match what's happened
- stripped `lexical`, `scratch`, and `secondaryLexical` from the context that is logged to Sentry because they aren't actionable there and just add noise
- added a diff to the logged context for the lexical change reason to more easily identify the changes that triggered an unexpected modal display
- previously we didn't get full objects in Sentry so couldn't do a comparison and the local workflow was to grab the logged scratch and secondaryLexical values and run a manual diff - this should help in both cases
ref https://linear.app/tryghost/issue/ENG-1466
ref https://linear.app/tryghost/issue/ENG-1484
- Previously, filtering members with multiple "Unsubscribed from
newsletter x" led to no filtering at all, all members were returned
- This was caused by a bug in NQL, that is fixed in version 0.12.5, cf.
[commit](dd18d1d6ca)
- We're also removing the safeguard in the product around bulk deletion
when multiple newsletter filters are in use, as the root problem has
been fixed
ref https://linear.app/tryghost/issue/ONC-323
- our fallback 404 error handler assumed we always had a transition along with the error
- this wasn't a bad assumption, it should be very unlikely that we see a 404 outside of navigating to a non-existent/deleted resource
- unfortunately we weren't handling the error thrown by our error handler which meant the error was silent as far as the user was concerned
- having a silent error meant that in very rare circumstances the editor could get into a state where saving was failing but there was no indication of that
- updated the fallback 404 error handler to only do something when navigation was occurring in which case it transitions to the 404 screen, otherwise let the error continue to our generic API error handling which will stay on the current screen but show an error alert
- adjusted the editor saving to actually trigger autosave-after-change when testing (albeit with 100ms wait compared to 3s) so the tests better reflect actual behaviour
In order to show replies in our notifications, we loop through all the
replies in our inbox, and filter them by the ones replying to an account
on our domain, however the check we were doing was on the admin domain -
which is sometimes the same as the frontend domain, but not always. This
fixes the check so that we check the frontend domain, which is the one
used by activitypub.
Pulled out the logic of finding the attachment(s) into a shared
function, which will only return an array if there are multiple
attachments, otherwise either null or an object will be returned.
This fixes an issue where the code assumed that an array meant
we have multiple attachments
ref
https://linear.app/tryghost/issue/ENG-1543/debounce-the-members-lastseenatupdater
- The `LastSeenAtUpdater.updateLastSeenAt` function is called in
response to a `MemberClickEvent` — when a member clicks a link in an
email with tracking enabled. This function can be called many times for
the same member in a short period of time if e.g. a link checker is
clicking all the links in an email they received.
- This function should only update a member's `last_seen_at` timestamp
once per day. To accomplish this, `updateLastSeenAt` runs a
`select...for update` query to find the member's current `last_seen_at`
timestamp, and only updates the timestamp if the current `last_seen_at`
is before the start of the current day. The `for update` is required to
avoid a race condition, which previously caused this function to update
the `last_seen_at` timestamp more frequently than needed, which results
in many unnecessary database queries. However, we still run the initial
`select...for update` query for each event, which seems to be resulting
in contention for locks on the member's row in the `members` table.
- This commit introduces a simple in-memory cache so that we avoid
calling `updateLastSeenAt` if the member's `last_seen_at` timestamp has
already been updated in the current day, which should avoid running so
many `select...for update` queries and locking the `members` table up.
refs https://linear.app/tryghost/issue/AP-388
The attributedTo property of objects refers to the author of the object,
wheras the actor of an activity refers to the author of the activity -
in the case of a `Create` activity - these are generally the same, but
in the case of an `Announce` they are different, the author refers to
the "announcer" and the attributedTo the author of the content! This is
a quick patch to use the true author when it's available, and in a
format we can handle.
ref https://linear.app/tryghost/issue/ONC-323
- we're seeing a rare situation where the editor can get into a bad state when creating a new post that means saves silently fail
- added logging to help debug
no issue
- OpenTelemetry has been problematic in a number of ways (boot time,
breaking the frontend). May revisit it at some point in the future, but
for now it is only exporting metrics via prometheus and not traces, so
there's currently nothing sending data to this jaeger container
- Cleaning it up for now as it's just sitting there idly consuming
resources
no issue
- This allows us to run `docker-compose down` or to restart docker
desktop without losing all our local databases
- Added a data volume to the MySQL service in the `docker-compose.yml`
file to persist the data between container restarts
- The `yarn docker:reset` command will still reset all the data in the
database since it uses `down -v` to remove the volumes as well
- Added activity icon for Replies
- Updated Replies design
- Updated hard-coded Profile values to more realistic ones
- Renamed ActivityPub nav item and moved it to the top of the navbar
- Added a check for post attachments
