- TODO: make these run in CI
- Right now you run them by running `yarn tb` and then `./script/branch_and_test.sh`
- These are snapshot tests that check we get the desired result
Co-authored-by: alejandromav <hi@alejandromav.com>
- Apparently several PRs for new languages were merged without the
corresponding additions to i18n.js, which has caused new strings not to
be generated for those languages, and missing translations in the
frontend, even though we had them in the .json file.
- I've added language codes for all languages that had at least some
strings translated in i18n/locales, and have run yarn translate to
create missing strings and files for these languages.
New language codes for:
* Arabic
* Swiss German
* Greek
* Estonian
* Macedonian
* Serbian (Cyrillic)
* Thai
no issue
- This commit removes all OpenTelemetry related code and dependencies
from Ghost.
- The initial implementation was done as a POC but it raised some
performance concerns with boot time, so we never actually enabled it
widely.
- We can revisit this in the future, but in the meantime it's just
adding unnecessary dependencies and bloating the codebase.
ref https://ghost-foundation.sentry.io/issues/5908152800/
- In the current state, we are maintaining an 'index' key for all
revisions in localStorage. This gives us quick and easy access to all
the revisions in localStorage, but it requires additional "bookkeeping"
to update the index each time we add/remove a key.
- In some obscure edge cases, this results in the `remove()` method
throwing a `QuotaExceededError` (since removing a revision also requires
updating the index with `localStorage.setItem()`). If the `remove()`
call fails, we are sort of stuck — the only way to reduce our storage
usage is to remove items, but if the `remove()` method throws errors, we
can't do that.
- This change removes the whole index concept, and instead loops over
all the keys in localStorage, filtering by the prefix to find all our
revisions. This makes the `keys()` method slightly more complex, as it
has to filter out keys in localStorage that aren't related to revisions,
but it simplifies saving and removing revisions.
- Critically, this also means that `remove()` should never throw a
`QuotaExceededError`, since it no longer needs to call
`localStorage.setItem()` — it now simply calls
`localStorage.removeItem()` for the revision, which should never fail.
ref https://linear.app/tryghost/issue/AP-438
This is going to allow us to load the activitypub package from the jsdelivr
cdn, which means we can release new versions without releasing the admin.
no issue
Either a node or macOS update resulted in our broken image upload tests causing native stack traces:
```
# /Users/kevin/.nvm/versions/node/v20.16.0/bin/node[8841]: static void node::Blob::ToSlice(const FunctionCallbackInfo<v8::Value> &) at ../src/node_blob.cc:248
# Assertion failed: args[1]->IsUint32()
```
- updated our `blob.slice()` calls to ensure the second argument is always an integer rather than possibly a float
This pull request adds Estonian (et) language translations to Ghost.
Translated files:
- comments.json
- ghost.json
- portal.json
- search.json
- signup-form.json
Changes:
- Created new 'et' folder in the locales directory
- Translated all strings from English to Estonian in the above files
I'm a native Estonian speaker, but if there are any questions about
specific translations, please let me know.
Thank you for considering this contribution to make Ghost more
accessible to Estonian-speaking users!
---------
Co-authored-by: Ronald Langeveld <hi@ronaldlangeveld.com>
A needed update of the Bulgarian version. Fixing some errors. Made some
improvements. And some new strings were translated.
---------
Co-authored-by: Ronald Langeveld <hi@ronaldlangeveld.com>
ref ONC-364
- Adds a condition to check whether the record is deleted or if deleting
is in progress before firing the `setFeatureImageCaption`.
- Adds tests. Managed to reproduce the issue using tests.
no issue
- Added Sentry logs to capture how often we are running into
`QuotaExceededErrors` when saving local revisions to localStorage, to
help in deciding if localStorage is sufficient, or if we need to expand
to e.g. IndexedDB.
- Also adds some handling to ignore errors when calling
`localStorage.setItem()` elsewhere in the admin app to avoid crashing if
localStorage isn't supported or the quota is exceeded.
ref https://app.incident.io/ghost/incidents/107
ref cc88757e2a
- added new path in admin `/restore`
- added basic ui for restoring posts from local storage
- added limits for # of revisions for posts with an `id` (5 revisions)
This commit adds a simple UI for restoring posts in case of data loss.
This is a backstop for very rare situations in which it seems Ember gets
into a conflicted state. See ref'd commit for more info. Clicking
'Restore' will create a new post with the saved off content.
ref
https://linear.app/tryghost/issue/ANAL-60/click-through-filtering-for-sources
- In our stats page we use the referrer without a protocol or www, that
is the pure domain as our source that we output
- Meanwhile all the data pipelines had the full url as the referrer
passed through
- When we come to add clickthroughs/filtering, we'll need to use this
value to filter the data. If we have a different value locally in the UI
to what is in the DB, we won't be able to make the filters match
- Also, we pay for everything we store, and this removes all the
https:// and www. data
- Whilst we are in development, we can safely make changes to all
aspects of our pipeline without worrying
- This is because currently, it's safe to delete all data and start over
- This script removes everything excepts the analytics_events
datasource, and then recreates everything fresh, repopulating from the
datasource where possible
- This shouldn't be used after tinybird is in production, we need a
better change process
ref https://github.com/TryGhost/Ghost/issues/16628
This adds translation support to search, which should be the last missing piece of i18n support for Ghost's frontend 🎉
- Translation (t) helper added to sodo-search.
- Ghost head tweaked to include data-locale.
- All (I hope) strings in sodo-search wrapped in the t helper.
- Possibly poor-quality French translation strings added.
---------
Co-authored-by: Vikas Potluri <vikaspotluri123.github@gmail.com>
ref
https://linear.app/tryghost/issue/ANAL-53/10-stats-page-engineering-stuff
- All of these changes are intended to improve developer experience
going forward, to make it easier to implement further changes to the
stats page.
- Moved the modal into the stats components, as it has a lot of shared
code, and it makes it easier to update them all
- Removed various comment blocks that were outdated or didn't really add
value
- Fixed all imports to use the same pattern starting `ghost-admin/`
- Ensured all the components had the correct name
- Dried up the generation of params for Tinybird charts into a utility
function as we'll need to add several more in the near future
- Tried to use a consistent pattern everywhere for the order of
operations
- Dried up the implementation of technical.js which handles the
device/browser charts
REF PLG-226
- Changed title copy from "You want to report this comment?" to "Report
this comment?"
- Changed button copy from "Report this comment" to "Report"
- Updated styles to be more responsive
DES-774
- The buttons in the Admin got very diverse over the last couple of
years. This PR updates the styles to use outline buttons, white
background for better contrast and slightly more rounded corners.
Additionally the right click and dropdown menu typography and spacing
has also been updated.
ref https://linear.app/tryghost/issue/ONC-323
When the store gets into a bad state for new posts that causes saves to fail we can detect that by looking at the `model.isNew` property. Currently our best approach to fix this state is to restart the app.
- added a `didTransition()` hook to our `lexical-edit.new` route
- detects the bad state, logs the error, and triggers a browser refresh
- logs with a `recreatedPostIsGood` property that will let us know if we could instead just try recreating the post and avoiding a full refresh (so far we have no reproduction case so we need to learn what we can)
- added `sinon-chai` dependency for better assertions on spies/stubs
- added `sentry-testkit` dependency so we can test our Sentry integration calls
- we can't use sinon for these calls because of the way Sentry's es6 imports work
- extracted our full Sentry config object generation to a util function so it can be re-used in unit tests
- updated our integrations list to disable the default `dedupe` integration because it can cause very unexpected/difficult to debug test failures when you're asserting using `sentry-testkit`
REF DES-770
- In certain email clients such as Protonmail, the newsletter title
line-height was inherited from the `body` rather than the parent `td`.
This commit adds line-height to the title link explicitly.
ref https://linear.app/tryghost/issue/ONC-323
- sometimes posts can be deleted by another user or in a different tab but then edited in an old tab that had the post loaded in the editor
- in this situation we were displaying our "Editor crashed" error put in place for the rarer situation where the editor is genuinely in a bad state
- added an extra conditional for the bad state and a custom error message for the deleted post state
ref https://app.incident.io/ghost/incidents/107
- We have a rare bug that causes the initial `POST` request to create a
new post from the editor to be skipped or fail. Subsequent `PUT`
requests then fail because there is no post ID, potentially resulting in
data loss. The aim of this commit is to start saving revisions of posts
in the editor to the browser's localStorage, as a last-ditch option to
restore lost work.
- Since we don't know where the bug is yet, and to protect against
future bugs, we've deliberately avoided depending too heavily on the
`lexical-editor` controller or the ember store. We've aimed to create a
direct route to the state in the editor, by hooking into the
`updateScratch` method (effectively the `onChange` handler for the
editor).
- The `scheduleSave` function on the new `local-revisions` service is
called immediately upon any changes to the state of the lexical editor,
which is effectively every keystroke. The service has some logic and
timeouts, so it doesn't actually save a revision on every change to the
editor.
- The "schema" of the datastore is a simple key-value store, where the
key is of the format: `post-revision-${postId}-${timestamp}` if the post
has an ID, or `post-revision-draft-${timestamp}` for an unsaved draft.
There is also an array of all the revisions' keys, which allows us to
clear all the revisions without having to loop over every key in
localStorage (along with some other conveniences, like filtering).
- There is currently no UI for viewing/restoring revisions. In the event
that you need to restore a revision, you can access the service in the
browser console. You can access all the saved revisions using the
`list()` method, which logs all the revisions to the console by title &
timestamp. You can then choose a revision to restore, and call
`restore(revision_key)`, which will `POST` the revision's data to the
server to create a new post.
- Since localStorage data is limited to a 5mb quota in most browsers,
the service has a mechanism for evicting the oldest revisions once it
meets the quota. If a save fails because it would exceed the quota, the
`performSave` method will evict the oldest revision, then recursively
try to save again.
---------
Co-authored-by: Steve Larson <9larsons@gmail.com>
ref https://linear.app/tryghost/issue/ONC-323
- added debug logs to print to console each time the post state changes and include a full list of post state changes within the editor session in the error reports when we hit the 404 error caused by a bad editor state
closes https://linear.app/tryghost/issue/ENG-1533
- the code to switch to "from analytics" state for the editor was applying when clicking the create post button in the nav menu whilst on the analytics screen which was confusing because you not only lost the `< Posts` link in the editor but you couldn't see the post's saving status
closes https://linear.app/tryghost/issue/ANAL-77/na-data-should-be-zero
ref https://www.tinybird.co/blog-posts/tips-9-filling-gaps-in-time-series-on-clickhouse
- Sometimes we have no matching data for a particular date/date range, which makes our charts look super janky
- Clickhouse has a feature to fill these in called WITH FILL, which makes it really easy to fix this!
- WITH FILL works except for on bounce rate. That seems to be due to the column being marked as nullable and so WITH FILL fills missing data with NULL instead of 0
- To fix that, I've updated the code that generates the bounce rate so that it doesn't generate nulls, and that seems to result in a not-nullable column, which then works with WITH FILL
- When I went off, I quickly recreated all our endpoints with some new functionality
- However, I forgot that I was manually managing tokens, this meant the UI for stats broke with a token error
- Adding the tokens to the endpoint definitions should prevent this happening again, by automating the management of the token scopes
ref https://linear.app/tryghost/issue/ONC-323
- the post model state appears to be in an odd situation when this issue occurs, the extra log context should help us determine if the bad state is occurring at the route level or inside the editor controller
Ref: https://linear.app/tryghost/issue/SLO-188/set-a-maximum-limit-for-get-members-api
Endpoint: ghost/api/admin/members/?limit=all
Change Overview: We are updating the GET ghost/api/admin/members/ endpoint to remove support for the limit=all parameter. Previously, a request like GET ghost/api/admin/members/?limit=all would return a list of all members. Going forward, any request with limit=all or a limit greater than 100 will only return up to 100 members per request.
This change aims to improve the performance and scalability of the API.
What changes for users? - They will have to implement pagination to retrieve the list of all members.
refs https://linear.app/tryghost/issue/DEV-23/workaround-for-yarn-caching-issues
- in our build pipeline, we add some more dependencies for our internal
adapters
- recently we've been seeing caching issues with these dependencies, not
sure why
- to workaround that, we'll just include them here and eventually bring
the adapters into the OSS repo
ref https://linear.app/tryghost/issue/ENG-661
Logging the full lexical objects to Sentry for the "showing leave editor modal" event isn't very useful because they almost always truncated due to size or stripped due to potentially sensitive data which makes the reports difficult to debug and action.
- added `code` to the context for each report to make identification easier
- updated `reason` for the lexical diverged reason to better match what's happened
- stripped `lexical`, `scratch`, and `secondaryLexical` from the context that is logged to Sentry because they aren't actionable there and just add noise
- added a diff to the logged context for the lexical change reason to more easily identify the changes that triggered an unexpected modal display
- previously we didn't get full objects in Sentry so couldn't do a comparison and the local workflow was to grab the logged scratch and secondaryLexical values and run a manual diff - this should help in both cases
ref https://linear.app/tryghost/issue/ENG-1466
ref https://linear.app/tryghost/issue/ENG-1484
- Previously, filtering members with multiple "Unsubscribed from
newsletter x" led to no filtering at all, all members were returned
- This was caused by a bug in NQL, that is fixed in version 0.12.5, cf.
[commit](dd18d1d6ca)
- We're also removing the safeguard in the product around bulk deletion
when multiple newsletter filters are in use, as the root problem has
been fixed
ref https://linear.app/tryghost/issue/ONC-323
- our fallback 404 error handler assumed we always had a transition along with the error
- this wasn't a bad assumption, it should be very unlikely that we see a 404 outside of navigating to a non-existent/deleted resource
- unfortunately we weren't handling the error thrown by our error handler which meant the error was silent as far as the user was concerned
- having a silent error meant that in very rare circumstances the editor could get into a state where saving was failing but there was no indication of that
- updated the fallback 404 error handler to only do something when navigation was occurring in which case it transitions to the 404 screen, otherwise let the error continue to our generic API error handling which will stay on the current screen but show an error alert
- adjusted the editor saving to actually trigger autosave-after-change when testing (albeit with 100ms wait compared to 3s) so the tests better reflect actual behaviour
ref
https://linear.app/tryghost/issue/ENG-1543/debounce-the-members-lastseenatupdater
- The `LastSeenAtUpdater.updateLastSeenAt` function is called in
response to a `MemberClickEvent` — when a member clicks a link in an
email with tracking enabled. This function can be called many times for
the same member in a short period of time if e.g. a link checker is
clicking all the links in an email they received.
- This function should only update a member's `last_seen_at` timestamp
once per day. To accomplish this, `updateLastSeenAt` runs a
`select...for update` query to find the member's current `last_seen_at`
timestamp, and only updates the timestamp if the current `last_seen_at`
is before the start of the current day. The `for update` is required to
avoid a race condition, which previously caused this function to update
the `last_seen_at` timestamp more frequently than needed, which results
in many unnecessary database queries. However, we still run the initial
`select...for update` query for each event, which seems to be resulting
in contention for locks on the member's row in the `members` table.
- This commit introduces a simple in-memory cache so that we avoid
calling `updateLastSeenAt` if the member's `last_seen_at` timestamp has
already been updated in the current day, which should avoid running so
many `select...for update` queries and locking the `members` table up.
ref https://linear.app/tryghost/issue/ONC-323
- we're seeing a rare situation where the editor can get into a bad state when creating a new post that means saves silently fail
- added logging to help debug
- Added activity icon for Replies
- Updated Replies design
- Updated hard-coded Profile values to more realistic ones
- Renamed ActivityPub nav item and moved it to the top of the navbar
- Added a check for post attachments
[ANAL-79](https://linear.app/tryghost/issue/ANAL-79/stats-page-v10-design-refinements)
- typography is inconsistent in table headings
- chart section headings are not unified
- spacings are off between section and data headings
- inactive KPI tab colors are too light
- tech data numbers are not formatted
no issue
- These tests were all in a single `describe` block, making it difficult
to navigate and add more tests.
- This commit adds a nested `describe` block for each method on the
`lastSeenAtUpdater`, making it easier to understand the test cases, and
to add more. None of the tests themselves have changed, just the
organization of them.
Ref:
https://linear.app/tryghost/issue/ENG-1526/errors-from-members-api-routercontroller-are-being-lost
The try/catch/re-throw pattern, that hides errors, is used throughout
the RouterController.js file.
I have not changed the try/catch/re-throw pattern because it helps in
sending clean messages to the users. We may not want to return internal
errors as API responses.
I have added logs and Sentry messages that will help us debug without
losing error messages.
[ANAL-43](https://linear.app/tryghost/issue/ANAL-43/implement-all-possible-ui-for-10)
- The BarList component wasn't using the parameters provided in its
latest release
- Number formatting was missing on all numbers
- "See all" links were missing in Content/Sources/Locations
- Empty/default values was showing [blank]
- Flags were missing for country values
Got some translated strings, and hope it's awesome 🎊!
A short description of my changes:
- [x] All lines have been translated
- [x] For lines that already had a translation into Russian - I
double-checked the accuracy of that translation and corrected the
translation if necessary
- [x] I tried to ensure uniformity in the translation of terms and even
compiled a small glossary (local on my computer)
- [x] Frequently referred to and checked the hints from the
`context.json` file
I hope, you will appreciate your contribution!
---------
Co-authored-by: Michael Barrett <mike@ghost.org>
no issue
- Moved business logic from `WebhookController` to dedicated service
classes (`SubscriptionEventService`, `InvoiceEventService`,
`CheckoutSessionEventService`).
- Reduced controller complexity.
- Added unit tests for individual services, increasing overall test
coverage.
- Improved maintainability and scalability by isolating responsibilities
in specific services, making future updates easier and safer.
ref
https://linear.app/tryghost/issue/ONC-217/implement-the-deliverytime-option-in-mailgun-api-calls
Ghost experiences its highest peak load immediately after sending out a
newsletter, as it recieves an influx of traffic from users clicking on
the links in the email, a burst of email analytics events to process
from mailgun, and an increase in organic traffic to the site's frontend
as well as the admin analytics pages. The `BatchSendingService`
currently sends all the batches to Mailgun as quickly as possible, which
may contribute to higher peak loads.
This commit adds a `deliverytime` parameter to our API calls to Mailgun,
which allows us to specify a time in the future when we want the email
to be delivered. This will allow us to moderate the rate at which emails
are delivered, and in turn that should moderate the peak traffic volume
that Ghost receives in the first 2-3 minutes after sending an email.
The `deliverytime` is calculated based on a configurable parameter:
`bulkEmail.targetDeliveryWindow`, which specifies the maximum allowable
time (in milliseconds) after the email is first sent for Ghost to
instruct Mailgun to deliver the emails. Ghost will attempt to space out
all the batches as evenly as possible throughout the specified window.
For example, if the targetDeliveryWindow is set to `300000` (5 minutes)
and there are 100 batches, Ghost will set the `deliveryTime` for each
batch ~3 seconds apart.
no issue
- These retries were added a while ago when these tests were flaking in
CI, but this only serves to cover up any potential problems with these
tests or the code they are testing
- Ran these tests in CI three times in a row without them failing, so I
don't think the retries are necessary
no issue
- One of the tests in this suite added a member and didn't clean it up
when it was finished.
- Because of this, the tests after this one depended on this test
running first, so running an individual test in isolation might fail,
despite passing when run in the whole test suite
- This commit removes the added member, so all the tests in this suite
should pass whether run independently or all together
[ANAL-43](https://linear.app/tryghost/issue/ANAL-50/update-colors-of-barlist)
- Copy is too technical, doesn't follow conventions on Stats page
- Range filter dropdown has to be updated with more meaningful values
- KPI charts need a granularity dropdown to display meaninful charts
depending on the context
- Typography details should be updated
- "Posts/pages" dropdown needs to be added to Content section. This is a
Ghost specific filter that brings high value to customers
- "Campaigns" dropdown needs to be added to Sources section to support
ad tracking and filtering in the future
- BarList colors should be updated to be less purple all over the place
ref https://github.com/TryGhost/Ghost/pull/20835
- reimplemented email analytics changes that prioritized opened events
over other events in order to speed up open analytics
- added db persistence to fetch missing job to ensure we re-fetch every
window of events, especially important if we restart following a large
email batch
We learned a few things with the previous trial run of this. Namely,
that event throughput is not as high as we initially saw in the data for
particularly large databases. This set of changes is more conservative,
while a touch more complicated, in ensuring we capture edge cases for
really large newsletter sends (100k+ members).
In general, we want to make sure we're fetching new open events at least
every 5 mins, and often much faster than that, unless it's a quiet
period (suggesting we haven't had a newsletter send or much outstanding
event data).
Following commit #20858, the following updates were made:
- Inserted the Brazilian standard phone number format for `+1 (123)
456-7890`.
- Translated missing static strings, such as "Enter your email address"
and "Invalid email address."
- Updated some translations to maintain consistency and proper context.
ref https://linear.app/tryghost/issue/ONC-296
Our `stripe_prices.nickname` field had a length of 50 chars which meant we could error out trying to save a donation Stripe price with a generated product nickname containing a long site title.
- updated db schema and added a migration to change column length to 255
- added truncation to nickname generation to enforce a limit of 250 chars to match Stripe's limit
no issue
Give your audience a simple way to support your work with one-time payments, no membership required.
- cleaned up `tipsAndDonations` labs flag
closes https://linear.app/tryghost/issue/PLG-205
The server trims whitespace from the title when saving but our unsaved changes detection was comparing the raw title input field value meaning there would be a mismatch after publishing if the title field contained leading/trailing whitespace.
- updated title comparison to compare trimmed values
- moved and improved unsaved changes modal tests from publish-flow to unsaved-changes acceptance test file
- added util for pasting content into the editor to test for content changes
no issue
- fixed eslint config so tests are correctly linted
- removed `.only` on stats tests preventing other tests from running
- removed unneccessary `return` from async setup functions
[ANAL-1](https://linear.app/tryghost/issue/ANAL-32/add-stats-kpis-charts)
The technical details section in Stats contains only the browser breakdown ATM. This PR adds the rest (devices, operating systems) and fixes a couple of minor UI details on the rest of the charts
ref https://linear.app/tryghost/issue/DEV-20/faster-builds
- `build` should be the overall script to build the package
- `build:ts` should only build TypeScript
- by having them switched around, `build:ts` would call `yarn build`,
and `yarn` adds several hundreds of milliseconds of time to each build
closes https://linear.app/tryghost/issue/ANAL-23/filtering-by-logged-out-logged-in-traffic
- Updated all of our tinybird datasources and pipes to handle member status
- Added member_status as an array query param to the API endpoints
- Added a really dodgy power select multiple to the stats page to demonstrate it works (needs styling)
- Added all of the wiring so each chart updates
- This was done pretty fast, and may not be 100% right yet
closes https://linear.app/tryghost/issue/PLG-190
- often when adding portal links to your own site pages the URLs are added as absolute on the site's homepage due to copy+paste from displayed URLs in Admin
- when clicking absolute portal URLs the homepage is first loaded before the Portal popup is shown resulting in a slower and flashier experience
- added a transform for all local portal URLs on the page when Portal is initialized so links open the Portal popup immediately on the current page
