ref
https://linear.app/tryghost/issue/ENG-1591/add-prometheus-and-grafana-services-to-docker-compose
This commit adds 2 new services to the docker compose file to enable
monitoring metrics from Ghost locally in real-time:
1. Prometheus - a service that scrapes Ghost's new `/metrics` endpoint
introduced in this
[commit](768336efad).
2. Grafana - a service that consumes the metrics from prometheus and
exposes them in a dashboard that you can view locally at
`localhost:3000`.
# Usage
Both of these services are selectively enabled using docker compose
[profiles](https://docs.docker.com/compose/how-tos/profiles/). This way,
if you don't opt-in to using these monitoring tools, they won't start
and consume resources on your host machine. To enable these services,
enable the `monitoring` profile by either setting the `COMPOSE_PROFILES`
environment variable to `monitoring`, or specifying the `--profile
monitoring` CLI argument to any `docker compose ...` commands.
I've found the easiest way to configure this in an 'always on' fashion
is to create a `.env` file in the project's root directory and add
`COMPOSE_PROFILES=monitoring` to it. As an added convenience, you can
also set `COMPOSE_FILE=.github/scripts/docker-compose.yml`, which will
allow you to run `docker compose ...` commands from the root directory
without specifying the full path each time.
# Intended for development only
These services are meant for local development only, and are not
configured for a production use-case. For example, the Grafana instance
is configured to have _no authorization_ so you won't need a
username/password to login at `localhost:3000`. Prometheus is also
configured to scrape the metrics once every second, which is likely
excessive for production use-cases, but may be useful for getting more
granular metrics while e.g. load testing locally.
# Dashboards
The Grafana instance includes a default dashboard including most of the
main default metrics provided by our prometheus client integration. The
dashboard is defined in a JSON file at
`.github/scripts/docker/grafana/dashboards/main-dashboard.json' and can
be modified & committed to add new visualizations that will be available
to anyone work on Ghost locally. You can also add other dashboards to
the same directory for specific use-cases, which should be picked up and
made available in the Grafana UI. [Read
more](https://grafana.com/docs/grafana/latest/dashboards/build-dashboards/view-dashboard-json-model/)
about Grafana's JSON schema for dashboards.
This PR contains the following updates:
| Package | Type | Update | Change |
|---|---|---|---|
|
[parse-prometheus-text-format](https://redirect.github.com/yunyu/parse-prometheus-text-format)
| devDependencies | pin | [`^1.1.1` ->
`1.1.1`](https://renovatebot.com/diffs/npm/parse-prometheus-text-format/1.1.1/1.1.1)
|
Add the preset `:preserveSemverRanges` to your config if you don't want
to pin your dependencies.
---
### Configuration
📅 **Schedule**: Branch creation - "every weekday" (UTC), Automerge - At
any time (no schedule defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.
♻ **Rebasing**: Whenever PR is behind base branch, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box
---
This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/TryGhost/Ghost).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOC45Ny4wIiwidXBkYXRlZEluVmVyIjoiMzguOTcuMCIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOltdfQ==-->
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
ref
https://linear.app/tryghost/issue/ENG-1505/add-prometheus-metrics-server-to-allow-monitoring-ghost-metrics
# Summary
This commit includes two main components: a prometheus client class to
collect metrics from Ghost, and a standalone metrics server that exposes
a /metrics endpoint at a separate port (9416 by default) from the main
Ghost app.
The prometheus client is a very thin wrapper around
[prom-client](https://github.com/siimon/prom-client). We could use
prom-client directly, but this approach should make it easier to switch
to a different prometheus client package (or make our own) if we ever
need to down the line.
The list of default metrics this enables is specified in an e2e test
[here](https://github.com/TryGhost/Ghost/pull/21192/files#diff-ebc52236be2cd14b40be89220ae961f48d3f837693f7d1da76db292348915941R66-R92).
This also gives us the ability to create and collect custom metrics,
although none are included in this commit yet.
# Configuration
The prometheus client and the metrics server are both disabled by
default, but can be enabled by setting the metrics_server:enabled flag
to true.
You can also define a custom host and port using `metrics_server:host`
and `metrics_server:port`.
## Why not expose the /metrics endpoint in one of the existing express
apps?
The standalone express app exists for two main reasons:
1. We don't want these metrics to be public, and the easiest way to
accomplish that is to expose the /metrics endpoint at a different port
that won't be exposed to the internet.
2. Creating a standalone express instance decouples the metrics endpoint
from the Ghost server, so if Ghost is not responding for whatever
reason, we should still be able to scrape metrics to understand what's
going on internally.
## Impact on Boot & Shut down time
The prometheus client is initialized early in the boot process so we can
collect metrics during the boot sequence. Testing locally has shown that
this increases boot time by ~20ms. The metrics server which exposes the
/metrics endpoint is not initialized until after the background
services, and it is not awaited, to avoid impacting boot time. None of
this code, including the requires, will run if the
metrics_server:enabled flag is set to false (or not set).
Shutting down the metrics server is added as a cleanup task for the main
Ghost server instance, and is setup to shut down with 0 grace period to
avoid impacting shut down time.
no issue
- These two tests were relying on the ordering of the posts in the posts
list, which can vary when running the test suite all together (as we do
in CI), since multiple tests can be running and creating/deleting posts
in parallel.
- This commit navigates directly to the post's URL to avoid this
problem.
no ref
Expose (some) Portal error strings for translations
💩This is a somewhat hacky (but test-passing and individual inspection
passing) solution to the way Portal handles errors. Or rather, the
half-dozen ways Portal handles errors.
Passing 't' around with context and state, and occasionally recreating
it from the site object. Yes, I am also somewhat horrified, but a better
implementation will need a major rewrite of Portal.
Addresses errors in both the popover React app and in the
data-attributes.
There are probably more. Since Portal exposes raw API responses in some
places, it's hard to enumerate everything that /might/ end up being
client-facing, but at least I've gotten the ones that I've commonly
seen.
Improvements very welcome.
no issue
Initial pass at refactoring the tests and making each independent so they aren't relying on data from previous tests.
- DRYed up some repeated API requests by extracting `testPostComment()`, `testGetComments()`, and `assertAuthorEmailSent()`
- removed data inter-dependency across tests
- truncated the tables under test before each test is run
- added suite of db fns to populate the database without having to do so via API requests
- updated all tests to include their own data setup calls rather than relying on fixtures and modifications from earlier tests
no ref
We had an instance where this was a ms off and I should've used mock
timers when I first wrote this. This should prevent any rare clock
mishaps.
no ref
Translated/updated all relevant strings in the search & portal feature
to Bahasa Indonesia.
---------
Co-authored-by: Cathy Sarisky <42299862+cathysarisky@users.noreply.github.com>
ref https://linear.app/tryghost/issue/ANAL-96/data-discrepancy-between-charts-when-filtering
- This adds a set of tests to describe what the data should look like when we filter on various values
- We have tests for source and browser which are pulled from different MVs
- The result files are generated using ./scripts/gen_test_results.sh, and then manually verified
- We know they are not yet fully correct
- Added yarn command to update TB CLI, as that needs doing frequently and I can never remember the command
- Improved safety & usability of tinybird test script by ensuring branches are correctly created before running & adding optional delete
- Updated tinybird test to warn only for sanity check as that's not always a valid check (Will prob remove soon)
- Improved output of tinybird test script on failure, so that the diff is readable and closer to what git shows you
- Added tool to convert tinybird ndjson to csv to make it easier to bring the data into google sheets for verifying numbers
no ref
- this caused some troubles with error representation and changed
defaulting behavior
Going back to the drawing board on this one. I've been working on a
larger scale refactor so that this could be a hook, which feels much
more appropriate, though a much more substantial change.
no issue
- This test file starts a Ghost server, but doesn't close it, which can
cause other tests to fail when they try to start an instance of Ghost,
with an `EADDRINUSE` error.
- This change closes the server in the `after` hook
no ref
This PR adds the translations for `search.js`. It's a bit opinionated in
that ...
- it uses the Anglicism "Posts" to ... "translate" ... "Posts" 😆;
- it uses an asterisk to make sure we address all genders when
translating "authors".
Happy to discuss. ☺️
no ref
This is a merge & tidy up of [[ayangizzat](https://github.com/ayangizzat)] 's original (https://github.com/TryGhost/Ghost/pull/20698), which was missing the changes to locale, causing folders & files not to generate correctly. All credit goes to @ayangizzat , and thank you for your work!
Kazakh-speaking help still needed for search.json, which is untranslated, and the addition of a few other new strings, and refinements anywhere needed. Please add comments if not yet merged, or open a new PR if already merged.
---------
Co-authored-by: ayangizzt
no ref
- Portal was not set up in a way to allow for easy use of the i18n
module for errors, as they weren't a React component
- moved away from the class model to a functional component that could
utilize React state (AppContext)
I'm working on a different refactor that would convert more of Portal to
hooks & functional components so that the codebase is more consistent
and easier to read. This will have to work for the moment while that is
being done, as that's no small task.
no ref
We've had several PRs for new locales that forgot to update i18n.js,
which causes them not to get regenerated by yarn translate. This PR
checks for that error, and also the reverse one where they update i18n
but we're missing folders. (Never expected to error, because 'yarn test'
is defined to include 'yarn translate', which generates the folders.)
no ref
- added dir prop, calculated by i18next from language (using the dir
function)
- tweaked a few styles to use me/ms/pe/ps instead of mr/ml/pr/pl
- added updated test that checks that stemming works in English, and added tests for partial and full-word searching with RTL content.
no ref
According to the flexsearch documentation, https://github.com/nextapps-de/flexsearch?tab=readme-ov-file#cjk-word-break-chinese-japanese-korean for searching CJK text, need to pass in a custom encode function for better search results.
This enhancement for CJK will only take effect when the ghost site locale is set to one of `zh`, `zh-Hans`, `zh-Hant`, `ja`, `ko`.
Co-authored-by: Cathy Sarisky <42299862+cathysarisky@users.noreply.github.com>
ref 86d61304b1
ref https://linear.app/tryghost/issue/ONC-323
- added `tracked()` to our proxy model object properties
- fixes default data always showing when opening the modal
- fixed data push after completing modal
- `post.tiers` is set up as an attribute in Admin rather than a relationship
- fixes incorrect tiers list showing when the change access modal is opened again after changing access before the post is re-fetched from the API
- fixed flash of failure button state when saving modal changes
- expanded tests to cover tiers selection
[ANAL-95](https://linear.app/tryghost/issue/ANAL-95/internal-beta-qa)
Various design refinements and fixes for the Stats page:
- Updated scroll area in detail modals so that the Close button and the footer is never outside the viewport
- The detail modal didn't close after clicking on the filter values
- "Show all" button was displayed also when there were no new items in the detail modal
- Dropdown styles needed a visual update: the toggles were way too huge and inconsistent with other dropdowns
- If no audience was selected we still showed stats. Now it's displaying the default empty screen in this case
- Click through filter indicators had low discoverability
- Technical data styles needed some love: changed the alignment and color scheme
- Mobile size viewports were not handled
- The google favicon API returned 404 many times for sources. Swapped the service for another one that returns favicons more reliably
- Default favicon was not handled. Now it comes from static.ghost.org
no ref
This very small helper adds {{content_api_key}} to the collection of handlebars helpers available to theme creators. This will make it easier for themes to access the content API key, without either requiring the user to get it from the integrations page and input it on the theme setting page or resorting to JavaScript to read it from one of the built-in script tag attributes -- both ugly workarounds.
ref https://linear.app/tryghost/issue/ONC-323
After changing a post's access via the posts list context menu, creating new posts or members would not work correctly.
- the issue stemmed from `this.post.set('currentState.parentState.isNew', false);` that was called when changing a post's access level, after that all Ember Data models created from the store would have `isNew: false` causing Ember Data to attempt a PUT request to update the not-yet-created model rather than a POST request to create it
- we were only using a real post model instance in order to run validations against the post access level settings but we can do that just as easily by creating a new object and injecting our validation mixin
no ref
This PR adds and adjusts various German translations in `portal.json`.
I'm a native speaker and have made adjustments to the best of my
knowledge. As in #21163, the gender variations are a bit opinionated and
I'm happy to discuss.
fixes#21104
This PR corrects a single translation. The German translations for "sign
up" and "sign in" _can_ both be "anmelden", but are confusing when used
together. So here we're switch to "register now" and "sign in" to make
the distinction clearer. IMHO even in English "register now" and "sign
in" might be a better (more distinguishable) combination, but it's much
harsher in German.
