refs https://github.com/TryGhost/Team/issues/2246
- This change helps avoid race conditions due to a lack of a transaction
in the email job. It also moves the status check before creating the
email batches (can take a while) to prevent other timing issues in case
the job got scheduled multiple times.
- Sets the patch option to true when changing the status of an email
batch. If we don't do this, the bookshelf-relations plugin might try to
save relations too. This could have caused a 'no rows updated' error.
- Added a test that tests if the email job can only run once
- Added logging to batching logic
- updated the cover image to be simpler
- made the change in text fixtures as well, just to keep the fixtures in sync
Co-authored-by: Hannah Wolfe <github.erisds@gmail.com>
fixes https://github.com/TryGhost/Team/issues/2085
Don't load relations we don't need anymore for the posts table. And
reload the individual post when we open the analytics page with more
relations that we actually need.
refs https://github.com/TryGhost/Team/issues/2216
The `membersActivity` flag was an alpha feature to test the first versions of member analytics, and is no longer active or in use. This change removes the remaining pieces of code that are setup behind that flag and are no longer in use or accessible.
refs: https://github.com/TryGhost/Ghost/issues/14882
This commit totally removes Bluebird from the importer. Updated `@tryghost/promise` to use native async/await and refactored importer logic to avoid the need of `reflect()`.
fixes https://github.com/TryGhost/Team/issues/1903
MembersAgent.loginAs sends email, asynchronously via events. Which
conflicts with tests that also test emails. We cannot properly await
these events, so this is currently fixed with a timeout of 200ms. But
this was too random and unreliable.
closes https://github.com/TryGhost/Team/issues/2222
Whilst we were checking for Stripe objects being active, we were not
checking for them existing in Stripe. This adds handling to all read
request to Stripe in the payment link flow, so that we can gracefully
handle deleted objects.
We've also included an automated test which fails without this fix.
We've also improved the query to find Stripe Prices which will result
in less request to the Stripe API to check if it is valid.
closes https://github.com/TryGhost/Team/issues/2207
- adds conditional to the post email serializer to switch between
`mobiledocLib` and `lexicalLib` depending on which format the post
contains
closes https://github.com/TryGhost/Team/issues/2211
We were allowing paid Tiers to be imported with non-integer prices which was
causing the Admin to be bricked when attempting to load them. This adds some
validation to the price data of Tiers.
refs https://github.com/TryGhost/Toolbox/issues/461
- The 'vary' header with 'Origin' value should only be set when an OPTIONS header is processed. Otherwise we are prone to leaking the vary header modification to further down in the request pipeline
refs https://github.com/TryGhost/Toolbox/issues/461
- The unit test was never using the "OPTIONS" request method, which did not actually trigger the full logic of the "cors" module used under the hood.
- Using the correct request method triggers all the right pathways and tests the state that's closer to the real world - for example the response does get "ended" instead of calling the "next" middleware.
refs https://github.com/TryGhost/Toolbox/issues/461
- Having a 'Origin' in vary header value present on each `OPTIONS` allows to correctly bucket "allowed CORS" and "disallowed CORS" responses in shared caches
refs https://github.com/TryGhost/Toolbox/issues/461
- Having a 'Origin' in vary header value present on each `OPTIONS` allows to correctly bucket "allowed CORS" and "disallowed CORS" responses in shared caches
refs https://github.com/TryGhost/Toolbox/issues/461
- The codebase has ambiguous behavior with OPTIONS request. Adding tests covering edge cases for all possible variations of OPTIONS responses is the first step to solving cahceability of these requests.
- The obvious question if you look into the changeset itself would also be: "WTF did you do with test suite naming? What are these changes in admin and click tracking suites? You having a bad day Naz?". The answer is "yes" (╯°□°)╯︵ ┻━┻
- On a serious note. I've introduced multiple hacks here that should be fixed:
1. Forced test suite execution order for options request - extreme blasphemy. This was last resort decision. I went deep into trying to fixup the server shutdown in the "admin" test suite, which cascaded into failing "click tracking" suite, which has shortcomings on it's own (see notes left in that suite)
2. Exposed "ghostServer" from the e2e-framework's "getAgentsWithFrontend" method. Exposing ghostServer to be able to shut it down (or do other manipulations) was one of the pitfalls we had in the previous test utils, which ended up plaguing the test codebase. Ideally the framework should only be exposing the agents and the rest would happen behind the scenes.
- To fix the hacks above I've raised a cleanup issue (https://github.com/TryGhost/Toolbox/issues/471). I'm very sorry for this mess. The issue at hand has very little to do with fixing the e2e framework, so leaving things "as is".
refs https://github.com/TryGhost/Toolbox/issues/461
- When testing OPTIONS requests there is a need to get all possible agents available in the system. The "getAgentsWithFrontend" serves exactly this purpose - create all possible agents while starting Ghost instance only once
- This is groundwork for OPTIONS request caching tests and improvements
refs https://github.com/TryGhost/Ghost/commit/1f300fb781f0
The full customer object was not being passed to the StripeAPI service
when it already exists, this was resulting in inconsistent behaviour when
sending the customerEmail param to the API, causing `invalid_email`
errors to be thrown from Stripe and breaking the checkout.
closes https://github.com/TryGhost/Team/issues/2196
We were incorrectly assuming that all requests would have the
`customerEmail` passed in the body. Instead we were incorrectly
passing `undefined` or `''` as the `customerEmail` property to stripe,
which resulted in a validation error.
We've updated the code to pass `null` in the case of a falsy value,
which the Stripe API handles without error.
closes https://github.com/TryGhost/Team/issues/2195
The issue here is two-fold, and specific to using Offers so was not
caught by any automated tests. First, we were incorrectly comparing
the tier.id to the offer.tier.id - this is because the Tier objects id
property is an instance of ObjectID rather than a string.
Secondly we were passing through the cadence parameter from the
request body, but when using Offers this is not including in the
request, so we must pull the data off of the Offer object instead and
pass that to the payments service.
closes https://github.com/TryGhost/Ghost/issues/15740
The validation function for a Tier description was not returning the
validated value, which meant we were unable to set the Tier
description.
refs https://github.com/TryGhost/Toolbox/issues/464
Bceause the import does not use the API, any backwards compat code we put in the
API does not get run for imports, this means we need to update the importer to
map the stripe_prices data onto the products table so that we have valid data in
the database.
refs: a8b1676734
- Extended the newly created handlebars test utils with a shouldCompileToError method
- Updated the price helper tests tp use shouldCompileToExpected and shouldCompileToError
- This allows us to test our handlebars helpers in a much more conisstent way
no issue
- There are currently two patterns in our handlebars helper unit tests:
1. Treating the helper as a function, and doing a function call
- This is the original way the tests were done, and they're not great as they're approximating how the helpers are really used
2. Using a template string, and rendering the string using a method called shouldCompileToExpected
- These tests are more realistic and powerful and also easier to read
- The new method is only being used in a few places so far, and each place had re-created the `shouldCompileToExpected` method
- Therefore I've moved this method into a util that should make it easier to write unit tests for handlebars helpers
- I also renamed the method in the excerpt tests, because it doesn't do the same thing, it's just a wrapper around a function call rather than compiling a string
The aim is to refactor all of our handlebars helper tests to use `shouldCompileToExpected`
- These tests are very slow, and make the build fail about 2/3 times
- Temporarily skipping until we can fix, as I want to get all our outstanding hacktoberfest PRs merged
fixes https://github.com/TryGhost/Team/issues/2175
- New event type `aggregated_click_event` that is disabled by default in all the existing activity feeds
- This returns click events, but only the first click events for each member/post combination.
- It includes the total count of unique link clicks for that member on that post combination
- Had to resort to some custom knex queries to make this work easily
- Requires `@tryghost/bookshelf-pagination@0.1.31`, included in `@tryghost/bookshelf-plugins@0.6.1` (this fixes an issue with custom selects breaking the total count query of pages)
- Went a bit overboard with the pagination tests to cover as much unknown edge cases as possible
refs https://github.com/TryGhost/Team/issues/2168
- site owners can now disable tracking sources from analytics settings.
- this change removes the loading of attribution script if tracking is
turned off so we don't capture any post/page or external source
attributions
refs https://github.com/TryGhost/Team/issues/2168
- the new setting allows site owners to control if they want to track
the sources for new member signups and subscriptions
- its switched on by default, but can be toggled off from new analytics
settings page
fixes https://github.com/TryGhost/Team/issues/2129
- This changes how the activity feed API parses the filter.
- We now parse the filter early to a MongoDB filter, and split it in two. One of the filters is applied to the pageActions, and the other one is used individually for every event type. We now allow to use grouping and OR's inside the filters because of this change. As long as we don't combine filters on 'type' with other filters inside grouped filters or OR, then it is allowed.
- We make use of mongoTransformer to manually inject a mongo filter without needing to parse it from a string value again (that would make it a lot harder because we would have to convert the splitted filter back to a string and we currently don't have methods for that).
- Added sorting by id for events with the same timestamp (required for reliable pagination)
- Added id to each event (required for pagination)
- Added more tests for filters
- Added test for pagination
- Removed unsued getSubscriptions and getVolume methods
Used new mongo utility methods introduced here: https://github.com/TryGhost/NQL/pull/49
refs https://github.com/TryGhost/Team/issues/2077
- Members and Posts test suites were using a broad tiers property matcher, which is an anti-pattern for snapshot tests. Without more specific snapshots it would be very hard to track down tier-related breaking changes!
- This change is groundwork for a refactor coming in tier usage at API's output serializers
closesTryGhost/Team#2159
- Added column to email table
- Hide the feedback tab on frontend depending on the column value
Co-authored-by: Daniel Lockyer <daniellockyer@fastmail.com>
refs: https://github.com/TryGhost/Ghost/issues/15537
- snapshot test created to add confidence to webhook stability and increase overall test coverage.
Co-authored-by: Kritika Sharma <kritikasharma@Kritikas-MacBook-Pro-2.local>
fixes https://github.com/TryGhost/Team/issues/2137
For the analytics page, we need the sent events to show up immediately
after sending an email. Otherwise we need to wait for emails to be
marked as received (which takes too long) before being able to show them
on the analytics page.
This adds the email_sent_event, which is hidden by default everywhere
and used on the analytics page.
refs: https://github.com/TryGhost/Ghost/issues/14882
- Opted to use the in-house `sequence` function when refactoring Bluebird's `Promise.each` to avoid deadlock issues (see 734ef66e6c).
-It's hard to know without tonnes of context if any `Promise.each` are safe to refactor to `Promise.all`.
refs https://github.com/TryGhost/Team/issues/2116
- allows site owners to edit a link in a post that has already been sent out, fixing any typos or other mistakes
- resets click counter for the edited link back to 0 so site owners can see the clicks on new link, doesn't change the overall click count
refs https://github.com/TryGhost/Team/issues/2135
The email link redirects on Pro are cached as 302 redirects in Varnish, so we're missing further clicks after the first one for each member, until the cache is invalidated. This change invalidates cache on link edits to ensure that we correctly redirect members to updated link everytime
refs https://github.com/TryGhost/Team/issues/2034
- this table will be used to link Stripe subscriptions to Ghost
subscriptions via a foreign key that we add at a later point
- this also includes `constraintName` as the auto-generated one would be
too long for MySQL 8
refs https://github.com/TryGhost/Team/issues/2104
- adds edit permissions for links endpoints to fixtures
- new `bulkEdit` endpoint will use the permissions and allow fixing newsletter links via Admin
refs 5fcf5098a8
- links browse endpoint had permissions switched off unintentionally and was also missing the necessary permissions in fixtures.
- enables permissions for browse endpoint and adds migration insert permissions in DB
fixes https://github.com/TryGhost/Team/issues/2090
- This changes how sentiment is exposed in the API. Now it is exposed as a `sentiment` relation, directly on the model (no longer in counts). Internally we still use `count.sentiment`.
- Content API users (and themes) can include the 'sentiment' relation and order by sentiment.
- Updated Admin to use sentiment instead of count.sentiment
fixes https://github.com/TryGhost/Team/issues/2091
fixes https://github.com/TryGhost/Team/issues/2089
- Added new fixtures to make testing easier for the activity feed
- Improved E2E test coverage of activity feed with separate test file
- Added data.post_id filter to enable filtering by events related to a
given post
- Fixed return types in JSDoc of test agents (TypeScript interprets
these as `typeof Agent` if we don't add `InstanceType<Agent>`)
- Added total pagination metadata to activity feed API (to allow a basic
type of pagination using filters)
fixes https://github.com/TryGhost/Team/issues/2096
When generating the recipient data for emails, the email clicks
implementation is resulting in a recipient variable being added called
replacement_xxx once for each link containing the same UUID.
This generates a lot of unnecessary data overhead for emails, and it
turns out that mailgun has a 25MB message limit. We wouldn't have come
close if we only included the uuid once.
fixes https://github.com/TryGhost/Team/issues/2102
- this column was added with `nullable: true` but it should never be
nullable, so we should drop the nullable status whilst it's easy to
fixes https://github.com/TryGhost/Team/issues/2084
- When audience feedback is enabled, we use a single 'conversions' count instead of having separate ones for signups and paid conversions.
- The analytics component is separated so we can change it without breaking the existing page.
refs https://github.com/TryGhost/Team/issues/2072
Google is indexing our redirects and storign the redirected content
against the redirect URL in search results. This seems to be caused by
us using a 302 redirect rather than 301. We don't want to switch to a
301 however, so that we can support the ability to update redirects in
the future.
refs https://github.com/TryGhost/Team/issues/1765
In order to better handle deleted objects in Stripe we want to decouple
Members from Stripe.
These changes allow us to have the Tier concept completely independent
of the Stripe tables, such that the Stripe data can be generated as/when
it's needed - which will help to protect against missing data.
closes: https://github.com/TryGhost/Ghost/issues/14973
- When fetching content using a non-standard charset, characters were notproperly decoded to utf-8 resulting in mangled text in the editor -> Detect charset and use iconv to decode the page text
- When requesting a non bookmark card, if no oembed data could be foundand we fallback to bookmark, a second network request to fetch the content was issued. This seemed unnecessary -> refactored to avoid that
refs: https://github.com/TryGhost/Ghost/issues/15537
- snapshot test created to add confidence to webhook stability and increase overall test coverage.
Co-authored-by: Kritika Sharma <kritikasharma@Kritikas-MacBook-Pro-2.local>
refs https://github.com/TryGhost/Team/issues/2030
- adds `subscriptions` table to the DB schema
- this new table is aimed to support a native "subscription" primitive in Ghost
that most resembles previously used `members_stripe_customers_subscriptions` table
refs https://github.com/TryGhost/Toolbox/issues/441
- whilst reviewing another PR, I noticed we were incorrectly using
`maxLength` instead of `maxlength` in the schema column definition
- it turns out we've already been doing this wrong for a while with
other columns
- this key is not acted upon, so the maximum column length was not applied
- fixing up the DB to the correct maximum length is something to fix in the
future but right now, the schema does not reflect the size of the
column that actually got created
- the fallback when `maxlength` is not provided is currently 191 [0], so
this commit switches the schema and migrations to using the correct
key name and column length that they are using when applied
[0]: 24670aa555/ghost/core/core/server/data/schema/commands.js (L27)
refs https://github.com/TryGhost/Toolbox/issues/441
- we tend to have a mix of `bool` and `boolean` in the schema and
migrations, which has become a real nit for me at this point
- we don't do any special handling between `bool` and `boolean`, it's
just something we pass to Knex
- `bool` is an alias for `boolean` but `boolean` is actually documented - https://knexjs.org/guide/schema-builder.html#boolean
- this commit switches Ghost to only using `boolean` in the schema and
migrations, and removes `bool` from the allowlist in tests to prevent
us from adding it again in the future
- this should make absolutely no difference to the DB because both
resulted in the same column
refs https://github.com/TryGhost/Toolbox/issues/441
- I'm currently working on cleaning up our uses of `bool` and `boolean`
in favor of `boolean`, and I've noticed we only handle converting
numbers into booleans when the type is `bool`, so validation would
otherwise fail
- given these can be used interchangeably, we should also support
converting the numbers into booleans when the type is `boolean`
- this is going to get cleaned up again when I remove `bool` but this
fixes the validation bug for now
refs: https://github.com/TryGhost/Ghost/issues/15537
- snapshot test created to add confidence to webhook stability and increase overall test coverage.
Co-authored-by: Kritika Sharma <kritikasharma@Kritikas-MacBook-Pro-2.local>
refs: https://github.com/TryGhost/Ghost/issues/15537
- this adds an e2e test and test snapshot for the `tag.edited` webhook so we can prevent regressions and bugs in the future
Co-authored-by: Hannah Wolfe <github.erisds@gmail.com>
closes: https://github.com/TryGhost/Ghost/issues/14981
- Taxonomy-specific sitemaps were invalid xml when there was no data
- These invalid empty sitemaps were referenced in the index sitemap causing SEO tools to report errors
closes: https://github.com/TryGhost/Ghost/issues/15470
- When multiple browser tabs are open, each manipulate a different copy of ember data model, changes to the model in one tab are not reflected in the model of the other tab.
- When updating some settings, all current settings were sent to the API.
- As a result, when updating two different categories of settings (navigation/code inspection) in different tabs, the second update was overriding the first one.
- From a user perspective, this is not a natural behaviour. Only settings visible on-screen when clicking save should be modified.
Co-authored-by: Kevin Ansfield <kevin@lookingsideways.co.uk>
refs https://github.com/TryGhost/Toolbox/issues/441
- this is only v1 of the test I would like but it validates the keys on
a column definition are part of an allowlist
- this has already uncovered a bug with `maxLength` (vs `maxlength`)
fixes https://github.com/TryGhost/Team/issues/2054
This change adds the sentiment and positive_feedback counts to the posts models. This change isn't really ideal because there are some problems here:
- sentiment isn't really a count
- we don't need to include the sentiment and positive_feedback as a default for posts (but the same is true for attribution)
It would make sense to move this to separate endpoints that only fetch the analytics for a given post when the analytics page is opened. But for our initial skateboard version of audience feedback this should be a good start to already see the data.
refs https://github.com/TryGhost/Team/issues/2047
- We anticipate upcoming changes in the PUT /members/:id/subscriptions/:subscription_id endpoint , so covered it with a snapshot test to track the differences more precisely.
- Note, the test case contains a more explicit outgoing HTTP request mocking.
refs https://github.com/TryGhost/Team/issues/1967
This tests the full flow of publishing a newsletter, and then checking
that clicked links will increase the click count, generate events for
the member which clicked the link as well as the redirects contain the
correct query params.
refs https://github.com/TryGhost/Toolbox/issues/320
- Added more complex mobiledoc structure in the post.published test to check for correct transformation of special purpose `__GHOST_URL__`. The snapshot has a correct URL transformation, which gives confidence it works properly
refs https://github.com/TryGhost/Toolbox/issues/320
- There noe "roles" attached to the post's author when the 'post.added' event is fired. Webhooks function based of the model events and differ slightly with it's output comparing to the API response. For example, in case of Posts API, there'a an additional 'findOne' call (ref.: https://github.com/TryGhost/Ghost/blob/main/ghost/core/core/server/models/post.js#L1224-L1227) before returning the post to the endpoint handler and then passing that to the output serializer.
- If we want to have 1:1 copy of webhooks outputs and API outputs, we should rethink how we rely on model event data which is never the same as API controller level data.
refs a499f866f3
refs d817e5830d
- The user-agent used in outgoing Ghost requests (webhooks mostly) is dependent on the Ghost version - snapshots break if the matcher is not dynamic.
- There will be a few more webhooks tests coming soon, so makes sense to have this matcher moved to a common "framework matchers"
fixes https://github.com/TryGhost/Ghost/issues/14508
This change requires the frontend to send an explicit `emailType` when sending a magic link. We default to `subscribe` (`signin` for invite only sites) for now to remain compatible with the existing behaviour.
**Problem:**
When a member tries to login and that member doesn't exist, we created a new member in the past.
- This caused the creation of duplicate accounts when members were guessing the email address they used.
- This caused the creation of new accounts when using an old impersonation token, login link or email change link that was sent before member deletion.
**Fixed:**
- Trying to login with an email address that doesn't exist will throw an error now.
- Added new and separate rate limiting to login (to prevent user enumeration). This rate limiting has a higher default limit of 8. I think it needs a higher default limit (because it is rate limited on every call instead of per email address. And it should be configurable independent from administrator rate limiting. It also needs a lower lifetime value because it is never reset.
- Updated error responses in the `sendMagicLink` endpoint to use the default error encoding middleware.
- The type (`signin`, `signup`, `updateEmail` or `subscribe`) is now stored in the magic link. This is used to prevent signups with a sign in token.
**Notes:**
- Between tests, we truncate the database, but this is not enough for the rate limits to be truly reset. I had to add a method to the spam prevention service to reset all the instances between tests. Not resetting them caused random failures because every login in every test was hitting those spam prevention middlewares and somehow left a trace of that in those instances (even when the brute table is reset). Maybe those instances were doing some in memory caching.
fixes https://github.com/TryGhost/Ghost/issues/14508
This change requires the frontend to send an explicit `emailType` when sending a magic link. We default to `subscribe` (`signin` for invite only sites) for now to remain compatible with the existing behaviour.
**Problem:**
When a member tries to login and that member doesn't exist, we created a new member in the past.
- This caused the creation of duplicate accounts when members were guessing the email address they used.
- This caused the creation of new accounts when using an old impersonation token, login link or email change link that was sent before member deletion.
**Fixed:**
- Trying to login with an email address that doesn't exist will throw an error now.
- Added new and separate rate limiting to login (to prevent user enumeration). This rate limiting has a higher default limit of 8. I think it needs a higher default limit (because it is rate limited on every call instead of per email address. And it should be configurable independent from administrator rate limiting. It also needs a lower lifetime value because it is never reset.
- Updated error responses in the `sendMagicLink` endpoint to use the default error encoding middleware.
- The type (`signin`, `signup`, `updateEmail` or `subscribe`) is now stored in the magic link. This is used to prevent signups with a sign in token.
**Notes:**
- Between tests, we truncate the database, but this is not enough for the rate limits to be truly reset. I had to add a method to the spam prevention service to reset all the instances between tests. Not resetting them caused random failures because every login in every test was hitting those spam prevention middlewares and somehow left a trace of that in those instances (even when the brute table is reset). Maybe those instances were doing some in memory caching.
no issue
- All content-length snapshots should be using the same matcher for consistency - anyContentLength. It's more explicit about what the matcher is all about and might be useful to have content-length matchers in one place if it ever changes (the header value should be a damn digit after all, not a string!) (ref. https://www.rfc-editor.org/rfc/rfc7230#section-3.3.2)
refs https://github.com/TryGhost/Team/issues/2024
Without validation it was possible to send a string of comma separated
email addresses to the endpoint, and an email would be sent to each
address, bypassing any rate limiting.
This bug does not allow for an authentication bypass exploit. It is purely a
spam email concern.
Credit: Sandip Maity <maitysandip925@gmail.com>
refs https://github.com/TryGhost/Toolbox/issues/320
- The URL matcher is very likely to be reused in the future, so having it abstracted away gives two benefits:
1. Central place to document hacky behavior and easier future cleanup
2. The implementer of the e2e test does not have to see the "hacky note" and just concentrate on the implementation of the test
refs https://github.com/TryGhost/Toolbox/issues/320
- Header snapshot matching was missing from webhook e2e tests. With a bumped version of webhook-mock-receiver it's now possible to record and match webhook request headers.
closesTryGhost/Team#2007
- uses request context to add referrer source and medium for a new member
- uses integration name as referrer medium if exists
fixes https://github.com/TryGhost/Team/issues/2008
- New column that stores email click tracking at the time it was created
- Improved frontend side checks for when to show analytics
refs https://github.com/TryGhost/Toolbox/issues/425
refs https://github.com/TryGhost/Toolbox/issues/280
- The versioned API responses vary based on requested version (passed in request's 'accept-version' header). shared caches that sit between Ghost's origin server and the browser would be putting responses with same Vary into the same caching bucket, which is incorrect.
- This change makes response's Vary more granular and tells caching mechanisms to take 'Accept-Version' request header into account when caching.
- Informative read on the topic - https://www.fastly.com/blog/getting-most-out-vary-fastly
- renames `refSource`, `refMedium` and `refUrl` to `referrerSource`, `referrerMedium` and `referrerUrl` respectively for consistent naming across files and usages
- bumps member attribution script from alpha feature to now load for all sites. The script captures recent url history in localstorage to capture correct attribution for members.
- script is only loaded on the site if members is enabled
refs https://github.com/TryGhost/Team/issues/1967
- Test is good to test if the whole flow works as expected, and works together
- We can test independent parts in separate tests that have better coverage of more edge cases
- Adds a basic helper to get an agent for the frontend (spent too much time on a better solution so I decided to keep the existing supertest agent)
refs https://github.com/TryGhost/Ghost/pull/15471#discussion_r979902374
- the accent color value used by default content cta was copying the global site property which is redundant, and can be directly used
- originally, the accentColor property was extended to allow a fallback value for content ctas, but was later removed as we added default value to global site property directly
- the accentColor property is now deprecated and will be removed in next version, as existing themes might be relying on it for custom cta helpers
closes https://github.com/TryGhost/Team/issues/1898
- the default content cta always used the terminology as `post` when showing message that users don't have access to some content
- this caused confusion when users were looking at a page and message showed "This post is for subscribers only"
- updates the message to correctly reflect `page` vs `post` on the default cta
refs https://github.com/TryGhost/Toolbox/issues/410
- Private cache control was preventing browser or shared caches from storing Content APIs response. The type of data served through the Content API is very much of a "public" nature, so should be cacheable.
- Right now the 'max-age' value of 'cache-control' header is hardcoded to '0', without 'must-revalidate' value, to allow browsers to cache content slightly more aggressively. In the future the 'max-age' value will most-likely become configurable to allow even more aggressive HTTP caching.
refs https://github.com/TryGhost/Toolbox/issues/410
- The 'private' value in 'Cache-Control' response header for all errors made it impossible for shared caches (e.g.: Fastly, Cloudflare) to cache 404 responses efficiently.
- The change substitutes 'max-age=0' which should not effect the browser cache behavior but would allow shared caches to process such requests efficiently.
- A more loose caching logic only applies to 404 responses from GET requests that are not user-specific (non-authenticated, non-cookie containing requests)
closes https://github.com/TryGhost/Toolbox/issues/372
- The admin assets are served with a unique hash depending on the build with a year-long "max-age" value in the response cache-control header. The client browsers still do send 'If-None-Match' requests when there is a hard-refresh on the client side. There's no need for 'If-None-Match' requests though!
- With 'immutable' value in the cache-control header, the browser caches are treating responses as "hard-fresh" without sending redundant requests.
- For more about 'immutable' value read https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control#immutable
closes https://github.com/TryGhost/Team/issues/1942
- Added data fixtures for referrers
- Added new endpoint to fetch referrer stats for a given post: `/stats/referrers/posts/:id`
- Added new ReferrersStatsService, responsible for calculating referrer stats
refs TryGhost/Team#1931
- referrer source, medium and url will be stored in the events table along with rest of attribution data
- stores referrer information on two tables
- `members_created_events` for signups
- `members_subscription_created_events` for paid conversions
closes https://github.com/TryGhost/Team/issues/1933
- Added click_events to activity feed
- Added support for parsing click_events in the frontend
- Moved url parsing (transform ready) to model layer of LinkRedirect
- Moved `getEventTimeline` method to the top of the event repository
- Added description field to parsed events in the frontend (because we need a second line)
- Fixed: member email not returned in comment_event
refs d5f03ec0b1
- underlying error message varies across node versions so the content-length can't be fixed
- applied any-content-length matcher to the right test this time
no issue
- bumped `@tryghost/url-utils` to get access to the new lexical transform utilities
- updated the Post model's `parse()` and `formatOnWrite()` methods to transform the `lexical` field contents when reading/writing to ensure any links in content point at the correct place with `site.url` config changes
refs TryGhost/Team#1907
- calculates final attribution source and medium using captured referrer information in history
- adds new referrer-translator that goes through available history and based to determine most valid referrer info
- includes referrer url, source and medium in the attribution data for storage
no issue
- The explore endpoint needs to expose the total amount of published posts
- To be more consistent, this PR creates a PostStats class which is exposed as `stats` method within the PostService; just like it's done with the MemberService
- Moved existing method to return the date of the most recently published post into the stats service
- Updated the explore service test to reflect the new return property
no issue
- added `PostRevsion` model
- duplicated `mobiledoc_revision` creation routine in Post model's onSaving hook to create `post_revision` when model's `lexical` field has changed
- updated `mobiledoc_revision` creation to skip when `lexical` field is populated
no issue
- initially this will perform the same function as `mobiledoc_revisions` but storing `lexical` instead of `mobiledoc`
- naming is intentionally generic ready for later expansions
closes https://github.com/TryGhost/Team/issues/1908
### Problem:
- We need tracking on the paywall links in each email. (we cannot ignore them because those buttons are probably gonna have a higher paid conversion attribution than others).
- Currently we only add the paywall HTML to an email when processing each batch. So if we batch an email to 1.000 recipients per 100, we'll generate the paywall HTML 10 times.
- We cannot replace links in `renderEmailForSegment` because that methods will get called multiple times. We don't want to have multiple redirect instances created for the same link in the same email.
### Solution:
- Move the generation of the paywall to the `serialize` method of the post email serializer.
- Surround the generated paywall with HTML-comments so we can remove it if required in `renderEmailForSegment` depending on the member segment we are sending the email to.
---
### Before:
**Serialize output:**
```html
<html>
<body>
<h1>Generated email header</h1>
<p>Generated text</p>
<div>
<!-- POST CONTENT START -->
<h1>Post title</h1>
<p>Content visible for all members</p>
<!--members-only-->
<p>Content visible for paid members only</p>
<!-- POST CONTENT END -->
</div>
</body>
</html>
```
To be modified later by `renderEmailForSegment`:
**Paid members (nothing changed):**
```html
<html>
<body>
<h1>Generated email header</h1>
<p>Generated text</p>
<div>
<!-- POST CONTENT START -->
<h1>Post title</h1>
<p>Content visible for all members</p>
<!--members-only-->
<p>Content visible for paid members only</p>
<!-- POST CONTENT END -->
</div>
</body>
</html>
```
**Free members (paywall _added_):**
```html
<html>
<body>
<h1>Generated email header</h1>
<p>Generated text</p>
<div>
<!-- POST CONTENT START -->
<h1>Post title</h1>
<p>Content visible for all members</p>
<h2>Generated paywall here</h2>
<a href="https://subscribe.com">Subscribe to read the full post</a>
<!-- POST CONTENT END -->
</div>
</body>
</html>
```
### After this change:
**Serialize output:**
```html
<html>
<body>
<h1>Generated email header</h1>
<p>Generated text</p>
<div>
<!-- POST CONTENT START -->
<h1>Post title</h1>
<p>Content visible for all members</p>
<!--members-only-->
<p>Content visible for paid members only</p>
<!-- PAYWALL -->
<h2>Generated paywall here</h2>
<a href="https://subscribe.com/?tracked">Subscribe to read the full post</a>
<!-- POST CONTENT END -->
</div>
</body>
</html>
```
To be modified later by `renderEmailForSegment`:
**Paid members (paywall removed):**
```html
<html>
<body>
<h1>Generated email header</h1>
<p>Generated text</p>
<div>
<!-- POST CONTENT START -->
<h1>Post title</h1>
<p>Content visible for all members</p>
<!--members-only-->
<p>Content visible for paid members only</p>
<!-- POST CONTENT END -->
</div>
</body>
</html>
```
**Free members (members-only content removed):**
```html
<html>
<body>
<h1>Generated email header</h1>
<p>Generated text</p>
<div>
<!-- POST CONTENT START -->
<h1>Post title</h1>
<p>Content visible for all members</p>
<!-- PAYWALL -->
<h2>Generated paywall here</h2>
<a href="https://subscribe.com/?tracked">Subscribe to read the full post</a>
<!-- POST CONTENT END -->
</div>
</body>
</html>
```
no issue
- added `@tryghost/kg-lexical-html-renderer` dependency
- added `lexical` lib following the same pattern as our `mobiledoc` lib
- updated the Post model's `onSaving` hook to generate the `html` value from `lexical` when present
fixes https://github.com/TryGhost/Team/issues/1909
- The feature image caption is already escaped on the frontend
- Doing it again in the backend breaks the possibility to add links to the caption
- I checked and the `feature_image_alt` is not escaped in the frontend.
fixes https://github.com/TryGhost/Team/issues/1900
refs https://github.com/TryGhost/Team/issues/1901
- Defaults to the same value as the current email_track_opens setting for existing installations, otherwise defaults to true
- Had to use a custom migration because the `addSetting` helper doesn't support using an existing setting as current value
- Added a minimal UI to change the setting, but this still needs some design magic 🪄✨
- Link replacement is disabled if `email_track_clicks` is disabled. In the future we might consider to still do parial additions, such as source attribution and maybe redirects (to discuss).
refs https://github.com/TryGhost/Team/issues/1899
- Added `addEmailAttributionToUrl` method to MemberAttributionService. This adds both the source attribution (`rel=newsletter`) and member attribution (`?attribution_id=123&attribution_type=post`) to a URL.
- The URLHistory can now contain a new sort of items: `{type: 'post', id: 'post-id', time: 123}`.
- Updated frontend script to read `?attribution_id=123&attribution_type=post` from the URL and add it to the URLHistory + clear it from the URL.
- Wired up some external dependencies to LinkReplacementService and added some dummy code.
- Increased test coverage of attribution service
- Moved all logic that removes the subdirectory from a URL to the UrlTranslator instead of the AttributionBuilder
- The UrlTranslator now parses a URLHistoryItem to an object that can be used to build an Attribution instance
- Excluded sites with different domain from member id and attribution tracking
We're going to be adding more redirection logic into Ghost and it's
going to get confusing if we have names this generic. This makes it
clear which feature this service is related to.
Ideally in the future we can combine all of these into one redirects
service, but for now we will be running a specific service per feature
no issue
- fixed API returning "Invalid mobiledoc structure" errors when `mobiledoc:null` is sent in the payload alongside `lexical: '{...}'`
- updated Admin's `posts` and `pages` adapters to always add `?formats=mobiledoc,lexical` because the API doesn't return `lexical` by default
- added `lexical` attribute to Admin's Post model
- updated `lexical-editor` controller and related components to work with `lexical` always being a JSON string rather than a parsed object
- updated `<KoenigLexicalEditor>` to pass through the lexical state string as initial state and wired up the `onChange` prop
no issue
- bumped `@tryghost/admin-api-schema` to allow passthrough of the `lexical` property on post and page API endpoints
- prevented saving of blank document in the `mobiledoc` field if `lexical` is provided
- prevented API input containing both `mobiledoc` and `lexical` fields to avoid issues when both are present:
- not possible to know which content is latest/has precedence
- not possible to know which editor should be displayed in Admin
refs 9471384020
- previously added tests (any subsequent matcher updates) for browse endpoint were not using matchers that sufficiently covered the dynamic portions of the body
no issue
- left `mobiledoc` as the only default format added in the post/page input serializers for now to minimize API/test churn during these early stages of lexical development
- tested that the `lexical` field is not returned by default but can be requested via `?formats=lexical`
