fixes https://github.com/TryGhost/Team/issues/2937
Bumps juice to 9.1.0:
- Support for 'auto' width and height attributes
- Fixed a bug with counter-reset styles
- Dependencies updates
fixes https://github.com/TryGhost/Team/issues/3541
The email preheader, which is only present in the html version of an
email, is also included in the plaintext version of all emails. This
results in all text being duplicated twice in plaintext emails.
fixes https://github.com/TryGhost/Ghost/issues/16131
Members only content was incorrectly being shown in a plaintext email
due to the email `preheader` using the post model `plaintext` field
directly (which contained the members-only content). This changes this
behaviour so that the post html content is utilised for the `preheader`
but has all members-only content (post-preview content + segmented
content) removed
refs https://github.com/TryGhost/Ghost/issues/15725
This pull request adds a new configuration option for the Mailgun email
provider that allows the user to set the maximum number of recipients
per email batch via a new config option `bulkEmail.batchSize`
refs: https://github.com/TryGhost/Toolbox/issues/595
We're rolling out new rules around the node assert library, the first of which is enforcing the use of assert/strict. This means we don't need to use the strict version of methods, as the standard version will work that way by default.
This caught some gotchas in our existing usage of assert where the lack of strict mode had unexpected results:
- Url matching needs to be done on `url.href` see aa58b354a4
- Null and undefined are not the same thing, there were a few cases of this being confused
- Particularly questionable changes in [PostExporter tests](c1a468744b) tracked [here](https://github.com/TryGhost/Team/issues/3505).
- A typo see eaac9c293a
Moving forward, using assert strict should help us to catch unexpected behaviour, particularly around nulls and undefineds during implementation.
refs 551532f874
refs https://github.com/TryGhost/Team/issues/3324
- After analyzing data dumps, the data revealed that we have extra data from a stray batch. The filtering logic manually filters out the data to the recipients that belong to a "current batch".
- Hunting down the root cause of the data mixup proved to be too expensive of an investigation, so this is a "good enough patch" to deal with the problem.
- Most likely cause is the concurrent batch sending, but reducing the concurrency would be too expensive of a performance price to pay instead of filtering the data rarely.
refs https://github.com/TryGhost/Team/issues/3337
Moved current email customization functionality that is behind the
`makeItRain` to its own flag (`emailCustomization`) and removed the now
redundant `makeItRain` flag
closes https://github.com/TryGhost/Team/issues/3324
- When the recipients batch size is larger than the limit in addition to logging the error we need extra data to figure out what exactly is inside those `2000` or `3000` records causing faulty behavior.
- This change grabs all available models and dumps them into a file inside of the `content/data` folder. The code is temporary and should be removed once the problem is narrowed down
refs TryGhost/Team#3229
- The issue we are observing that even though the returned amount of email recipients should not ever accede the max batch size (1000 in case of MailGun), there are rare glitches when this number is doubled and we fetch 2000 records instead.
- The fix takes it's best guess in de-duping data in the batch and then truncates it if the amount of records is still above the threshold. This ensures we at least end up sending the emails out to some of the recipients instead of none.
As discussed with the product team we want to enforce kebab-case file names for
all files, with the exception of files which export a single class, in which
case they should be PascalCase and reflect the class which they export.
This will help find classes faster, and should push better naming for them too.
Some files and packages have been excluded from this linting, specifically when
a library or framework depends on the naming of a file for the functionality
e.g. Ember, knex-migrator, adapter-manager
refs TryGhost/Team#2974
- currently the unsubscribeFromNewsletters event is failing with 'member
not found' in elastic
- this change catches the error and logs it, which should allow the rest
of the event(s) to be processed
refs TryGhost/Team#2891
- test was flaking frequently enough that we had to remove it — not a perfect fix but figure it's better to enable retries than to completely remove the test
- ran CI 5 times (x 4 environments) and it passed 5 times in a row
- we previously used `@stdlib/utils` instead of the child package
`@stdlib/copy`, which is a lot smaller and contains our only use of
the parent
- this saves 140+MB of dependencies
refs https://github.com/TryGhost/Team/issues/2671
The inline style display: none; isn't applied to the images in Outlook for some reason. This change manually removes the images in the backend.
- we keep ending up with multiple versions of the depedency in our tree,
and it's causing problems when comparing instances
- the workaround I'm implementing for now is to bump the package
everywhere and set a resolution so we only have 1 shared instance
- hopefully we can come up with a better method down the line
refs https://github.com/TryGhost/Team/issues/2845
We needed to update the html out of the cards to include images for light
and dark mode, and then we've used CSS to show/hide them
Co-authored-by: Fabien "egg" O'Carroll <fabien@allou.is>
Refs https://github.com/TryGhost/Team/issues/2845
- In order to have more control over koenig card styling, we're moving some of the inline styles from the koenig repo over to the dedicated email style files.
refs https://github.com/TryGhost/Team/issues/2845
We've copied over the existing styles and html to new files and added a feature
flag based switch to choose which to render. We've also had to remove the
caching of the render function so that the switch can be dynamic and not
require a restart.
refs https://github.com/TryGhost/Team/issues/2845
Ideally the calculation of these values would be handled by a Newsletter entity
but we don't have one yet, we can look to fix this if we have time. For now
we're calculating them in separate methods to make it easier to extract in
future.
no issue
Bookshelf by default returns an empty model when requesting .related('email') for a post without an email. So we need to be a bit smarter to know if a post has an email or not. This fixed an issue where we always showed 'published and emailed' instead of 'published only'.
Since this change also included some changes to test helpers, it also made some changes to the email service because coverage dropped below 100% as a result of fixing the .related method mocking. Ideally we want to move test test helpers to a seperate package in the future.
refs TryGhost/Team#2840
- moves the `entities.decode()` step to the `LinkReplacer` class so that
it's applied to all links, not just the ones that are replaced in the
email service
- adds a test case to `LinkReplacer` to ensure that the
`entities.decode()` step is applied to all links correctly, decoding any
URLs with HTML entities in them
---------
Co-authored-by: Chris Raible <chris@ghost.org>
Refs https://github.com/TryGhost/Team/issues/2801
- It was not possible to click latest post links in Outlook due to <a>
tag wrapping around a table
- The post meta data wouldn't display properly when centered in Outlook
---------
Co-authored-by: Simon Backx <simon@ghost.org>
refs https://github.com/TryGhost/Team/issues/2805
When we render mobiledoc to HTML, it automatically escapes HTML entities in the process, so a button or directly pasted link with href="https://example.com?code=test" will be rendered as href="https://example.com?code=test" as the url is encoded in the rendered HTML. Our link tracking was using the encoded URL as the redirect URL in newsletters, causing certain links to break.
This change updates the link tracking to decode the URL with `entities.decode(url)` so we store the correct redirect URL in our DB and ensure link tracking redirects to the correct url from newsletters.
---------
Co-authored-by: Rishabh <zrishabhgarg@gmail.com>
refs https://github.com/TryGhost/Team/issues/2674
- The segment detection doesn't work outside the main post content. So the data-gh-segment attribute didn't work. It is now replaced with just a simple email replacement that is empty for a free member.
- Fixed that a trialing member was shown as 'paid'. This is now replaced with 'trialing'.
This commit also includes E2E tests for a couple of member statusses.
no issue
Some things break in some email clients with this new setting. Disabled it for now and moved the required css style to hide the member name row to @media all.
refs https://github.com/TryGhost/Team/issues/2736
If the name is not known for a member, we'll hide the name row in the subscription details in an email. This method is supported in most email clients, and requires the support of `<style>` in `<head>`.
refs https://github.com/TryGhost/Team/issues/2617
- some sites have SVG favicons and it made bookmark card favicons look broken in some email clients as the SVG support is poor across email clients
- this fix simply hides the SVG favicons in emails
fixes https://github.com/TryGhost/Team/issues/2672
- Moves the feature from behind the flag
- Also hides the comment CTA for email only posts
Co-authored-by: Sanne de Vries <sannedv@protonmail.com>
fixes https://github.com/TryGhost/Team/issues/2725
- Added to emails if labs flag enabled, comments enabled and comment CTA button enabled
- Links to comment section
- Design and styling not added yet
fixes https://github.com/TryGhost/Team/issues/2705
- Added showPostTitleSection to newsletter model in admin
- Wired up UI to admin model so it saves to the database
- Implemented showPostTitleSection in newsletter preview and added some
minor temporary css styling
- Implemented showPostTitleSection in newsletter template in backend,
and added some extra CSS styling to fix spacing
fixes https://github.com/TryGhost/Team/issues/2560
When an email fails, and you reschedule the post, the error dialog was
shown (from the previous try). The retry button on that page allowed you
to retry sending the email immediately, which could be very confusing.
- The email error dialog is no longer shown for scheduled emails
- The email status is no longer polled for scheduled emails
- Retrying an email is not possible via the API if the post status is
not published or sent
- Added some extra snapshot tests
- When retrying an email, we immediately update the email status to
'pending' to have a better API response (instead of still returning
failed).
- Disabled email sending retrying in development (otherwise very hard to
test failed emails if it takes 10 mins before it gives up automatic
retrying)
fixes https://github.com/TryGhost/Team/issues/2683
When sending a newsletter with a replacement that has a fallback, the
replacement only happens in the HTML version of the newsletter. The
plaintext version isn't replaced.
This commit fixes the issue and adds some tests to make sure it doesn't
happen again.
The cause of the issue was that we used the original matched Regex text
to replace. But that was calculated on the HTML version, so double
quotes were encoded. This change updates the generated 'token' regex to
also match on both a double quote as the escaped double quote.
no issue
Disables email sending retrying in test mode by default. This is to prevent test timeouts and to make testing more reliable in case where we manually let a batch fail.
- there's a weird situation when we have mixed versions of the
dependency because different libraries try to compare instances
- this brings the usage up to 1.2.21 so we can fix the build for now
fixes https://github.com/TryGhost/Team/issues/2562
New event fetching loops:
- Reworked the analytics fetching algorithm. Instead of starting again
where we stopped during the last fetching minus 30 minutes, we now just
continue where we stopped. But with ms precision (because no longer
database dependent after first fetch), and we stop at NOW - 1 minute to
reduce chance of missing events.
- Apart from that, a missing fetching loop is introduced. This fetches
events that are older than 30 minutes, and just processes all events a
second time to make sure we didn't skip any because of storage delays in
the Mailgun API.
- A new scheduled fetching loop, that allows us to schedule between a
given start/end date (currently only persisted in memory, so stops after
a reboot)
UI and endpoint changes:
- New UI to show the state of the analytics 'loops'
- New endpoint to request the analytics loop status
- New endpoint to schedule analytics
- New endpoint to cancel scheduled analytics
- Some number formatting improvements, and introduction of 'opened'
count in debug screen
- Live reload of data in the debug screen
Other changes:
- This also improves the support for maxEvents. We can now stop a
fetching loop after x events without worrying about lost events. This is
used to reduce the fetched events in the missing and scheduled event
loop (e.g. when the main one is fetching lots of events, we skip the
other loops).
- Prevents fetching the same events over and over again if no new events
come in (because we always started at the same begin timestamp). The
code increases the begin timestamp with 1 second if it is safe to do so,
to prevent the API from returning the same events over and over again.
- Some optimisations in handing the processing results (less merges to
reduce CPU usage in cases we have lots of events).
Testing:
- You can test with lots of events using the new mailgun mocking server
(Toolbox repo `scripts/mailgun-mock-server`). This can also simulate
events that are only returned after x minutes because of storage delays.
fixes https://github.com/TryGhost/Team/issues/2433
- Moved all outbound link tagging code to separate OutboundLinkTagger
- Because a site can easily enable/disable this feature, we don't store
the ?refs in the HTML but add them on the fly for now in the Content
API.
