Commit Graph

162 Commits

Author SHA1 Message Date
Naz
5d5b77d32f
Fixed comped tier assignment
closes https://github.com/TryGhost/Team/issues/1869

- When there were "archived" tiers in the system the importer incorrectly fetched them instead of only taking "active" ones into account. The "getDefaultProduct" on product repository does exactly that.
- Additionally, reusing the "getDefaultProduct" makes testing the importer slightly less complex.
2022-10-20 17:19:52 +08:00
Naz
b589a66cd4
Fixed broken CSV importer tests
refs 90768e9985

- With introduction of strict field mapping the regression test testing for "imports of not mapped fields" failed.
2022-10-19 18:33:47 +08:00
Naz
90768e9985
Added strict field mapping to member CSV importer
closes https://github.com/TryGhost/Toolbox/issues/430

- The members importer used to  import all fields present in the uploaded CSV if the headers match, even if they're not mapped in the UI. This behavior has lead to have misleading consequences and "hidden" features. For example, if the field was present but intentionally left as "Not imported" in the UI the field would still get imported.
- Having a strict list of supported import fields also allows for manageable long-term maintenance of the CSV Import API and detect/communicate changes when they happen.
- The list of the current default field mapping is:

    email: 'email',
    name: 'name',
    note: 'note',
    subscribed_to_emails: 'subscribed',
    created_at: 'created_at',
    complimentary_plan: 'complimentary_plan',
    stripe_customer_id: 'stripe_customer_id',
    labels: 'labels',
    products: 'products'
2022-10-19 18:10:40 +08:00
Naz
748ef87954
Made running the import outside of job on test env
- Allows to write tests for the importer easier when there is a "subscription" or a "product" present
2022-10-19 18:10:40 +08:00
Naz
9389064ae2
Removed unused error message
no issue

- The job-related code was ripped out form the importer and this message was just an overlooked leftover
2022-10-19 18:10:40 +08:00
Naz
f0b68846cc
Moved header mapping configuration to importer
refs https://github.com/TryGhost/Toolbox/issues/430

- To be able to introduce strict mapping rules (exclude unknown fields) we need to control the CSV header mapping on the importer level. This change moves the configuration up from CSV parser to the importer
- Also adds tests covering correct inserts for specially treated "subscribed_to_emails" field
2022-10-19 18:10:40 +08:00
Naz
c08d4f8ad1
Added basic test coverage for perform method
refs https://github.com/TryGhost/Toolbox/issues/430

- "perform()" is what gets executed by the import job for both immediate import and "inline job" import. Testing it on granular level will allow to change it with more confidence when introducing strict field mapping rules
2022-10-19 18:10:40 +08:00
Naz
8dc630b3a0
Removed cleaned up use of "Job" object
refs https://github.com/TryGhost/Toolbox/issues/430

- Importer code was filled with an unnecessarily complex "job" object that was passed around. It had an "id" property, which confusingly was a path to a file at all times.
- Simplified the logic significantly by keeping and passing around the path to a "prepared" members CSV.
2022-10-19 18:10:40 +08:00
Naz
812973d962
Removed unused concept of "status" in importer job
refs https://github.com/TryGhost/Toolbox/issues/430

- The job "status" is never anything different than "pending" and never leaves the module itself. It's an outdated concept that only takes up lines of code!
2022-10-19 18:10:40 +08:00
Naz
2175d64095
Added future investigation note
refs https://github.com/TryGhost/Toolbox/issues/430

- This is a strange hardcoded value that seems like some legacy leftover concept we could do without or improve
2022-10-19 18:10:40 +08:00
Naz
d165a0017c
Extracted MembersCSVImporter to a builder method
refs https://github.com/TryGhost/Toolbox/issues/430
refs https://github.com/TryGhost/Ghost/issues/14882

- The MembersCSVImporter constructor is way to complex and needs refactoring. This complexity makes initialization in tests too bulky and makes tests hard to read.
- Having a builder method is a stopgap solution to avoid going into MembersCSVImporter refactoring too deep.
2022-10-19 18:10:40 +08:00
Naz
4300f14d79
Removed hidden row mapping in csv parser
refs https://github.com/TryGhost/Toolbox/issues/430
refs https://github.com/TryGhost/Ghost/issues/14882

- Having an explicit mappings passed into the members CSV parser makes it easier to control and understand the transforms for package clients
- Eventually the parser will receive a strict map with the fields it should parse - skipping all unknown & unmapped fields
2022-10-19 18:10:40 +08:00
renovate[bot]
e9587e02d0
Update dependency mocha to v10.1.0 2022-10-17 08:41:28 +00:00
Naz
9e20544927
Fixed typo 2022-10-13 09:53:16 +08:00
Naz
71bed4f926
Fixed jsdoc to only mention used parameters
refs https://github.com/TryGhost/Toolbox/issues/430

- It's a minor cleanup while exploring the referenced bug surface area. Fixes distracting "missing parameters" warnings
2022-10-13 09:53:16 +08:00
renovate[bot]
6fc497743d
Updated @tryghost dependencies (#15479)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2022-10-12 10:16:05 +07:00
Rishabh Garg
8a598fe721
🐛 Fixed member importer crash for failed imports (#15560)
refs/closes https://github.com/TryGhost/Team/issues/2004

- for imports, members are created inside a transaction, which causes the member created events to be dispatched.
- its possible that transactions for import can be rolled back if for some reason there is an error down the line while inserting other member properties. The rollback doesn't commit the member to DB, but the event dispatched earlier will still try to create the member created event which fails due to missing member id.
- knex transactions resolve the `executionPromise` both in case of explicit commit or rollback from the user, so just the transaction end check will not be good enough to make sure the member exists in DB
- adds explicit config to knex to reject transaction in case of rollback, which is then caught and event is not dispatched
2022-10-07 19:15:18 +05:30
renovate[bot]
2c2ee81adb
Update Test & linting packages 2022-10-05 00:36:08 +00:00
Simon Backx
99e6f8ddda Moved analytics page to separate component
refs https://github.com/TryGhost/Team/issues/1976
refs https://ghost.slack.com/archives/C02G9E68C/p1664446234131629

- @tracked properties in controllers are persisted, should use components instead
2022-09-29 12:41:16 +02:00
renovate[bot]
9eb3c84a23
Updated @tryghost dependencies (#15434)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2022-09-27 08:31:35 +07:00
renovate[bot]
225765241c
Updated @tryghost dependencies (#15404)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2022-09-19 12:09:33 +01:00
Simon Backx
f9e5d57f0d 🎨 Added number formatting to members import email
refs https://ghost.slack.com/archives/C02G9E68C/p1663318892246929?thread_ts=1663317382.429489&cid=C02G9E68C
2022-09-16 11:13:12 +02:00
Peter Zimon
2769578dbd Various Members importer UX improvements
no refs.

- CSV overview table headings were selectable
- spinner on the last screen was confusing: users might think they had to wait for something to finish, but that's not true
- spacings in import confirmation emails were off
2022-09-16 10:50:54 +02:00
Rishabh Garg
c16abbf085
🐛 Fixed empty error csv file for member imports (#15274)
closes https://github.com/TryGhost/Team/issues/1828

- members importer was sending empty error files in case of invalid rows in CSV, hiding both error and affected rows
- fixes typo in `content` option(was `contents` before) passed to attachment (ref - https://nodemailer.com/message/attachments )
2022-08-24 00:49:30 +05:30
Daniel Lockyer
f51226e5fb Organized package dependencies
- cleaned up unused dependencies
- adds missing dependencies that are used in the code
- this should help us be more explicit about the dependencies a package
  uses
2022-08-18 11:55:49 +02:00
Daniel Lockyer
54aa9f016b Fixed full Admin test suite running during unit tests
- because of how the npm scripts were set up, we were running the full
  Admin integration tests during the unit tests phase of CI
- this commit renames the majority of `test` to `test:unit` in the
  package.json files, and aliases `test` to `test:unit`
- special packages like Admin have no-op'd `test:unit` scripts so we
  don't end up running its tests
2022-08-15 15:34:52 +02:00
Simon Backx
82a3133ace
Added replies/likes count and limited replies to comments (#15206)
refs https://github.com/TryGhost/Team/issues/1723

- Added count.replies to comments
- Added replies endpoint
- Limited returned replies to 3.
- Replaced likes_count with count.likes in comments
- Instead of fetching all the likes of a comment to determine the total count, we'll now use count.likes
- Instead of fetching all the likes of a comment to determine whether a member liked a comment, we'll now use count.liked (which returns the amount of likes of the current member, being 0 or 1). This is mapped to `liked` to make it more natural to work with.

The `members.test.snap` file changed because we no longer include `liked: false` if we didn't fetch the liked relation. And in the comments events of the activity feed the liked property is therefore removed.

These changes requires an update to the `bookshelf-include-count` plugin:
- Updated to also work for nested relations
- This moves the count queries from the `bookshelf-include-count` plugin to the `countRelations` method of each model.
- Updated to keep the counts after saving a model (crud.edit didn't return the counts before)
2022-08-10 16:12:35 +02:00
Daniel Lockyer
a9cffd9550
Pinned all dependencies
- we should pin all dependencies so we know what is getting shipped in
  the application
2022-07-27 08:44:50 +02:00
Daniel Lockyer
255eb0726b
Cleaned up package metadata
refs https://github.com/TryGhost/Toolbox/issues/354

- set packages to `private: true`
- removed repository link - these packages won't be published so this
  link won't be seen anywhere
- removed `publishConfig`
2022-07-26 15:08:05 +02:00
Daniel Lockyer
308a28d31a
Tidied up package READMEs
refs https://github.com/TryGhost/Toolbox/issues/354

- these READMEs were migrated over from when each package was in a
  different repo
- they also assume you're going to be publishing the packages because it
  mentions install instructions
- only a few of them contain custom content
- this commit deletes the majority of these files because they're now
  not useful
- any that contained other instructions have been cut down
2022-07-25 15:17:12 +02:00
Daniel Lockyer
61125d7605
Removed repository from component package.json files
refs https://github.com/TryGhost/Toolbox/issues/354

- these repository links made sense when they were in different repos
  and published to NPM but we don't publish these packages any more
- this commit deletes those keys from the files
2022-07-25 11:15:16 +02:00
Daniel Lockyer
d73d7da7ef
Deleted non-root LICENSE files
- these were copied over during the monorepo conversion but we're not
  going to be publishing these packages so the top-level LICENSE file
  covers all packages here
2022-07-25 08:35:59 +02:00
Daniel Lockyer
17bf358b47
Pinned internal packages
- we're going to be pinning all dependencies within the monorepo
- this shouldn't change anything anyway because we're using the same
  version across all packages
2022-07-21 16:35:19 +02:00
renovate[bot]
d228144c27 Pin dependencies 2022-07-21 16:27:57 +02:00
Daniel Lockyer
48fd5ca8cb
Removed posttest linting step
- linting is done as a separate step in CI and it's a git hook upon
  pushing locally, so we don't need to run it after tests
2022-07-21 10:09:36 +02:00
Daniel Lockyer
9ec83a6e21
Disabled publishing for Members packages
- we don't want to publish these anymore so this commit disables the
  ability to
- also fixes up a missing version that wasn't reset
2022-07-21 09:26:04 +02:00
Daniel Lockyer
376ee24600
Switched to unversioned Members packages
- these packages are split apart for local development, but will be
  bundled into Ghost when publishing
- therefore, these packages won't be published so we are resetting the
  versions to make them cleaner
2022-07-21 09:15:29 +02:00
renovate[bot]
bc4aebf163 Update Test & linting packages 2022-07-19 18:55:22 +00:00
Simon Backx
5bef2a1068 Published new versions
- @tryghost/members-api@8.1.2
 - @tryghost/members-importer@0.5.16
2022-05-25 13:15:17 +02:00
Simon Backx
14a7d1f00f Cleaned up multipleProducts and multipleNewsletters flags 2022-05-25 10:25:02 +02:00
Simon Backx
90fc7f5064 Published new versions
- @tryghost/members-csv@1.2.16
 - @tryghost/members-importer@0.5.15
2022-05-19 18:11:03 +02:00
Fabien "egg" O'Carroll
c41f067ea8 Published new versions
- @tryghost/domain-events@0.1.14
 - @tryghost/express-dynamic-redirects@0.2.13
 - @tryghost/magic-link@1.0.26
 - @tryghost/member-analytics-service@0.1.16
 - @tryghost/member-events@0.4.6
 - @tryghost/members-analytics-ingress@0.1.17
 - @tryghost/members-api@8.0.0
 - @tryghost/members-csv@1.2.15
 - @tryghost/members-events-service@0.4.3
 - @tryghost/members-importer@0.5.14
 - @tryghost/members-ssr@1.0.28
 - @tryghost/members-offers@0.11.6
 - @tryghost/members-payments@0.3.6
 - @tryghost/members-stripe-service@0.10.5
 - @tryghost/verification-trigger@0.2.5
2022-05-16 19:29:05 +01:00
Renovate Bot
a599830920 Update dependency c8 to v7.11.3 2022-05-16 05:20:36 +00:00
Aileen Nowak
152a2ea41f Published new versions
- @tryghost/domain-events@0.1.13
 - @tryghost/express-dynamic-redirects@0.2.12
 - @tryghost/magic-link@1.0.25
 - @tryghost/member-analytics-service@0.1.15
 - @tryghost/member-events@0.4.5
 - @tryghost/members-analytics-ingress@0.1.16
 - @tryghost/members-api@7.0.1
 - @tryghost/members-csv@1.2.14
 - @tryghost/members-events-service@0.4.2
 - @tryghost/members-importer@0.5.13
 - @tryghost/members-ssr@1.0.27
 - @tryghost/members-offers@0.11.5
 - @tryghost/members-payments@0.3.5
 - @tryghost/members-stripe-service@0.10.4
 - @tryghost/verification-trigger@0.2.4
2022-05-10 11:40:41 -04:00
Renovate Bot
7af0eb6fdf Update dependency sinon to v14 2022-05-09 04:41:01 +00:00
Simon Backx
0111e23fb7 Published new versions
- @tryghost/domain-events@0.1.12
 - @tryghost/express-dynamic-redirects@0.2.11
 - @tryghost/magic-link@1.0.24
 - @tryghost/member-analytics-service@0.1.14
 - @tryghost/member-events@0.4.4
 - @tryghost/members-analytics-ingress@0.1.15
 - @tryghost/members-api@6.3.1
 - @tryghost/members-csv@1.2.13
 - @tryghost/members-events-service@0.4.1
 - @tryghost/members-importer@0.5.12
 - @tryghost/members-ssr@1.0.26
 - @tryghost/members-offers@0.11.4
 - @tryghost/members-payments@0.3.4
 - @tryghost/members-stripe-service@0.10.3
 - @tryghost/verification-trigger@0.2.3
2022-05-03 16:40:39 +02:00
Renovate Bot
bddc9a5159 Update dependency c8 to v7.11.2 2022-05-02 21:39:21 +00:00
Fabien "egg" O'Carroll
b8545d2912 Published new versions
- @tryghost/domain-events@0.1.11
 - @tryghost/express-dynamic-redirects@0.2.10
 - @tryghost/magic-link@1.0.23
 - @tryghost/member-analytics-service@0.1.13
 - @tryghost/member-events@0.4.3
 - @tryghost/members-analytics-ingress@0.1.14
 - @tryghost/members-api@6.3.0
 - @tryghost/members-csv@1.2.12
 - @tryghost/members-events-service@0.4.0
 - @tryghost/members-importer@0.5.11
 - @tryghost/members-ssr@1.0.25
 - @tryghost/members-offers@0.11.3
 - @tryghost/members-payments@0.3.3
 - @tryghost/members-stripe-service@0.10.2
 - @tryghost/verification-trigger@0.2.2
2022-05-02 19:11:55 +01:00
Renovate Bot
cddf87863e Update dependency mocha to v10 2022-05-02 02:29:13 +00:00
Aileen Nowak
b6c48bb8e2 Published new versions
- @tryghost/members-api@6.2.1
 - @tryghost/members-importer@0.5.10
2022-04-27 15:44:04 -04:00