Commit Graph

20 Commits

Author SHA1 Message Date
Naz
eca4b142d2
Cleaned up csv parse test suite
refs https://github.com/TryGhost/Toolbox/issues/430

- Removed unnecessary "readCSV" leftover code.
2022-10-19 18:10:40 +08:00
Naz
f1e45f4632
Cleaned up test case names 2022-10-19 18:10:40 +08:00
Naz
4300f14d79
Removed hidden row mapping in csv parser
refs https://github.com/TryGhost/Toolbox/issues/430
refs https://github.com/TryGhost/Ghost/issues/14882

- Having an explicit mappings passed into the members CSV parser makes it easier to control and understand the transforms for package clients
- Eventually the parser will receive a strict map with the fields it should parse - skipping all unknown & unmapped fields
2022-10-19 18:10:40 +08:00
Naz
1c746c0ca0
Added test coverage for 'subscribed' transform
refs https://github.com/TryGhost/Toolbox/issues/430

- Not having any extra logic in the mapper will allow to have a generalized "mapping" concept for CSV input serialization
- This is groundwork for stricter header value filtering on the parsing stage
2022-10-19 18:10:40 +08:00
Naz
bc70835890
Increased test coverage
refs https://github.com/TryGhost/Toolbox/issues/430

- The bonus of using the module exports file is that it also gets included in the test coverage statistics ^_^
2022-10-19 18:10:40 +08:00
Naz
5b8b18ded1
Moved csv parser utility method
refs https://github.com/TryGhost/Toolbox/issues/430

- The 'readCSV' method was only exposed to be used in the unit tests. To keep the module code to the minimum moved readCSV to the unit test itself - the only place where it's used and belongs to.
2022-10-13 09:53:16 +08:00
Fabien "egg" O'Carroll
b1cc2a8ea8 Fixed CSV unparsing for subscribed flag
no-issue

We have a special mapping for subscribed_to_emails -> subscribed in the
parse method, but were not mapping it in the unparse method, which meant
we were losing information during CSV imports.
2022-01-11 16:26:11 +02:00
Fabien O'Carroll
6083e4825f Removed trailing commas from .eslintrc.js
refs https://github.com/TryGhost/Team/issues/879
2021-07-14 12:04:46 +01:00
Fabien O'Carroll
4d8c2ebb1f Supported products column for parse & unparse
refs https://github.com/TryGhost/Team/issues/765

Support for multiple products means we can no longer map a members state
to a csv row using just the `complimentary_plan` option. Instead we must
include the product(s) that a member has. This ensures that we can read
and write this data from/to csv files.
2021-06-23 10:14:30 +01:00
Fabien O'Carroll
0a5f91f3d0 Refactored members-csv to remove unused columnsToExtract
no-issue

 - Was not used by the importer and removed for simplicity.
 - Updates the header mapping to happen in place, rather than in a loop
 - Updates the parsing of values to give correct types
2020-12-04 10:10:59 +00:00
Nazar Gargol
b8c1aeee35 Added empty string '' to null transform when parsing CSVs
no issue

- When items are parsed from CSV empty values were interpreted as empty strings - ''. Empty strings are always transformed into 'null' values in Ghost's model layer and are much more problematic to validate comparing to plain `null`. Specifically validation was imossible for 'format: date-time' with JSON schema validation through ajv when the value of date property was an empty string
- This behavior resemples one present in Ghost's model layer  - 95880dddeb
- When testing performance overhead for this change did not spot any statistically significant change in performance (tested set was 50K rows)
2020-08-17 17:57:49 +12:00
Nazar Gargol
86ff376c99 Reworked unparse test to use less dependencies 2020-07-08 20:09:21 +12:00
Nazar Gargol
4c21bb9d65 Added basic test coverage for JSON to CSV serialization
no issue
2020-07-08 19:40:48 +12:00
Nazar Gargol
7f25548be5 Removed unneeded test suite
no issue

- This is a leftover from initial project scaffolding
2020-07-08 19:40:10 +12:00
Nazar Gargol
3a97d39d72 Substituted csv-parser with papaparse as CSV parsing package
refs 5c46786ebc

- This is continuation of work removing csv-parser as main CSV handling library with more suitable papaparse library
- Referenced commit introduced papaparse as a library to serialize JSON to CSV, this changeset takes it a step further and replaces CSV to JSON seriazliation logic
2020-07-08 19:39:37 +12:00
Nazar Gargol
fd982fef73 Added mapping support to CSV parser
no issue

- When processing CSV files `parse` function now allows for the client to specify "mapping" parameter in format of a hash as follows:
{ destination_property_name: 'source_column_name'}
e.g.:
{
  name: 'weird_name_column',
  email: 'email_column'
}
- It is done so to allow for the end user to provide exact mapping of the fields to be transformed into JSON.
2020-07-02 14:53:56 +12:00
Nazar Gargol
93ab8ae946 Moved modules into lib folder
no issue

- This fixes a problem where files are skiped form the @tryghost/members-csv package
- Also this follows the file structure convention with other packages
2020-06-19 19:49:39 +12:00
Nazar Gargol
5c47d725cb Fixed tests and added fixtures from Ghost repo 2020-06-19 19:33:43 +12:00
Nazar Gargol
5c46786ebc Colocated readCSV util with members input serializer
refs 8185b42d9e5fd9e9051f08ce3395a648ec02f3a4

- Allows to move files and keep the history in one go.
- 'csv-parser' will be upgraded to 'papaparse' lib in a new package
2020-06-19 19:33:43 +12:00
Nazar Gargol
ca2ab5bbce Initialized members-csv package 2020-06-19 18:54:48 +12:00