refs: https://github.com/TryGhost/Toolbox/issues/595
We're rolling out new rules around the node assert library, the first of which is enforcing the use of assert/strict. This means we don't need to use the strict version of methods, as the standard version will work that way by default.
This caught some gotchas in our existing usage of assert where the lack of strict mode had unexpected results:
- Url matching needs to be done on `url.href` see aa58b354a4
- Null and undefined are not the same thing, there were a few cases of this being confused
- Particularly questionable changes in [PostExporter tests](c1a468744b) tracked [here](https://github.com/TryGhost/Team/issues/3505).
- A typo see eaac9c293a
Moving forward, using assert strict should help us to catch unexpected behaviour, particularly around nulls and undefineds during implementation.
refs https://github.com/TryGhost/Toolbox/issues/430
- Previously the CSV parser had "map whatever you can and pass on unknown properties further" approach to CSV parsing. This logic has led to unwanted fields leaking through CSV imports - messy, dangerous.
- The strict mapping rules act as a "validator" to the user input, only passing through the fields we expect explicitly - safer clean cut solution with no unintended side-effects.
refs https://github.com/TryGhost/Toolbox/issues/430
refs https://github.com/TryGhost/Ghost/issues/14882
- Having an explicit mappings passed into the members CSV parser makes it easier to control and understand the transforms for package clients
- Eventually the parser will receive a strict map with the fields it should parse - skipping all unknown & unmapped fields
refs https://github.com/TryGhost/Toolbox/issues/430
- Not having any extra logic in the mapper will allow to have a generalized "mapping" concept for CSV input serialization
- This is groundwork for stricter header value filtering on the parsing stage
refs https://github.com/TryGhost/Toolbox/issues/430
- The 'readCSV' method was only exposed to be used in the unit tests. To keep the module code to the minimum moved readCSV to the unit test itself - the only place where it's used and belongs to.
no-issue
- Was not used by the importer and removed for simplicity.
- Updates the header mapping to happen in place, rather than in a loop
- Updates the parsing of values to give correct types
no issue
- When items are parsed from CSV empty values were interpreted as empty strings - ''. Empty strings are always transformed into 'null' values in Ghost's model layer and are much more problematic to validate comparing to plain `null`. Specifically validation was imossible for 'format: date-time' with JSON schema validation through ajv when the value of date property was an empty string
- This behavior resemples one present in Ghost's model layer - 95880dddeb
- When testing performance overhead for this change did not spot any statistically significant change in performance (tested set was 50K rows)
refs 5c46786ebc
- This is continuation of work removing csv-parser as main CSV handling library with more suitable papaparse library
- Referenced commit introduced papaparse as a library to serialize JSON to CSV, this changeset takes it a step further and replaces CSV to JSON seriazliation logic
no issue
- When processing CSV files `parse` function now allows for the client to specify "mapping" parameter in format of a hash as follows:
{ destination_property_name: 'source_column_name'}
e.g.:
{
name: 'weird_name_column',
email: 'email_column'
}
- It is done so to allow for the end user to provide exact mapping of the fields to be transformed into JSON.
no issue
- This fixes a problem where files are skiped form the @tryghost/members-csv package
- Also this follows the file structure convention with other packages
refs 8185b42d9e5fd9e9051f08ce3395a648ec02f3a4
- Allows to move files and keep the history in one go.
- 'csv-parser' will be upgraded to 'papaparse' lib in a new package
