I have investigated the performance of our frontend vite build:
`npx nx run twenty:start` of `npx nx run twenty:build`
RAM usage:
- 160Mb: vite serve
- background typescript checker: 2.5GB
- background eslint checker: 3.5GB
I'm introducing two environment variables in FE .env to disable these
checkers on lower configuration (and to disable them from CD build):
```
# VITE_DISABLE_TYPESCRIPT_CHECKER=true
# VITE_DISABLE_ESLINT_CHECKER=true
```
Fixes: #6436
Changes made:
- Added typecheck step before twenty-ui build to check stories TS errors
- Added a tsconfig.dev.json to add stories and tests to typecheking when
in dev mode
- Added tsconfig.dev.json to storybook dev command of twenty-ui to
typecheck stories while developing
- Fixed twenty-ui stories that were broken
- Added a serve command to serve front build
- Fixed unit test from another PR
---------
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: Lucas Bordeau <bordeau.lucas@gmail.com>
In this PR, I'm simplifying storybook setup:
1) Remove build --test configuration that prevent autodocs. We are not
using autodocs at all (the dev experience is not good enough), so I have
completely disabled it.
2) Clarify `serve` vs `test` vs `serve-and-test` configurations
After this PR:
- you can serve storybook in two modes: `npx nx run
twenty-front:storybook:serve:dev` and `npx nx run
twenty-front:storybook:serve:static`
- you can run tests agains an already served storybook (this is useful
in dev so you don't have to rebuild everytime to run tests): `npx nx run
twenty-front:storybook:test`
- you can conbine both: `npx nx run
twenty-front:storybook:serve-and-test:static`
Makes sure the `twenty-front:chromatic:ci` task in the CI job
`front-chromatic-deployment` reuses the cache of the Storybook built in
the CI job `front-sb-build` instead of re-building Storybook so
Chromatic is deployed faster in the CI.
TL;DR:
- removed `--configuration={args.scope}` from `storybook:static:test`
for the `storybook:static` part, as it was making `front-sb-test` jobs
in CI not reuse the cache from the `front-sb-build` job and re-build
storybook every time.
- replaced it with a new `test` configuration which optimizes storybook
build for tests and builds storybook 2x faster.
## Fix storybook:build cache usage in CI
`storybook:static:test` executes two scripts in parallel:
1. `storybook:static`, which depends on `storybook:build`
1.a. it builds storybook first with `storybook:build`, the output
directory is `storybook-static`.
1.b. then it launches an `http-server`, using what has been built in
`storybook-static`
2. `storybook:test` to execute tests (needs the storybook http-server to
be running)
When passing `--configuration=pages` or `--configuration=modules` to
`storybook:static` from step 1, those configurations are passed to the
`storybook:build` script from step 1.a as well.
But for Nx `storybook:build` and `storybook:build --configuration=pages`
(or `modules`) are not the same command, therefore one does not reuse
the cache of the other because they could output completely different
things.
As `front-sb-test` jobs are passing `--configuration={args.scope}` to
`storybook:static`, the cache of the previously executed
`storybook:build` (from `front-sb-build`) is not reused and therefore
each job re-builds Storybook with its own scope, which increases CI
time.
### Solution
- Removed scope configurations from `storybook:static` and
`storybook:build` scripts to avoid confusion.
- `storybook:test` and `storybook:dev` can keep scope configurations as
they can be useful and this doesn't impact storybook build cache in CI.
### Improve Storybook build time for testing
Added the `test` configuration to `storybook:build` and
`storybook:static` which makes Storybook build time 2x faster. It
disables addons that slow down build time and are not used in tests.
This PR introduces a Profiling feature for our story book tests.
It also implements a new CI job : front-sb-test-performance, that only
runs stories suffixed with `.perf.stories.tsx`
## How it works
It allows to wrap any component into an array of React Profiler
components that will run tests many times to have the most replicable
average render time possible.
It is simply used by calling the new `getProfilingStory` util.
Internally it creates a defined number of tests, separated by an
arbitrary waiting time to allow the CPU to give more stable results.
It will do 3 warm-up and 3 finishing runs of tests because the first and
last renders are always a bit erratic, so we want to measure only the
runs in-between.
On the UI side it gives a table of results :
<img width="515" alt="image"
src="https://github.com/twentyhq/twenty/assets/26528466/273d2d91-26da-437a-890e-778cb6c1f993">
On the programmatic side, it stores the result in a div that can then be
parsed by the play fonction of storybook, to expect a defined threshold.
```tsx
play: async ({ canvasElement }) => {
await findByTestId(
canvasElement,
'profiling-session-finished',
{},
{ timeout: 60000 },
);
const profilingReport = getProfilingReportFromDocument(canvasElement);
if (!isDefined(profilingReport)) {
return;
}
const p95result = profilingReport?.total.p95;
expect(
p95result,
`Component render time is more than p95 threshold (${p95ThresholdInMs}ms)`,
).toBeLessThan(p95ThresholdInMs);
},
```
- Fixes storybook coverage command: the coverage directory path was
incorrect, but instead of failing `storybook:test --configuration=ci`,
it was hanging indefinitely.
- Switches back to `concurrently` to launch `storybook:static` and
`storybook:test` in parallel, which allows to use options to explicitly
kill `storybook:static` when `storybook:test` fails.
- Moves `storybook:test --configuration=ci` to its own command
`storybook:static:test`: used in the CI, and can be used locally to run
storybook tests without having to launch `storybook:dev` first.
- Creates command `storybook:coverage` and enables cache for this
command.
- Fixes Jest tests that were failing.
- Improves caching conditions for some tasks (for instance, no need to
invalidate Jest test cache if only Storybook story files were modified).
Closes#5097
- Uses "nx affected" to detect what projects need to be checked in the
current PR (for now, `ci-front` and `ci-server` workflows only).
- Caches results of certain tasks (`lint`, `typecheck`, `test`,
`storybook:build`) when a PR pipeline runs. The next runs of the same
PR's pipeline will then be able to reuse the PR's task cache to execute
tasks faster.
- Caches Yarn's cache folder to install dependencies faster in CI jobs.
- Rewrites the node modules cache/install steps as a custom, reusable
Github action.
- Distributes `ci-front` jobs with a "matrix" strategy.
- Sets common tasks config at the root `nx.json`. For instance, to
activate the `typecheck` task in a project, add `typecheck: {}` to its
`project.json` and it'll use the default config set in `nx.json` for the
`typecheck` task. Options can be overridden in each individual
`project.json` if needed.
- Adds "scope" tags to some projects: `scope:frontend`, `scope:backend`,
`scope:shared`. An eslint rule ensures that `scope:frontend` only
depends on `scope:frontent` or `scope:shared` projects, same for
`scope:backend`. These tags are used by `nx affected` to filter projects
by scope and generates different task cache keys according to the
requested scope.
- Enables checks for twenty-emails in the `ci-server` workflow.
# This PR
- Moves dev and ci scripts to the `project.json` file in the
twenty-front package
- Adds a project.json file in the root of the project with the main
start command that start both twenty-server and twenty-front
applications concurrently
- Updates the script command of the root project with the start:prod
command (replacing the start command which will be used in dev with the
help of nx)
- Add a start:prod command in the twenty-front app, replacing the start
command (now used for dev purpose)
Issue ref #4645
@charlesBochet @FelixMalfait please let me know how can I improve it
---------
Co-authored-by: Thaïs Guigon <guigon.thais@gmail.com>
Split from https://github.com/twentyhq/twenty/pull/4518
- Setup `@ui/*` as an internal alias to reference `twenty-ui/src`.
- Configures twenty-front to understand the `@ui/*` alias on development
mode, so twenty-ui can be hot reloaded.
- When building on production mode, twenty-front needs twenty-ui to be
built beforehand (which is automatic with the `dependsOn` option).
- Configures twenty-front to understand the `@ui/*` alias when launching
tests, so there is no need to re-build twenty-ui for tests.
---------
Co-authored-by: Lucas Bordeau <bordeau.lucas@gmail.com>
Split from https://github.com/twentyhq/twenty/pull/4518
- Upgrades dependencies and applies automatic config migrations with the
command: `npx nx migrate nx` (see
https://nx.dev/nx-api/nx/documents/migrate)
- Fixes lint errors after upgrading `@typescript-eslint`
Note: it was not possible (for now) to migrate Nx to the latest stable
version (v18.2.1) because it upgrades Typescript to v5.4.3, which seems
to cause a bug on install when Yarn tries to apply its native patches.
Might be a bug on the Yarn side.
* feat: create a separate package for twenty-ui, extract the pill component with hard-coded theme values into it, and use the component inside twenty-front to complete the setup
* feat: extract the light and the dark theme into twenty-ui and update the AppThemeProvider component inside twenty-front to consume themes from twenty-ui
* fix: create a decorator inside preview.tsx to provide a default theme to storybook development server
* fix: remove redundant type declarations and revert back the naming convention for theme declarations
* fix: introduce a default value for pill label within the story for development server
* fix: introduce the nx script into package.json for twenty-ui and resolve imports for theme type within the package
* fix: remove the pill component from the twenty-front package along with the story for it
* fix: revert the package versions to those before running the nx cli command for storybook init
* feat: update readme to include details for building the ui library and starting the storybook development server
* fix: include details about twenty-ui inside jest.config for twenty-front to complete front-jest job
* - Added preview head for font
- Added theme addon for light/dark switch
- Added ComponentDecorator
---------
Co-authored-by: Lucas Bordeau <bordeau.lucas@gmail.com>
* wip
* trying to parse display names and emails
* add nodemailer mailparser
* mail parsing is working
* add personId and workspaceMemberId
* add date to messages
* Fix PR
* Run tsc on bigger machine
* Fix lint
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
