- Improve snackbar to enable displaying multi-line message (so far we
only displayed the first few words which was very frustrating)
- Followup on previous issue to enable tim@apple.dev on the demo
workspace (prefilled automatically)
- Fix sentry tracing which had been broken when migrating from v7 to v8
### Summary
This PR introduces several integration tests, a mix of manually written
tests and those generated using the `generate-integration-tests` Python
script located in the `scripts` folder.
### Tests Added:
- **Authentication tests**: Validating login, registration, and token
handling.
- **FindMany queries**: Fetching multiple records for all existing
entities that do not require input arguments.
### How the Integration Tests Work:
- A `setupTest` function is called during the Jest test run. This
function initializes a test instance of the application and exposes it
on a dedicated port.
- Since tests are executed in isolated workers, they do not have direct
access to the in-memory app instance. Instead, the tests query the
application through the exposed port.
- A static accessToken is used, this one as a big expiration time so it
will never expire (365 years)
- The queries are executed, and the results are validated against
expected outcomes.
### Current State and Next Steps:
- These tests currently run using the existing development seed data. We
plan to introduce more comprehensive test data using `faker` to improve
coverage.
- At the moment, the only mutation tests implemented are for
authentication. Future updates should include broader mutation testing
for other entities.
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
- Add message deletion and thread cleaning during full message list
fetch
- Add thread cleaning during partial message list fetch
- Delete provider from cache key
We had to remove soft-deletion on default filters due to the missing
indexes. We now generate composite indexes with the foreign key
containing the deletedAt column as well which should improve
performances
From PR: #6626Resolves#6763Resolves#6055Resolves#6782
## GTK
I retain the 'Invite by link' feature to prevent any breaking changes.
We could make the invitation by link optional through an admin setting,
allowing users to rely solely on personal invitations.
## Todo
- [x] Add an expiration date to an invitation
- [x] Allow to renew an invitation to postpone the expiration date
- [x] Refresh the UI
- [x] Add the new personal token in the link sent to new user
- [x] Display an error if a user tries to use an expired invitation
- [x] Display an error if a user uses another mail than the one in the
invitation
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
Looks like insert() does not return foreign keys. We could eventually
call findMany after but it seems that's what save() is doing so I'm
replacing insert with save.
```typescript
/**
* Flag to determine whether the entity that is being persisted
* should be reloaded during the persistence operation.
*
* It will work only on databases which does not support RETURNING / OUTPUT statement.
* Enabled by default.
*/
reload?: boolean;
```
Note: save() also does an upsert by default with no way to configure
that so if we want to keep that behaviour we will need to add a check
before
```typescript
if (args.upsert) {
const existingRecords = await repository.findBy({
id: Any(args.data.map((record) => record.id)),
});
...
```
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
We have a few issues on demo seeding:
- redis metdata cache was not flushed
- server ram graphql schema cache was not cleared on metadata version
increment
- avoid failing when missing cache (used for command)
- remove unused load cache function. Cache will be always re-created
when trying to fetch if not existing
## Context
We currently have a race condition when dealing with datasource
creation. This happen when multiple queries arrive at the same time (for
example graphql dataloaders) and the datasource is not created yet.
Since the datasource is stored in memory this can happen more often as
well and they were all triggering the datasource creation at the same
time.
I'm trying to fix the issue with promise memoization. Now, instead of
caching the datasource only, we also want to cache the promise of the
datasource creation and make the creation itself synchronous.
More info about promise memoization in this article for example:
https://www.jonmellman.com/posts/promise-memoization
Co-authored-by: Charles Bochet <charlesBochet@users.noreply.github.com>
Some message channels are stuck in an ongoing `syncStage` because
`syncStartedAt` was not set correctly at the beginning of the sync.
This command resets message channels with an ongoing `syncStage` and
`syncStartedAt` set to null.
This command was supposed to set all custom objects as softDeletable.
After some discussion we realised this bool was not used as intended so
we are removing it all together until we find a better usage (remote
objects for example). This PR removes the command which won't be needed
anymore
## Context
This PR introduces createOne/createMany through the new graphql query
runner.
Trying to use twentyOrm wrapper as much as possible, in this case here
the args are already converted from "metadata-like" structure (including
composite fields) as graphql input to typeorm / raw columns (I had to
introduce a little fix there).
Keep in mind that I'm not using the new graphql query runner parsing
classes here, especially the selected-fields part, because typeorm
already returns all the record columns in the InsertResult object
(including default values such as id, createdAt, ...). That also means
relation objects will be returned as NULL in the gql response but we
don't handle nested creation for the moment so it should be fine.
Note: also removing the feature flag from findOne/findMany
In this PR:
- removing ugprade-0.24 commands as we are releasing 0.30
- introducing cache:flush command
- refactoring upgrade command and sync-metadata command to use the
ActiveWorkspacesCommand so they consistently run on all workspaces or
selected workspaces
Fixes:
- clear localStorage on sign out
- fix missing workspaceMember in verify resolver
- do not throw on datasource already destroyed exception which can
happen with race condition when several resolvers are resolving in
parallel
In this PR:
1. Refactor guards to avoid duplicated queries: WorkspaceAuthGuard and
UserAuthGuard only check for existence of workspace and user in the
request without querying the database
This https://github.com/twentyhq/twenty/pull/7006 introduced a
regression.
The goal was to set "isSoftDeletable" to all standard objects but it was
done at the wrong level, meaning it was setting the boolean correctly
but not creating the corresponding fieldMetadata.
I took the occasion to update the new graphql query runner to use that
boolean and automatically add a filter on soft delete in case it's true.
Also adding **IsQueryRunnerTwentyORMEnabled** by default in the seeds
This is the second PR on TWNTY-6261 which handlesdata migration of Email
field to Emails field.\
\
How to Test?\
Firstly make sure that you have completed the testing steps on first PR
then follow the below steps:
- Checkout to TWNTY-6261-emails-migrations branch
- Rebuild typescript using "npx nx build twenty-server"
- Run command "yarn command:prod upgrade-0.25" to do migration\
\
Loom Video:\
<https://www.loom.com/share/f82b8d29f8f64f92abe3c59c01147b45?sid=9f8ccc05-aa38-4c49-b139-fd0823066273>
**Testing Messaging Sync functionality:**
Please watch the below video to see that the synchronization of contacts
is working fine after migrating Email field to Emails field:\
<https://www.loom.com/share/400949464b244272b78c25e338cc6ab2?sid=103f6625-5933-4b99-9825-0fed33782f36>
**Question to the client**
should we rename email to emails here? in the DomainName PR, the name
did not change.
```typescript
@WorkspaceField({
standardId: PERSON_STANDARD_FIELD_IDS.email,
type: FieldMetadataType.EMAILS,
label: 'Email',
description: 'Contact’s Email',
icon: 'IconMail',
})
email: EmailsMetadata;
```
**Test Messaging Sync**
This pr will update messaging sync files so the changes shouldn't break
existing functionality of importing people and companies in the app.\
To test messaging sync you should follow the below steps:\
1. you need to connect a google account to see the importing
functionality. For this purpose you
have to create a project inside Google Cloud. But to make things easier
you can use the below credentials of an already created project. Put
them in .env of twenty-server package:
```properties
MESSAGING_PROVIDER_GMAIL_ENABLED=true
CALENDAR_PROVIDER_GOOGLE_ENABLED=true
AUTH_GOOGLE_ENABLED=true
AUTH_GOOGLE_CLIENT_ID=951231465939-h61tg6nkpkv1821qi899fjbj9looquto.apps.googleusercontent.com
AUTH_GOOGLE_CLIENT_SECRET=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
AUTH_GOOGLE_CALLBACK_URL=http://localhost:3000/auth/google/redirect
AUTH_GOOGLE_APIS_CALLBACK_URL=http://localhost:3000/auth/google-apis/get-access-token
MESSAGE_QUEUE_TYPE=bull-mq
```
Alternative env
```properties
MESSAGING_PROVIDER_GMAIL_ENABLED=true
CALENDAR_PROVIDER_GOOGLE_ENABLED=true
AUTH_GOOGLE_ENABLED=true
AUTH_GOOGLE_CLIENT_ID=622006708006-dc4n3vrtf3cs2h6k7hgbborudme7ku9l.apps.googleusercontent.com
AUTH_GOOGLE_CLIENT_SECRET=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
AUTH_GOOGLE_CALLBACK_URL=http://localhost:3000/auth/google/redirect
AUTH_GOOGLE_APIS_CALLBACK_URL=http://localhost:3000/auth/google-apis/get-access-token
MESSAGE_QUEUE_TYPE=bull-mq
```
1. Launch your worker with `npx nx run twenty-server:worker`
2. npx nx run twenty-server:command cron:messaging:messages-import
3. npx nx run twenty-server:command cron:messaging:message-list-fetch
4. npx nx run twenty-server:command
cron📆calendar-event-list-fetch
5. Run the app and navigate to Settings/Accounts then connect your
Google account
---------
Co-authored-by: gitstart-twenty <gitstart-twenty@users.noreply.github.com>
Co-authored-by: Marie Stoppa <marie.stoppa@essec.edu>
Co-authored-by: Weiko <corentin@twenty.com>
- Create a workflow version when the user visits an empty workflow.
- If the trigger is not defined yet and the user selects either the
standard object type or the event type first, we automatically select
the first option of the other value. Indeed, every state update is
automatically saved on the backend and we need both standard object and
event types to save the event name.
- Introduces a change in the backend. I removed the assertions that
throw when a workflow version is not complete, that is, when it doesn't
have a defined trigger, which is the case when scaffolding a new
workflow with a first empty workflow version.
- We should keep validating the workflow versions, at least when we
publish them. That should be done in a second step.
This PR introduces the following changes:
- add the metadataVersion to all our metadata cache keys to ease
troubleshooting:
<img width="1146" alt="image"
src="https://github.com/user-attachments/assets/8427805b-e07f-465e-9e69-1403652c8b12">
- introduce a cache recompute lock to avoid overloading the database to
recompute the cache many time
### Description
- This is the first PR on Phones field;
- We are introducing new field type(Phones)
- We are Forbidding creation of Phone field
- We Added support for filtering and sorting on Phones field
- We are using the same display mode as used on the Links field type
(chips), check the Domain field of the Company object
- We are also using the same logic of the link when editing the field
**How to Test**
1. Checkout to TWNTY-6260 branch
2. Reset database using "npx nx database:reset twenty-server" command
3. Add custom field of type Phones in settings/data-model
**Loom Video:**\
<https://www.loom.com/share/3c981260be254dcf851256d020a20ab0?sid=58507361-3a3b-452c-9de8-b5b1abda70ac>
### Refs
#6260
Co-authored-by: gitstart-twenty <gitstart-twenty@users.noreply.github.com>
- make member nullable on favorites
- add potential relation with view entity
- add a new type of favorite list in front : workspace favorite
- build a new component for retrieving workspace favorite to display +
refacto the existing one
Bonus:
- removing activities seed since this is deprecated
from @BOHEUS comments in #6640
- fix bad 500 error when authentication invalid
- remove "id", "createdAt", "updatedAt", etc from creation and update
paths schema
- improve error message
- remove "id" from test body
- improve secondaryLink schema description
- improve depth parameter description
- remove required from response body
- improve examples
- improve error message formatting
- fix filter by position
- answered to negative float position @BOHEUS comment
Also:
- fix secondary field openapi field description
- remove schema display in playground
Screenshots
Workflow version :
- prevent status to be update manually
- prevent creation / deletion in another status than draft
- prevent creation if a draft already exists
Workflow:
- prevent statuses to set manually
WorkflowRun:
- prevent all manual operations
Closes https://github.com/twentyhq/twenty/issues/6840
- Add query-hooks folder in common. Will be followed by hooks for
workflows and runs
- When updating a version, ensure the status is draft and that the
status is not manually updated
- add layer for lambda execution
- add layer for local execution
- add package resolve for the monaco editor
- add route to get installed package for serverless functions
- add layer versioning
Closes#6657
- Fix listeners
- Refactor jobs to take array of events
- Fix calendar events and messages deletion
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
Add listener to keep status on workflows up to date:
- version draft => statuses should contain draft
- version active => statuses should contain active
- version deactivated => if no version active, statuses should contain
deactivated
Renaming also the endpoints because it was not reflecting the full
behaviour.
Finally, adding a new status Archived for versions. Will be used when a
version is deactivated, but is not the last published version anymore.
It means this version cannot be re-activated.
The code removed in the PR was flagged as unused by the JetBrains
inspector.
I did a QA on the dev environment but other checks are highly
recommended.
There is one commit by scope to make the review easier.
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
### Description
1.
- We are introducing new field type(Emails)
- We are Forbiding creation of Email field
- We Added support for filtering and sorting on Emails field
- We are using the same display mode as used on the Links field type
(chips), check the Domain field of the Company object
- We are also using the same logic of the link when editing the field
\
How To Test\
Follow the below steps for testing locally:\
1. Checkout to TWENTY-6261\
2. Reset database using "npx nx database:reset twenty-server" command\
3. Run both the backend and frontend app\
4. Go to Settings/Data model and choose one of the standard objects like
people\
5. Click on Add Field button and choose Emails as the field type
\
### Refs
#6261\
\
### Demo
\
<https://www.loom.com/share/22979acac8134ed390fef93cc56fe07c?sid=adafba94-840d-4f01-872c-dc9ec256d987>
Co-authored-by: gitstart-twenty <gitstart-twenty@users.noreply.github.com>
Following figma updates
https://www.figma.com/design/PNBfTgOVraw557OXChYagk/Explo?node-id=21872-7929&t=DOUzd6rzwr6lprcs-0
- No activity targets for workflow entities for now
- Adding a direct relation between workflow run et workflow
- Adding a status on the version (draft, active, deactivated)
- Adding a list of statuses on workflow
- publishedVersionId => lastPublishedVersionId
Also adding:
- the endpoint to deactivate a version
## Context
The goal is to replace pg_graphql with our own ORM wrapper (TwentyORM).
This PR tries to add some parsing logic to convert graphql requests to
send to the ORM to replace pg_graphql implementation.
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
Website CD has been broken by the recent addition of typeorm patch in
root package.json
Our current vision is to add npm package to each twenty-package
package.json directly
I have investigated the performance of our frontend vite build:
`npx nx run twenty:start` of `npx nx run twenty:build`
RAM usage:
- 160Mb: vite serve
- background typescript checker: 2.5GB
- background eslint checker: 3.5GB
I'm introducing two environment variables in FE .env to disable these
checkers on lower configuration (and to disable them from CD build):
```
# VITE_DISABLE_TYPESCRIPT_CHECKER=true
# VITE_DISABLE_ESLINT_CHECKER=true
```
At field creation we are checking the availability of the name by
comparing it to the other fields' names' on the object; but for
composite fields the fields' names' as indicated in the repository do
not exactly match the column names' on the tables (e.g "createdBy" field
is actually represented by columns createdByName, createdBySource etc.).
In this PR we prevent the conflict with the standard composite fields'
names.
There is still room for errors with the custom composite fields: for
example a custom composite field "address" of type address on a custom
object "listing" will introduce the columns addressAddressStreet1,
addressAddressStreet2 etc. while we won't prevent the user from later
creating a custom field named "addressAddressStreet1".
For now I decided not to tackle this as this seem extremely edgy + would
impact performance on creation of all fields while never actually useful
(I think).
- improvements on serverless function behavior (autosave performances,
deploy on execution only)
- add versioning to serverless functions
- add a publish endpoint to create a new version of a serverless
function
- add deploy and reset to lastVersion button in the settings section:
<img width="736" alt="image"
src="https://github.com/user-attachments/assets/2001f8d2-07a4-4f79-84dd-ec74b6f301d3">
We want to avoid the nested structure of active pieces. Steps to execute
will now be separated from the trigger. It will be an array executed
sequentially.
For now a step can only be an action. But at some point it will also be
a branch or a loop
WorkspaceMemberId is mandatory in the jwt token generated for a given
user on a given workspace.
However, when a user signs up, it does not have a workspaceMemberId yet.
Fix https://github.com/twentyhq/twenty/issues/6669
- create a commun function `startWorkflowRun` that both create the run
object and the job for executing the workflow
- use it in both the `workflowEventJob` and the `runWorkflowVersion`
endpoint
Bonus:
- use filtering for exceptions instead of a util. It avoids doing a try
catch in all endpoint
Fix
[#web](https://github.com/orgs/twentyhq/projects/1/views/3?pane=issue&itemId=75329194)
This PR does 2 things:
- migrate webhooks to TwentyORM
- Fix inversion between objectNameSingular and operation in webhook
eventName. It is stored as {objectNameSingular}.{operation} and we were
querying {operation}.{objectNameSingular}
We have found the root cause of the issue:
- when using a datasource (including the cached ones), we are fetching
ObjectMetadataCollection from cache (700kB). Datasource usage is
happening any time we are using twentyORM, which is everywhere in the
jobs and in some resolvers (including the GetCurrentUser one). This is
leading to a high load on redis and leading to the performance issues we
are seeing.
- we actually don't need to fetch this objectMetadataCollection while
using a cached datasource, only when we instantiate a new one
This PR was created by [GitStart](https://gitstart.com/) to address the
requirements from this ticket:
[TWNTY-5370](https://clients.gitstart.com/twenty/5449/tickets/TWNTY-5370).
This ticket was imported from:
[TWNTY-5370](https://github.com/twentyhq/twenty/issues/5370)
---
### Description
- We updated the logic in
packages/twenty-server/src/engine/metadata-modules/object-metadata/object-metadata.service.ts
Test cases:
1. Ensure that when an object is disabled, all related relationships are
also disabled.
a. Example disable the people object
b. Check the company object and verify that the people field has been
disabled too
c. Check the opportunity object and check that the point of contact
field has been disabled too
2. Verify that when a previously disabled object is restored, the
relationships are also restored.
3. Ensure that previously disabled relationships remain disabled when
the object is disabled and later restored.
4. Verify that relationships of a disabled object are not visible in the
UI.
5. Ensure that relationships to a disabled object are marked as inactive
in the data models screen
### Refs
#5370
### Demo
<https://www.loom.com/share/2b0a91f463ca4e02a6963f9a8796a0d9?sid=1e9c4fb8-8fb9-4c6c-b43a-c50f3776e1d3>
Fixes#5370
---------
Co-authored-by: gitstart-twenty <gitstart-twenty@users.noreply.github.com>
Co-authored-by: Marie Stoppa <marie.stoppa@essec.edu>
## Context
As we grow, the messaging scripts are experiencing performance issues
forcing us to temporarily disable them on the cloud.
While investigating the performance, I have noticed that generating the
entity schema (for twentyORM) in the repository is taking ~500ms locally
on my Mac M2 so likely more on pods. Caching the entitySchema then!
I'm also clarifying naming around schemaVersion and cacheVersions ==>
both are renamed workspaceMetadataVersion and migrated to the workspace
table (the workspaceCacheVersion table is dropped).
