Commit Graph

694 Commits

Author SHA1 Message Date
bosiraphael
f458322303
Refactor calendar to use new sync statuses and stages (#6141)
- Refactor calendar modules and some messaging modules to better
organize them by business rules and decouple them
- Work toward a common architecture for the different calendar providers
by introducing interfaces for the drivers
- Modify cron job to use the new sync statuses and stages
2024-07-08 17:01:06 +02:00
Charles Bochet
1c3ea9b106
Bump version to 0.21 (#6161) 2024-07-08 16:57:35 +02:00
Charles Bochet
9ba211055a
Add message import granulary on non-pro emails, group emails and received contact creation (#6156)
1) Remove featureFlag
2) Base contactCreation on messageChannel.autoContactCreationPolicy
4) add excludeProfessionalEmails + excludeGroupEmails logic
2024-07-08 14:33:48 +02:00
Marie
b33b468679
Add command to update boolean fields null values (#6113)
We have recently decided that boolean fields should only accept truthy
or falsy value, with users deciding of a default value at creation.

This command helps cleaning the existing data, by
1. updating all boolean fields default values from null to false
2. updating all boolean fields values for records from null to false

---------

Co-authored-by: Weiko <corentin@twenty.com>
2024-07-05 12:27:12 +02:00
Marie
6cd154aac6
Forbid names above 63 characters to comply with pg identifier limit (#6095)
Fixes #6032.

Pg has a char limit on identifiers (= table, columns, enum names) of 63
bytes.
Let's limit the metadata names that will be converted to identifiers
(objects names, fields names, relation names, enum values) to 63 chars.
For the sake of simplicity in the FE we will limit the input length of
labels.

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-07-04 15:32:42 +02:00
Jérémy M
5df0ea6466
fix: message queue injection issue (#6126) 2024-07-04 12:22:58 +02:00
Félix Malfait
5b4d2d989a
Hotfix tests after AI PR (#6124)
I merged https://github.com/twentyhq/twenty/pull/5788 too quickly and
didn't noticed it was broken with
https://github.com/twentyhq/twenty/pull/6069
2024-07-04 10:21:33 +02:00
ad-elias
4c642a0bb8
Text-to-SQL proof of concept (#5788)
Added:
- An "Ask AI" command to the command menu.
- A simple GraphQL resolver that converts the user's question into a
relevant SQL query using an LLM, runs the query, and returns the result.

<img width="428" alt="Screenshot 2024-06-09 at 20 53 09"
src="https://github.com/twentyhq/twenty/assets/171685816/57127f37-d4a6-498d-b253-733ffa0d209f">

No security concerns have been addressed, this is only a
proof-of-concept and not intended to be enabled in production.

All changes are behind a feature flag called `IS_ASK_AI_ENABLED`.

---------

Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
2024-07-04 08:57:26 +02:00
Charles Bochet
25fce27fe3 Improve performance/robustness of worker 2024-07-03 22:54:39 +02:00
Charles Bochet
14cd6f8b6b Decrease messaging import batch size 2024-07-03 22:24:21 +02:00
Jérémy M
921b311c65
fix: small PR fixes workspace-sync-fields (#6107)
Small fixes of PR #6069
2024-07-03 19:19:37 +02:00
Charles Bochet
c2da7c4c6e
Fix workspace sync issue (#6121)
Fixing an issue with typeORM not fetching existing metadata collection
while syncing fields and relations
2024-07-03 19:18:57 +02:00
Charles Bochet
4c57e838ca
Fix pg-boss worker not working with dynamic injection (#6119) 2024-07-03 19:17:29 +02:00
Thomas Trompette
4183e5460d
Use return await to catch exceptions (#6109)
So the exceptions are handled properly and filtered in sentry
2024-07-03 11:14:28 +02:00
Jérémy M
5b26452649
feat: refactor workspace sync fields (#6069)
This PR was first here to fix the issue related to ticket #5004, after
some testing it seems that changing the name of a relation is actually
properly working, if we rename `ONE-TO-MANY` side, the only things that
is going to be updated is the FieldMetadata as the `joinColumn` is
stored on the opposite object.
For `MANY-TO-ONE` relations, the `joinColumn` migration is properly
generated. We need to take care that if we rename a side of a relation,
sometimes the opposite side doesn't have `inverseSideFieldKey`
implemented and used by default the name of the opposite object, so this
is going to throw an error as the field can't be found in the object.

---------

Co-authored-by: Marie <51697796+ijreilly@users.noreply.github.com>
2024-07-02 17:21:13 +02:00
bosiraphael
a163ccced6
Fix calendar import cron job (#6096)
An error was introduced in the calendar cron job because we tried to
inject the workspace context inside the calendarChannelRepository where
we didn't have access to that context.
2024-07-02 16:51:10 +02:00
Charles Bochet
f8dd2cc733
Reorganise calendar module (#6089)
Refactor Calendar into functional sub modules
<img width="437" alt="image"
src="https://github.com/twentyhq/twenty/assets/12035771/d9de3285-a226-4fe8-b3ef-2d8a21def2a5">

---------

Co-authored-by: bosiraphael <raphael.bosi@gmail.com>
2024-07-02 13:55:11 +02:00
Charles Bochet
88915291d9
Fix Active Workspaces check (#6084)
We have recently deprecated our subscriptionStatus on workspace to
replace it by a check on existing subscription (+ freeAccess
featureFlag) but the logic was not properly implemented
2024-07-01 16:45:56 +02:00
Thomas Trompette
bb627a91e2
Use invalid field input error for invalid object metadata input (#6083)
As title
2024-07-01 16:33:48 +02:00
Charles Bochet
fbbcdb20e5 Fix demo seed script by using usage of deprecated sub status 2024-07-01 15:13:31 +02:00
Charles Bochet
b371c77284 Change Messaging import frequency 2024-07-01 14:22:31 +02:00
bosiraphael
8c33d91734
5748 Create contacts for emails sent and received by email aliases (#5855)
Closes #5748
- Create feature flag
- Add scope `https://www.googleapis.com/auth/profile.emails.read` when
connecting an account
- Get email aliases with google people API, store them in
connectedAccount and refresh them before each message-import
- Update the contact creation logic accordingly
- Refactor

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-07-01 14:21:34 +02:00
Thomas Trompette
a15884ea0a
Add exceptions for metadata modules (#6070)
Class exception for each metadata module + handler to map on graphql
error

TODO left :
- find a way to call handler on auto-resolvers nestjs query (probably
interceptors)
- discuss what should be done for pre-hooks errors
- discuss what should be done for Unauthorized exception
2024-07-01 13:49:17 +02:00
Jérémy M
632e34fc7b
fix: message cleaner find operator (#6080)
This PR fix an issue with the `IsNull()` find operator applied on
one-to-many relation, this one is not supported by TypeORM.
We can instead filter by an empty array to retrieve object with empty
relations.
2024-07-01 11:23:22 +02:00
Jérémy M
13f213a05e
feat: message cleaner drop repository (#6052)
This PR use the new `TwentyORM` for the message-cleaner module by using
the new injection system with `@InjectWorkspaceRepository`.
2024-06-30 21:57:18 +02:00
brendanlaschke
afb3f4b7b7
Allow s3 credentials via env (#6066)
closes #5072
2024-06-30 21:47:51 +02:00
martmull
cce9bf5730
Improve use set next onboarding state (#6076)
querying workspaceMembers may be slow leads to wrong
setNextOnboardingStatus value. So we added a resolved field in workspace
to get workspaceMemberCount directly
2024-06-30 21:00:20 +02:00
martmull
b8f33f6f59
5095 move onboardingstatus computation from frontend to backend (#5954)
- move front `onboardingStatus` computing to server side
- add logic to `useSetNextOnboardingStatus`
- update some missing redirections in
`usePageChangeEffectNavigateLocation`
- separate subscriptionStatus from onboardingStatus
2024-06-28 17:32:02 +02:00
bosiraphael
1a66db5bff
Refactor messaging refresh access token (#6034)
- Put error handling outside of `refreshAndSaveAccessToken`
- return after failing to refresh access token in
`processMessageBatchImport`
- remove unnecessary token refresh in `processMessageListFetch`
2024-06-27 17:07:45 +02:00
bosiraphael
4f9527c860
5901 refactor email and calendar auto contact creation to create them by batch (#6038)
Closes #5901
2024-06-27 16:37:34 +02:00
bosiraphael
d1bb0fb822
Change messaging batch size and cron pattern (#6063)
Change messaging batch size and cron pattern to accelerate messages
import
2024-06-27 16:23:47 +02:00
bosiraphael
ac7d905c84
Create fields for calendar and messaging settings v2 (#6049)
Create fields for calendar and messaging settings v2
2024-06-27 16:23:25 +02:00
bosiraphael
845fcb65ce
Replace ObjectRecord<MessageChannelWorkspaceEntity> with MessageChannelWorkspaceEntity (#6059)
Replace ObjectRecord<MessageChannelWorkspaceEntity> with
MessageChannelWorkspaceEntity
2024-06-27 14:35:25 +02:00
Jérémy M
95c5602a4e
feat: manually implement joinColumn (#6022)
This PR introduce a new decorator named `@WorkspaceJoinColumn`, the goal
of this one is to manually declare the join columns inside the workspace
entities, so we don't have to rely on `ObjectRecord` type.

This decorator can be used that way:

```typescript
  @WorkspaceRelation({
    standardId: ACTIVITY_TARGET_STANDARD_FIELD_IDS.company,
    type: RelationMetadataType.MANY_TO_ONE,
    label: 'Company',
    description: 'ActivityTarget company',
    icon: 'IconBuildingSkyscraper',
    inverseSideTarget: () => CompanyWorkspaceEntity,
    inverseSideFieldKey: 'activityTargets',
  })
  @WorkspaceIsNullable()
  company: Relation<CompanyWorkspaceEntity> | null;

  // The argument is the name of the relation above
  @WorkspaceJoinColumn('company')
  companyId: string | null;
```
2024-06-27 11:41:22 +02:00
bosiraphael
97822533db
Fix cache flush in messaging-channel-sync-status.service (#6024)
Fix cache flush in messaging-channel-sync-status.service
2024-06-27 09:58:37 +02:00
bosiraphael
03b00c49d5
Improve gmail error handling by catching and throttling for 400 failedPrecondition (#6044)
Closes #5897
2024-06-27 09:57:19 +02:00
Weiko
1eb9c582f3
Rename mutation maximum affected records (#6042)
As per my last comment on https://github.com/twentyhq/twenty/pull/6039,
we decided to rename this var
2024-06-26 18:00:25 +02:00
Weiko
6599bc136e
Add mutationMaximumRecordAffected to clientConfig (#6039)
We are exposing the server mutationMaximumRecordAffected value via
clientConfig so it can be used by the FE.

This is a first step for https://github.com/twentyhq/twenty/issues/6025

<img width="610" alt="Screenshot 2024-06-26 at 14 58 26"
src="https://github.com/twentyhq/twenty/assets/1834158/9d192976-fd22-45d2-bdaa-a8ff6bb90ca2">
2024-06-26 15:21:13 +02:00
Félix Malfait
cf67ed09d0
Upsert endpoint and CSV import upsert (#5970)
This PR introduces an `upsert` parameter (along the existing `data`
param) for `createOne` and `createMany` mutations.

When upsert is set to `true`, the function will look for records with
the same id if an id was passed. If not id was passed, it will leverage
the existing duplicate check mechanism to find a duplicate. If a record
is found, then the function will perform an update instead of a create.

Unfortunately I had to remove some nice tests that existing on the args
factory. Those tests where mostly testing the duplication rule
generation logic but through a GraphQL angle. Since I moved the
duplication rule logic to a dedicated service, if I kept the tests but
mocked the service we wouldn't really be testing anything useful. The
right path would be to create new tests for this service that compare
the JSON output and not the GraphQL output but I chose not to work on
this as it's equivalent to rewriting the tests from scratch and I have
other competing priorities.
2024-06-26 11:39:16 +02:00
Charles Bochet
1736aee7ff
Remove message-import cache when connectedAccount is removed (#6021) 2024-06-26 11:23:08 +02:00
Weiko
78865ee73e
Fix billing signup when workspace does not exist (#6018) 2024-06-25 15:51:01 +02:00
Jérémy M
7c2e745b45
feat: Dynamic hook registration for WorkspaceQueryHooks (#6008)
#### Overview

This PR introduces a new API for dynamically registering and executing
pre and post query hooks in the Workspace Query Hook system using the
`@WorkspaceQueryHook` decorator. This approach eliminates the need for
manual provider registration, and fix the issue of `undefined` or `null`
repository using `@InjectWorkspaceRepository`.

#### New API

**Define a Hook**

Use the `@WorkspaceQueryHook` decorator to define pre or post hooks:

```typescript
@WorkspaceQueryHook({
  key: `calendarEvent.findMany`,
  scope: Scope.REQUEST,
})
export class CalendarEventFindManyPreQueryHook implements WorkspaceQueryHookInstance {
  async execute(userId: string, workspaceId: string, payload: FindManyResolverArgs): Promise<void> {
    if (!payload?.filter?.id?.eq) {
      throw new BadRequestException('id filter is required');
    }

    // Implement hook logic here
  }
}
```

This API simplifies the registration and execution of query hooks,
providing a more flexible and maintainable approach.

---------

Co-authored-by: Weiko <corentin@twenty.com>
2024-06-25 12:41:46 +02:00
bosiraphael
4dfca45fd3
5615 create messageongoingstalecron (#6005)
Closes #5615
2024-06-25 11:57:02 +02:00
Charles Bochet
f8c057deea
Fix sign up broken because of missing workspace schema (#6013)
Allow workspace datasource factory to return null if the workspace
schema has not been created yet
2024-06-25 10:59:07 +02:00
martmull
7fb5c9b60f
Remove useless api position parameter (#6010)
- remove buggy addition of position parameter
- check created records are in first position by default
2024-06-25 10:30:19 +02:00
bosiraphael
a001bf1514
5951 create a command to trigger the import of a single message (#5962)
Closes #5951

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-06-24 18:01:22 +02:00
bosiraphael
f3701281e9
Create new sync statuses and stages for calendar (#5997)
Create fields:
- syncStatus
- syncStage
- syncStageStartedAt
2024-06-24 16:39:56 +02:00
Charles Bochet
ad61efe6ed
Remove multi select usage (#6004)
As per title!

Also, I'm removing an incorrect logic in the enum migration runner that
takes care of the case where we have no defaultValue but non nullable
which is not a valid business case.
2024-06-24 16:39:17 +02:00
bosiraphael
77f9f6473b
Create feature flag for calendar V2 (#5998)
Create feature flag for calendar V2
2024-06-24 13:54:52 +02:00
Charles Bochet
158e7a31f4
Improve tests (#5994)
Our tests on FE are red, which is a threat to code quality. I'm adding a
few unit tests to improve the coverage and lowering a bit the lines
coverage threshold
2024-06-23 20:12:18 +02:00