Commit Graph

685 Commits

Author SHA1 Message Date
Charles Bochet
14cd6f8b6b Decrease messaging import batch size 2024-07-03 22:24:21 +02:00
Jérémy M
921b311c65
fix: small PR fixes workspace-sync-fields (#6107)
Small fixes of PR #6069
2024-07-03 19:19:37 +02:00
Charles Bochet
c2da7c4c6e
Fix workspace sync issue (#6121)
Fixing an issue with typeORM not fetching existing metadata collection
while syncing fields and relations
2024-07-03 19:18:57 +02:00
Charles Bochet
4c57e838ca
Fix pg-boss worker not working with dynamic injection (#6119) 2024-07-03 19:17:29 +02:00
Thomas Trompette
4183e5460d
Use return await to catch exceptions (#6109)
So the exceptions are handled properly and filtered in sentry
2024-07-03 11:14:28 +02:00
Jérémy M
5b26452649
feat: refactor workspace sync fields (#6069)
This PR was first here to fix the issue related to ticket #5004, after
some testing it seems that changing the name of a relation is actually
properly working, if we rename `ONE-TO-MANY` side, the only things that
is going to be updated is the FieldMetadata as the `joinColumn` is
stored on the opposite object.
For `MANY-TO-ONE` relations, the `joinColumn` migration is properly
generated. We need to take care that if we rename a side of a relation,
sometimes the opposite side doesn't have `inverseSideFieldKey`
implemented and used by default the name of the opposite object, so this
is going to throw an error as the field can't be found in the object.

---------

Co-authored-by: Marie <51697796+ijreilly@users.noreply.github.com>
2024-07-02 17:21:13 +02:00
bosiraphael
a163ccced6
Fix calendar import cron job (#6096)
An error was introduced in the calendar cron job because we tried to
inject the workspace context inside the calendarChannelRepository where
we didn't have access to that context.
2024-07-02 16:51:10 +02:00
Charles Bochet
f8dd2cc733
Reorganise calendar module (#6089)
Refactor Calendar into functional sub modules
<img width="437" alt="image"
src="https://github.com/twentyhq/twenty/assets/12035771/d9de3285-a226-4fe8-b3ef-2d8a21def2a5">

---------

Co-authored-by: bosiraphael <raphael.bosi@gmail.com>
2024-07-02 13:55:11 +02:00
Charles Bochet
88915291d9
Fix Active Workspaces check (#6084)
We have recently deprecated our subscriptionStatus on workspace to
replace it by a check on existing subscription (+ freeAccess
featureFlag) but the logic was not properly implemented
2024-07-01 16:45:56 +02:00
Thomas Trompette
bb627a91e2
Use invalid field input error for invalid object metadata input (#6083)
As title
2024-07-01 16:33:48 +02:00
Charles Bochet
fbbcdb20e5 Fix demo seed script by using usage of deprecated sub status 2024-07-01 15:13:31 +02:00
Charles Bochet
b371c77284 Change Messaging import frequency 2024-07-01 14:22:31 +02:00
bosiraphael
8c33d91734
5748 Create contacts for emails sent and received by email aliases (#5855)
Closes #5748
- Create feature flag
- Add scope `https://www.googleapis.com/auth/profile.emails.read` when
connecting an account
- Get email aliases with google people API, store them in
connectedAccount and refresh them before each message-import
- Update the contact creation logic accordingly
- Refactor

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-07-01 14:21:34 +02:00
Thomas Trompette
a15884ea0a
Add exceptions for metadata modules (#6070)
Class exception for each metadata module + handler to map on graphql
error

TODO left :
- find a way to call handler on auto-resolvers nestjs query (probably
interceptors)
- discuss what should be done for pre-hooks errors
- discuss what should be done for Unauthorized exception
2024-07-01 13:49:17 +02:00
Jérémy M
632e34fc7b
fix: message cleaner find operator (#6080)
This PR fix an issue with the `IsNull()` find operator applied on
one-to-many relation, this one is not supported by TypeORM.
We can instead filter by an empty array to retrieve object with empty
relations.
2024-07-01 11:23:22 +02:00
Jérémy M
13f213a05e
feat: message cleaner drop repository (#6052)
This PR use the new `TwentyORM` for the message-cleaner module by using
the new injection system with `@InjectWorkspaceRepository`.
2024-06-30 21:57:18 +02:00
brendanlaschke
afb3f4b7b7
Allow s3 credentials via env (#6066)
closes #5072
2024-06-30 21:47:51 +02:00
martmull
cce9bf5730
Improve use set next onboarding state (#6076)
querying workspaceMembers may be slow leads to wrong
setNextOnboardingStatus value. So we added a resolved field in workspace
to get workspaceMemberCount directly
2024-06-30 21:00:20 +02:00
martmull
b8f33f6f59
5095 move onboardingstatus computation from frontend to backend (#5954)
- move front `onboardingStatus` computing to server side
- add logic to `useSetNextOnboardingStatus`
- update some missing redirections in
`usePageChangeEffectNavigateLocation`
- separate subscriptionStatus from onboardingStatus
2024-06-28 17:32:02 +02:00
bosiraphael
1a66db5bff
Refactor messaging refresh access token (#6034)
- Put error handling outside of `refreshAndSaveAccessToken`
- return after failing to refresh access token in
`processMessageBatchImport`
- remove unnecessary token refresh in `processMessageListFetch`
2024-06-27 17:07:45 +02:00
bosiraphael
4f9527c860
5901 refactor email and calendar auto contact creation to create them by batch (#6038)
Closes #5901
2024-06-27 16:37:34 +02:00
bosiraphael
d1bb0fb822
Change messaging batch size and cron pattern (#6063)
Change messaging batch size and cron pattern to accelerate messages
import
2024-06-27 16:23:47 +02:00
bosiraphael
ac7d905c84
Create fields for calendar and messaging settings v2 (#6049)
Create fields for calendar and messaging settings v2
2024-06-27 16:23:25 +02:00
bosiraphael
845fcb65ce
Replace ObjectRecord<MessageChannelWorkspaceEntity> with MessageChannelWorkspaceEntity (#6059)
Replace ObjectRecord<MessageChannelWorkspaceEntity> with
MessageChannelWorkspaceEntity
2024-06-27 14:35:25 +02:00
Jérémy M
95c5602a4e
feat: manually implement joinColumn (#6022)
This PR introduce a new decorator named `@WorkspaceJoinColumn`, the goal
of this one is to manually declare the join columns inside the workspace
entities, so we don't have to rely on `ObjectRecord` type.

This decorator can be used that way:

```typescript
  @WorkspaceRelation({
    standardId: ACTIVITY_TARGET_STANDARD_FIELD_IDS.company,
    type: RelationMetadataType.MANY_TO_ONE,
    label: 'Company',
    description: 'ActivityTarget company',
    icon: 'IconBuildingSkyscraper',
    inverseSideTarget: () => CompanyWorkspaceEntity,
    inverseSideFieldKey: 'activityTargets',
  })
  @WorkspaceIsNullable()
  company: Relation<CompanyWorkspaceEntity> | null;

  // The argument is the name of the relation above
  @WorkspaceJoinColumn('company')
  companyId: string | null;
```
2024-06-27 11:41:22 +02:00
bosiraphael
97822533db
Fix cache flush in messaging-channel-sync-status.service (#6024)
Fix cache flush in messaging-channel-sync-status.service
2024-06-27 09:58:37 +02:00
bosiraphael
03b00c49d5
Improve gmail error handling by catching and throttling for 400 failedPrecondition (#6044)
Closes #5897
2024-06-27 09:57:19 +02:00
Weiko
1eb9c582f3
Rename mutation maximum affected records (#6042)
As per my last comment on https://github.com/twentyhq/twenty/pull/6039,
we decided to rename this var
2024-06-26 18:00:25 +02:00
Weiko
6599bc136e
Add mutationMaximumRecordAffected to clientConfig (#6039)
We are exposing the server mutationMaximumRecordAffected value via
clientConfig so it can be used by the FE.

This is a first step for https://github.com/twentyhq/twenty/issues/6025

<img width="610" alt="Screenshot 2024-06-26 at 14 58 26"
src="https://github.com/twentyhq/twenty/assets/1834158/9d192976-fd22-45d2-bdaa-a8ff6bb90ca2">
2024-06-26 15:21:13 +02:00
Félix Malfait
cf67ed09d0
Upsert endpoint and CSV import upsert (#5970)
This PR introduces an `upsert` parameter (along the existing `data`
param) for `createOne` and `createMany` mutations.

When upsert is set to `true`, the function will look for records with
the same id if an id was passed. If not id was passed, it will leverage
the existing duplicate check mechanism to find a duplicate. If a record
is found, then the function will perform an update instead of a create.

Unfortunately I had to remove some nice tests that existing on the args
factory. Those tests where mostly testing the duplication rule
generation logic but through a GraphQL angle. Since I moved the
duplication rule logic to a dedicated service, if I kept the tests but
mocked the service we wouldn't really be testing anything useful. The
right path would be to create new tests for this service that compare
the JSON output and not the GraphQL output but I chose not to work on
this as it's equivalent to rewriting the tests from scratch and I have
other competing priorities.
2024-06-26 11:39:16 +02:00
Charles Bochet
1736aee7ff
Remove message-import cache when connectedAccount is removed (#6021) 2024-06-26 11:23:08 +02:00
Weiko
78865ee73e
Fix billing signup when workspace does not exist (#6018) 2024-06-25 15:51:01 +02:00
Jérémy M
7c2e745b45
feat: Dynamic hook registration for WorkspaceQueryHooks (#6008)
#### Overview

This PR introduces a new API for dynamically registering and executing
pre and post query hooks in the Workspace Query Hook system using the
`@WorkspaceQueryHook` decorator. This approach eliminates the need for
manual provider registration, and fix the issue of `undefined` or `null`
repository using `@InjectWorkspaceRepository`.

#### New API

**Define a Hook**

Use the `@WorkspaceQueryHook` decorator to define pre or post hooks:

```typescript
@WorkspaceQueryHook({
  key: `calendarEvent.findMany`,
  scope: Scope.REQUEST,
})
export class CalendarEventFindManyPreQueryHook implements WorkspaceQueryHookInstance {
  async execute(userId: string, workspaceId: string, payload: FindManyResolverArgs): Promise<void> {
    if (!payload?.filter?.id?.eq) {
      throw new BadRequestException('id filter is required');
    }

    // Implement hook logic here
  }
}
```

This API simplifies the registration and execution of query hooks,
providing a more flexible and maintainable approach.

---------

Co-authored-by: Weiko <corentin@twenty.com>
2024-06-25 12:41:46 +02:00
bosiraphael
4dfca45fd3
5615 create messageongoingstalecron (#6005)
Closes #5615
2024-06-25 11:57:02 +02:00
Charles Bochet
f8c057deea
Fix sign up broken because of missing workspace schema (#6013)
Allow workspace datasource factory to return null if the workspace
schema has not been created yet
2024-06-25 10:59:07 +02:00
martmull
7fb5c9b60f
Remove useless api position parameter (#6010)
- remove buggy addition of position parameter
- check created records are in first position by default
2024-06-25 10:30:19 +02:00
bosiraphael
a001bf1514
5951 create a command to trigger the import of a single message (#5962)
Closes #5951

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-06-24 18:01:22 +02:00
bosiraphael
f3701281e9
Create new sync statuses and stages for calendar (#5997)
Create fields:
- syncStatus
- syncStage
- syncStageStartedAt
2024-06-24 16:39:56 +02:00
Charles Bochet
ad61efe6ed
Remove multi select usage (#6004)
As per title!

Also, I'm removing an incorrect logic in the enum migration runner that
takes care of the case where we have no defaultValue but non nullable
which is not a valid business case.
2024-06-24 16:39:17 +02:00
bosiraphael
77f9f6473b
Create feature flag for calendar V2 (#5998)
Create feature flag for calendar V2
2024-06-24 13:54:52 +02:00
Charles Bochet
158e7a31f4
Improve tests (#5994)
Our tests on FE are red, which is a threat to code quality. I'm adding a
few unit tests to improve the coverage and lowering a bit the lines
coverage threshold
2024-06-23 20:12:18 +02:00
Weiko
e13dc7a1fc
[FlexibleSchema] Add IndexMetadata decorator (#5981)
## Context
Our Flexible Schema engine dynamically generates entities/tables/APIs
for us but was not flexible enough to build indexes in the DB. With more
and more features involving heavy queries such as Messaging, we are now
adding a new WorkspaceIndex() decorator for our standard objects (will
come later for custom objects). This decorator will give enough
information to the workspace sync metadata manager to generate the
proper migrations that will create or drop indexes on demand.
To be aligned with the rest of the engine, we are adding 2 new tables:
IndexMetadata and IndexFieldMetadata, that will store the info of our
indexes.

## Implementation

```typescript
@WorkspaceEntity({
  standardId: STANDARD_OBJECT_IDS.person,
  namePlural: 'people',
  labelSingular: 'Person',
  labelPlural: 'People',
  description: 'A person',
  icon: 'IconUser',
})
export class PersonWorkspaceEntity extends BaseWorkspaceEntity {
  @WorkspaceField({
    standardId: PERSON_STANDARD_FIELD_IDS.email,
    type: FieldMetadataType.EMAIL,
    label: 'Email',
    description: 'Contact’s Email',
    icon: 'IconMail',
  })
  @WorkspaceIndex()
  email: string;
```
By simply adding the WorkspaceIndex decorator, sync-metadata command
will create a new index for that column.
We can also add composite indexes, note that the order is important for
PSQL.
```typescript
@WorkspaceEntity({
  standardId: STANDARD_OBJECT_IDS.person,
  namePlural: 'people',
  labelSingular: 'Person',
  labelPlural: 'People',
  description: 'A person',
  icon: 'IconUser',
})
@WorkspaceIndex(['phone', 'email'])
export class PersonWorkspaceEntity extends BaseWorkspaceEntity {
```

Currently composite fields and relation fields are not handled by
@WorkspaceIndex() and you will need to use this notation instead
```typescript
@WorkspaceIndex(['companyId', 'nameFirstName'])
export class PersonWorkspaceEntity extends BaseWorkspaceEntity {
```
<img width="700" alt="Screenshot 2024-06-21 at 15 15 45"
src="https://github.com/twentyhq/twenty/assets/1834158/ac6da1d9-d315-40a4-9ba6-6ab9ae4709d4">

Next step: We might need to implement more complex index expressions,
this is why we have an expression column in IndexMetadata.
What I had in mind for the decorator, still open to discussion
```typescript
@WorkspaceIndex(['nameFirstName', 'nameLastName'], { expression: "$1 || ' ' || $2"})
export class PersonWorkspaceEntity extends BaseWorkspaceEntity {
```

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-06-22 12:39:57 +02:00
Jérémy M
0b4bfce324
feat: drop calendar repository (#5824)
This PR is replacing and removing all the raw queries and repositories
with the new `TwentyORM` and injection system using
`@InjectWorkspaceRepository`.
Some logic that was contained inside repositories has been moved to the
services.
In this PR we're only replacing repositories for calendar feature.

---------

Co-authored-by: Weiko <corentin@twenty.com>
Co-authored-by: bosiraphael <raphael.bosi@gmail.com>
Co-authored-by: Charles Bochet <charles@twenty.com>
2024-06-22 09:26:58 +02:00
Pacifique LINJANJA
9228667a57
Add the support of Empty and Non-Empty filter (#5773) 2024-06-20 18:18:12 +02:00
bosiraphael
86f95c0870
5898 Create a cron to monitor messageChannelSyncStatus (#5933)
Closes #5898
2024-06-19 16:04:01 +02:00
bosiraphael
016132ecf6
Fix reconnect google account bug (#5905)
Update syncStage to FULL_MESSAGE_LIST_FETCH_PENDING when reconnecting
the account to trigger a full sync on the next cron iteration.
2024-06-19 16:00:39 +02:00
Thomas Trompette
96da777107
Handle no concurrency option (#5952)
Fix error in local `teamConcurrency must be an integer between 1 and
1000`
2024-06-19 11:54:11 +02:00
Thomas Trompette
d045bcbb94
Add http status to graphql errors (#5896)
Graphql errors are not properly filtered by our handler. We still
receive errors like `NOT_FOUND` in sentry while they should be filtered.
Example
[here](https://twenty-v7.sentry.io/issues/5490383016/?environment=prod&project=4507072499810304&query=is%3Aunresolved+issue.priority%3A%5Bhigh%2C+medium%5D&referrer=issue-stream&statsPeriod=7d&stream_index=6).

We associate statuses with errors in our map
`graphQLPredefinedExceptions` but we cannot retrieve the status from the
error.

This PR lists the codes that should be filtered.

To test:
- call `findDuplicates` with an invalid id
- before, server would breaks
- now the error is simply returned
2024-06-19 10:39:09 +02:00
martmull
6fd8dab552
5582 get httpsapitwentycomrestmetadata objects filters dont work (#5906)
- Remove filters from metadata rest api
- add limite before and after parameters for metadata
- remove update from metadata relations
- fix typing issue
- fix naming
- fix before parameter

---------

Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
2024-06-18 18:55:13 +02:00
Thomas Trompette
de2b0527a3
Fix secondaryLinks field input (#5911)
PR https://github.com/twentyhq/twenty/pull/5785/files broke links
update.

Also, dropdown "Add link" will be displayed only if a link is already
added. Otherwise, it should be a normal input.
2024-06-17 18:09:46 +02:00