Commit Graph

595 Commits

Author SHA1 Message Date
Thomas Trompette
25f4e44aec
Add backfill position job by workspace (#5725)
- Removing existing listener that was backfilling created records
without position
- Switch to a job that backfill all objects within workspace
- Adapting `FIND_BY_POSITION` so it can fetch objects without position.
Currently we needed to input a number
2024-06-04 14:10:58 +02:00
Thomas Trompette
b1f12d7257
Fix input position backfill (#5731)
Some objects do not have position field so they should not be backfilled
2024-06-04 13:26:40 +02:00
bosiraphael
3f9f2c3ba6
5620 implement throttle logic for message and calendar sync (#5718)
Closes #5620 and improve messages filters
2024-06-04 10:29:05 +02:00
Thomas Trompette
2886664b62
Backfill position when not input (#5696)
- refactor record position factory and record position query factory
- override position if not present during createMany

To avoid overriding the same positions for all data in createMany, the
logic is:
- if inserted last, use last position + arg index + 1
- if inserted first, use first position - arg index - 1
2024-06-03 15:18:01 +02:00
Weiko
e2dc660d18
Fix exception handler capturing graphql errors (#5714)
Our exception handler has to filter out some errors/exceptions so they
are not caught by the ExceptionHandlerDriver (Logged or Sentry for
example). This is done for Http errors in the range of 4xx and also
makes sure they are converted back to Graphql validation errors.
However, graphql validation errors that are already managed by Yoga
(with Schema validation) should also be filtered out, this PR should fix
that behaviour
2024-06-03 14:36:03 +02:00
bosiraphael
2d12984db4
5613 add throttlepauseuntil and throttlefailurecount fields to messagechannel and calendarchannel (#5713)
Closes #5613
2024-06-03 13:22:49 +02:00
Charles Bochet
eab8deb211
Rework messaging modules (#5710)
In this PR, I'm refactoring the messaging module into smaller pieces
that have **ONE** responsibility: import messages, clean messages,
handle message participant creation, instead of having ~30 modules (1
per service, jobs, cron, ...). This is mandatory to start introducing
drivers (gmails, office365, ...) IMO. It is too difficult to enforce
common interfaces as we have too many interfaces (30 modules...). All
modules should not be exposed

Right now, we have services that are almost functions:
do-that-and-this.service.ts / do-that-and-this.module.ts
I believe we should have something more organized at a high level and it
does not matter that much if we have a bit of code duplicates.

Note that the proposal is not fully implemented in the current PR that
has only focused on messaging folder (biggest part)

Here is the high level proposal:
- connected-account: token-refresher
- blocklist
- messaging: message-importer, message-cleaner, message-participants,
... (right now I'm keeping a big messaging-common but this will
disappear see below)
- calendar: calendar-importer, calendar-cleaner, ...

Consequences:
1) It's OK to re-implement several times some things. Example:
- error handling in connected-account, messaging, and calendar instead
of trying to unify. They are actually different error handling. The only
things that might be in common is the GmailError => CommonError parsing
and I'm not even sure it makes a lot of sense as these 3 apis might have
different format actually
- auto-creation. Calendar and Messaging could actually have different
rules

2) **We should not have circular dependencies:** 
- I believe this was the reason why we had so many modules, to be able
to cherry pick the one we wanted to avoid circular deps. This is not the
right approach IMO, we need architect the whole messaging by defining
high level blocks that won't have circular dependencies by design. If we
encounter one, we should rethink and break the block in a way that makes
sense.
- ex: connected-account.resolver is not in the same module as
token-refresher. ==> connected-account.resolver => message-importer (as
we trigger full sync job when we connect an account) => token-refresher
(as we refresh token on message import).

connected-account.resolver and token-refresher both in connected-account
folder but should be in different modules. Otherwise it's a circular
dependency. It does not mean that we should create 1 module per service
as it was done before

In a nutshell: The code needs to be thought in term of reponsibilities
and in a way that enforce high level interfaces (and avoid circular
dependencies)

Bonus: As you can see, this code is also removing a lot of code because
of the removal of many .module.ts (also because I'm removing the sync
scripts v2 feature flag end removing old code)
Bonus: I have prefixed services name with Messaging to improve dev xp.
GmailErrorHandler could be different between MessagingGmailErrorHandler
and CalendarGmailErrorHandler for instance
2024-06-03 11:16:05 +02:00
Charles Bochet
c4b6b1e076 Fix thread cleaner forgetting to clean some thread and messages 2024-06-01 16:51:20 +02:00
Charles Bochet
182ab66eef Fix messaging import issue when all batch messages are excluded 2024-05-31 23:58:33 +02:00
bosiraphael
e0103bbcdc
5015 make gmail filters work for partial sync (#5695)
Closes #5015

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-05-31 23:20:57 +02:00
Charles Bochet
99dff43579
Add new enum options to messageChannel syncStatus (#5694)
As per title!
This command will enable smooth migrations between 0.12 and 0.20
2024-05-31 18:02:52 +02:00
Thomas Trompette
fbd8714c76
Make positions possibly negatives (#5690)
Closes https://github.com/twentyhq/twenty/issues/5427
2024-05-31 14:17:49 +02:00
bosiraphael
f166171a1c
5531 update gmail full sync to v2 (#5674)
Closes #5531

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-05-31 13:29:58 +02:00
Thomas Trompette
c60a3e49cd
Catch query timeout exceptions (#5680)
Query read timeouts happen when a remote server is not available. It
breaks:
- the remote server show page
- the record table page of imported remote tables

This PR will catch the exception so it does not go to Sentry in both
cases.

Also did 2 renaming.
2024-05-31 10:39:35 +02:00
Aditya Pimpalkar
a12c1aad5e
fix: user has to login every time chrome sidepanel is opened (#5544)
We can pass the auth tokens to our front app via post message, which
will also allow us to pass route names to navigate on it
2024-05-30 12:58:45 +02:00
Marie
339aee6dbb
Run queries within queryRunner transaction sequentially (#5668)
Within a queryRunner transaction, it is important that migrations are
run subsequently and not concurrently: otherwise if an error is thrown
by one of the query, it will abort the transaction; any subsequent query
running on the same queryRunner will cause the error _current
transaction is aborted, commands ignored until end of transaction
block_.

Using an async function in a map as below does not guarantee that each
query terminates before iterating over the next one, which can be an
issue as described above, and which seems to cause [this
sentry](https://twenty-v7.sentry.io/issues/5258406553/?environment=prod&project=4507072499810304&query=is%3Aunresolved+issue.priority%3A%5Bhigh%2C+medium%5D&referrer=issue-stream&statsPeriod=7d&stream_index=4).
2024-05-30 10:45:46 +02:00
Marie
5bb205bd6a
Fix update remote field metadata (#5638)
Closes #5610.

& update fetch-policy when fetching database on the remote databases
show page to get freshest status.
2024-05-28 18:01:05 +02:00
Thomas Trompette
ebb1aa0377
Add label to remote server (#5637)
Added label on remote server entity. 

Also added the possibility to update schema. 

<img width="688" alt="Capture d’écran 2024-05-28 à 15 36 31"
src="https://github.com/twentyhq/twenty/assets/22936103/c9786122-8459-4876-833e-c9a1d7d27829">
2024-05-28 15:54:57 +02:00
Thomas Trompette
ae6d5afdfc
Add missing stripe tables (#5621)
As title

Still adding not working tables / columns commented so we know why these
are not available.
2024-05-28 11:32:57 +02:00
Marie
930237e778
Bump to version v0.12.1 (#5608) 2024-05-27 16:07:38 +02:00
Marie
f58c961d98
Remove feature flag for Links field (#5606) 2024-05-27 16:05:22 +02:00
Marie
857971458a
Bump version to v0.12.0 (#5604) 2024-05-27 15:16:50 +02:00
martmull
2f52e0fdb6
5505 forgot password feature broken (#5602)
- add missing `excludedOperations` in
`packages/twenty-server/src/engine/middlewares/graphql-hydrate-request-from-token.middleware.ts`
- update generated graphql file
- Add missing redirection to index after password update
2024-05-27 15:13:11 +02:00
Marie
def1774bf0
[Fix] Object names should be camel cased (#5571)
as per title
2024-05-25 10:29:00 +02:00
bosiraphael
936ac4027a
Introduce a new feature flag for contact creation (#5570)
Introduce new feature flag
`IS_CONTACT_CREATION_FOR_SENT_AND_RECEIVED_EMAILS_ENABLED` to allow
contacts to be created for sent and received emails.
2024-05-24 18:55:21 +02:00
bosiraphael
3680647c9a
Fix sync token is no longer valid in calendar sync (#5563)
Fix sync token is no longer valid in calendar sync.


https://developers.google.com/apps-script/add-ons/calendar/conferencing/sync-calendar-changes#implement_a_sync_trigger_function
_Caution: Occasionally sync tokens are invalidated by the server,
resulting in a Sync token is no longer valid error. When this happens,
your code should conduct a full sync and replace any stored sync tokens
you have._
2024-05-24 18:33:44 +02:00
bosiraphael
87465b13ee
5507 modify the partial sync cron to work with the new statuses (#5512)
Closes #5507
2024-05-24 18:27:54 +02:00
Thomas Trompette
18fafbdeb5
Rename findAvailableTables endpoint (#5557)
As title
2024-05-24 10:57:46 +02:00
Marie
4bd0aafb8e
[fix] Update remote table sync status in cache after schema update (#5553)
Upon schema update, sync status can change from synced to non_synced in
case the update regards a table that was deleted. Let's update the sync
status too to avoid displaying the table as still synchronized.


https://github.com/twentyhq/twenty/assets/51697796/7ff2342b-ce9f-4179-9b76-940617cf1292
2024-05-24 10:20:08 +02:00
Thomas Trompette
fede721ba8
Add sorter for distant tables (#5546)
As title
2024-05-23 22:36:50 +02:00
Marie
fe5b558477
[FE] Update remote table schema + refactor Tables list (#5548)
Closes #5062.

Refactoring tables list to avoid rendering all toggles on each sync or
schema update while using fresh data:
- introducing id for RemoteTables in apollo cache
- manually updating the cache for the record that was updated after a
sync or schema update instead of fetching all tables again
2024-05-23 17:00:24 +02:00
Thomas Trompette
0d6fe7b2b4
Handle relations separately for remotes (#5538)
Remote object id columns are not removed anymore when a remote object is
unsynced.
This is because we do not use relations anymore. We only created the id
field. So the current behavior that was implemented for custom objects,
to retrieve the fields to deleted, does not work.

Since remote object relations are really different, I extracted the
logic from `objectMetadataService`. It now handles only the relations
for custom objects creation and deletion (this part should be extracted
as well).

I create a new remote table relation service that will:
- fetch objects metadata linked to remotes (favorites,
activityTargets...)
- look for columns based on remote object name
- delete the fields and columns
2024-05-23 14:59:34 +02:00
Jérémy M
453525ca25
fix: workspace health showing error for multi select (#5547)
Fix `workspace:health` command not working properly with `MULTI_SELECT`
field metadata type.
2024-05-23 12:02:40 +02:00
Charles Bochet
7b1bea3a8a Release patch v0.11.3 2024-05-23 08:41:37 +02:00
Marie
6b1d4e0744
[Fix] Do not allow names with whitespaces (#5542)
As per title
2024-05-23 07:43:09 +02:00
Thaïs
04bf697b25
feat: add feature flag to activate Links field creation (#5535)
Related issue: #3607
2024-05-22 18:06:32 +02:00
Weiko
4e533bf2ef
fix pgGraphqlQuery with concurent search path (#5537) 2024-05-22 17:14:33 +02:00
Thomas Trompette
5448512bdc
Add quotes for table name (#5533)
As title
2024-05-22 14:21:32 +02:00
Jérémy M
bb6df43d17
fix: twentyORM datasource configuration for ssl (#5529)
We need to specify ssl configuration for TwentyORM datasources when
needed, otherwise connection will be broken.
2024-05-22 11:11:05 +02:00
bosiraphael
beaaf33544
5498 create a feature flag is gmail sync v2 enabled (#5501)
Closes #5498
2024-05-22 09:25:06 +02:00
Marie
3deda2f29a
Update foreign table to distant table schema (#5508)
Closes #5069 back-end part

And:
- do not display schemaPendingUpdates status on remote server lists as
this call will become too costly if there are dozens of servers
- (refacto) create foreignTableService

After this is merged we will be able to delete remoteTable's
availableTables column
2024-05-21 21:25:38 +02:00
gitstart-twenty
36b467d301
Fix storybook tests (#5487)
Fixes #5486

---------

Co-authored-by: gitstart-twenty <gitstart-twenty@users.noreply.github.com>
Co-authored-by: v1b3m <vibenjamin6@gmail.com>
Co-authored-by: Charles Bochet <charles@twenty.com>
2024-05-21 20:24:08 +02:00
bosiraphael
e47101e08b
5483 modify messagechannel syncstatus (#5484)
- Closes #5483
- Fix seeds
- Add default value to syncSubStatus
2024-05-21 13:31:39 +02:00
Jérémy M
66a23a852f
fix: unwanted change moving back datetime to date (#5499)
Moving back datetime to date, due to an unwanted change.
2024-05-21 13:29:59 +02:00
martmull
ec248f8605
Remove dumb code placement (#5494)
Fix a bug introduced in [this
PR](https://github.com/twentyhq/twenty/pull/5254/files)

When a subscription is created, we need to create the subscription,
#5254 return if no subscription is created so the sub can never be
created at all

This PR fixes that
2024-05-21 12:08:51 +02:00
martmull
4fcdfbff7d
Fix unhandled exception (#5474)
Solves exception.getStatus is not a function error logs in twenty-server

Catch all errors in order to have no error log at all
2024-05-21 11:31:03 +02:00
Marie
0d16051ded
[fix] Re-introduce beforeUpdateOneObject hook (#5495)
... and disable name edition in object edition form. This feature will
be introduced by #5491
2024-05-21 10:46:49 +02:00
Félix Malfait
4907ae5a74
Improve docs (#5492)
Fix #4382 and remove useless pages to make docs more readable
2024-05-21 09:09:19 +02:00
rostaklein
a9813447f3
feat: fetch and parse full gmail message (#5160)
first part of https://github.com/twentyhq/twenty/issues/4108
related PR https://github.com/twentyhq/twenty/pull/5081

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-05-20 17:29:35 +02:00
bosiraphael
b5d3396ea9
5477 - Introduce syncsubstatus in db to refactor gmail sync behavior (#5479)
Closes #5477
2024-05-20 17:19:21 +02:00