- add missing `excludedOperations` in
`packages/twenty-server/src/engine/middlewares/graphql-hydrate-request-from-token.middleware.ts`
- update generated graphql file
- Add missing redirection to index after password update
Closes#5062.
Refactoring tables list to avoid rendering all toggles on each sync or
schema update while using fresh data:
- introducing id for RemoteTables in apollo cache
- manually updating the cache for the record that was updated after a
sync or schema update instead of fetching all tables again
Remote object id columns are not removed anymore when a remote object is
unsynced.
This is because we do not use relations anymore. We only created the id
field. So the current behavior that was implemented for custom objects,
to retrieve the fields to deleted, does not work.
Since remote object relations are really different, I extracted the
logic from `objectMetadataService`. It now handles only the relations
for custom objects creation and deletion (this part should be extracted
as well).
I create a new remote table relation service that will:
- fetch objects metadata linked to remotes (favorites,
activityTargets...)
- look for columns based on remote object name
- delete the fields and columns
Closes#5069 back-end part
And:
- do not display schemaPendingUpdates status on remote server lists as
this call will become too costly if there are dozens of servers
- (refacto) create foreignTableService
After this is merged we will be able to delete remoteTable's
availableTables column
Fix a bug introduced in [this
PR](https://github.com/twentyhq/twenty/pull/5254/files)
When a subscription is created, we need to create the subscription,
#5254 return if no subscription is created so the sub can never be
created at all
This PR fixes that
For remotes, we will only create the foreign key, without the relation
metadata. Expected behavior will be:
- possible to create an activity. But the remote object will not be
displayed in the relations of the activity
- the remote objects should not be available in the search for relations
Also switched the number settings to an enum, since we now have to
handle `BigInt` case.
---------
Co-authored-by: Thomas Trompette <thomast@twenty.com>
- Check if a table with the same name already exists
- If yes, add a number suffix, and check again
Co-authored-by: Thomas Trompette <thomast@twenty.com>
## Context
Fixes#5403
Transliteration is now integrated to form validation through the schema.
While it does not impede inputting an invalid value, it impedes
submitting a form that will fail as the transliteration is not possible.
Until then we were only performing the transliteration at save time in
the front-end, but it's best to provide the information as soon as
possible. Later we will add helpers to guide the user (eg "This name is
not valid": https://github.com/twentyhq/twenty/issues/5428).
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
We do not update the comment on the local table when a foreign table key
is deleted.
This was not breaking, which is why we did not see it. But comments
should be kept up to date.
---------
Co-authored-by: Thomas Trompette <thomast@twenty.com>
This PR is updating all object metadata entities with the new
decorators, and deleting the old ones.
This way we can use the new TwentyORM with all the standard objects.
---------
Co-authored-by: Weiko <corentin@twenty.com>
In this PR
1. Enable deletion of relation fields in the product and via the api
(migration part was missing in the api)
3. Change wording, only use "deactivate" and "delete" everywhere (and
not a mix of the two + "disable", "erase")
When user is deleting its account on a specific workspace, we remove it
as if it was a workspaceMember, and if no workspaceMember remains, we
delete the workspace and the associated stripe subscription
A user has reported an issue with REST API.
We have recently migrated the graphql IDs from UUID to ID type. As Rest
API is leveraging the graphql API under the hood, the Rest API query
builder should be updated accordingly
## Context
Since pg_graphql does not return specific error/exception, we have to
map the error message and throw validation errors when needed
This PR adds a check on unicity constraint error returned by pg_graphql
when we are trying to insert duplicate records and returns a 400 instead
of being handled by the exceptionHandler as a 500.
## Context
Yoga can catch its own errors and we don't want to convert them again.
Moreover those errors don't have an "originalError" property and should
be schema related only (400 validation) so we only want to send them
back to the API caller without going through the exception handler.
Also fixed an issue in the createMany which was throwing a 500 when id
was missing from the creation payload. It seems the FE is always sending
an ID but it should actually be optional since the DB can generate one.
This is a regression from the new UUID validation introduced a few weeks
ago.
Fixes#5276.
Updates were not triggering a cache version incrementation because they
do not trigger migrations while that is where the caching version logic
was.
We have decided to move the cache incrementation logic to the services.
## Context
JobsModule is hard to maintain because we provide all the jobs there,
including their dependencies. This PR aims to split jobs in dedicated
modules.
In this PR, I'm refactoring the way we associate messageParticipant post
person/company creation. Instead of looking a all person without
participant, we are passing the one that were just created.
Also, I'm making sure the message and messageParticipant creation
transaction is commited before creating person/company creation (and
then messageParticipant association)
This PR fixes several issues:
- enum naming should be: {tableName}_{fieldName}_enum and respecting the
case
- defaultValue format handled in the FE should respect the one in the BE
In my opinion we should refactor the defaultValue:
- we should respect backend format: "'myDefault'" for constant default
and "0" for float, "now" for expressions, "true" for booleans. we can
rename it to defaultValueExpression if it is more clear but we should
not maintain a parallel system
- we should deprecate option: isDefaultValue which is confusing
- we should re-work backend to have a more unified approach between
fields and avoid having if everywhere about select, multiselect, and
currency cases. one unified "computeDefaultValue" function should do the
job
What is still broken:
- currency default Value on creation. I think we should do the refactor
first
- select default value edition.
These cases do not break the schema but are ignored currently
# This PR
- Fix#5021
- Migrates `passwordResetToken` and `passwordResetTokenExpiresAt` fields
from `core.users` to `core.appToken`
- Marks those fields as `deprecated` so we can remove them later if we
are happy with the transition -- I took this decision on my own,
@FellipeMTX let me know what you think about it, we can also remove them
straight away if you think it's better
- Fixed the `database:migration` script from the `twenty-server` to:
```json
"database:migrate": {
"executor": "nx:run-commands",
"dependsOn": ["build"], // added this line
"options": {
"cwd": "packages/twenty-server",
"commands": [
"nx typeorm -- migration:run -d src/database/typeorm/metadata/metadata.datasource",
"nx typeorm -- migration:run -d src/database/typeorm/core/core.datasource"
],
"parallel": false
}
},
```
The migration script wasn't running because the builds were not executed
- [x] Added unit tests for the token.service file's changes
Looking forward to hearing feedback from you
cc: @charlesBochet
---------
Co-authored-by: Weiko <corentin@twenty.com>
This PR is a follow up of PR #5153.
This one introduce some changes on how we're querying composite fields.
We can do:
```typescript
export class CompanyService {
constructor(
@InjectWorkspaceRepository(CompanyObjectMetadata)
private readonly companyObjectMetadataRepository: WorkspaceRepository<CompanyObjectMetadata>,
) {}
async companies(): Promise<CompanyObjectMetadata[]> {
// Old way
// const companiesFilteredByLinkLabel = await this.companyObjectMetadataRepository.find({
// where: { xLinkLabel: 'MyLabel' },
// });
// Result will return xLinkLabel property
// New way
const companiesFilteredByLinkLabel = await this.companyObjectMetadataRepository.find({
where: { xLink: { label: 'MyLabel' } },
});
// Result will return { xLink: { label: 'MyLabel' } } property instead of { xLinkLabel: 'MyLabel' }
return companiesFilteredByLinkLabel;
}
}
```
Also we can now inject `TwentyORMManage` class to manually create a
repository based on a given `workspaceId` using
`getRepositoryForWorkspace` function that way:
```typescript
export class CompanyService {
constructor(
// TwentyORMModule should be initialized
private readonly twentyORMManager,
) {}
async companies(): Promise<CompanyObjectMetadata[]> {
const repository = await this.twentyORMManager.getRepositoryForWorkspace(
'8bb6e872-a71f-4341-82b5-6b56fa81cd77',
CompanyObjectMetadata,
);
const companies = await repository.find();
return companies;
}
}
```
## Context
Because creating an object in metadata also generates a graphql type and
because graphql does not allow 2 types with the same name, we have to
manage a list of reserved keywords that can't be used as object names.
Currently we were maintaining a list of the core objects but we also
have to introduce composite fields that are also generated as gql types.
## Context
Positions are used within a view to display and sort the different
records of standard/custom object.
When we add a new record and want to put it before the existing first
record, we have to use float values to insert them in the DB and respect
the desired order. We are adding a new command that can be executed to
flatten those positions.
---------
Co-authored-by: bosiraphael <raphael.bosi@gmail.com>
While using middleware (executed pre-graphql) for graphql endpoint, we
need to swallow exception and return errors with a 200. Otherwise it's
not a valid graphql response
Various fixes
- Remote objects are read-only for now, we already hide and block most
of the write actions but the button that allows you to add a new record
in an empty collection was still visible.
- CreatedAt is not mandatory on remote objects (at least for now) so it
was breaking the show page, it now checks if createdAt exists and is not
null before trying to display the human readable format `Added x days
ago`
- The filters are overwritten in query-runner-args.factory.ts to handle
NUMBER field type, this was only working with filters like
```
{
"id": {
"in": [
1
]
}
```
but not with more depth such as
```
"and": [
{},
{
"id": {
"in": [
1
]
}
}
]
```
- Fixes CREATE FOREIGN TABLE raw query which was missing ",".
## Context
We recently introduced a change that now throws a 401 if the token is
invalid or expired.
The first implementation is using an allow list and 'IntrospectionQuery'
was missing so the playground was broken.
The check has been updated and we now only check the excludedOperations
list if a token is not present. This is because some operations can be
both used as loggedIn and loggedOut so we want to validate the token for
those sometimes (and set the workspace, user, cache version, etc). Still
not a very clean solution imho.
## Context
We have a non-nullable constraint on authorId in attachments and
documents, until we have soft-deletion we need to handle deletion of
workspace-members and their attachments/documents.
This PR introduces pre-hooks to deleteOne/deleteMany
This is called when a user deletes a workspace-member from the members
page
Next: needs to be done on user level as well. This is called when users
try to delete their own accounts. I've seen other issues such as
re-creating a user with a previously used email failing.
Adding stripe integration by making the server logic independent of the
input fields:
- query factories (remote server, foreign data wrapper, foreign table)
to loop on fields and values without hardcoding the names of the fields
- adding stripe input and type
- add the logic to handle static schema. Simply creating a big object to
store into the server
Additional work:
- rename username field to user. This is the input intended for postgres
user mapping and we now need a matching by name
---------
Co-authored-by: Thomas Trompette <thomast@twenty.com>
## Context
Currently, this middleware validates the token and stores the user,
workspace and cacheversion in the request object.
It only does so when a token is provided and ignores the middleware
logic if not. If the token is invalid or expired, the exception is
swallowed.
This PR removes the try/catch and adds an allowlist to skip the token
validation for operations executed while not signed-in.
I don't know a better way to do that with Nestjs. We can't easily add
the middleware per resolver without refactoring the flexible schema
engine so I'm doing it the other way around.
Fixes https://github.com/twentyhq/twenty/issues/5224
## Context
#4774
## How was it tested
Locally
## In further PRs
- Update connection status upon page change
- Adapt Info banner to dark mode
- placeholders for form
## Context
Currently we have an unicity constraint in the DB but we don't return a
clear error to the frontend before reaching the DB (which then throws a
500). This PR adds a validation check similar to what we have with field
creation
Closes#5097
- Uses "nx affected" to detect what projects need to be checked in the
current PR (for now, `ci-front` and `ci-server` workflows only).
- Caches results of certain tasks (`lint`, `typecheck`, `test`,
`storybook:build`) when a PR pipeline runs. The next runs of the same
PR's pipeline will then be able to reuse the PR's task cache to execute
tasks faster.
- Caches Yarn's cache folder to install dependencies faster in CI jobs.
- Rewrites the node modules cache/install steps as a custom, reusable
Github action.
- Distributes `ci-front` jobs with a "matrix" strategy.
- Sets common tasks config at the root `nx.json`. For instance, to
activate the `typecheck` task in a project, add `typecheck: {}` to its
`project.json` and it'll use the default config set in `nx.json` for the
`typecheck` task. Options can be overridden in each individual
`project.json` if needed.
- Adds "scope" tags to some projects: `scope:frontend`, `scope:backend`,
`scope:shared`. An eslint rule ensures that `scope:frontend` only
depends on `scope:frontent` or `scope:shared` projects, same for
`scope:backend`. These tags are used by `nx affected` to filter projects
by scope and generates different task cache keys according to the
requested scope.
- Enables checks for twenty-emails in the `ci-server` workflow.
Now that we have persistent cache for schemas, we want to be able to
reset its state when users run the database:reset db otherwise schemas
won't be synced with the new DB state.
Note: In an upcoming PR, we want to be able to invalidate the cache on a
workspace level when we change the metadata schema through twenty
version upgrade
## Context
Messaging and calendar cron jobs are only working for workspace that
have sub status different than incomplete, this is because currently
this is the simplest way to know if a user is onboarded. This should not
be the source of truth and this will be updated in a later version. In
the meantime, to make self-hosting easier, we are adding an extra check
on IS_BILLING_ENABLED env var since sub status is not relevant for
people not using billing.
We should not depend on the foreign data wrapper type to manage distant
table. The remote server should be enough to handle the table creation.
Here is the new flow to fetch available tables:
- check if the remote server have available tables already stored
- if not, import full schema in a temporary schema
- copy the tables into the available tables field
- delete the schema
Left todo:
- update remote server input for postgres so we receive the schema
---------
Co-authored-by: Thomas Trompette <thomast@twenty.com>
## Introduction
This PR introduces "TwentyORM," a custom ORM module designed to
streamline database interactions within our workspace schema, reducing
the need for raw SQL queries. The API mirrors TypeORM's to provide a
familiar interface while integrating enhancements specific to our
project's needs.
To facilitate this integration, new decorators prefixed with `Workspace`
have been implemented. These decorators are used to define entity
metadata more explicitly and are critical in constructing our schema
dynamically.
## New Features
- **Custom ORM System**: Named "TwentyORM," which aligns closely with
TypeORM for ease of use but is tailored to our application's specific
requirements.
- **Decorator-Driven Configuration**: Entities are now configured with
`Workspace`-prefixed decorators that clearly define schema mappings and
relationships directly within the entity classes.
- **Injectable Repositories**: Repositories can be injected similarly to
TypeORM, allowing for flexible and straightforward data management.
## Example Implementations
### Decorated Entity Definitions
Entities are defined with new decorators that outline table and field
metadata, relationships, and constraints. Here are examples of these
implementations:
#### Company Metadata Object
```typescript
@WorkspaceObject({
standardId: STANDARD_OBJECT_IDS.company,
namePlural: 'companies',
labelSingular: 'Company',
labelPlural: 'Companies',
description: 'A company',
icon: 'IconBuildingSkyscraper',
})
export class CompanyObjectMetadata extends BaseObjectMetadata {
@WorkspaceField({
standardId: COMPANY_STANDARD_FIELD_IDS.name,
type: FieldMetadataType.TEXT,
label: 'Name',
description: 'The company name',
icon: 'IconBuildingSkyscraper',
})
name: string;
@WorkspaceField({
standardId: COMPANY_STANDARD_FIELD_IDS.xLink,
type: FieldMetadataType.LINK,
label: 'X',
description: 'The company Twitter/X account',
icon: 'IconBrandX',
})
@WorkspaceIsNullable()
xLink: LinkMetadata;
@WorkspaceField({
standardId: COMPANY_STANDARD_FIELD_IDS.position,
type: FieldMetadataType.POSITION,
label: 'Position',
description: 'Company record position',
icon: 'IconHierarchy2',
})
@WorkspaceIsSystem()
@WorkspaceIsNullable()
position: number;
@WorkspaceRelation({
standardId: COMPANY_STANDARD_FIELD_IDS.accountOwner,
label: 'Account Owner',
description: 'Your team member responsible for managing the company account',
type: RelationMetadataType.MANY_TO_ONE,
inverseSideTarget: () => WorkspaceMemberObjectMetadata,
inverseSideFieldKey: 'accountOwnerForCompanies',
onDelete: RelationOnDeleteAction.SET_NULL,
})
@WorkspaceIsNullable()
accountOwner: WorkspaceMemberObjectMetadata;
}
```
#### Workspace Member Metadata Object
```typescript
@WorkspaceObject({
standardId: STANDARD_OBJECT_IDS.workspaceMember,
namePlural: 'workspaceMembers',
labelSingular: 'Workspace Member',
labelPlural: 'Workspace Members',
description: 'A workspace member',
icon: 'IconUserCircle',
})
@WorkspaceIsSystem()
@WorkspaceIsNotAuditLogged()
export class WorkspaceMemberObjectMetadata extends BaseObjectMetadata {
@WorkspaceField({
standardId: WORKSPACE_MEMBER_STANDARD_FIELD_IDS.name,
type: FieldMetadataType.FULL_NAME,
label: 'Name',
description: 'Workspace member name',
icon: 'IconCircleUser',
})
name: FullNameMetadata;
@WorkspaceRelation({
standardId: WORKSPACE_MEMBER_STANDARD_FIELD_IDS.accountOwnerForCompanies,
label: 'Account Owner For Companies',
description: 'Account owner for companies',
icon: 'IconBriefcase',
type: RelationMetadataType.ONE_TO_MANY,
inverseSideTarget: () => CompanyObjectMetadata,
inverseSideFieldKey: 'accountOwner',
onDelete: RelationOnDeleteAction.SET_NULL,
})
accountOwnerForCompanies: Relation
<CompanyObjectMetadata[]>;
}
```
### Injectable Repository Usage
Repositories can be directly injected into services, allowing for
streamlined query operations:
```typescript
export class CompanyService {
constructor(
@InjectWorkspaceRepository(CompanyObjectMetadata)
private readonly companyObjectMetadataRepository: WorkspaceRepository<CompanyObjectMetadata>,
) {}
async companies(): Promise<CompanyObjectMetadata[]> {
// Example queries demonstrating simple and relation-loaded operations
const simpleCompanies = await this.companyObjectMetadataRepository.find({});
const companiesWithOwners = await this.companyObjectMetadataRepository.find({
relations: ['accountOwner'],
});
const companiesFilteredByLinkLabel = await this.companyObjectMetadataRepository.find({
where: { xLinkLabel: 'MyLabel' },
});
return companiesFilteredByLinkLabel;
}
}
```
## Conclusions
This PR sets the foundation for a decorator-driven ORM layer that
simplifies data interactions and supports complex entity relationships
while maintaining clean and manageable code architecture. This is not
finished yet, and should be extended.
All the standard objects needs to be migrated and all the module using
the old decorators too.
---------
Co-authored-by: Weiko <corentin@twenty.com>
## Context
We recently enabled the option to bypass SSL certificate authority
validation when establishing a connection to PostgreSQL. Previously, if
this validation failed, the server would revert to unencrypted traffic.
Now, it maintains encryption even if the SSL certificate check fails. In
the process, we overlooked a few DataSource setups, prompting a review
of DataSource creation within our code.
## Current State
Our DataSource initialization is distributed as follows:
- **Database folder**: Contains 'core', 'metadata', and 'raw'
DataSources. The 'core' and 'metadata' DataSources manage migrations and
static resolver calls to the database. The 'raw' DataSource is utilized
in scripts and commands that require handling both aspects.
- **typeorm.service.ts script**: These DataSources facilitate
multi-schema connections.
## Vision for Discussion
- **SystemSchema (formerly core) DataSource**: Manages system schema
migrations and system resolvers/repos. The 'core' schema will be renamed
to 'system' as the Core API will include parts of the system and
workspace schemas.
- **MetadataSchema DataSource**: Handles metadata schema migrations and
metadata API resolvers/repos.
- **(Dynamic) WorkspaceSchema DataSource**: Will be used in the Twenty
ORM to access a specific workspace schema.
We currently do not support cross-schema joins, so maintaining these
DataSources separately should be feasible. Core API resolvers will
select the appropriate DataSource based on the field context.
- **To be discussed**: The potential need for an AdminDataSource (akin
to 'Raw'), which would be used in commands, setup scripts, and the admin
panel to connect to any database schema without loading any model. This
DataSource should be reserved for cases where utilizing metadata,
system, or workspace entities is impractical.
## In This PR
- Ensuring all existing DataSources are compliant with the SSL update.
- Introducing RawDataSource to eliminate the need for declaring new
DataSource() instances in commands.
## Context
@lucasbordeau introduced a new Yoga plugin that allows us to cache our
requests (👏), see https://github.com/twentyhq/twenty/pull/5189
I'm simply updating the implementation to allow us to use different
cache storage types such as redis
Also adding a check so it does not use cache for other operations than
ObjectMetadataItems
## Test
locally, first call takes 340ms, 2nd takes 30ms with 'redis' and 13ms
with 'memory'
