Commit Graph

63 Commits

Author SHA1 Message Date
Sasank Chilamkurthy
d90c3110cf
Fail entrypoint if migration fails (#8590)
If migration fails for some reason, the script runs anyway and creates
the file which indicates that migration is complete. This is obviously
not cool. If database is not available for some reason and migrations
fail, the container should exit, not continue.

Relevant stack overflow: https://stackoverflow.com/a/2871034
2024-11-19 18:57:15 +01:00
Félix Malfait
736635a94b
Begin moving to postgres spilo + adding pgvector (#8309)
We will remove the `twenty-postgres` image that was used for local
development and only use `twenty-postgres-pilo` (which we use in prod),
bringing the development environment closer to prod and avoiding having
to maintain 2 images.


Instead of provisioning the super user after the db initialization, we
directly rely on the superuser provided by Spilo for simplicity. We also
introduce a change that tries to create the right database (`default` or
`test`) based on the context.
  

How to test:
```
docker build -t twentycrm/twenty-postgres-spilo:latest -f ./packages/twenty-docker/twenty-postgres-spilo/Dockerfile .
docker images --no-trunc | grep twenty-postgres-spilo
postgres-on-docker:
	docker run \
	--name twenty_pg \
	-e PGUSER_SUPERUSER=twenty \
	-e PGPASSWORD_SUPERUSER=twenty \
	-e ALLOW_NOSSL=true \
	-v twenty_db_data:/home/postgres/pgdata \
	-p 5432:5432 \
	REPLACE_WITH_IMAGE_ID
```
2024-11-15 09:38:30 +01:00
ZiaCodes
57d9b8e8b4
feat: generate secret function and replaced few instances (#7810)
This PR fixes #4588

---------

Co-authored-by: Félix Malfait <felix@twenty.com>
Co-authored-by: Charles Bochet <charles@twenty.com>
2024-10-30 12:07:11 +01:00
Thomas Mol
ba2ee0da72
Refactor Redis connections to use Redis URL - closes #7421 (#7736)
Closes #7421

This pull request consolidates Redis connection parameters into a single
`REDIS_URL` environment variable across various configuration files and
code modules. The most important changes include updates to environment
variable files, Docker and Kubernetes configurations, and code modules
to utilize the new `REDIS_URL` format.

### Environment Variable Updates:
*
[`packages/twenty-docker/.env.example`](diffhunk://#diff-f4b5e7acc0dde630eafb2228390ca56bd56db0d183582be6433a9ee808088a4cL8-R8):
Replaced `REDIS_HOST` and `REDIS_PORT` with `REDIS_URL`.
*
[`packages/twenty-server/.env.example`](diffhunk://#diff-c06e244338b33286ea90221635809785352a971de53f647ea65650124ae74ad2L53-R53):
Replaced `REDIS_HOST`, `REDIS_PORT`, `REDIS_USERNAME`, and
`REDIS_PASSWORD` with `REDIS_URL`.
*
[`packages/twenty-server/.env.test`](diffhunk://#diff-def44a08e699c9deca2f72c9f87951de5d33d9ccf5621eab2f34978df8ad0954L16-R16):
Replaced `REDIS_HOST`, `REDIS_PORT`, `REDIS_USERNAME`, and
`REDIS_PASSWORD` with `REDIS_URL`.

### Docker and Kubernetes Configuration Updates:
*
[`packages/twenty-docker/docker-compose.yml`](diffhunk://#diff-545fb51ad66f93f727bb8f8b375dc6858b33348d91d5e51ad931fb0bbc1affeeL28-R28):
Replaced `REDIS_HOST` and `REDIS_PORT` with `REDIS_URL` in multiple
service definitions.
[[1]](diffhunk://#diff-545fb51ad66f93f727bb8f8b375dc6858b33348d91d5e51ad931fb0bbc1affeeL28-R28)
[[2]](diffhunk://#diff-545fb51ad66f93f727bb8f8b375dc6858b33348d91d5e51ad931fb0bbc1affeeL62-R61)
*
[`packages/twenty-docker/k8s/manifests/deployment-server.yaml`](diffhunk://#diff-91623ed4e8b2088947cfa9a5dad76b6013e8db0c150d84347a215fa9ad78bf04L44-R45):
Replaced `REDIS_HOST` and `REDIS_PORT` with `REDIS_URL`.
*
[`packages/twenty-docker/k8s/manifests/deployment-worker.yaml`](diffhunk://#diff-8532debd131ce168a2527fa6a9be6405792178576ee47e2eef0cd9f3ff4a8f8cL43-R44):
Replaced `REDIS_HOST` and `REDIS_PORT` with `REDIS_URL`.
*
[`packages/twenty-docker/k8s/terraform/deployment-server.tf`](diffhunk://#diff-b4c468660ab00bd03589d0d47502c477ca83f7d876ff196534880b02ba46dce0L64-R65):
Replaced `REDIS_HOST` and `REDIS_PORT` with `REDIS_URL`.
*
[`packages/twenty-docker/k8s/terraform/deployment-worker.tf`](diffhunk://#diff-e5d6a1f68d5391e2120aef6261f22f905aa1ce0a567e31b21a27d44f313eaecaL62-R63):
Replaced `REDIS_HOST` and `REDIS_PORT` with `REDIS_URL`.

### Code Module Updates:
*
[`packages/twenty-server/src/engine/core-modules/cache-storage/cache-storage.module-factory.ts`](diffhunk://#diff-06e787a7c8a48022d5909b5df9b8c6ca192521cf32f51d7f561cee937bed6678L23-R35):
Replaced `REDIS_HOST` and `REDIS_PORT` with `REDIS_URL`.
*
[`packages/twenty-server/src/engine/core-modules/environment/environment-variables.ts`](diffhunk://#diff-26ce615693b053eda02aa48aa2e30400381a2588dcb08d4a9dc3b0bf5bdd6fe7L378-R384):
Added validation for `REDIS_URL`.
*
[`packages/twenty-server/src/engine/core-modules/message-queue/message-queue.module-factory.ts`](diffhunk://#diff-7bd644d28bdd86c159c1d71242753df5d5acd91c73c6e60b2e35caa53ed3836bR6):
Replaced `REDIS_HOST`, `REDIS_PORT`, `REDIS_USERNAME`, and
`REDIS_PASSWORD` with `REDIS_URL`.
[[1]](diffhunk://#diff-7bd644d28bdd86c159c1d71242753df5d5acd91c73c6e60b2e35caa53ed3836bR6)
[[2]](diffhunk://#diff-7bd644d28bdd86c159c1d71242753df5d5acd91c73c6e60b2e35caa53ed3836bL35-R41)

### Documentation Updates:
*
[`packages/twenty-website/src/content/developers/self-hosting/self-hosting-var.mdx`](diffhunk://#diff-c4cc78a3ce18b6edb10f1aee8990271e1d2796a8c06c1c6ae3b68db8d52278a3L37-R37):
Updated documentation to reflect the change to `REDIS_URL`.
*
[`packages/twenty-website/src/content/developers/self-hosting/upgrade-guide.mdx`](diffhunk://#diff-c7d757829f6128e1f47aa1955bde561292dce558280511fab66281afa99042a5R102-R112):
Added upgrade instructions for the new `REDIS_URL` variable.

---------

Co-authored-by: Weiko <corentin@twenty.com>
Co-authored-by: Charles Bochet <charles@twenty.com>
2024-10-16 17:17:44 +02:00
Charles Bochet
1b99a05dec Put back twenty-website dockerfile 2024-10-15 18:36:10 +02:00
Weiko
7e808cf529
Remove pgGraphql from setup (#7716)
## Context
This PR removes pg_graphql from the setup. It also updates the local
setup documentation accordingly.

Note: We removed local setup scripts to align with redis installation,
the setup should be much simpler since we don't rely on pg_graphql
anymore.

## Test
tested locally with docker + mac (brew) setup
2024-10-15 16:59:44 +02:00
BOHEUS
a7a4052109
Docker cleanup (#7655)
Related to #7648 

@charlesBochet I've deleted only dev related commands in Makefile, if
it's wrong, I'll change it ASAP
2024-10-13 20:40:27 +02:00
Ciara Hatcher
db9ec58f5d
Updating terraform and k8s files adding redis... (#7425)
Also updated the way secrets are generated with Terraform and some code
cleanup
2024-10-07 10:23:42 +02:00
Charles Bochet
e4959ad534
Add 0.30 release notes (#7300)
In this PR:
- update your environment variables to default `CACHE_STORAGE_TYPE` to
`redis` and `MESSAGE_QUEUE_TYPE` to `bull-mq`
- add redis container to our default docker-compose
- add `REDIS_HOST` and `REDIS_PORT` to docker-compose yaml
- add upgrade instructions
2024-09-27 19:10:26 +02:00
Quentin G.
d1871a4a64
feat: replace ts-node with tsx and pre-install it globaly in docker (#7027)
Implement https://github.com/twentyhq/twenty/issues/5976 suggestion.
It leads to faster boot time since we don't have to install
dependencies.

It should also solve https://github.com/twentyhq/twenty/issues/4772 as
we don't need to run ressource-heavy `npm install` (the mentioned
migration step does not seem to exist anymore anyway.
2024-09-14 11:05:01 +02:00
Quentin G.
b0bc839a49
feat: improve self hosting documentation (#7024)
This PR aims to resolve common misunderstanding while deploying Twenty
with Docker compose.

I've made the documentation clearer, and more detailed.

This should solve https://github.com/twentyhq/twenty/issues/5184, #6140
and similar issues
2024-09-14 11:03:28 +02:00
Ciara Hatcher
c3bf94e4cc
Lumosviridi v20 kubernetes updates (#6356)
Updates for v20+ and misc terraform bug fixes. Also refactored to use
terraform variables instead of locals which helps with readability and
ease of use for new users.

Terraform validation is currently passing:
![Screenshot 2024-07-21 at 13 18
37](https://github.com/user-attachments/assets/02aadc2d-d3f6-4e8b-9315-64e25191d9e6)

Additionally added [terraform-docs](https://terraform-docs.io/) to
generate a more helpful README for terraform specific configuration.

Raw K8s manifests were updated with changes for v20+ as well.

---------

Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
2024-08-08 13:55:45 +02:00
Sandab Gharti GC
7e80cd5366
mod: entrypoint for docker to setup db even in server with less memory (#5815)
Hello team,

- Modified File: `packages/twenty-docker/twenty/entrypoint.sh`
- Reason: The current `entrypoint.sh` works fine for server with RAM
more than 8GB. But the server with less RAM will give error while
running `scripts/setup-db.ts`.

![setup-db-failed](https://github.com/twentyhq/twenty/assets/52379037/5e27240a-23ef-4d52-93bb-337e1a004feb)

- My commit: It will set `NODE_OPTIONS="--max-old-space-size=8192"` in
docker entrypoint which guarantee to run the script successfully.

---------

Co-authored-by: Lucas Bordeau <bordeau.lucas@gmail.com>
2024-07-19 14:54:00 +02:00
Ciara Hatcher
43016db801
Support for Kubernetes via Terraform and Manifests (#5721)
# Support for Kubernetes via Terraform and Manifests

Adding basic support for the following Kubernetes resources

- persistent volume
  - server
  - database
- persistent volume claim
  - server
  - database
- deployment
  - server
  - database
- ingress
  - server
- service
  - server
  - database
- secret
  - server

---------

Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
2024-07-10 13:54:27 +02:00
martmull
cff8561597
Upgrade pg graphql version to 1.5.6 (#5937)
- update `pg_graphql` version doc
- update `pg_graphql` version to 1.5.6
2024-06-18 17:34:16 +02:00
Jérémy M
04edf2bf7b
feat: add resolve absolute path util (#5836)
Add a new util called `resolveAbsolutePath` to allow providing absolute
path for environment variable like `STORAGE_LOCAL_PATH`.
If the path in the env start with `/` we'll not prefix it with
`process.cwd()`.

Also we're using a static path for the old `db_initialized` file now
named `db_status` and stop using the env variable for this file as this
one shouldn't ne stored in the `STORAGE_LOCAL_PATH`.

Fix #4794

---------

Co-authored-by: Quentin Galliano <qgalliano@gmail.com>
2024-06-12 21:17:31 +02:00
Charles Bochet
a57e251208
Fix docs build in CI (#5826) 2024-06-11 19:06:37 +02:00
bosiraphael
f166171a1c
5531 update gmail full sync to v2 (#5674)
Closes #5531

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-05-31 13:29:58 +02:00
Quentin G
c1740e3395
Feat/add postgres spilo (#5049)
This is a new Docker image that extends the Zalando Spilo image in order
to install the extensions needed by Twenty : pg_graphql, wrappers,
mysql_fdw
2024-04-29 17:09:41 +02:00
Thaïs
d3170fc1ea
fix: fix root start script (#5032)
Fixes #5022

See https://nx.dev/recipes/running-tasks/root-level-scripts#setup
2024-04-19 18:28:02 +02:00
Thaïs
7065495223
fix: attempt to fix Dockerfile front build (#5020) 2024-04-18 11:24:39 +02:00
Thaïs
9f83cc1426
refactor: move @/ui/display/icon to twenty-ui (#4820)
Split from https://github.com/twentyhq/twenty/pull/4518

Part of https://github.com/twentyhq/twenty/issues/4766
2024-04-12 15:30:48 +02:00
Quentin G
bf60227d67
feat: add SENTRY_RELEASE env (#4912)
Add support for a new SENTRY_RELEASE and SENTRY_ENVIRONMENT env.
It is optional and allows to init sentry with a Release version and an
env (used internally at Twenty).
Docker image have been updated do intergrate the new env as an Argument
2024-04-11 16:53:15 +02:00
Quentin G
e48960afbe
chore: remove old deployments (#4922)
We migrated all Twenty's env to the new Twenty docker image. We can now
remove old deployments of front and back.
This PR takes care of that!
2024-04-11 16:35:11 +02:00
Félix Malfait
7a34dc4910
Simplify docs and remove Docker local setup (#4783)
Having 2 different dev setups caused confusion, let's remove the Docker
local setup and recommend people install yarn locally.

Also simplified some docs by merging pages together, the recommend
self-hosting option is now the docker-compose / to adapt the
docker-compose.
2024-04-03 16:38:28 +02:00
Quentin G
35fb77d9a8
feat: reinforce one liner experience (#4688)
* feat: add more dependencies check, randomize postgres admin password, tail logs of server container

* feat: improve retro compatibility

* feat: comment POSTGRES_ADMIN_PASSWORD as it will be generated by the one liner
2024-03-29 15:25:41 +01:00
Quentin G
d6de380e02
feat: add one liner install command (#4613)
* feat: add one liner

* fix: interactive issue & add support for both linux & mac

* feat: move quick start documentation

* feat: catch errors

* feat: check if directory exists

* feat: default to yes for prompt

* feat: open in browser

* fix: format

* feat: do not expose STORAGE_LOCAL_PATH env but handle the case where it would be set

* fix: db reset command wasn't working out of the box

* Update install.sh

Co-authored-by: Darek Desu <4459421+darekdesu@users.noreply.github.com>

* feat: harden the whole UX with one-liner

* fix: small logical order adjustment

* Update packages/twenty-docs/docs/start/self-hosting/docker-compose.mdx

---------

Co-authored-by: Darek Desu <4459421+darekdesu@users.noreply.github.com>
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
2024-03-27 21:28:03 +01:00
Quentin G
90ce7709dd
fix: update docker-compose database volumes (#4677) 2024-03-27 08:47:58 +01:00
Darek Desu
22d17d855c
File token chores (#4664)
* Missing file token chores

* Make whole idea folder ignored
2024-03-26 13:42:09 +01:00
Quentin G
ef8867e552
chore: debug Twenty dockerfile failing to build on Github Actions (#4658)
* chore: debug Twenty dockerfile failing to build on Github Actions

* fix: remove commented code
2024-03-26 12:32:53 +01:00
Darek Desu
0549313c43
Update .env.example (#4661)
Added missing FILE_TOKEN_SECRET variable
2024-03-26 12:31:57 +01:00
martmull
e2af5b8628
4525 upgrade pg graphql (#4646)
* TO remove: add multiple workspace with command

* TO remove: update build script

* Update script and add doc

* TO remove: add more seed workspaces

* Build pg_graphql files for 1.5.1

* Build pg_graphql files for 1.5.1 macos arm

* Remove 600 seed workspaces

* Build pg_graphql files for 1.5.1 macos intel
2024-03-25 15:08:17 +01:00
Quentin G
aee6d49ea9
feat: add a docker-compose file for production (#4609)
* feat: add a docker-compose file for production

* fix: remove unused user filed

* fix: do not provide default token secrets
2024-03-22 09:16:39 +01:00
Quentin G
1aa48d3bf7
feat: merge front and server dockerfiles and optimize build (#4589)
* feat: merge front and server dockerfiles and optimize build

* fix: update image label

* fix: bring back support for REACT_APP_SERVER_BASE_URL injection at runtime

* fix: remove old entries & add nx cache in dockerignore

* feat: generate frontend config at runtime using Nest

* fix: format and filename

* feat: use the EnvironmentService and leave default blank

* feat: add support for DB migrations
2024-03-21 19:22:21 +01:00
Félix Malfait
b6e8bb1a6c
Delete auth/file front variables (#4455) 2024-03-19 22:19:40 +01:00
Charles Bochet
eb07b373a7 Fix storage s3 endpoint not being optional 2024-03-15 23:08:30 +01:00
Charles Bochet
feebc45d31
Fix storybook tests on IconPicker (#4510) 2024-03-15 21:47:07 +01:00
Charles Bochet
5e5ae0b2ca Fix server container build 2024-03-15 21:38:40 +01:00
Charles Bochet
e0ae12ffa1
Fix server deploy (#4509)
* Fix server deployment broken by nx

* Fix server deployment broken by nx

* Fix server deployment broken by nx

* Fix

* Fix

* Fix

* Fix
2024-03-15 21:15:14 +01:00
Charles Bochet
4eae3b8290
Fix front production container build (#4478)
* Fix deploy main

* Fix deploy main
2024-03-14 16:13:27 +01:00
Charles Bochet
aa13b8338d Fix website build 2024-02-25 22:47:07 +01:00
Charles Bochet
fa9edad311 Fix website build 2024-02-25 22:38:12 +01:00
Charles Bochet
88990144cf
Fix website build (#3986)
* Fix website build

* Try fix

* Try fix
2024-02-15 11:03:29 +01:00
Charles Bochet
9777c5fbce
Fix website build (#3983) 2024-02-14 21:28:26 +01:00
Thaïs
e011ecbd6f
POC: generate twenty-server package.json with Nx (#3654)
* POC: generate twenty-server package.json with Nx

* Re-add passport

* Fix instal

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-02-13 22:16:21 +01:00
Thaïs
a654205dbc
chore: set up twenty-emails config so build isn't needed in development (#3619)
* chore: set up twenty-emails config so build isn't needed in development

* fix: fix script dependency

* chore: use @vitejs/plugin-react-swc

* Remove useless dependancy

* Fix typing

* chore: use baseUrl in twenty-emails

* chore: fix docker server prod build

* refactor: optimize Docker file and tsconfig

* fix: fix WORKDIR in docker

---------

Co-authored-by: martmull <martmull@hotmail.fr>
2024-01-29 06:17:12 -03:00
Charles Bochet
3f0743493b
Fix Continuous Deployment script after adding twenty-emails package (#3589)
* Fix cd

* Fix server

* Fix server

* Fix server

* Fix server

* Fix

* Fix docs

* Fix
2024-01-23 12:30:03 +01:00
Charles Bochet
762a56782c Fix build server 2024-01-23 10:46:36 +01:00
Charles Bochet
a7265fa3b4
Remove flag relation select (#3588)
* Remove feature flag on relation and select

* Move packages back to twenty-server to enable smaller build without using nx

* Fix package.json
2024-01-23 09:59:00 +01:00
Charles Bochet
03bf597301 Re-add docker-compose up command 2024-01-13 11:02:00 +01:00