quivr/backend/worker/Dockerfile
Stan Girard 380cf82706
feat: quivr core 0.1 (#2970)
# Description


# Testing backend 

## Docker setup
1. Copy `.env.example` to `.env`. Some env variables were added :
EMBEDDING_DIM
2. Apply supabase migratrions : 
```sh
supabase stop
supabase db reset
supabase start
```
3. Start backend containers
```
make dev
```
## Local setup 
You can also run backend without docker.
1. Install [`rye`](https://rye.astral.sh/guide/installation/). Choose
the managed python version and set the version to 3.11
2. Run the following: 
```
cd quivr/backend
rye sync
```
3. Source `.venv` virtual env : `source .venv/bin/activate`
4. Run the backend, make sure you are running redis and supabase
API: 
```
LOG_LEVEL=debug uvicorn quivr_api.main:app --log-level debug --reload --host 0.0.0.0 --port 5050 --workers 1
```
Worker: 
```
LOG_LEVEL=debug celery -A quivr_worker.celery_worker worker -l info -E --concurrency 1
```
Notifier: 
```
LOG_LEVEL=debug python worker/quivr_worker/celery_monitor.py
```

---------

Co-authored-by: chloedia <chloedaems0@gmail.com>
Co-authored-by: aminediro <aminedirhoussi1@gmail.com>
Co-authored-by: Antoine Dewez <44063631+Zewed@users.noreply.github.com>
Co-authored-by: Chloé Daems <73901882+chloedia@users.noreply.github.com>
Co-authored-by: Zewed <dewez.antoine2@gmail.com>
2024-09-02 10:20:53 +02:00

56 lines
1.3 KiB
Docker

FROM python:3.11.6-slim-bullseye AS BUILDER
ARG DEV_MODE
ENV DEV_MODE=$DEV_MODE
# Install GEOS library, Rust, and other dependencies, then clean up
RUN apt-get clean && apt-get update && apt-get install -y \
libgeos-dev \
libcurl4-openssl-dev \
libssl-dev \
binutils \
curl \
git \
gcc \
autoconf \
automake \
build-essential \
libtool \
python-dev \
wget \
# Additional dependencies for document handling
libmagic-dev \
tesseract-ocr \
poppler-utils \
tesseract-ocr \
libreoffice \
libpq-dev \
pandoc && \
rm -rf /var/lib/apt/lists/*
RUN curl -sSL https://install.python-poetry.org | POETRY_HOME=/opt/poetry python && \
cd /usr/local/bin && \
ln -s /opt/poetry/bin/poetry && \
poetry config virtualenvs.create false
WORKDIR /code/worker
# Add Rust binaries to the PATH
ENV PATH="/root/.cargo/bin:${PATH}" \
POETRY_CACHE_DIR=/tmp/poetry_cache \
PYTHONDONTWRITEBYTECODE=1 \
POETRY_VIRTUALENVS_PATH=/code/worker/.venv-docker
COPY . /code/
# Cache playwright dependency
RUN poetry run pip install playwright && playwright install --with-deps
# Run install
RUN poetry install && rm -rf $POETRY_CACHE_DIR
RUN poetry run python -c "from unstructured.nlp.tokenize import download_nltk_packages; download_nltk_packages()"
ENV PYTHONPATH=/code/worker