mirror of
https://github.com/StanGirard/quivr.git
synced 2024-12-14 21:21:56 +03:00
8bbe6e7054
To enable the ingestion of copy protected PDF via OCR instead of text extraction # Description Copy protected PDF can't be properly imported via the standard langchain loader. See the following errors: ``` 2023-11-15 14:16:31,927 [INFO] models.files: Computing documents from file Cradle to Cradle Criteria for the built environmen.pdf [nltk_data] Downloading package punkt to [nltk_data] /home/pascal_gula_luccid_ai/nltk_data... [nltk_data] Unzipping tokenizers/punkt.zip. [nltk_data] Downloading package averaged_perceptron_tagger to [nltk_data] /home/pascal_gula_luccid_ai/nltk_data... [nltk_data] Unzipping taggers/averaged_perceptron_tagger.zip. Error processing file: detectron2 is not installed, pytesseract is not installed and the text of the PDF is not extractable. To process this file, install detectron2, install pytesseract, or remove copy protection from the PDF. ``` ``` 2023-11-15 15:04:14,624 [INFO] models.files: Computing documents from file Cradle to Cradle Criteria for the built environmen.pdf Error processing file: Unable to get page count. Is poppler installed and in PATH? ``` ``` 023-11-15 15:59:11,886 [INFO] models.files: Computing documents from file Cradle to Cradle Criteria for the built environmen.pdf Error processing file: tesseract is not installed or it's not in your PATH. See README file for more information. ``` ## Checklist before requesting a review Please delete options that are not relevant. - [x] My code follows the style guidelines of this project - [x] I have performed a self-review of my code - [x] I have commented hard-to-understand areas - [x] I have ideally added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] Any dependent changes have been merged ## Screenshots (if appropriate): None Co-authored-by: Stan Girard <girard.stanislas@gmail.com>
42 lines
1012 B
Docker
42 lines
1012 B
Docker
# Using a slim version for a smaller base image
|
|
FROM python:3.11.6-slim-bullseye
|
|
|
|
ARG DEV_MODE
|
|
ENV DEV_MODE=$DEV_MODE
|
|
|
|
# Install GEOS library, Rust, and other dependencies, then clean up
|
|
RUN apt-get clean && apt-get update && apt-get install -y \
|
|
libgeos-dev \
|
|
libcurl4-openssl-dev \
|
|
libssl-dev \
|
|
binutils \
|
|
pandoc \
|
|
curl \
|
|
git \
|
|
poppler-utils \
|
|
tesseract-ocr \
|
|
build-essential && \
|
|
rm -rf /var/lib/apt/lists/* && apt-get clean
|
|
|
|
# Add Rust binaries to the PATH
|
|
ENV PATH="/root/.cargo/bin:${PATH}"
|
|
|
|
WORKDIR /code
|
|
|
|
# Copy just the requirements first
|
|
COPY ./requirements.txt .
|
|
|
|
# Upgrade pip
|
|
RUN pip install --upgrade pip
|
|
|
|
# Increase timeout to wait for the new installation
|
|
RUN pip install --no-cache-dir -r requirements.txt --timeout 200
|
|
|
|
RUN if [ "$DEV_MODE" = "true" ]; then pip install --no-cache debugpy --timeout 200; fi
|
|
|
|
# Copy the rest of the application
|
|
COPY . .
|
|
|
|
EXPOSE 5050
|
|
|
|
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "5050", "--workers", "6"] |