Commit Graph

27 Commits

Author SHA1 Message Date
Jacopo Chevallard
ef90e8e672
feat: introducing configurable retrieval workflows (#3227)
# Description

Major PR which, among other things, introduces the possibility of easily
customizing the retrieval workflows. Workflows are based on LangGraph,
and can be customized using a [yaml configuration
file](core/tests/test_llm_endpoint.py), and adding the implementation of
the nodes logic into
[quivr_rag_langgraph.py](1a0c98437a/backend/core/quivr_core/quivr_rag_langgraph.py)

This is a first, simple implementation that will significantly evolve in
the coming weeks to enable more complex workflows (for instance, with
conditional nodes). We also plan to adopt a similar approach for the
ingestion part, i.e. to enable user to easily customize the ingestion
pipeline.

Closes CORE-195, CORE-203, CORE-204

## Checklist before requesting a review

Please delete options that are not relevant.

- [X] My code follows the style guidelines of this project
- [X] I have performed a self-review of my code
- [X] I have commented hard-to-understand areas
- [X] I have ideally added tests that prove my fix is effective or that
my feature works
- [X] New and existing unit tests pass locally with my changes
- [X] Any dependent changes have been merged

## Screenshots (if appropriate):
2024-09-23 09:11:06 -07:00
Stan Girard
9cbb47f75a
feat: Add external Supabase URL support for generating file signed URL (#3179)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2024-09-10 07:53:44 -07:00
Stan Girard
380cf82706
feat: quivr core 0.1 (#2970)
# Description


# Testing backend 

## Docker setup
1. Copy `.env.example` to `.env`. Some env variables were added :
EMBEDDING_DIM
2. Apply supabase migratrions : 
```sh
supabase stop
supabase db reset
supabase start
```
3. Start backend containers
```
make dev
```
## Local setup 
You can also run backend without docker.
1. Install [`rye`](https://rye.astral.sh/guide/installation/). Choose
the managed python version and set the version to 3.11
2. Run the following: 
```
cd quivr/backend
rye sync
```
3. Source `.venv` virtual env : `source .venv/bin/activate`
4. Run the backend, make sure you are running redis and supabase
API: 
```
LOG_LEVEL=debug uvicorn quivr_api.main:app --log-level debug --reload --host 0.0.0.0 --port 5050 --workers 1
```
Worker: 
```
LOG_LEVEL=debug celery -A quivr_worker.celery_worker worker -l info -E --concurrency 1
```
Notifier: 
```
LOG_LEVEL=debug python worker/quivr_worker/celery_monitor.py
```

---------

Co-authored-by: chloedia <chloedaems0@gmail.com>
Co-authored-by: aminediro <aminedirhoussi1@gmail.com>
Co-authored-by: Antoine Dewez <44063631+Zewed@users.noreply.github.com>
Co-authored-by: Chloé Daems <73901882+chloedia@users.noreply.github.com>
Co-authored-by: Zewed <dewez.antoine2@gmail.com>
2024-09-02 10:20:53 +02:00
Stan Girard
78355eda6a
feat: Add NEXT_PUBLIC_PROJECT_NAME environment variable (#3070)
The `.env.example` and `Dockerfile` have been modified to include the
`NEXT_PUBLIC_PROJECT_NAME` environment variable. This variable is used
in the frontend code to display the project name dynamically.
2024-08-22 12:50:10 -07:00
Stan Girard
e2df8b37d4
feat(smtp&share): implemented smtp support and fixed share brain (#3049)
completes ENT-53 & fixes ENT-52
2024-08-22 02:20:21 -07:00
Antoine Dewez
d844f66e0d
feat(frontend): show or hide tokens relative stuff (#3017)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2024-08-19 10:22:56 -07:00
Stan Girard
1d08ebfff8
chore(env): add deactivate stripe env variable (#2986)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2024-08-08 15:40:44 -07:00
AmineDiro
a90a01ccf2
feat: quivr api send notification (#2865)
# Description

- Quivr notification service. listens to celery events and updates
knowledge status.
- Process file notification will wait for retried tasks
2024-07-15 06:37:42 -07:00
AmineDiro
ca93cb9062
refacto(backend): poetry package manager and chat route refactoring (#2684)
# Description
- Added package manager
- Added precommit checks
- Rewrote dependency injection of Services and Repositories
- Integrate async SQL alchemy engine
- Migrate Chat  repository to SQLModel 
- Migrated ChatHistory repository to SQLModel
- User SQLModel
- Unit test methodology with db rollback
- Unit tests ChatRepository
- Test ChatService get_history
- Brain entity SQL Model
- Promp SQLModel
- Rewrite chat/{chat_id}/question route
- updated docker files and docker compose in dev and production

Added `quivr_core` subpackages:
- Refactored KnowledgebrainQa
- Added Rag service to interface with non-rag dependencies

---------

Co-authored-by: aminediro <aminediro@github.com>
2024-06-26 00:58:55 -07:00
Stan Girard
d41a0b4be4
Feat/auth-playground (#2605)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2024-05-21 13:20:35 -07:00
Stan Girard
848aed46ea
Revert "feat: Add Google Drive & Sharepoint sync in backend" (#2603)
Reverts QuivrHQ/quivr#2592
2024-05-21 08:37:08 -07:00
Stan Girard
8303aca9d9
feat: Add Google Drive & Sharepoint sync in backend (#2592)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2024-05-21 07:53:04 -07:00
Stan Girard
a1b74d00f5 Update BRAVE_SEARCH_API_KEY in .env.example 2024-05-11 16:56:54 +02:00
Stan Girard
a4e2104cad
feat(brave-search): Update GPT4Brain tools and add WebSearchTool (#2576)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2024-05-10 03:22:15 -07:00
Stan Girard
466eee09e9
fix(front): Add NEXT_PUBLIC_AUTH_MODES to .env.example (#2570)
This pull request adds the `NEXT_PUBLIC_AUTH_MODES` variable to the
`.env.example` file. This variable allows the application to specify the
authentication modes that are supported.
2024-05-10 00:17:19 -07:00
Stan Girard
5717fdc1d5 Update .env.example and backend/models/settings.py 2024-04-29 10:36:10 +02:00
Stan Girard
f656dbcb42
feat(reranker): Add flashrank and contextual compression retriever (#2480)
This pull request adds the flashrank and contextual compression
retriever to the codebase. The flashrank reranker model is used for
compression, and the contextual compression retriever combines the base
compressor and base retriever to improve document retrieval.
2024-04-24 10:44:31 -07:00
Stan Girard
b62297341f
feat(assistants): Add PDF generation functionality and nice emails (#2436)
This pull request adds PDF generation functionality and improves the
formatting of emails. It includes a new module for generating PDFs using
the fpdf2 library and updates the email templates to use markdown
formatting for better readability.

<!--
ELLIPSIS_HIDDEN
-->
----

| <a href="https://ellipsis.dev" target="_blank"><img
src="https://avatars.githubusercontent.com/u/80834858?s=400&u=31e596315b0d8f7465b3ee670f25cea677299c96&v=4"
alt="Ellipsis" width="30px" height="30px"/></a> | 🚀 This PR
description was created by [Ellipsis](https://www.ellipsis.dev) for
commit ccecff77a5. |
|--------|--------|

### Summary:
This PR introduces PDF generation functionality, improves email
formatting, sanitizes filenames, updates various files and dependencies,
and updates `docker-compose.dev.yml`.

**Key points**:
- Added PDF generation functionality using the `fpdf2` library
- Improved email formatting
- Sanitized filenames using the `unidecode` library
- Updated `backend/modules/assistant/ito/ito.py` to generate PDFs and
sanitize filenames
- Added `pdf_generator.py` in `backend/modules/assistant/ito/utils/` for
PDF generation
- Updated `Pipfile` and `Pipfile.lock` in
`backend/modules/assistant/ito/utils/` to include `unidecode`
- Updated `requirements.txt` with new dependencies
- Updated `docker-compose.dev.yml`


----
Generated with ❤️ by [ellipsis.dev](https://www.ellipsis.dev)

<!--
ELLIPSIS_HIDDEN
-->
2024-04-17 06:00:31 -07:00
Adam John
d82e2cc478
Patch 1 (#2388)
# Description

Two small comment changes in the .env file. One to reposition the main
file comment to the top of the file and the other to add a comment to
the OpenAI config line per https://github.com/QuivrHQ/quivr/issues/2384

## Checklist before requesting a review

- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [na] I have commented hard-to-understand areas
2024-03-29 18:55:42 -07:00
Stan Girard
19c11944bc
feat: implement elasticache (#2234)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2024-02-20 19:52:14 -08:00
Stan Girard
597fb4711a
feat: 🎸 telemetry (#2169)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2024-02-07 20:44:37 -08:00
Stan Girard
b2d1d10212 feat: 🎸 supabase
migrated to supabase cli
2024-01-07 16:50:38 +01:00
Stan Girard
43502221da docs(.env): removed unused 2023-12-07 17:32:48 +01:00
Yon
5f23bcfcb8
fix: celery config typo (#1776)
# Description

There was a typo on one of the celery's config variables.
The `CELEBRY_BROKER_QUEUE_NAME` variable has been replaced by
`CELERY_BROKER_QUEUE_NAME`

## Checklist before requesting a review

Please delete options that are not relevant.

- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
2023-12-05 09:40:40 +01:00
Stan Girard
8382dde575 feat: 🎸 local
all local in 60 seconds
2023-12-02 15:43:05 +01:00
Stan Girard
9e0bc0749f
feat: 🎸 install (#1784)
now made easy

# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2023-12-02 14:50:57 +01:00
Stan Girard
87138635aa
feat(install): it now takes 30 seconds to install Quivr (#1780)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2023-12-02 00:26:06 +01:00