Commit Graph

721 Commits

Author SHA1 Message Date
AmineDiro
89d9e2fcaf
fix: remove knowledge and idle conn (#3165)
# Description

- Add set session idle conn
- Remove brain vectore in knowledge routees ( knowledge cascades to
vectors now), should speed up th
- Add session rollback/commit to worker

TODO: Should refactor session in worker . The design async+ celery is
starting to be really bad
2024-09-06 06:56:07 -07:00
Stan Girard
b3a78a2fd2
fix: remove unused 'models' field in check_premium.py (#3164)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2024-09-06 02:30:01 -07:00
AmineDiro
f6fb868d23
fix: tests pytest-asyncio (#3157)
# Description


FIX TESTS !
2024-09-05 07:39:27 -07:00
Stan Girard
f11b98ffc6 fix: Fix invalid user check in oauth2callback_azure
The commit fixes an issue in the `oauth2callback_azure` function where the check for a valid user was incorrect. It now correctly compares the `user_id` from the `sync_user_state` object with the `current_user` value.
2024-09-05 11:18:05 +02:00
AmineDiro
9a4ee1506b
fix: url knowledge multiple brain (#3145)
# Description

- Find knowledge path in storage
2024-09-05 01:15:51 -07:00
Stan Girard
7209500d0b
fix(chat): order of chat history was reversed (#3148)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2024-09-04 07:29:08 -07:00
Stan Girard
8e29218865
feat(anthropic): add llm (#3146)
This commit adds the langchain_openai and langchain_anthropic
dependencies to the `llm_endpoint.py` file.

# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2024-09-04 06:24:21 -07:00
AmineDiro
d959927d33
fix: knowledge multiple brains - update knowledge rollback (#3143)
# Description

- rollback update
2024-09-03 18:40:21 +02:00
AmineDiro
43fe0d1539
fix: knowledge multiple brains (#3142)
# Description

- Same knowledge in multiple brains
- Delete knowledge works but removing the brain/knowledge association if
knowledge in multiple brains or remove knowledge if only one brain
exists
- Fix bugs in knowledge service

**NOTE:** 
- There is a risk of race condition but this shouldn't occur that often
- Sync updates could have side effects on sync knowledge that are
updated and are in multiple brains ...

---------

Co-authored-by: Stan Girard <girard.stanislas@gmail.com>
2024-09-03 17:47:57 +02:00
Jacopo Chevallard
8cfdf53fe7
feat: using langgraph in our RAG pipeline (#3130)
# Description

Using LangGraph instead of LangChain LCEL to build and run the RAG
pipeline, as LangGraph enables greater flexibility and an easier
maintainability of complex (agentic) pipelines

Completes CORE-175

## Checklist before requesting a review

Please delete options that are not relevant.

- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [x] New and existing unit tests pass locally with my changes
- [x] Any dependent changes have been merged

## Screenshots (if appropriate):

---------

Co-authored-by: Stan Girard <girard.stanislas@gmail.com>
2024-09-03 06:23:23 -07:00
AmineDiro
b59fc4c0c9
fix: knowledge user (#3124)
# Description

- Add `user_id` foreign to knowledge table
- Updates knowledges service 
- Updates sha1 contraint to be on (`user_id`,`file_sha1`) on knowledge
- Fix event_loop fixture
- Adds knowledge tests

---------

Co-authored-by: chloedia <chloedaems0@gmail.com>
Co-authored-by: Stan Girard <stan@quivr.app>
Co-authored-by: aminediro <aminedirhoussi@gmail.com>
Co-authored-by: Antoine Dewez <44063631+Zewed@users.noreply.github.com>
Co-authored-by: Stan Girard <girard.stanislas@gmail.com>
Co-authored-by: Chloé Daems <73901882+chloedia@users.noreply.github.com>
Co-authored-by: porter-deployment-app[bot] <87230664+porter-deployment-app[bot]@users.noreply.github.com>
Co-authored-by: Zewed <dewez.antoine2@gmail.com>
2024-09-02 15:07:30 +02:00
Stan Girard
380cf82706
feat: quivr core 0.1 (#2970)
# Description


# Testing backend 

## Docker setup
1. Copy `.env.example` to `.env`. Some env variables were added :
EMBEDDING_DIM
2. Apply supabase migratrions : 
```sh
supabase stop
supabase db reset
supabase start
```
3. Start backend containers
```
make dev
```
## Local setup 
You can also run backend without docker.
1. Install [`rye`](https://rye.astral.sh/guide/installation/). Choose
the managed python version and set the version to 3.11
2. Run the following: 
```
cd quivr/backend
rye sync
```
3. Source `.venv` virtual env : `source .venv/bin/activate`
4. Run the backend, make sure you are running redis and supabase
API: 
```
LOG_LEVEL=debug uvicorn quivr_api.main:app --log-level debug --reload --host 0.0.0.0 --port 5050 --workers 1
```
Worker: 
```
LOG_LEVEL=debug celery -A quivr_worker.celery_worker worker -l info -E --concurrency 1
```
Notifier: 
```
LOG_LEVEL=debug python worker/quivr_worker/celery_monitor.py
```

---------

Co-authored-by: chloedia <chloedaems0@gmail.com>
Co-authored-by: aminediro <aminedirhoussi1@gmail.com>
Co-authored-by: Antoine Dewez <44063631+Zewed@users.noreply.github.com>
Co-authored-by: Chloé Daems <73901882+chloedia@users.noreply.github.com>
Co-authored-by: Zewed <dewez.antoine2@gmail.com>
2024-09-02 10:20:53 +02:00
Stan Girard
d619b0acce
feat: Update send_email.py to conditionally login with SMTP credentials (#3069)
The `send_email` function in `send_email.py` has been updated to
conditionally login with SMTP credentials. This change allows for more
flexibility in handling SMTP authentication.
2024-08-22 11:01:30 -07:00
Stan Girard
bfd309efd8
feat(smtp): not enforcing tls (#3067)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2024-08-22 10:40:51 -07:00
Stan Girard
e2df8b37d4
feat(smtp&share): implemented smtp support and fixed share brain (#3049)
completes ENT-53 & fixes ENT-52
2024-08-22 02:20:21 -07:00
Antoine Dewez
6b07a63e4e
feat(frontend): brain snippet selector (#3038)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):

---------

Co-authored-by: Stan Girard <stan@quivr.app>
2024-08-21 17:23:59 +02:00
Stan Girard
2a79be8e42
fix: Update Supabase configuration (#3014)
Fix CORE-152
2024-08-16 02:07:11 -07:00
Stan Girard
b5f31a83d4
feat(azure): quivr compatible with it (#3005)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2024-08-14 06:00:19 -07:00
AmineDiro
a388cf8896
fix: commit on try sqlalchemy (#3004)
- Fixed bug of try/except rollback- > Sqlalchemy catching exceptions
without raising
- Commit on each success
- Idle transaction in commit state but are resolved by db
2024-08-14 02:01:34 -07:00
AmineDiro
91ada8c3ed
fix: autocommit isolation level (#2999)
# Description

- Reduce pool size
- Sets isolation level to autocommit. Sqlalchemy keeps transactions
running after

Co-authored-by: Stan Girard <girard.stanislas@gmail.com>
2024-08-13 03:05:01 -07:00
Stan Girard
8e7263ca59 feat: supabase 2024-08-09 09:47:56 +02:00
Stan Girard
4ef5f30aa9
feat(models): all models by default (#2983)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2024-08-08 15:36:31 -07:00
Antoine Dewez
ef6037e665
feat(frontend): talk with models and handle code markdown (#2980)
# Description

complete ENT-35

---------

Co-authored-by: Stan Girard <girard.stanislas@gmail.com>
2024-08-08 07:21:28 -07:00
Stan Girard
111200184b
feat: Add brain_id and brain_name to ChatLLMMetadata model (#2968)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2024-08-07 11:47:33 -07:00
Stan Girard
13e9fc490b
feat: Add get_model method to ModelRepository (#2949)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2024-08-06 08:44:12 -07:00
Stan Girard
fccd197511
feat: add chat with models (#2933)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):

---------

Co-authored-by: AmineDiro <aminedirhoussi1@gmail.com>
2024-08-06 05:51:27 -07:00
Chloé Daems
9934a7a8ce
feat: Add GitHub sync functionality to sync router (#2871)
The code changes in `sync_routes.py` add the GitHub sync functionality
to the sync router. This allows users to sync their GitHub repositories
with Quivr.

# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):

---------

Co-authored-by: Stan Girard <girard.stanislas@gmail.com>
2024-08-05 05:17:53 -07:00
Stan Girard
b3debeefee
chore(main): release core 0.0.13 (#2930)
🤖 I have created a release *beep* *boop*
---


##
[0.0.13](https://github.com/QuivrHQ/quivr/compare/core-0.0.12...core-0.0.13)
(2024-08-01)


### Features

* quivr core tox test + parsers
([#2929](https://github.com/QuivrHQ/quivr/issues/2929))
([6855585](685558560c))


### Bug Fixes

* processor quivr version
([#2934](https://github.com/QuivrHQ/quivr/issues/2934))
([2d64962](2d64962ca4))
* quivr core fix tests
([#2935](https://github.com/QuivrHQ/quivr/issues/2935))
([d9c1f3a](d9c1f3add4))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
2024-08-02 15:24:21 +02:00
AmineDiro
d9c1f3add4
fix: quivr core fix tests (#2935)
# Description
- Update CI
- Fix DockerFile for quivr-core tests
- Tox parallel tests

@StanGirard  @chloedia 
- For running quivr-core tests:
```bash
cd backend/core
./scripts/run_tests.sh
```

---------

Co-authored-by: aminediro <aminedirhoussi@gmail.com>
Co-authored-by: Stan Girard <girard.stanislas@gmail.com>
2024-08-01 06:54:43 -07:00
AmineDiro
2d64962ca4
fix: processor quivr version (#2934) 2024-07-31 21:57:28 +02:00
AmineDiro
685558560c
feat: quivr core tox test + parsers (#2929) 2024-07-30 18:49:12 +02:00
Chloé Daems
206e25ffd2
Feat/handle_azure_sites (#2921)
# Description

Fix get all sites
2024-07-29 06:49:04 -07:00
Stan Girard
11e443411d
chore: update sync notification status to INFO (#2918)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2024-07-29 05:47:53 -07:00
Chloé Daems
8d0e07601b
feat(integrations): Add Azure Drive Sites support (#2919)
This PR adds Sites of azure drives navigation possibility
2024-07-29 05:33:12 -07:00
Stan Girard
43c47d83d4
feat(notifications): improved requirements (#2915)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2024-07-26 07:43:32 -07:00
Antoine Dewez
4dfb3049c8
fix(backend): fix error messages (#2917)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2024-07-26 16:38:01 +02:00
Chloé Daems
e5a1319ca4
fix: error dict google drive (#2912) 2024-07-25 07:24:16 -07:00
Stan Girard
ce40cc9e70
chore(main): release core 0.0.12 (#2889)
🤖 I have created a release *beep* *boop*
---


##
[0.0.12](https://github.com/QuivrHQ/quivr/compare/core-0.0.11...core-0.0.12)
(2024-07-23)


### Features

* **dead-code:** removed composite & api
([#2902](https://github.com/QuivrHQ/quivr/issues/2902))
([a2721d3](a2721d3926))
* **frontend:** add knowledge icon when integration
([#2888](https://github.com/QuivrHQ/quivr/issues/2888))
([733d083](733d083e33))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
2024-07-23 15:21:35 -07:00
Chloé Daems
2093fb9407
fix(sync): folder upload to Google Drive (#2909)
# Description

small fix to upload folder with google drive
2024-07-23 14:40:40 -07:00
Stan Girard
5a7fc21163
feat: Update AzureDriveSync name to "Share Point" (#2907)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2024-07-23 12:10:06 -07:00
Stan Girard
a2721d3926
feat(dead-code): removed composite & api (#2902)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2024-07-22 14:12:54 -07:00
Stan Girard
cf5c56d74c
feat: Remove unused onboarding code (#2901)
Remove unused code related to onboarding functionality. This includes
removing the `OnboardingUpdatableProperties` class and the
`OnboardingStates` class. These classes are no longer used in the
codebase and can be safely deleted.

# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2024-07-22 10:54:41 -07:00
Stan Girard
d9e7ba8657
feat(premium): add trialing (#2900)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2024-07-22 09:15:13 -07:00
Stan Girard
10bcec862f
feat(api): add logging for inactive subscriptions (#2897)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2024-07-22 08:24:06 -07:00
Stan Girard
cf5ad25896
feat(azure): changed auth method (#2890)
Auth method changed to web
2024-07-22 06:32:30 -07:00
Stan Girard
ade4802c34
feat: add integration and integration link to Sources class (#2893)
The Sources class in `chats.py` now includes optional fields for
integration and integration link. This allows for better tracking and
integration of external sources in the chat module.
2024-07-22 06:09:57 -07:00
Chloé Daems
7eba571306
fix: google conversion (#2891)
# Description

Fix google file conversion problem
2024-07-22 05:02:59 -07:00
Antoine Dewez
733d083e33
feat(frontend): add knowledge icon when integration (#2888)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2024-07-22 12:21:39 +02:00
Stan Girard
09d8d50422
chore(main): release core 0.0.11 (#2886)
🤖 I have created a release *beep* *boop*
---


##
[0.0.11](https://github.com/QuivrHQ/quivr/compare/core-0.0.10...core-0.0.11)
(2024-07-22)


### Features

* move parsers quivr core
([#2884](https://github.com/QuivrHQ/quivr/issues/2884))
([d3c53e6](d3c53e6353))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
2024-07-22 10:13:42 +02:00
AmineDiro
d3c53e6353
feat: move parsers quivr core (#2884)
# Description

- Moved `quivr-api` parser to `quivr_core.processor.implementations` by
Dynamically creating classes on the fly that inherit from
`ProcessorBase`
- Defined a priority based based system to automagically register the
"important" processor that we can import at runtime
- Wrote extensive tests for the registry 
- Added support file extensions

### Next steps
- Find a way to have correct LSP autocomplete on the dynamically
generated processors
- Test that processor are imported correctly based on the installed
packages in environment ( using tox) ?
2024-07-22 01:09:02 -07:00