Commit Graph

401 Commits

Author SHA1 Message Date
Mamadou DICKO
512b9b4f37
feat: add custom rag first abstraction layer (#1858)
- Add `QAInterface` which should be implemented by all custom answer
generator to be compatible with Quivr
- Add `RAGInterface` which should be implemented by all RAG classes
2023-12-11 16:46:45 +01:00
Mamadou DICKO
7e6209a30b
feat: add generate_answer function to support non streamed response for api brain (#1847)
Issue: https://github.com/StanGirard/quivr/issues/1841

Demo:


https://github.com/StanGirard/quivr/assets/63923024/56dfa85c-6c2e-4e55-8eda-4723e58ced1d
2023-12-07 14:52:37 +01:00
Zineb El Bachiri
90c4a44525
feat: composite brains get (#1837)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2023-12-06 15:44:36 +01:00
Stan Girard
5619f2e602 feat(sentry): added on celery 2023-12-06 12:20:46 +01:00
Zineb El Bachiri
5bbc72eb09
feat: composite brains db (#1826)
# Description

- db for composite brains & repository
- composite brains creation
2023-12-06 08:40:18 +01:00
Stan Girard
8ba376e0a1
feat(requirements): update aws version (#1819)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2023-12-05 13:49:34 +01:00
Yon
5f23bcfcb8
fix: celery config typo (#1776)
# Description

There was a typo on one of the celery's config variables.
The `CELEBRY_BROKER_QUEUE_NAME` variable has been replaced by
`CELERY_BROKER_QUEUE_NAME`

## Checklist before requesting a review

Please delete options that are not relevant.

- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
2023-12-05 09:40:40 +01:00
Stan Girard
b8f8851ccd feat(docker): multi platform build and prebuilt 2023-12-05 00:10:49 +01:00
Stan Girard
d0d74cecbb chore(ci): trigger it 2023-12-05 00:02:48 +01:00
Zineb El Bachiri
436e49a5e7
refactor: chat for multibrains (#1812)
# Description

- Chat Module
- External Api Secrets Interface, exposed through brain service
2023-12-04 18:38:54 +01:00
Stan Girard
ef53590b58
feat(embedding): now 100 times faster 🔥 (#1807)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2023-12-04 11:59:08 +01:00
Stan Girard
31c1802084
feat(api-keys): added customization (#1802)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2023-12-03 23:32:37 +01:00
Stan Girard
4c89812832 feat(supabase): upgraded to latest version 2023-12-03 20:26:26 +01:00
Stan Girard
a5ef5a0454 fix(supabase): fixed 2023-12-03 01:35:01 +01:00
Stan Girard
882a10c459 fix(prompt): fixed issue with wrong api key 2023-12-03 01:22:38 +01:00
Stan Girard
c82a8cc8b1
fix(url): crawling fixed (#1785)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2023-12-02 18:08:22 +01:00
Stan Girard
9e0bc0749f
feat: 🎸 install (#1784)
now made easy

# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2023-12-02 14:50:57 +01:00
Stan Girard
72aaec2a1b
feat(supabase): local installation made easy (#1777)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2023-12-01 19:43:23 +01:00
Stan Girard
c0f8f50c49 feat(telemetry): implmented local telemetry for local runs 2023-12-01 17:02:46 +01:00
Zineb El Bachiri
f48dab4a7d
refactor: to modules (#1754)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2023-11-30 22:29:28 +01:00
Stan Girard
a84219f193
fix(api): fixed issue with name function and ilmproved promtp (#1759)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2023-11-29 23:54:39 +01:00
Stan Girard
e1cde0fcb4
Feat/local llm bug fix (#1758)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2023-11-29 19:17:16 +01:00
Mamadou DICKO
07618877a9
fix: revert implement local llms (#1749) 2023-11-29 13:19:22 +01:00
Mamadou DICKO
d5b45cbad1
chore: downgrade versions (#1748) 2023-11-29 13:18:52 +01:00
Zineb El Bachiri
9766befb53
refactor: knowledge module (#1743)
# Description

- Refactor knowledge to a module

- This PR breaks the Github Processor function -> need to comment brain
and file imports as it creates a circular dependency issue. Should be
fixed and reverted in next PR.
2023-11-29 09:04:03 +01:00
Stan Girard
7e4f711f07
feat: implement local llms (#1745)
upgraded

# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2023-11-29 00:07:04 +01:00
Zineb El Bachiri
9858a484a1
refactor: remove explore route from back & front (#1741)
# Description

- Removes the explore route from back & front as it has been replaced by
the knowledge route
2023-11-28 14:28:05 +01:00
Zineb El Bachiri
0782df5e37
refactor: Notification module (#1740)
# Description

- Notification module

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2023-11-28 14:27:39 +01:00
Stan Girard
928e6b6ab2 chore(sentry): sampling 2023-11-28 08:46:42 +01:00
Stan Girard
7ff081cf40
fix: 🐛 crawler (#1735)
fixed

# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2023-11-27 17:36:46 +01:00
Mamadou DICKO
10e94e5a91
feat: improve delete knowledge performance (#1733)
Issue: https://github.com/StanGirard/quivr/issues/1724
2023-11-27 16:47:13 +01:00
Mamadou DICKO
f1ddaca7e2
feat(publicBrains): use join queries for better performance (#1730)
Issue https://github.com/StanGirard/quivr/issues/1720
2023-11-27 14:16:27 +01:00
Mamadou DICKO
b252aa1794
fix: update max token logic (#1725)
Fix headless and Doc based brains max token overwriting
2023-11-27 11:21:26 +01:00
Stan Girard
56a5c81fe4
fix: 🐛 sentry (#1716)
reduce sampling

# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2023-11-27 10:08:00 +01:00
Stan Girard
e3a6f86d7b fix: 🐛 upload
wrong ctrl f replace
2023-11-24 23:12:49 +01:00
Stan Girard
3ffe6a8d5a chore: 🤖 sentry
reduce sampling
2023-11-24 23:12:18 +01:00
Mamadou DICKO
65e656f04e
feat: add api brain steps log (backend) (#1705)
Issue https://github.com/StanGirard/quivr/issues/1697
2023-11-24 14:58:33 +01:00
Mamadou DICKO
65a4057272
feat: display notification when file size is too big (#1704)
Issue: 

https://github.com/StanGirard/quivr/issues/1703

Demo:


https://github.com/StanGirard/quivr/assets/63923024/76c9a294-ff0b-4d38-8683-2fefb710fd48
2023-11-24 14:54:16 +01:00
Stan Girard
a5518e6b2a chore(sentry): smaller sampling rate 2023-11-24 14:46:25 +01:00
Zineb El Bachiri
f578b2fa00
refactor: onboarding module (#1702)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2023-11-24 10:25:02 +01:00
Mamadou DICKO
4bb1a0dc8a
fix: update max token overwrite logic (#1694)
Issue: https://github.com/StanGirard/quivr/issues/1690
2023-11-23 17:36:11 +01:00
Pascal Gula
d893ec7d97
Fixes string formatting when logging knowledge table (#1691)
# Description

```
======================================================================
backend-core  | --- Logging error ---
backend-core  | Traceback (most recent call last):
backend-core  |   File "/usr/local/lib/python3.11/logging/__init__.py", line 1110, in emit
backend-core  |     msg = self.format(record)
backend-core  |           ^^^^^^^^^^^^^^^^^^^
backend-core  |   File "/usr/local/lib/python3.11/logging/__init__.py", line 953, in format
backend-core  |     return fmt.format(record)
backend-core  |            ^^^^^^^^^^^^^^^^^^
backend-core  |   File "/usr/local/lib/python3.11/logging/__init__.py", line 687, in format
backend-core  |     record.message = record.getMessage()
backend-core  |                      ^^^^^^^^^^^^^^^^^^^
backend-core  |   File "/usr/local/lib/python3.11/logging/__init__.py", line 377, in getMessage
backend-core  |     msg = msg % self.args
backend-core  |           ~~~~^~~~~~~~~~~
backend-core  | TypeError: not all arguments converted during string formatting
backend-core  | Call stack:
backend-core  |   File "<string>", line 1, in <module>
backend-core  |   File "/usr/local/lib/python3.11/multiprocessing/spawn.py", line 122, in spawn_main
backend-core  |     exitcode = _main(fd, parent_sentinel)
backend-core  |   File "/usr/local/lib/python3.11/multiprocessing/spawn.py", line 135, in _main
backend-core  |     return self._bootstrap(parent_sentinel)
backend-core  |   File "/usr/local/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap
backend-core  |     self.run()
backend-core  |   File "/usr/local/lib/python3.11/multiprocessing/process.py", line 108, in run
backend-core  |     self._target(*self._args, **self._kwargs)
backend-core  |   File "/usr/local/lib/python3.11/site-packages/uvicorn/_subprocess.py", line 76, in subprocess_started
backend-core  |     target(sockets=sockets)
backend-core  |   File "/usr/local/lib/python3.11/site-packages/uvicorn/server.py", line 61, in run
backend-core  |     return asyncio.run(self.serve(sockets=sockets))
backend-core  |   File "/usr/local/lib/python3.11/asyncio/runners.py", line 190, in run
backend-core  |     return runner.run(main)
backend-core  |   File "/usr/local/lib/python3.11/asyncio/runners.py", line 118, in run
backend-core  |     return self._loop.run_until_complete(task)
backend-core  |   File "/usr/local/lib/python3.11/asyncio/base_events.py", line 640, in run_until_complete
backend-core  |     self.run_forever()
backend-core  |   File "/usr/local/lib/python3.11/asyncio/base_events.py", line 607, in run_forever
backend-core  |     self._run_once()
backend-core  |   File "/usr/local/lib/python3.11/asyncio/base_events.py", line 1922, in _run_once
backend-core  |     handle._run()
backend-core  |   File "/usr/local/lib/python3.11/asyncio/events.py", line 80, in _run
backend-core  |     self._context.run(self._callback, *self._args)
backend-core  |   File "/usr/local/lib/python3.11/site-packages/uvicorn/protocols/http/h11_impl.py", line 428, in run_asgi
backend-core  |     result = await app(  # type: ignore[func-returns-value]
backend-core  |   File "/usr/local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__
backend-core  |     return await self.app(scope, receive, send)
backend-core  |   File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 276, in __call__
backend-core  |     await super().__call__(scope, receive, send)
backend-core  |   File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 122, in __call__
backend-core  |     await self.middleware_stack(scope, receive, send)
backend-core  |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 162, in __call__
backend-core  |     await self.app(scope, receive, _send)
backend-core  |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 91, in __call__
backend-core  |     await self.simple_response(scope, receive, send, request_headers=headers)
backend-core  |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 146, in simple_response
backend-core  |     await self.app(scope, receive, send)
backend-core  |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
backend-core  |     await self.app(scope, receive, sender)
backend-core  |   File "/usr/local/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
backend-core  |     await self.app(scope, receive, send)
backend-core  |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 718, in __call__
backend-core  |     await route.handle(scope, receive, send)
backend-core  |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 276, in handle
backend-core  |     await self.app(scope, receive, send)
backend-core  |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 66, in app
backend-core  |     response = await func(request)
backend-core  |   File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 237, in app
backend-core  |     raw_response = await run_endpoint_function(
backend-core  |   File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 163, in run_endpoint_function
backend-core  |     return await dependant.call(**values)
backend-core  |   File "/code/routes/knowledge_routes.py", line 37, in list_knowledge_in_brain_endpoint
backend-core  |     logger.info("List of knowledge from knowledge table", knowledges)
backend-core  | Message: 'List of knowledge from knowledge table'
backend-core  | Arguments: ([Knowledge(id=UUID('b5ae6a25-1e8e-4fb5-a0c1-0039a8318ec3'), brain_id=UUID('7af64537-61cb-48e1-b3d1-c700d748b373'), file_name='Cradle to Cradle Criteria for the built environmen.pdf', url=None, extension='.pdf'), Knowledge(id=UUID('efb7ad5a-a13e-46f3-a580-4c574710ad15'), brain_id=UUID('7af64537-61cb-48e1-b3d1-c700d748b373'), file_name='GEG_2023_gebaeudeenergiegesetz_kompakt_und_praktisch.pdf', url=None, extension='.pdf'), Knowledge(id=UUID('9442f898-0fbe-46d2-b6ae-a726d62717bd'), brain_id=UUID('7af64537-61cb-48e1-b3d1-c700d748b373'), file_name='dgnb-kriterienkatalog-gebaeude-neubau-version-2023-auflage-2.pdf', url=None, extension='.pdf')],)
======================================================================
```

## Checklist before requesting a review

Please delete options that are not relevant.

- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented hard-to-understand areas
- [x] I have ideally added tests that prove my fix is effective or that
my feature works
- [x] New and existing unit tests pass locally with my changes
- [x] Any dependent changes have been merged
2023-11-23 17:25:57 +01:00
Zineb El Bachiri
1bf67e3640
refactor: Prompt module (#1688)
# Description

Prompt module with Service

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2023-11-23 14:13:21 +01:00
Stan Girard
85913c5db0 feat: 🎸 brainasapi
fixed size
2023-11-22 22:14:53 +01:00
Pascal Gula
8bbe6e7054
Adds pytesseract, tesseract and poopler-utils (#1648)
To enable the ingestion of copy protected PDF via OCR instead of text
extraction

# Description

Copy protected PDF can't be properly imported via the standard langchain
loader.

See the following errors:

```
2023-11-15 14:16:31,927 [INFO] models.files: Computing documents from file Cradle to Cradle Criteria for the built environmen.pdf
[nltk_data] Downloading package punkt to
[nltk_data]     /home/pascal_gula_luccid_ai/nltk_data...
[nltk_data]   Unzipping tokenizers/punkt.zip.
[nltk_data] Downloading package averaged_perceptron_tagger to
[nltk_data]     /home/pascal_gula_luccid_ai/nltk_data...
[nltk_data]   Unzipping taggers/averaged_perceptron_tagger.zip.
Error processing file: detectron2 is not installed, pytesseract is not installed and the text of the PDF is not extractable. To process this file, install detectron2, install pytesseract, or remove copy protection from the PDF.
```

```
2023-11-15 15:04:14,624 [INFO] models.files: Computing documents from file Cradle to Cradle Criteria for the built environmen.pdf
Error processing file: Unable to get page count. Is poppler installed and in PATH?
```

```
023-11-15 15:59:11,886 [INFO] models.files: Computing documents from file Cradle to Cradle Criteria for the built environmen.pdf
Error processing file: tesseract is not installed or it's not in your PATH. See README file for more information.
```


## Checklist before requesting a review

Please delete options that are not relevant.

- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented hard-to-understand areas
- [x] I have ideally added tests that prove my fix is effective or that
my feature works
- [x] New and existing unit tests pass locally with my changes
- [x] Any dependent changes have been merged

## Screenshots (if appropriate):

None

Co-authored-by: Stan Girard <girard.stanislas@gmail.com>
2023-11-22 17:26:11 +01:00
Mamadou DICKO
d9a72b368a
feat: allow updating api brain definition (#1682)
Issue: 
https://github.com/StanGirard/quivr/issues/1683


Demo:


https://github.com/StanGirard/quivr/assets/63923024/67cf2a0f-4cfe-420c-9181-baffaa584c78




https://github.com/StanGirard/quivr/assets/63923024/eee57598-1520-4c11-bd64-887869878f46
2023-11-22 11:15:14 +01:00
Stan Girard
1a4c6c8741
feat: 🎸 tokens (#1678)
increased default value to 1000

# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2023-11-22 08:47:51 +01:00
Stan Girard
536a7ae37c
fix: 🐛 api (#1676)
truncating long response

# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2023-11-21 15:34:12 +01:00
Mamadou DICKO
d902e78ffa
feat(apiBrain): improve ux (#1672)
Demo:


https://github.com/StanGirard/quivr/assets/63923024/71efe888-285a-41ed-aa89-44e5f0b99162
2023-11-20 18:22:06 +01:00
Mamadou DICKO
3da64c0b0e
feat: update brain modal in chat input (#1668)
Issue: https://github.com/StanGirard/quivr/issues/1571

Demo:



https://github.com/StanGirard/quivr/assets/63923024/e7a6118e-29f8-44aa-af4b-0ea1b8013a33
2023-11-20 18:04:26 +01:00
Stan Girard
6a041b6f6d
feat: 🎸 openai (#1658)
cleaning old code to introduce better patern

# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2023-11-20 01:22:03 +01:00
Stan Girard
d955e31f50
feat: 🎸 marketplace (#1657)
added explore button and removed unused feature openai key

# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2023-11-19 18:46:12 +01:00
Stan Girard
744eea6d43
feat: 🎸 docker reduced size by 2 (#1653)
reduced size by 2

# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2023-11-18 19:23:56 +01:00
Zineb El Bachiri
efe4e8ccb0
fix(brainManagement): fix shared brain access issue (#1641)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

fix/brain-management-shared-brain-access
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2023-11-16 19:57:02 +01:00
Zineb El Bachiri
18946ad329
fix: delete brain on users click in brains management (#1638)
# Description

Fix subscription routes on get users
Revert changes deleting the function to get user email from its id in
SQL


https://www.loom.com/share/46534965a83f4296898495ac370b3451?sid=7f582838-32f6-4812-960e-c80d71517a6d
2023-11-16 12:49:10 +01:00
Mamadou DICKO
fb4cfdcf9c
feat: add public api brain subscription (#1636)
Issue: https://github.com/StanGirard/quivr/issues/1570
2023-11-16 11:04:12 +01:00
Mamadou DICKO
1a5fa8ba0f
feat: update settings tab add api brain definition (#1635)
Issue: https://github.com/StanGirard/quivr/issues/1627

Demo:
<img width="1512" alt="Screenshot 2023-11-15 at 13 09 59"
src="https://github.com/StanGirard/quivr/assets/63923024/532c45ce-32d7-4af7-a71f-02eb6099538d">
2023-11-16 11:02:58 +01:00
Zineb El Bachiri
375f50356c
refactor: add modules folder (#1633)
# Description

New Modules folder with "user" module:
- controller: contains the current route
- entity: contains the current Models (TO be renamed DTO)
- repository: contains the current repo
- service: methods used by other modules 

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2023-11-15 13:17:51 +01:00
Mamadou DICKO
223af1d3f6
feat: add api brain creation frontend (#1631)
Issue: https://github.com/StanGirard/quivr/issues/1607

Demo:



https://github.com/StanGirard/quivr/assets/63923024/a087d856-e201-4b88-80b3-b001d7bb6b16
2023-11-15 12:42:19 +01:00
Zineb El Bachiri
1f21c6d7f3
refactor: packages folder be 2 (#1628)
# Description

Move some folders to packages
Remove unused code
2023-11-14 14:31:02 +01:00
Zineb El Bachiri
8f693bc92a
refactor: create "files" package (#1626)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2023-11-14 09:52:44 +01:00
Stan Girard
5a3f284785
test(all): added (#1624)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2023-11-14 08:36:44 +01:00
Mamadou DICKO
8ed0adf7b2
feat: remove api brain secrets and schemas on delete (#1621)
Issue: https://github.com/StanGirard/quivr/issues/1573
2023-11-13 19:08:47 +01:00
Stan Girard
ee864e6441
test(added): misc prompt onboarding (#1622)
Added tests for:

- Prompt
- Misc
- Onboarding
2023-11-13 19:02:10 +01:00
Stan Girard
6017fa2e9c test(contact): added contact route with mock 2023-11-13 17:58:11 +01:00
Stan Girard
ec60d3dbff ci(backend): added tests for user 2023-11-13 15:28:42 +01:00
Stan Girard
1e00f6929f fix: 🐛 docker
fixed multi stage
2023-11-13 15:03:55 +01:00
Mamadou DICKO
5e61445fb2
fix: update delete brain logic (#1619) 2023-11-13 15:02:10 +01:00
Stan Girard
6bc9dd1894
ci: 🎡 tests (#1615)
fixed

# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2023-11-13 13:53:25 +01:00
Mamadou DICKO
f24779ff14
fix: add user id while creating default brain (#1616) 2023-11-13 12:16:38 +01:00
Mohamed Messaad
71d4a63a17
feat(docker): use multi-stage Docker builds for smaller images (#1614)
# Description

Currently, the production Docker images are very large, sitting at 4.17
GB for the frontend image, and 3.49 GB for backend images. This change
adds multi-stage builds, to optimize the image sizes, which results in
the following improvements:

- frontend image size: 4.17 GB -> 1.64 GB
- backend image size: 3.49 GB -> 1.71 GB

I hope this is appropriate as there is no open issue for this that I
know of.
I implemented this change and tested it locally, and would be glad to
discuss this and open an issue if necessary.

## Checklist before requesting a review

Please delete options that are not relevant.

- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented hard-to-understand areas
- [x] New and existing unit tests pass locally with my changes

## Screenshots (if appropriate):
Image sizes before:
<img width="1416" alt="image"
src="https://github.com/StanGirard/quivr/assets/8296549/fcbb020f-8165-4549-ae30-823318691ec6">

Image sizes after:
<img width="1416" alt="image"
src="https://github.com/StanGirard/quivr/assets/8296549/d3f43c78-be26-4c38-9d23-9c1b0e9e37f2">
2023-11-13 10:13:56 +01:00
Mamadou DICKO
db5a6e4b9b
feat: allow users to chat with apis (#1612)
You can now create a brain which can fetch data from external APIs with
or without authentification

- POST query example with authentification 


https://github.com/StanGirard/quivr/assets/63923024/15013ba9-dedb-4f24-9e06-49daad9de7f3


- Get query example with authentification and search params



https://github.com/StanGirard/quivr/assets/63923024/1763875d-a8e9-4478-b07c-e99ca7337942


- Get query without authentification and search params



https://github.com/StanGirard/quivr/assets/63923024/f2742963-790d-4cb2-864a-8173979b650a
2023-11-09 16:58:51 +01:00
Mamadou DICKO
addcd27fce
feat: add APIBrainQA (#1606)
Issue: https://github.com/StanGirard/quivr/issues/1566
2023-11-08 16:07:21 +01:00
Zineb El Bachiri
ed5de8b80c
feat: supabase vault (#1605)
# Description

https://github.com/StanGirard/quivr/issues/1551

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2023-11-07 16:57:25 +01:00
Mamadou DICKO
dd2bc1f6fc
feat: add brain_type column to brain table (#1603)
Issue: https://github.com/StanGirard/quivr/pull/1601

Demo:


https://github.com/StanGirard/quivr/assets/63923024/a7fe76a7-8809-4ec7-b1e6-d6fe36928a5c
2023-11-07 16:23:52 +01:00
Mamadou DICKO
9382762d01
feat: add api_brain_definition table (#1601)
Issue: https://github.com/StanGirard/quivr/issues/1549
2023-11-07 14:03:50 +01:00
Stan Girard
1d3554cb92
feat: 🎸 source documents (#1598)
are kept only once

# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2023-11-06 17:19:26 +01:00
Mamadou DICKO
3c0819de09
fix: allow users to delete brains (#1596)
Issue: https://github.com/StanGirard/quivr/issues/1362

Demo:


https://github.com/StanGirard/quivr/assets/63923024/aa5e41f8-a738-41f2-b037-975788ad8611
2023-11-06 16:58:03 +01:00
Stan Girard
192610d008
feat: 🎸 sources (#1591)
added sources in front and backend

<img width="1536" alt="image"
src="https://github.com/StanGirard/quivr/assets/19614572/eb997288-282d-4f6e-b489-08ab3db400c6">
2023-11-06 12:09:18 +01:00
Stan Girard
ab11b384f7 fix: 🐛 parsers
pdf and odt to unstructured
2023-11-05 22:43:46 +01:00
Stan Girard
35f5fe0958 feat: 🎸 telegram
fixed a few bugs
2023-11-01 22:33:47 +01:00
Stan Girard
5eed1a05cb
feat: 🎸 telegram (#1555)
added loaders and compatibility

# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2023-11-01 22:09:04 +01:00
Zineb El Bachiri
0c58a4bb0e
refactor: extract prompt from settings (#1546)
# Description

- Refactor useSettingsTab by extracting usePrompt and methods in utils
- Fix bug #1545 

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2023-11-01 16:45:22 +01:00
Stan Girard
b330370d8c
feat: 🎸 max-token (#1538)
added limit to 4k for gpt4

# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2023-11-01 08:52:49 +01:00
Zineb El Bachiri
e3a99d1ace
refactor: chat_routes (#1512)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2023-10-30 10:18:23 +01:00
Zineb El Bachiri
82bcf38b16
refactor: fix bad smells (#1399)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2023-10-30 10:18:04 +01:00
Mamadou DICKO
c47548d3cd
feat: setup premium feature backend (#1467)
Issue: https://github.com/StanGirard/quivr/issues/1468
2023-10-23 18:19:04 +02:00
Matthieu Jacq
cee050a2e4
feat: POST on /contact to contact quivr team by email (#1466)
# Description

Issue: #1445
Epic: #1232

## ⚠️ À rajouter en variables d'environnement

```env
RESEND_CONTACT_SALES_FROM=contact_sales@resend.dev
RESEND_CONTACT_SALES_TO=<change-me>
```
2023-10-23 17:58:45 +02:00
Stan Girard
ca1ef8ccbd
refactor(backend): cleaning dead and unused code (#1432)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2023-10-19 15:52:20 +02:00
Mamadou DICKO
a9d2145fe6
feat: add a cron to remove onboarding more than 7 days (#1397)
https://github.com/StanGirard/quivr/issues/1385
2023-10-16 16:11:34 +02:00
Mamadou DICKO
ea227df30e
feat(onboarding): add suggested questions answer (#1390)
https://github.com/StanGirard/quivr/assets/63923024/08554dd5-2765-436a-b900-48b38d582f6c



https://github.com/StanGirard/quivr/issues/1389
2023-10-12 09:39:56 +02:00
HamzaKhalid
024dd5a264
fix(user identity): User identity dict has no attribute user_id and open_api_key (#1351)
# Description
Trying to access access 'user identity' attributes as attribute of
dictionary which throws an error since the attributes are actually the
key in the dictionary not an attribute of the dictionary object.

## Checklist before requesting a review

Please delete options that are not relevant.

- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code

## Screenshots (if appropriate):
<img width="795" alt="image"
src="https://github.com/StanGirard/quivr/assets/88370405/7db55e74-159c-43e1-b9f7-8384dad9be30">

Co-authored-by: HamzaKhalid-bboxx <127825841+HamzaKhalid-bboxx@users.noreply.github.com>
Co-authored-by: Zineb El Bachiri <100568984+gozineb@users.noreply.github.com>
2023-10-11 16:06:37 +02:00
Mamadou DICKO
b9172b7442
feat: track onboarding events (#1388)
https://github.com/StanGirard/quivr/issues/1386
2023-10-11 15:56:28 +02:00
Mamadou DICKO
1baf9ee5b0
feat: remove user onboarding on complete (#1387)
https://github.com/StanGirard/quivr/issues/1384


https://github.com/StanGirard/quivr/assets/63923024/ac05456d-2817-4196-8afc-51b34a4d8c5f
2023-10-11 09:46:57 +02:00
Matthieu Jacq
fa92243a18
feat: ⚙️🐞 configure debugger for the backend (#1345) 2023-10-09 15:23:13 +02:00
Stan Girard
b5c01efb78
fix(question): fixed with user_settings (#1349)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2023-10-06 20:32:23 +02:00
Mamadou DICKO
16cea2c86b
feat: add onboarding_a column to onboarding table (#1340)
https://github.com/StanGirard/quivr/issues/1338

<img width="1215" alt="Screenshot 2023-10-05 at 18 05 30"
src="https://github.com/StanGirard/quivr/assets/63923024/67562c50-ddc1-4eaa-8b68-d374b2eb4b67">
2023-10-06 12:37:12 +02:00
Mamadou DICKO
28ef515458
feat: update onboarding controller and fix typo (#1333)
<img width="525" alt="Screenshot 2023-10-05 at 10 55 29"
src="https://github.com/StanGirard/quivr/assets/63923024/d7a4e7ef-e73f-46d0-8499-06c5e951f64a">


<img width="1207" alt="Screenshot 2023-10-05 at 10 49 45"
src="https://github.com/StanGirard/quivr/assets/63923024/50eba38a-b2cb-4e7d-82b1-ceca60abdad4">

https://github.com/StanGirard/quivr/issues/1328
2023-10-05 11:31:26 +02:00
Mamadou DICKO
fbdc07af2c
feat: add onboarding table (#1327)
* feat: add onboarding table

* feat: add Onboarding repository

* feat: add onboarding controller
2023-10-05 09:47:29 +02:00
Stan Girard
e62c3e0579 feat(litellm): adding huggingface compatibility mistral 2023-10-03 17:18:50 +02:00