Commit Graph

1626 Commits

Author SHA1 Message Date
Zewed
3e95f1a5aa small 2024-04-19 10:21:19 +02:00
Zewed
2a7d465f93 Merge branch 'main' into feat/assistants 2024-04-19 10:06:46 +02:00
Zewed
da7bf55046 small 2024-04-19 10:05:56 +02:00
Stan Girard
5876bcfcdc
feat(aws): Update CPU and memory configurations for task definitions (#2447)
This pull request updates the CPU and memory configurations for the task
definitions in the AWS service. The CPU has been changed from 2048 to
1024, and the memory has been changed from 8192 to 2048. These changes
optimize the resource allocation for the task definitions.

<!--
ELLIPSIS_HIDDEN
-->


----

| 🚀 This description was created by
[Ellipsis](https://www.ellipsis.dev) for commit
c095179818 |
|--------|

### Summary:
This PR optimizes resource allocation by reducing CPU and memory
configurations for task definitions in five AWS service JSON files.

**Key points**:
- Reduced CPU and memory configurations in `task_definition.json`,
`task_definition_preview.json`, `task_definition_preview_beat.json`,
`task_definition_preview_worker.json`, and
`task_definition_prod_worker.json`.
- Optimized resource allocation for task definitions.


----
Generated with ❤️ by [ellipsis.dev](https://www.ellipsis.dev)



<!--
ELLIPSIS_HIDDEN
-->
2024-04-18 15:17:46 -07:00
Stan Girard
c3e0c2f2ee
feat(summary): improve prompt to get more insights (#2446)
This pull request includes a commit that improves the prompt used to get
more insights from the document. The commit modifies the map and reduce
templates to provide clearer instructions for analyzing each section of
the document and generating a consolidated summary. The changes also
include adjustments to the text splitter to improve the chunk overlap.

<!--
ELLIPSIS_HIDDEN
-->


----

| 🚀 This description was created by
[Ellipsis](https://www.ellipsis.dev) for commit
adf5541dab |
|--------|

### Summary:
This PR enhances the document analysis and summary generation
instructions in the `SummaryAssistant` class and improves text splitting
by adjusting the `chunk_overlap` parameter.

**Key points**:
- Updated `map_template` and `reduce_template` in `process_assistant`
method of `SummaryAssistant` class in
`/backend/modules/assistant/ito/summary.py`.
- Adjusted `chunk_overlap` parameter of `CharacterTextSplitter` instance
from 0 to 100.


----
Generated with ❤️ by [ellipsis.dev](https://www.ellipsis.dev)



<!--
ELLIPSIS_HIDDEN
-->
2024-04-18 14:04:06 -07:00
Stan Girard
3e53ff0dce
fix(assistants): brain id is null (#2445)
This pull request fixes an issue where the brain activation and value
handling were not working correctly. The `BrainInput` model now allows
for optional activation and value fields, and an empty string is
converted to `None` using a root validator. This ensures that brain
activation and value can be properly handled in the application.

<!--
ELLIPSIS_HIDDEN
-->


----

| 🚀 This description was created by
[Ellipsis](https://www.ellipsis.dev) for commit
d63ed6e651 |
|--------|

### Summary:
This PR modifies the `BrainInput` class in
`/backend/modules/assistant/dto/inputs.py` to make `activated` and
`value` fields optional and adds a root validator to convert empty
strings to `None`.

**Key points**:
- Modified `BrainInput` class in
`/backend/modules/assistant/dto/inputs.py`
- Made `activated` and `value` fields optional with default values
- Added root validator `empty_string_to_none` to convert empty strings
to `None`


----
Generated with ❤️ by [ellipsis.dev](https://www.ellipsis.dev)



<!--
ELLIPSIS_HIDDEN
-->
2024-04-18 14:03:47 -07:00
porter-deployment-app[bot]
e0edaedbd0
Enable Porter Application quivr-demo-front (#2444)
Hello 👋 from Porter! Please merge this PR to finish setting up your
application.

Co-authored-by: porter-deployment-app[bot] <87230664+porter-deployment-app[bot]@users.noreply.github.com>
2024-04-18 09:23:39 -07:00
Zewed
c156d6dc9e accept only files 2024-04-18 18:12:34 +02:00
Zewed
2d2e90340d wip 2024-04-18 18:04:15 +02:00
porter-deployment-app[bot]
c84e15dcff
Enable Porter Application quivr-back (#2443)
Hello 👋 from Porter! Please merge this PR to finish setting up your
application.

Co-authored-by: porter-deployment-app[bot] <87230664+porter-deployment-app[bot]@users.noreply.github.com>
2024-04-18 08:55:42 -07:00
Zewed
0d5d7827ef wip 2024-04-18 16:57:29 +02:00
Zewed
a2c70cee15 small 2024-04-18 14:57:31 +02:00
Zewed
b0709b55ce Merge branch 'main' into feat/assistants 2024-04-18 14:46:59 +02:00
Zewed
df8c780ef1 wip 2024-04-18 14:46:41 +02:00
Zewed
fb70fbc5e1 wip 2024-04-18 14:23:45 +02:00
Stan Girard
1e6ff8fce0
feat(rag): Refactor DEFAULT_DOCUMENT_PROMPT in quivr_rag.py (#2442)
This pull request refactors the DEFAULT_DOCUMENT_PROMPT in quivr_rag.py
to improve code readability and maintainability. The unnecessary file
name information has been removed from the prompt template as it is now
included in the vector embedding

<!--
ELLIPSIS_HIDDEN
-->


----

| <a href="https://ellipsis.dev" target="_blank"><img
src="https://avatars.githubusercontent.com/u/80834858?s=400&u=31e596315b0d8f7465b3ee670f25cea677299c96&v=4"
alt="Ellipsis" width="30px" height="30px"/></a> | 🚀 This PR
description was created by [Ellipsis](https://www.ellipsis.dev) for
commit 81939457e6. |
|--------|--------|

### Summary:
This PR refactors the `DEFAULT_DOCUMENT_PROMPT` in `quivr_rag.py` by
removing the file name information from the prompt template, enhancing
code readability and maintainability.

**Key points**:
- Refactored `DEFAULT_DOCUMENT_PROMPT` in `quivr_rag.py`.
- Removed file name information from the prompt template.
- File name information is now included in the vector embedding.


----
Generated with ❤️ by [ellipsis.dev](https://www.ellipsis.dev)



<!--
ELLIPSIS_HIDDEN
-->
2024-04-17 15:26:24 -07:00
Stan Girard
309666f3fd
feat: Update Docker build commands and dependencies (#2441)
This pull request updates the Docker build commands and dependencies.
2024-04-17 14:37:32 -07:00
Stan Girard
311d9bb15c
feat(assistant): check if key of file is same as filename uploaded (#2439)
This pull request fixes the file key validation in the `check_input`
method. Previously, the method only checked if a file was uploaded and
if the key of the file was "doc_to_summarize". However, it did not check
if the key of the file was the same as the name of the file. This PR
adds a check to ensure that the key of the file is the same as the name
of the file. This improves the accuracy of the file validation process.

<!--
ELLIPSIS_HIDDEN
-->


----

| <a href="https://ellipsis.dev" target="_blank"><img
src="https://avatars.githubusercontent.com/u/80834858?s=400&u=31e596315b0d8f7465b3ee670f25cea677299c96&v=4"
alt="Ellipsis" width="30px" height="30px"/></a> | 🚀 This PR
description was created by [Ellipsis](https://www.ellipsis.dev) for
commit 8f04d832ac. |
|--------|--------|

### Summary:
This PR enhances the file validation process in the `SummaryAssistant`
class by ensuring the key of the uploaded file matches its filename.

**Key points**:
- Added a new file key validation in `check_input` method of
`SummaryAssistant` class.
- The validation checks if the key of the uploaded file matches the
filename.
- This is an additional check to the existing file upload and key
validations.


----
Generated with ❤️ by [ellipsis.dev](https://www.ellipsis.dev)



<!--
ELLIPSIS_HIDDEN
-->
2024-04-17 13:27:15 -07:00
Stan Girard
1bd7e81b3d
fix(assistant): summary now can output 2000 tokens (#2440)
This pull request increases the token limit for the summary output to
2000 in order to accommodate larger documents.

<!--
ELLIPSIS_HIDDEN
-->


----

| <a href="https://ellipsis.dev" target="_blank"><img
src="https://avatars.githubusercontent.com/u/80834858?s=400&u=31e596315b0d8f7465b3ee670f25cea677299c96&v=4"
alt="Ellipsis" width="30px" height="30px"/></a> | 🚀 This PR
description was created by [Ellipsis](https://www.ellipsis.dev) for
commit 4412ecde83. |
|--------|--------|

### Summary:
This PR increases the token limit for the summary output to 2000 in the
`SummaryAssistant` class to accommodate larger documents.

**Key points**:
- Increased `max_tokens` parameter in `ChatLiteLLM` instance to 2000 in
`process_assistant` function of `SummaryAssistant` class in
`/backend/modules/assistant/ito/summary.py` file.


----
Generated with ❤️ by [ellipsis.dev](https://www.ellipsis.dev)



<!--
ELLIPSIS_HIDDEN
-->
2024-04-17 08:34:04 -07:00
Stan Girard
3400b8633f
feat(analytics): rely on sql rather that python loop for brains (#2437)
This pull request refactors the get_brains_usages method in the
Analytics class. The changes include improving code readability,
optimizing database queries, and fixing potential bugs.

<!--
ELLIPSIS_HIDDEN
-->


----

| <a href="https://ellipsis.dev" target="_blank"><img
src="https://avatars.githubusercontent.com/u/80834858?s=400&u=31e596315b0d8f7465b3ee670f25cea677299c96&v=4"
alt="Ellipsis" width="30px" height="30px"/></a> | 🚀 This PR
description was created by [Ellipsis](https://www.ellipsis.dev) for
commit f0ad664673. |
|--------|--------|

### Summary:
This PR optimizes the `get_brains_usages` method in the `Analytics`
class by reducing database calls and improving code readability.

**Key points**:
- Refactored `get_brains_usages` method in `Analytics` class.
- Optimized database query to fetch chat history in one go.
- Improved code readability and formatting.


----
Generated with ❤️ by [ellipsis.dev](https://www.ellipsis.dev)



<!--
ELLIPSIS_HIDDEN
-->
2024-04-17 07:24:03 -07:00
Stan Girard
b62297341f
feat(assistants): Add PDF generation functionality and nice emails (#2436)
This pull request adds PDF generation functionality and improves the
formatting of emails. It includes a new module for generating PDFs using
the fpdf2 library and updates the email templates to use markdown
formatting for better readability.

<!--
ELLIPSIS_HIDDEN
-->
----

| <a href="https://ellipsis.dev" target="_blank"><img
src="https://avatars.githubusercontent.com/u/80834858?s=400&u=31e596315b0d8f7465b3ee670f25cea677299c96&v=4"
alt="Ellipsis" width="30px" height="30px"/></a> | 🚀 This PR
description was created by [Ellipsis](https://www.ellipsis.dev) for
commit ccecff77a5. |
|--------|--------|

### Summary:
This PR introduces PDF generation functionality, improves email
formatting, sanitizes filenames, updates various files and dependencies,
and updates `docker-compose.dev.yml`.

**Key points**:
- Added PDF generation functionality using the `fpdf2` library
- Improved email formatting
- Sanitized filenames using the `unidecode` library
- Updated `backend/modules/assistant/ito/ito.py` to generate PDFs and
sanitize filenames
- Added `pdf_generator.py` in `backend/modules/assistant/ito/utils/` for
PDF generation
- Updated `Pipfile` and `Pipfile.lock` in
`backend/modules/assistant/ito/utils/` to include `unidecode`
- Updated `requirements.txt` with new dependencies
- Updated `docker-compose.dev.yml`


----
Generated with ❤️ by [ellipsis.dev](https://www.ellipsis.dev)

<!--
ELLIPSIS_HIDDEN
-->
2024-04-17 06:00:31 -07:00
Stan Girard
ed814de1c6
feat(assistant): improve prompt summary (#2435)
This pull request improves the prompt summary in the assistant code. It
updates the map_template and reduce_template to provide clearer
instructions for identifying main themes, key points, and important
information in each section of a document. The reduce_template also
specifies that the final summary should include the main themes, key
points, and important information of the entire document.

<!--
ELLIPSIS_HIDDEN
-->


----

| <a href="https://ellipsis.dev" target="_blank"><img
src="https://avatars.githubusercontent.com/u/80834858?s=400&u=31e596315b0d8f7465b3ee670f25cea677299c96&v=4"
alt="Ellipsis" width="30px" height="30px"/></a> | 🚀 This PR
description was created by [Ellipsis](https://www.ellipsis.dev) for
commit 456736a41d. |
|--------|--------|

### Summary:
This PR enhances the document summarization process in the
`SummaryAssistant` class by providing clearer instructions for
identifying main themes, key points, and important information in each
section of a document.

**Key points**:
- Updated `map_template` and `reduce_template` in `process_assistant`
function of `SummaryAssistant` class.
- Changes provide clearer instructions for summarizing documents.
- Final summary should include main themes, key points, and important
information of the entire document.


----
Generated with ❤️ by [ellipsis.dev](https://www.ellipsis.dev)



<!--
ELLIPSIS_HIDDEN
-->
2024-04-16 14:03:36 -07:00
Stan Girard
1931848262
feat(assistants): Add user usage update and pricing calculation to ITO assistant (#2433)
This pull request adds functionality to update user usage and calculate
pricing in the ITO assistant. It includes a new method
`increase_usage_user()` that raises an error if the user has consumed
all of their credits, and a new method `calculate_pricing()` that
returns a fixed pricing value of 20.

<!--
ELLIPSIS_HIDDEN
-->
----

| <a href="https://ellipsis.dev" target="_blank"><img
src="https://avatars.githubusercontent.com/u/80834858?s=400&u=31e596315b0d8f7465b3ee670f25cea677299c96&v=4"
alt="Ellipsis" width="30px" height="30px"/></a> | 🚀 This PR
description was created by [Ellipsis](https://www.ellipsis.dev) for
commit ee5fdf70f6. |
|--------|--------|

### Summary:
This PR adds functionality to update user usage and calculate pricing in
the ITO assistant, with new methods in the `ITO` class and an update to
where the `increase_usage_user()` method is called.

**Key points**:
- Added `increase_usage_user()` and `calculate_pricing()` methods to
`ITO` class in `/backend/modules/assistant/ito/ito.py`
- `increase_usage_user()` updates user usage and raises an error if all
credits are consumed
- `calculate_pricing()` returns a fixed pricing value of 20
- `increase_usage_user()` is now called in the `ITO` class constructor


----
Generated with ❤️ by [ellipsis.dev](https://www.ellipsis.dev)

<!--
ELLIPSIS_HIDDEN
-->
2024-04-16 09:14:38 -07:00
Zewed
75291e023c wip outputs 2024-04-16 17:28:31 +02:00
Zewed
c7728a99ab wip 2024-04-16 16:28:10 +02:00
Zewed
fb075be280 wip 2024-04-16 16:21:44 +02:00
Stan Girard
c4106dbb86
chore(main): release 0.0.230 (#2431)
🤖 I have created a release *beep* *boop*
---


## 0.0.230 (2024-04-16)

## What's Changed
* feat(backend): add RAG evaluation using Ragas by @StanGirard in
https://github.com/QuivrHQ/quivr/pull/2429
* feat(assistants): Add new input models for boolean, number, select
text, and select number by @StanGirard in
https://github.com/QuivrHQ/quivr/pull/2432


**Full Changelog**:
https://github.com/QuivrHQ/quivr/compare/v0.0.229...v0.0.230

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

<!--
ELLIPSIS_HIDDEN
-->


----

| <a href="https://ellipsis.dev" target="_blank"><img
src="https://avatars.githubusercontent.com/u/80834858?s=400&u=31e596315b0d8f7465b3ee670f25cea677299c96&v=4"
alt="Ellipsis" width="30px" height="30px"/></a> | 🚀 This PR
description was created by [Ellipsis](https://www.ellipsis.dev) for
commit 9aa6637296. |
|--------|--------|

### Summary:
This PR bumps the version to 0.0.230, adds RAG evaluation using Ragas in
the backend, introduces new input models in the assistants module, and
reflects these changes in `CHANGELOG.md`.

**Key points**:
- Version bumped to 0.0.230
- Added RAG evaluation using Ragas in the backend
- Introduced new input models for boolean, number, select text, and
select number in the assistants module
- Changes reflected in `CHANGELOG.md`


----
Generated with ❤️ by [ellipsis.dev](https://www.ellipsis.dev)



<!--
ELLIPSIS_HIDDEN
-->
2024-04-16 06:15:53 -07:00
Stan Girard
c92ccb5543
feat(assistants): Add new input models for boolean, number, select text, and select number (#2432)
This pull request adds new input models for boolean, number, select
text, and select number. These models will allow for more flexible and
dynamic input options in the software. The changes include the addition
of the `InputBoolean`, `InputNumber`, `InputSelectText`, and
`InputSelectNumber` classes to the `Inputs` model. These classes provide
properties for key, required, description, and additional options
specific to each input type. This enhancement will improve the overall
functionality and user experience of the software.

<!--
ELLIPSIS_HIDDEN
-->


----

| <a href="https://ellipsis.dev" target="_blank"><img
src="https://avatars.githubusercontent.com/u/80834858?s=400&u=31e596315b0d8f7465b3ee670f25cea677299c96&v=4"
alt="Ellipsis" width="30px" height="30px"/></a> | 🚀 This PR
description was created by [Ellipsis](https://www.ellipsis.dev) for
commit 6fa583b6a8. |
|--------|--------|

### Summary:
This PR enhances the `Inputs` model by adding new input models for
boolean, number, select text, and select number, improving the
software's functionality and user experience.

**Key points**:
- Added `InputBoolean`, `InputNumber`, `InputSelectText`, and
`InputSelectNumber` classes to `Inputs` model in `inputs.py` and
`outputs.py`.
- Each new class provides properties for key, required, description, and
additional options specific to each input type.


----
Generated with ❤️ by [ellipsis.dev](https://www.ellipsis.dev)



<!--
ELLIPSIS_HIDDEN
-->
2024-04-16 00:29:37 -07:00
Zewed
97dbbb3c35 wip 2024-04-15 21:49:33 +02:00
Stan Girard
861931f231 tests: added new way of testing less prone to errors 2024-04-15 18:22:21 +02:00
Zewed
05176305fb Merge branch 'main' into feat/assistants 2024-04-15 17:07:24 +02:00
Zewed
d61d37e02b stash commit 2024-04-15 17:07:04 +02:00
Stan Girard
38589d32cf
feat(backend): add RAG evaluation using Ragas (#2429)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):


<!--
ELLIPSIS_HIDDEN
-->


----

| <a href="https://ellipsis.dev" target="_blank"><img
src="https://avatars.githubusercontent.com/u/80834858?s=400&u=31e596315b0d8f7465b3ee670f25cea677299c96&v=4"
alt="Ellipsis" width="30px" height="30px"/></a> | 🚀 This PR
description was created by [Ellipsis](https://www.ellipsis.dev) for
commit 5350e0a071. |
|--------|--------|

### Summary:
This PR adds a new script for evaluating the RAG model using the Ragas
library, with results saved as a JSON file and printed to the console.

**Key points**:
- New script `run_evaluation.py` added to
`backend/tests/ragas_evaluation/`.
- The script processes documents, generates replies using a QuivrRAG
chain, and evaluates the replies using specified metrics.
- Results are saved as a JSON file and printed to the console.
- The script can be run from the command line with various options.


----
Generated with ❤️ by [ellipsis.dev](https://www.ellipsis.dev)



<!--
ELLIPSIS_HIDDEN
-->

---------

Co-authored-by: Damien Mourot <damien.mourot@gmail.com>
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2024-04-15 06:53:48 -07:00
Stan Girard
7c9f67e539
chore(main): release 0.0.229 (#2419)
🤖 I have created a release *beep* *boop*
---


## 0.0.229 (2024-04-12)

## What's Changed
* feat: optimization calls by @StanGirard in
https://github.com/QuivrHQ/quivr/pull/2417
* feat: Add assistant module and remove ingestion module by @StanGirard
in https://github.com/QuivrHQ/quivr/pull/2420
* feat: assistants by @StanGirard in
https://github.com/QuivrHQ/quivr/pull/2421
* feat: Add tags to AssistantOutput classes by @StanGirard in
https://github.com/QuivrHQ/quivr/pull/2425
* feat: Add icon and description to assistant by @StanGirard in
https://github.com/QuivrHQ/quivr/pull/2426
* feat: llamaparse & diff agent by @StanGirard in
https://github.com/QuivrHQ/quivr/pull/2427


**Full Changelog**:
https://github.com/QuivrHQ/quivr/compare/v0.0.228...v0.0.229

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
2024-04-15 01:51:43 -07:00
Zewed
d5a1697752 wip 2024-04-12 19:23:47 +02:00
Zewed
520e945052 wip 2024-04-12 19:22:22 +02:00
Zewed
2adb0872ce stepper works 2024-04-12 19:08:59 +02:00
Zewed
9618012ef0 works 2024-04-12 19:06:58 +02:00
Zewed
f8fa45a681 begin to implement modal 2024-04-12 18:11:02 +02:00
Stan Girard
c773677b46
feat: llamaparse & diff agent (#2427)
This pull request updates the dependencies in the Pipfile to their
latest versions.
2024-04-12 09:06:50 -07:00
Zewed
a31b2e70dc wip 2024-04-12 10:20:41 +02:00
Zewed
96d113794b wip 2024-04-11 22:40:13 +02:00
Zewed
74f950af28 Merge branch 'main' into feat/assistants 2024-04-11 22:36:20 +02:00
Stan Girard
a8c5e186be
feat: Add icon and description to assistant (#2426)
This pull request adds an icon and description to the assistant. The
icon is displayed in the user interface and the description provides
additional information about the assistant.
2024-04-11 13:35:35 -07:00
Zewed
f2fd9625ed dont need provider 2024-04-11 22:32:11 +02:00
Zewed
a5eb9b3ad3 generate provider 2024-04-11 22:29:48 +02:00
Zewed
01bb3aaced some scss and message box 2024-04-11 22:02:35 +02:00
Zewed
8aecca8420 brainCard 2024-04-11 21:44:34 +02:00
Zewed
05e3c32a36 Merge branch 'main' into feat/assistants 2024-04-11 18:49:51 +02:00
Stan Girard
9db8b63bb9
feat: Add tags to AssistantOutput classes (#2425)
This pull request adds a new optional field "tags" to the
AssistantOutput classes. The "tags" field allows for categorizing the
outputs and can be used for filtering or organizing purposes.
2024-04-11 09:42:01 -07:00