This pull request refactors the DEFAULT_DOCUMENT_PROMPT in quivr_rag.py
to improve code readability and maintainability. The unnecessary file
name information has been removed from the prompt template as it is now
included in the vector embedding
<!--
ELLIPSIS_HIDDEN
-->
----
| <a href="https://ellipsis.dev" target="_blank"><img
src="https://avatars.githubusercontent.com/u/80834858?s=400&u=31e596315b0d8f7465b3ee670f25cea677299c96&v=4"
alt="Ellipsis" width="30px" height="30px"/></a> | 🚀 This PR
description was created by [Ellipsis](https://www.ellipsis.dev) for
commit 81939457e6. |
|--------|--------|
### Summary:
This PR refactors the `DEFAULT_DOCUMENT_PROMPT` in `quivr_rag.py` by
removing the file name information from the prompt template, enhancing
code readability and maintainability.
**Key points**:
- Refactored `DEFAULT_DOCUMENT_PROMPT` in `quivr_rag.py`.
- Removed file name information from the prompt template.
- File name information is now included in the vector embedding.
----
Generated with ❤️ by [ellipsis.dev](https://www.ellipsis.dev)
<!--
ELLIPSIS_HIDDEN
-->
This pull request fixes the file key validation in the `check_input`
method. Previously, the method only checked if a file was uploaded and
if the key of the file was "doc_to_summarize". However, it did not check
if the key of the file was the same as the name of the file. This PR
adds a check to ensure that the key of the file is the same as the name
of the file. This improves the accuracy of the file validation process.
<!--
ELLIPSIS_HIDDEN
-->
----
| <a href="https://ellipsis.dev" target="_blank"><img
src="https://avatars.githubusercontent.com/u/80834858?s=400&u=31e596315b0d8f7465b3ee670f25cea677299c96&v=4"
alt="Ellipsis" width="30px" height="30px"/></a> | 🚀 This PR
description was created by [Ellipsis](https://www.ellipsis.dev) for
commit 8f04d832ac. |
|--------|--------|
### Summary:
This PR enhances the file validation process in the `SummaryAssistant`
class by ensuring the key of the uploaded file matches its filename.
**Key points**:
- Added a new file key validation in `check_input` method of
`SummaryAssistant` class.
- The validation checks if the key of the uploaded file matches the
filename.
- This is an additional check to the existing file upload and key
validations.
----
Generated with ❤️ by [ellipsis.dev](https://www.ellipsis.dev)
<!--
ELLIPSIS_HIDDEN
-->
This pull request increases the token limit for the summary output to
2000 in order to accommodate larger documents.
<!--
ELLIPSIS_HIDDEN
-->
----
| <a href="https://ellipsis.dev" target="_blank"><img
src="https://avatars.githubusercontent.com/u/80834858?s=400&u=31e596315b0d8f7465b3ee670f25cea677299c96&v=4"
alt="Ellipsis" width="30px" height="30px"/></a> | 🚀 This PR
description was created by [Ellipsis](https://www.ellipsis.dev) for
commit 4412ecde83. |
|--------|--------|
### Summary:
This PR increases the token limit for the summary output to 2000 in the
`SummaryAssistant` class to accommodate larger documents.
**Key points**:
- Increased `max_tokens` parameter in `ChatLiteLLM` instance to 2000 in
`process_assistant` function of `SummaryAssistant` class in
`/backend/modules/assistant/ito/summary.py` file.
----
Generated with ❤️ by [ellipsis.dev](https://www.ellipsis.dev)
<!--
ELLIPSIS_HIDDEN
-->
This pull request refactors the get_brains_usages method in the
Analytics class. The changes include improving code readability,
optimizing database queries, and fixing potential bugs.
<!--
ELLIPSIS_HIDDEN
-->
----
| <a href="https://ellipsis.dev" target="_blank"><img
src="https://avatars.githubusercontent.com/u/80834858?s=400&u=31e596315b0d8f7465b3ee670f25cea677299c96&v=4"
alt="Ellipsis" width="30px" height="30px"/></a> | 🚀 This PR
description was created by [Ellipsis](https://www.ellipsis.dev) for
commit f0ad664673. |
|--------|--------|
### Summary:
This PR optimizes the `get_brains_usages` method in the `Analytics`
class by reducing database calls and improving code readability.
**Key points**:
- Refactored `get_brains_usages` method in `Analytics` class.
- Optimized database query to fetch chat history in one go.
- Improved code readability and formatting.
----
Generated with ❤️ by [ellipsis.dev](https://www.ellipsis.dev)
<!--
ELLIPSIS_HIDDEN
-->
This pull request adds PDF generation functionality and improves the
formatting of emails. It includes a new module for generating PDFs using
the fpdf2 library and updates the email templates to use markdown
formatting for better readability.
<!--
ELLIPSIS_HIDDEN
-->
----
| <a href="https://ellipsis.dev" target="_blank"><img
src="https://avatars.githubusercontent.com/u/80834858?s=400&u=31e596315b0d8f7465b3ee670f25cea677299c96&v=4"
alt="Ellipsis" width="30px" height="30px"/></a> | 🚀 This PR
description was created by [Ellipsis](https://www.ellipsis.dev) for
commit ccecff77a5. |
|--------|--------|
### Summary:
This PR introduces PDF generation functionality, improves email
formatting, sanitizes filenames, updates various files and dependencies,
and updates `docker-compose.dev.yml`.
**Key points**:
- Added PDF generation functionality using the `fpdf2` library
- Improved email formatting
- Sanitized filenames using the `unidecode` library
- Updated `backend/modules/assistant/ito/ito.py` to generate PDFs and
sanitize filenames
- Added `pdf_generator.py` in `backend/modules/assistant/ito/utils/` for
PDF generation
- Updated `Pipfile` and `Pipfile.lock` in
`backend/modules/assistant/ito/utils/` to include `unidecode`
- Updated `requirements.txt` with new dependencies
- Updated `docker-compose.dev.yml`
----
Generated with ❤️ by [ellipsis.dev](https://www.ellipsis.dev)
<!--
ELLIPSIS_HIDDEN
-->
This pull request improves the prompt summary in the assistant code. It
updates the map_template and reduce_template to provide clearer
instructions for identifying main themes, key points, and important
information in each section of a document. The reduce_template also
specifies that the final summary should include the main themes, key
points, and important information of the entire document.
<!--
ELLIPSIS_HIDDEN
-->
----
| <a href="https://ellipsis.dev" target="_blank"><img
src="https://avatars.githubusercontent.com/u/80834858?s=400&u=31e596315b0d8f7465b3ee670f25cea677299c96&v=4"
alt="Ellipsis" width="30px" height="30px"/></a> | 🚀 This PR
description was created by [Ellipsis](https://www.ellipsis.dev) for
commit 456736a41d. |
|--------|--------|
### Summary:
This PR enhances the document summarization process in the
`SummaryAssistant` class by providing clearer instructions for
identifying main themes, key points, and important information in each
section of a document.
**Key points**:
- Updated `map_template` and `reduce_template` in `process_assistant`
function of `SummaryAssistant` class.
- Changes provide clearer instructions for summarizing documents.
- Final summary should include main themes, key points, and important
information of the entire document.
----
Generated with ❤️ by [ellipsis.dev](https://www.ellipsis.dev)
<!--
ELLIPSIS_HIDDEN
-->
This pull request adds functionality to update user usage and calculate
pricing in the ITO assistant. It includes a new method
`increase_usage_user()` that raises an error if the user has consumed
all of their credits, and a new method `calculate_pricing()` that
returns a fixed pricing value of 20.
<!--
ELLIPSIS_HIDDEN
-->
----
| <a href="https://ellipsis.dev" target="_blank"><img
src="https://avatars.githubusercontent.com/u/80834858?s=400&u=31e596315b0d8f7465b3ee670f25cea677299c96&v=4"
alt="Ellipsis" width="30px" height="30px"/></a> | 🚀 This PR
description was created by [Ellipsis](https://www.ellipsis.dev) for
commit ee5fdf70f6. |
|--------|--------|
### Summary:
This PR adds functionality to update user usage and calculate pricing in
the ITO assistant, with new methods in the `ITO` class and an update to
where the `increase_usage_user()` method is called.
**Key points**:
- Added `increase_usage_user()` and `calculate_pricing()` methods to
`ITO` class in `/backend/modules/assistant/ito/ito.py`
- `increase_usage_user()` updates user usage and raises an error if all
credits are consumed
- `calculate_pricing()` returns a fixed pricing value of 20
- `increase_usage_user()` is now called in the `ITO` class constructor
----
Generated with ❤️ by [ellipsis.dev](https://www.ellipsis.dev)
<!--
ELLIPSIS_HIDDEN
-->
This pull request adds new input models for boolean, number, select
text, and select number. These models will allow for more flexible and
dynamic input options in the software. The changes include the addition
of the `InputBoolean`, `InputNumber`, `InputSelectText`, and
`InputSelectNumber` classes to the `Inputs` model. These classes provide
properties for key, required, description, and additional options
specific to each input type. This enhancement will improve the overall
functionality and user experience of the software.
<!--
ELLIPSIS_HIDDEN
-->
----
| <a href="https://ellipsis.dev" target="_blank"><img
src="https://avatars.githubusercontent.com/u/80834858?s=400&u=31e596315b0d8f7465b3ee670f25cea677299c96&v=4"
alt="Ellipsis" width="30px" height="30px"/></a> | 🚀 This PR
description was created by [Ellipsis](https://www.ellipsis.dev) for
commit 6fa583b6a8. |
|--------|--------|
### Summary:
This PR enhances the `Inputs` model by adding new input models for
boolean, number, select text, and select number, improving the
software's functionality and user experience.
**Key points**:
- Added `InputBoolean`, `InputNumber`, `InputSelectText`, and
`InputSelectNumber` classes to `Inputs` model in `inputs.py` and
`outputs.py`.
- Each new class provides properties for key, required, description, and
additional options specific to each input type.
----
Generated with ❤️ by [ellipsis.dev](https://www.ellipsis.dev)
<!--
ELLIPSIS_HIDDEN
-->
# Description
Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.
## Checklist before requesting a review
Please delete options that are not relevant.
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged
## Screenshots (if appropriate):
<!--
ELLIPSIS_HIDDEN
-->
----
| <a href="https://ellipsis.dev" target="_blank"><img
src="https://avatars.githubusercontent.com/u/80834858?s=400&u=31e596315b0d8f7465b3ee670f25cea677299c96&v=4"
alt="Ellipsis" width="30px" height="30px"/></a> | 🚀 This PR
description was created by [Ellipsis](https://www.ellipsis.dev) for
commit 5350e0a071. |
|--------|--------|
### Summary:
This PR adds a new script for evaluating the RAG model using the Ragas
library, with results saved as a JSON file and printed to the console.
**Key points**:
- New script `run_evaluation.py` added to
`backend/tests/ragas_evaluation/`.
- The script processes documents, generates replies using a QuivrRAG
chain, and evaluates the replies using specified metrics.
- Results are saved as a JSON file and printed to the console.
- The script can be run from the command line with various options.
----
Generated with ❤️ by [ellipsis.dev](https://www.ellipsis.dev)
<!--
ELLIPSIS_HIDDEN
-->
---------
Co-authored-by: Damien Mourot <damien.mourot@gmail.com>
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
This pull request adds an icon and description to the assistant. The
icon is displayed in the user interface and the description provides
additional information about the assistant.
This pull request adds a new optional field "tags" to the
AssistantOutput classes. The "tags" field allows for categorizing the
outputs and can be used for filtering or organizing purposes.
…dio_transcript and crawler assistants
# Description
Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.
## Checklist before requesting a review
Please delete options that are not relevant.
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged
## Screenshots (if appropriate):
# Description
Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.
## Checklist before requesting a review
Please delete options that are not relevant.
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged
## Screenshots (if appropriate):
# Description
Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.
## Checklist before requesting a review
Please delete options that are not relevant.
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged
## Screenshots (if appropriate):
# Description
Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.
## Checklist before requesting a review
Please delete options that are not relevant.
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged
## Screenshots (if appropriate):
# Description
Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.
## Checklist before requesting a review
Please delete options that are not relevant.
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged
## Screenshots (if appropriate):
---------
Co-authored-by: Stan Girard <girard.stanislas@gmail.com>
# Description
Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.
## Checklist before requesting a review
Please delete options that are not relevant.
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged
## Screenshots (if appropriate):
# Description
Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.
## Checklist before requesting a review
Please delete options that are not relevant.
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged
## Screenshots (if appropriate):
# Description
Added ability to upload .bib files to brain using langchain bibtex
document loader. Also changed frontend to allow the choosing of said
file.
Looking for guidance on how to run unit tests locally!
## Checklist before requesting a review
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented hard-to-understand areas
- [x] I have ideally added tests that prove my fix is effective or that
my feature works
- [x] New and existing unit tests pass locally with my changes
- [x] Any dependent changes have been merged
This pull request adds a new field, `url_required`, to the
`IngestionEntity` class. This field is required for the ingestion
process and will be used to store the URL of the entity.
This pull request adds the ingestion module and routes to the project.
It includes the necessary files and code changes to implement the
ingestion functionality.
# Description
Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.
## Checklist before requesting a review
Please delete options that are not relevant.
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged
## Screenshots (if appropriate):
# Description
Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.
## Checklist before requesting a review
Please delete options that are not relevant.
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged
## Screenshots (if appropriate):
# Description
TODO BEFORE RELEASE
- [ ] Onboarded true for all existing users
## Checklist before requesting a review
Please delete options that are not relevant.
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged
## Screenshots (if appropriate):
---------
Co-authored-by: Stan Girard <girard.stanislas@gmail.com>
# Description
Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.
## Checklist before requesting a review
Please delete options that are not relevant.
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged
## Screenshots (if appropriate):
# Description
Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.
## Checklist before requesting a review
Please delete options that are not relevant.
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged
## Screenshots (if appropriate):
---------
Co-authored-by: Stan Girard <girard.stanislas@gmail.com>
# Description
Optimize the docker image for cpu-only environment. (PyTorch in cpu
mode, cuda not installed etc) This should reduce the backend docker
image size by ~2
Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.
## Checklist before requesting a review
Please delete options that are not relevant.
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged
## Screenshots (if appropriate):
# Description
Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.
## Checklist before requesting a review
Please delete options that are not relevant.
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged
## Screenshots (if appropriate):
This pull request adds optional fields to the UserIdentity and
UserUpdatableProperties classes in order to allow for more flexibility
when updating user properties. The new fields include username, company,
and onboarded. Additionally, the database schema has been updated to
reflect these changes.
# Description
Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.
## Checklist before requesting a review
Please delete options that are not relevant.
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged
## Screenshots (if appropriate):
This pull request updates the langchain.prompts and
langchain_core.messages modules. It includes changes to the code that
improve functionality and fix any existing issues.
# Description
Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.
## Checklist before requesting a review
Please delete options that are not relevant.
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged
## Screenshots (if appropriate):
# Description
Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.
## Checklist before requesting a review
Please delete options that are not relevant.
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged
## Screenshots (if appropriate):
This pull request includes a feature update to the newest version of
litellm. This update brings in the latest enhancements and bug fixes to
improve the functionality and performance of the software.
# Description
With this pull request, I add a fix that **enables Ollama**.
Running quivr in a Docker container, Ollama needs to be accessed via
http://host.docker.internal:11434.
This URL is defined as `api_base` of the `ChatLiteLLM` constructor. But
there are calls that are missing `api_base`. This leads to fallback URLs
being used. This is now fixed.
Fixes#2270#2267#2217#2204
## Checklist before requesting a review
Please delete options that are not relevant.
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [x] New and existing unit tests pass locally with my changes
- [x] Any dependent changes have been merged
## Screenshots (if appropriate):
None
Co-authored-by: Niels Garve <niels.garve@experienceone.com>
This pull request updates the integration types and classes in the
codebase. It includes the following commits:
- Update logo_url field in brain entity and integration_brain entity
- Update integration references in BrainsUsers and IntegrationBrain
classes
- Revert "Update integration references in BrainsUsers and
IntegrationBrain classes"
- Revert "Update logo_url field in brain entity and integration_brain
entity"
- Add 'doc' integration type and KnowledgeBrainQA class
Please review and merge these changes.
This pull request adds a custom integration feature and sync
functionality to the application. It includes the following changes:
- Added a new integration entity for custom integrations.
- Implemented the ability to load and poll the custom integration.
- Added a task to sync the custom integration with the user's brain.
- Updated the celery beat schedule to include the new task.
Please review and merge this pull request.
This pull request updates the origins in cors.py, origin in
resend_invitation_email.py, and referring_site in
add-new-email/index.ts. It also refactors the code in home/page.tsx and
next.config.js.
This pull request removes the healthz endpoint from the Sentry
integration. The `before_send` function has been updated to drop
transaction events that contain 'healthz' in the transaction name. This
change ensures that healthz transactions are not sent to Sentry.
This pull request adds the `max_input` and `max_tokens` parameters to
the `KnowledgeBrainQA` constructor. These parameters allow for setting
the maximum input length and maximum number of tokens for the
KnowledgeBrainQA model. This provides more flexibility and control over
the input size and tokenization process.
This pull request reduces the sampling rate for Sentry traces to 0.1.
This change ensures that only a small percentage of traces are captured,
which helps to reduce the impact on performance and storage.
This pull request fixes the `get_brain_details` API by adding a
`user_id` parameter. Previously, the API did not include the `user_id`
parameter, which caused issues when retrieving brain details. This fix
ensures that the `user_id` is passed correctly to the API, allowing for
accurate retrieval of brain details.
# Description
Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.
## Checklist before requesting a review
Please delete options that are not relevant.
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged
## Screenshots (if appropriate):
