This pull request adds support for recursive folder retrieval in the
`get_files_folder_user_sync` method. Previously, the method only
retrieved files from the specified folder, but now it can also retrieve
files from all subfolders recursively. This enhancement improves the
functionality and flexibility of the method, allowing for more
comprehensive file retrieval in sync operations.
# Description
Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.
## Checklist before requesting a review
Please delete options that are not relevant.
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged
## Screenshots (if appropriate):
This pull request includes updates to the `docker-compose.dev.yml` and
`Dockerfile.dev` files. The changes aim to improve performance and fix
bugs. The updates include:
- Removing unnecessary workers configuration in the
`docker-compose.dev.yml` file.
- Updating the base image in the `Dockerfile.dev` to use a slim version.
- Adjusting the schedule for a specific task in the code.
- Modifying the time interval for retrieving active syncs.
- Changing the loader class for processing PowerPoint files.
- Refactoring the file existence check logic.
- Adding debug logs for file existence check and file removal.
- Adjusting the file synchronization logic.
These changes are intended to enhance the performance and stability of
the application.
# Description
Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.
## Checklist before requesting a review
Please delete options that are not relevant.
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged
## Screenshots (if appropriate):
Co-authored-by: Stan Girard <stan@quivr.app>
# Description
Hey,
Here's a breakdown of what I've done:
- Reducing the number of opened fd and memory footprint: Previously, for
each uploaded file, we were opening a temporary NamedTemporaryFile to
write existing content read from Supabase. However, due to the
dependency on `langchain` loader classes, we couldn't use memory buffers
for the loaders. Now, with the changes made, we only open a single
temporary file for each `process_file_and_notify`, cutting down on
excessive file opening, read syscalls, and memory buffer usage. This
could cause stability issues when ingesting and processing large volumes
of documents. Unfortunately, there is still reopening of temporary files
in some code paths but this can be improved further in later work.
- Removing `UploadFile` class from File: The `UploadFile` ( a FastAPI
abstraction over a SpooledTemporaryFile for multipart upload) was
redundant in our `File` setup since we already downloaded the file from
remote storage and read it into memory + wrote the file into a temp
file. By removing this abstraction, we streamline our code and eliminate
unnecessary complexity.
- `async` function Adjustments: I've removed the async labeling from
functions where it wasn't truly asynchronous. For instance, calling
`filter_file` for processing files isn't genuinely async, ass async file
reading isn't actually asynchronous—it [uses a threadpool for reading
the
file](9f16bf5c25/starlette/datastructures.py (L458))
. Given that we're already leveraging `celery` for parallelism (one
worker per core), we need to ensure that reading and processing occur in
the same thread, or at least minimize thread spawning. Additionally,
since the rest of the code isn't inherently asynchronous, our bottleneck
lies in CPU operations rather than asynchronous processing.
These changes aim to improve performance and streamline our codebase.
Let me know if you have any questions or suggestions for further
improvements!
## Checklist before requesting a review
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have ideally added tests that prove my fix is effective or that
my feature works
---------
Signed-off-by: aminediro <aminediro@github.com>
Co-authored-by: aminediro <aminediro@github.com>
Co-authored-by: Stan Girard <girard.stanislas@gmail.com>
# Description
Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.
## Checklist before requesting a review
Please delete options that are not relevant.
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged
## Screenshots (if appropriate):
# Description
Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.
## Checklist before requesting a review
Please delete options that are not relevant.
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged
## Screenshots (if appropriate):
This pull request updates the parsing instructions in the `common.py`
file for the `llamaparse` feature. The previous parsing instruction for
transforming checkboxes into text has been modified to also extract
tables and transform them into key-value pairs. Additionally, the
instruction now allows for duplicate keys if needed. The example
instructions have also been updated to provide clearer examples for both
tables and checkboxes.
# Description
Change the prompt of the thoughts feature to have more steps.
## Checklist before requesting a review
Please delete options that are not relevant.
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged
## Screenshots (if appropriate):
# Description
Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.
## Checklist before requesting a review
Please delete options that are not relevant.
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged
## Screenshots (if appropriate):
---------
Co-authored-by: chloedia <chloedaems0@gmail.com>
This pull request adds a new component called ThoughtsButton for
displaying thoughts in the chat messages. The ThoughtsButton component
is used to show a tooltip with the thoughts text when the user hovers
over the button.
# Description
Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.
## Checklist before requesting a review
Please delete options that are not relevant.
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged
## Screenshots (if appropriate):
# Description
Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.
## Checklist before requesting a review
Please delete options that are not relevant.
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged
## Screenshots (if appropriate):
Adds a new "Privacy & Compliance" page to the Quivr documentation.
- **Introduces a comprehensive overview** of Quivr's commitment to
privacy and compliance, including the open-source security model,
responsible telemetry use, row-level policy enforcement, local data
handling, and LLM compatibility.
- **Details the open-source security model**, emphasizing transparency
and community-driven security enhancements.
- **Explains the responsible use of telemetry**, highlighting minimal
data collection, anonymization, secure storage, and opt-out options.
- **Describes row-level policy enforcement** for database security,
ensuring controlled data access.
- **Clarifies local data handling**, ensuring all data remains local
with no external data transfer without user consent.
- **Highlights compatibility with any LLM**, including local server-run
models, offering flexibility for user needs.
- **Includes information on SOC2 compliance** through partnership with
Porter & Oneleet, aiming for compliance within 90 days for managed
instances.
---
For more details, open the [Copilot Workspace
session](https://copilot-workspace.githubnext.com/QuivrHQ/quivr?shareId=e9ecbf64-d06d-45b6-ab73-c04b9ed7916d).
This pull request refactors the generate_answer and generate_stream
functions in order to improve code readability and maintainability. It
also adds new fields to the cited_answer model and updates the system
message template.
# Description
Applying dark mode to icons in tiptap.
## Checklist before requesting a review
Please delete options that are not relevant.
- [X] My code follows the style guidelines of this project
- [X] I have performed a self-review of my code
## Screenshots:
<img width="906" alt="image"
src="https://github.com/QuivrHQ/quivr/assets/33584382/c8d86fa2-0f58-4c44-82d2-5162a5e025cd">
# Description
Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.
## Checklist before requesting a review
Please delete options that are not relevant.
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged
## Screenshots (if appropriate):
# Description
Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.
## Checklist before requesting a review
Please delete options that are not relevant.
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged
## Screenshots (if appropriate):
# Description
Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.
## Checklist before requesting a review
Please delete options that are not relevant.
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged
## Screenshots (if appropriate):
# Description
Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.
## Checklist before requesting a review
Please delete options that are not relevant.
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged
## Screenshots (if appropriate):
# Description
Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.
## Checklist before requesting a review
Please delete options that are not relevant.
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged
## Screenshots (if appropriate):
# Description
Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.
## Checklist before requesting a review
Please delete options that are not relevant.
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged
## Screenshots (if appropriate):
# Description
Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.
## Checklist before requesting a review
Please delete options that are not relevant.
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged
## Screenshots (if appropriate):
# Description
Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.
## Checklist before requesting a review
Please delete options that are not relevant.
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged
## Screenshots (if appropriate):
# Description
Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.
## Checklist before requesting a review
Please delete options that are not relevant.
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged
## Screenshots (if appropriate):
This pull request fixes the sender email address in the
`resend_invitation_email` function in the `resend_invitation_email.py`
file. The `from` field has been changed to `sender` to ensure that the
correct email address is used when sending the invitation email.
This pull request updates the ChatLiteLLM model to "gpt-4o" and adds a
row-level security (RLS) optimization for notifications. It also
includes a new SQL script to drop and create a policy for allowing user
access to all notifications.
# Description
Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.
## Checklist before requesting a review
Please delete options that are not relevant.
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged
## Screenshots (if appropriate):
# Description
Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.
## Checklist before requesting a review
Please delete options that are not relevant.
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged
## Screenshots (if appropriate):
# Description
Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.
## Checklist before requesting a review
Please delete options that are not relevant.
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] I have commented hard-to-understand areas
- [x] I have ideally added tests that prove my fix is effective or that
my feature works
- [x] New and existing unit tests pass locally with my changes
- [x] Any dependent changes have been merged
## Screenshots (if appropriate):
<!--
ELLIPSIS_HIDDEN
-->
----
| <a href="https://ellipsis.dev" target="_blank"><img
src="https://avatars.githubusercontent.com/u/80834858?s=400&u=31e596315b0d8f7465b3ee670f25cea677299c96&v=4"
alt="Ellipsis" width="30px" height="30px"/></a> | 🚀 This PR
description was created by [Ellipsis](https://www.ellipsis.dev) for
commit bb45566f8a. |
|--------|--------|
### Summary:
This PR updates the `useLanguageHook` and `UserSettingsProvider`
functions to use the `shortName` of the selected language when changing
the language and to retrieve and store the dark mode setting from and to
local storage, respectively.
**Key points**:
- Updated `useLanguageHook` in
`/frontend/app/user/components/LanguageSelect/hooks/useLanguageHook.ts`
to use `shortName` of selected language when changing language.
- Updated `UserSettingsProvider` in
`/frontend/lib/context/UserSettingsProvider/User-settings.provider.tsx`
to retrieve dark mode setting from local storage.
- Updated listener for `change` event on `mediaQueryList` in
`UserSettingsProvider` to store updated dark mode setting in local
storage.
----
Generated with ❤️ by [ellipsis.dev](https://www.ellipsis.dev)
<!--
ELLIPSIS_HIDDEN
-->
---------
Co-authored-by: Zewed <dewez.antoine2@gmail.com>
This pull request adds support for the gpt-4o model to the existing
codebase. It includes changes to the BrainConfig, openAiFreeModels,
defineMaxTokens, model_compatible_with_function_calling, create_graph,
main, and process_assistant functions.
This pull request fixes the import statements for OllamaEmbeddings in
multiple files. The import statements are updated to use the correct
package name "langchain_community.embeddings" instead of
"langchain.embeddings.ollama". This ensures that the code can be
compiled and executed without any import errors.
This pull request adds comprehensive docstrings to the Brain classes
within the `backend/modules/brain/integrations` directory, enhancing
code documentation and readability. The changes include:
- **BigBrain (`Big/Brain.py`)**: Adds a class-level docstring explaining
the purpose and functionality of the BigBrain class, along with
method-level docstrings detailing the operations performed by each
method.
- **ClaudeBrain (`Claude/Brain.py`)**: Introduces a class-level
docstring that describes the ClaudeBrain class's integration with the
Claude model for conversational AI capabilities, and method-level
docstrings that clarify the purpose of each method.
- **GPT4Brain (`GPT4/Brain.py`)**: Updates include a detailed
class-level docstring outlining the GPT4Brain's integration with GPT-4
for real-time answers and tool support, along with method-level
docstrings explaining the functionality of each method.
- **NotionBrain (`Notion/Brain.py`)**: Adds a class-level docstring that
describes the NotionBrain's role in leveraging Notion data for
knowledge-based responses.
- **ProxyBrain (`Proxy/Brain.py`)**: Incorporates a class-level
docstring explaining the ProxyBrain's function as a dynamic language
model selector and method-level docstrings detailing the operations of
each method.
These additions ensure that each Brain class and its methods are
well-documented, providing clear insights into their purposes and
functionalities.
---
For more details, open the [Copilot Workspace
session](https://copilot-workspace.githubnext.com/QuivrHQ/quivr?shareId=b4e301ad-828e-4424-95ec-6e378d5d3849).
Updates the GPT-4 documentation and the `GPT4Brain` class to include
detailed information about the tools available for GPT4Brain and their
use cases.
- **Documentation (`docs/brains/gpt4.mdx`):**
- Adds a new section titled "Tools Available for GPT4Brain" that
describes specific tools: WebSearchTool, ImageGeneratorTool,
URLReaderTool, and EmailSenderTool.
- Provides use cases for each tool, demonstrating how they can be
utilized within GPT4Brain for various scenarios, such as generating
images, reading content from URLs, and sending emails.
- **Code (`backend/modules/brain/integrations/GPT4/Brain.py`):**
- Updates the class documentation to include information about the tools
available for GPT4Brain and outlines use cases for WebSearchTool,
ImageGeneratorTool, URLReaderTool, and EmailSenderTool.
- Maintains the existing functionality of the `GPT4Brain` class,
ensuring compatibility with the newly documented tools and use cases.
---
For more details, open the [Copilot Workspace
session](https://copilot-workspace.githubnext.com/QuivrHQ/quivr?shareId=2c2c1666-e5fb-4a06-bb08-ca967f4fe276).
