quivr/backend
Stan Girard 38589d32cf
feat(backend): add RAG evaluation using Ragas (#2429)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):


<!--
ELLIPSIS_HIDDEN
-->


----

| <a href="https://ellipsis.dev" target="_blank"><img
src="https://avatars.githubusercontent.com/u/80834858?s=400&u=31e596315b0d8f7465b3ee670f25cea677299c96&v=4"
alt="Ellipsis" width="30px" height="30px"/></a> | 🚀 This PR
description was created by [Ellipsis](https://www.ellipsis.dev) for
commit 5350e0a071. |
|--------|--------|

### Summary:
This PR adds a new script for evaluating the RAG model using the Ragas
library, with results saved as a JSON file and printed to the console.

**Key points**:
- New script `run_evaluation.py` added to
`backend/tests/ragas_evaluation/`.
- The script processes documents, generates replies using a QuivrRAG
chain, and evaluates the replies using specified metrics.
- Results are saved as a JSON file and printed to the console.
- The script can be run from the command line with various options.


----
Generated with ❤️ by [ellipsis.dev](https://www.ellipsis.dev)



<!--
ELLIPSIS_HIDDEN
-->

---------

Co-authored-by: Damien Mourot <damien.mourot@gmail.com>
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2024-04-15 06:53:48 -07:00
..
.vscode feat: ⚙️🐞 configure debugger for the backend (#1345) 2023-10-09 15:23:13 +02:00
llm feat(lcel): migrated to lcel and pydantic (#2185) 2024-02-14 14:01:35 -08:00
middlewares feat: new landing page (#2264) 2024-02-25 18:09:01 -08:00
models fix: logger level and telemetry function calls (#2409) 2024-04-06 18:35:57 -07:00
modules feat: llamaparse & diff agent (#2427) 2024-04-12 09:06:50 -07:00
packages feat(backend): add RAG evaluation using Ragas (#2429) 2024-04-15 06:53:48 -07:00
repository feat(notion): added custom integration (#2268) 2024-02-27 21:30:25 -08:00
routes fix: Add integration brain to subscription route (#2410) 2024-04-06 19:34:46 -07:00
supabase/functions/add-new-email feat: new landing page (#2264) 2024-02-25 18:09:01 -08:00
tests feat(backend): add RAG evaluation using Ragas (#2429) 2024-04-15 06:53:48 -07:00
vectorstore feat(custom): big brain (#2198) 2024-02-16 17:14:30 -08:00
.dockerignore feat(notion): added custom integration (#2268) 2024-02-27 21:30:25 -08:00
celery_config.py fix: Update Celery config to remove SSL certificate requirement 2024-02-20 19:56:14 -08:00
celery_worker.py fix: logger level and telemetry function calls (#2409) 2024-04-06 18:35:57 -07:00
conftest.py fix: 🐛 tests (#2095) 2024-01-26 16:10:16 -08:00
Dockerfile Add libpq-dev and gcc to Dockerfile 2024-02-14 20:29:57 -08:00
Dockerfile.dev fix: logger level and telemetry function calls (#2409) 2024-04-06 18:35:57 -07:00
logger.py fix: logger level and telemetry function calls (#2409) 2024-04-06 18:35:57 -07:00
main.py feat: Add assistant module and remove ingestion module (#2420) 2024-04-10 04:28:22 -07:00
manage_services.sh feat(integration): implementation (#2191) 2024-02-14 20:07:53 -08:00
pyrightconfig.json feat(refacto): changed a bit of things to make better dx (#984) 2023-08-19 13:32:16 +02:00
requirements.txt feat(backend): add RAG evaluation using Ragas (#2429) 2024-04-15 06:53:48 -07:00