quivr/backend/tests/ragas_tests/experiment.json
Stan Girard ff4ecb86e5
feat: Update QuivrRAG and run_evaluation.py files (#2615)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):

---------

Co-authored-by: chloedia <chloedaems0@gmail.com>
2024-05-28 07:47:31 -07:00

57 lines
2.6 KiB
JSON

[
{
"question": "When is the current contract with Contoso ending?",
"evolution_type": "simple",
"ground_truth": "The current contract with Contoso is ending on December 31, 2028."
},
{
"question": "What's the maximum Production Fonts Contoso have?",
"evolution_type": "simple",
"ground_truth": "Contoso has an unlimited Production Fonts licensed under the Monotype Fonts License Agreement."
},
{
"question": "How many contracts we have with Contoso?",
"evolution_type": "simple",
"ground_truth": "5"
},
{
"question": "Have we got any change in number of production fonts by addendum in previous contracts, before the current contract?",
"evolution_type": "simple",
"ground_truth": "Yes, there have been changes in the number of production fonts in previous contracts. For example, in the Addendum No. 1 to the Design and Deploy License Agreement, the Permitted Usage per Term table"
},
{
"question": "What's the number of production fonts for previous contract before current contract with Contoso?",
"evolution_type": "simple",
"ground_truth": "The number of production fonts for the contract with Contoso that preceded the current contract was 140."
},
{
"question": "What will be the next renewal date for Contoso?",
"evolution_type": "simple",
"ground_truth": "The next renewal date for Contoso will be December 31, 2028,"
},
{
"question": "How many Software License Order Form contract we ever had with Contoso?",
"evolution_type": "simple",
"ground_truth": "1"
},
{
"question": "What's the Number of Licensed Mobile Applications in different contract with Contoso?",
"evolution_type": "simple",
"ground_truth": "1"
},
{
"question": "Is Single Sign-On Option available in the latest contract with Contoso?",
"evolution_type": "simple",
"ground_truth": "Yes, the Single Sign-On Option is available in the latest contract with Contoso."
},
{
"question": "What's the Licensed Page Views in the latest contract with Contoso?",
"evolution_type": "simple",
"ground_truth": "The latest contract with Contoso specifies that the Licensed Page Views are unlimited"
},
{
"question": "What's the Licensed Impressions (Digital Marketing Communications) in the latest contract with Contoso?",
"evolution_type": "simple",
"ground_truth": "The Licensed Impressions (Digital Marketing Communications) in the latest contract with Contoso is Unlimited."
}
]