StanGirard/quivr
1
1
Fork 0
mirror of https://github.com/StanGirard/quivr.git synced 2024-11-23 12:26:03 +03:00
Code Issues Packages Projects Releases Wiki Activity
1,317 Commits 835 Branches 345 Tags 200 MiB
cc39f9e3ba
Commit Graph

395 Commits

Author SHA1 Message Date
Stan Girard
cbc8ac4946
Revert "fix: update backend tests (#975)" (#983) 
This reverts commit c746eb1830.
 2023-08-19 12:31:15 +02:00
Stan Girard
ce6b45e1ac
feat(preview): added service upload (#979) 
* feat(docker): improved size image

* feat(preview): added upload service

* ci(aws): using matrix
 2023-08-18 14:49:48 +02:00
Mamadou DICKO
c746eb1830
fix: update backend tests (#975) 
* fix: update backend tests

* fix(pytest): update types
 2023-08-18 14:01:14 +02:00
Stan Girard
aa623c4039
feat(docker): improved size image (#978) 2023-08-18 12:57:16 +02:00
Stan Girard
b894f8e7e5
ci(preview): fixed name service (#977) 
* feat(chat-service): added task definition

* feat(ci): added ci

* feat(ci): added preview chat
 2023-08-18 12:20:47 +02:00
Stan Girard
d1dfd6c7ec
Feat/microservice aws (#976) 
* feat(chat-service): added task definition

* feat(ci): added ci
 2023-08-18 12:02:57 +02:00
Zineb El Bachiri
74da7dde2d
Crawl/update env variables (#931) 
* 📝 add env variable for crawl

* 🐛 make CharQuestion with optional attributes

* 💡 make chatQustion config optional
 2023-08-18 11:59:20 +02:00
Stepan Lebedev
600ff1ede0
feat(Unplug): chatting without brain streaming (#970) 
* feat(Unplug): Adds new basic headless llm

* feat(Unplug): adds chatting without brain option when no streaming

* feat(Unplug): adds chatting without brain option when streaming
 2023-08-18 10:32:22 +02:00
Stan Girard
7281fd905a
feat(microservices): split into 4 quivr to better handle long services (#972) 
* feat(verbose): removed

* feat(microservices): split quivr into micro services

* test(main): fixed
 2023-08-18 10:18:29 +02:00
Stan Girard
ad7ac1516d
fix(windows): removed unused start script (#962) 2023-08-17 19:10:30 +02:00
Zineb El Bachiri
30060944d7
🐛 delete brain with custom prompt (#959) 2023-08-16 13:59:19 +02:00
Stan Girard
d7c5c79043
fix(crawler): using newspaper and fixed recursive by merging content (#955) 
* fix(crawler): using newspaper and fixed recursive by merging content

* feat(code-review): added feedback from code review
 2023-08-15 18:09:09 +02:00
Joey Wang
1fef9b0783
fix(chatMessages): Fix error on answering question (#953) 2023-08-15 15:59:30 +02:00
joelsprunger
59ddfb4882
fix(chat routes): use brain model, temp, and token (#902) 2023-08-11 20:44:48 +02:00
Tural Sadık
d4d19bbf0b
feat (926): make crawl recursively navigate linked pages (#927) 2023-08-11 08:20:12 +02:00
Mamadou DICKO
21db719796
fix(chat): update data keys (#923) 2023-08-10 17:35:30 +01:00
Mamadou DICKO
6e777327aa
feat(chatMessages): add brain_id and prompt_id columns (#912) 
* feat: add prompt_id and brain_id to chat history)

* feat: add prompt_id and brain_id to chat routes
 2023-08-10 10:25:08 +02:00
Stan Girard
9ba724168e
fix(sentry): some unhandled errors (#894) 
* fix(answers): fixed with self.qa not existing anymore

* fix(crawling): fixed when it bugs
 2023-08-08 16:15:43 +01:00
joelsprunger
b31924f71d
debug by running main.py in the backend (#888) 
Co-authored-by: Joel Sprunger <joel@lendertoolkit.com>
 2023-08-08 16:01:31 +01:00
Stan Girard
7028505571
feat(qa): improve code (#886) 
* feat(qa): improve code

* feat: 🎸 customprompt

now in system
 2023-08-07 19:53:04 +02:00
ChloeMouret
e9ebeef72a
feat: gpt4 is not available for brains if there is no given openAiKey (#850) 
* rename defineMaxToken

* use gpt-3.5-turbo instead of gpt-3.5-turbo-0613

* gpt4 not available if no open ai key
 2023-08-07 16:35:23 +02:00
Mamadou DICKO
61cd0a6bde
feat(backend): add custom prompt (#885) 2023-08-07 16:31:00 +02:00
Stan Girard
6d74aea729
Revert "Revert "feat: remove private prompts on related brain delete (#842)" (#870)" (#876) 
This reverts commit 6184dfadfe.
 2023-08-07 10:15:38 +02:00
renovate[bot]
8eb564f5bc
chore(deps): update dependency openai to v0.27.8 (#867) 
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
 2023-08-07 10:11:48 +02:00
renovate[bot]
43597b53d8
chore(deps): update dependency strenum to v0.4.15 (#875) 
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
 2023-08-07 10:09:42 +02:00
renovate[bot]
9f02d9480d
chore(deps): update dependency sqlalchemy to v2.0.19 (#874) 
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
 2023-08-07 10:09:18 +02:00
renovate[bot]
d3bf1bb1a5
chore(deps): update dependency markdown to v3.4.4 (#866) 
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
 2023-08-07 10:08:09 +02:00
Stan Girard
10d4d65c1e
perf(deps): removed (#873) 2023-08-07 10:06:58 +02:00
Stan Girard
6184dfadfe
Revert "feat: remove private prompts on related brain delete (#842)" (#870) 
This reverts commit 4c15fe2bfd.
 2023-08-07 09:16:43 +02:00
Mamadou DICKO
4c15fe2bfd
feat: remove private prompts on related brain delete (#842) 
* feat: remove private prompts on brain deletion

* ui: improve personnality picker
 2023-08-07 08:13:03 +02:00
renovate[bot]
3ebfc6868f
chore(deps): update dependency pypdf to v3.9.0 [security] (#861) 
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
 2023-08-07 00:55:43 +02:00
Mamadou DICKO
b92d058fbe
test(backend): fix failing tests (#851) 2023-08-04 20:25:09 +02:00
Stan Girard
69e2c289e5
fix(pg-database): by default variable is not implemented (#848) 2023-08-04 10:17:26 +02:00
ChloeMouret
eaa574dd19
fix upload (#847) 2023-08-04 09:58:43 +02:00
ChloeMouret
711e9fb8c9
refactor: delete common_dependencies function (#843) 
* use function for get_documents_vector_store

* use function for get_embeddings

* use function for get_supabase_client

* use function for get_supabase_db

* delete lasts common_dependencies
 2023-08-03 20:24:42 +02:00
ChloeMouret
fdcdf581a8
refactor: clean backend, add typing, delete dead code (#835) 
* type common dependencies

* delete unused code
 2023-08-03 16:20:36 +02:00
Laurens Van Keer
659e585145
fix(chats): delete chats from correct table (#834) 2023-08-03 12:15:33 +02:00
Laurens Van Keer
68f03b2416
fix(chat routes): HTTPException import correction (#833) 
* fix(chat routes): HTTPException import correction

Solves "TypeError: HTTPException() takes no keyword arguments"

* fix(chat routes): error in user id comparison in metadata endpoint

current_user.id is a UUID and must be converted to a string before comparison with the chat.user_id string
 2023-08-03 12:11:25 +02:00
Mamadou DICKO
4ca6c667da
feat(backend): implement brain-prompt link (#831) 
* feat: add prompt_id field to brain

* feat(Prompt controller): update prompt routes

* feat: remove unused private prompts

* refactor: add BrainEntity and repo and service

* tests: partially type main Repository

* feat: add PromptStatusEnum enum

* feat: change delete prompt repository return type
 2023-08-03 10:37:13 +02:00
Mamadou DICKO
e3b6114248
feat(prompt): add prompt table, entity and repository (#823) 
* feat: add prompts table

* feat: add Prompt entity

* feat: add prompt router

* refactor(promptRepository): use common reposority
 2023-08-03 09:53:38 +02:00
Stan Girard
c21797905d
feat(prompt): added instructions in standalone question & a bit more things (#826) 
* fix(backend): temperature signature

* feat(qa): fixed temperature question generator and added doc

* feat(prompt): added instructions to standalone question

* fix(brains-rights): fixed function missing
 2023-08-02 10:31:42 +02:00
Noe
303ba72028
feat: Introduce repository pattern to prepare adding other database providers (#646) 
* add sqlalchemy models

* add neon settings

* add insert brain

* abstract supabase from Brain class

* abstract supabase from Brain class

* abstract supabase from /models

* update Database to Repository

* update neon_tables to pg_tables

* update chat, api-key and message

* update vector class

* update settings

* update env vars for test

* Update backend-tests.yml

* fix test

* fix fetch_user_requests_count()

* fix fetch_user_requests_count()

* fix increment_user_request_count

* fix increment_user_request_count

* fix asset upload_response message

* fix pyright

* fix brain_subscription

* fix brain_subscription

* fix brain_subscription

* fix get user request stat

* update create_brain_user

* add delete brain vector and user

* add delete brain vector and user

* correctly call function

---------

Co-authored-by: Noé Pion <noe.pion@onfido.com>
Co-authored-by: raoufchebri <raouf@chebri.com>
Co-authored-by: Stan Girard <girard.stanislas@gmail.com>
 2023-08-01 23:03:47 +02:00
Mamadou DICKO
edcbb30e97
fix: bugs (#818) 
* feat: add chat config modal

* feat: save chat config in localStorage

* feat: remove <ConfigPage/>

* feat: overwrite chat config with brain

* fix(SettingsPage): upload payload keys

* fix: update default brain marker logic

* feat: set new created brain as current selected brain
 2023-08-01 16:25:02 +02:00
Stan Girard
c01433c841 fix(openai): user key now used for llm model 2023-08-01 13:46:53 +02:00
Mamadou DICKO
7532b558c7
feat: add user level open ai key management (#805) 
* feat: add user user identity table

* feat: add user openai api key input

* feat: add encryption missing message

* chore: log more details about 422 errors

* docs(API): update api creation path

* feat: use user openai key if defined
 2023-08-01 09:24:57 +02:00
Stan Girard
3166d089ee
feat(chat): added streaming (#808) 
* feat(tmp): added streaming

* feat(streaming): implemented by changing order
 2023-07-31 21:34:34 +02:00
Stan Girard
db40f3cccd
feat(llm): removing all llms to prepare for genoss (#804) 
* feat(routes): removed all except openai

* feat(deadcode): removed some deadcode and summarization feature that wasn't used

* feat(streaming): removed privateGPT from it

* chore(requirements): increased version
 2023-07-31 16:01:34 +02:00
Mamadou DICKO
3529222b95
Brain management 4 (#762) 
* feat: add <ApiKeyConfig/>

* feat(SDK): add update brain

* feat: add removeUndefined helper

* feat: remove unnecessary autofocus flag

* add brain settings tab

* ui: add tab delimitor

* feat: improve ux
 2023-07-25 23:12:46 +02:00
Zineb El Bachiri
e05f25b025
Feat/update brain fields (#756) 
* 🗃️ update and add fields in brains table

* ✨ update endpoints for more brain attribute

* ✨ new set as default brain endpoint

* 🔥 remove update brain with file

* ✏️ fix wrong  auto imports

* 🐛 fix max tokens for model in front

* 🚑 post instead of put to set default brain

* 🚑 update brain creation endpoint with new fields
 2023-07-25 15:22:17 +02:00
Mamadou DICKO
25f859a31e
fix: make max_brain_size overwritable (#751) 2023-07-24 14:17:33 +02:00
Braden Mars
914689957d
fix(backend): FastAPI docs broken due to missing schema for NullableUUID (#739) 
Just updates `NullableUUID` to extend from `UUID`. This lets FastAPI infer the correct field schema and resolves the /docs and /redocs endpoint failing to render

Signed-off-by: Braden Mars <bradenmars@bradenmars.me>
 2023-07-23 11:14:42 +02:00
Mamadou DICKO
d27504f735
Feat/rate limiting (#719) 
* feat: add max brain count

* fix: prevent page cashing when invitation is invalid

* feat: rename rights to role in frontend
 2023-07-20 18:17:55 +02:00
Braden Mars
b3455d3686
fix(parsers.github): AttributeError when crawling github repository. (#667) 
When attempting to add new files from a crawled github repository, quivr will always fail due to a reference to a non-existant `File.add_files_to_brain` method.

Signed-off-by: Braden Mars <bradenmars@bradenmars.me>
 2023-07-20 16:13:25 +02:00
Mamadou DICKO
eb779f9e58
Minor UX improvments (#717) 
* feat: display user rights on invitation page

* feat: add brain name in invitation email
 2023-07-20 15:15:43 +02:00
Mamadou DICKO
d7ca11f5d1
feat: prevent editor to share as owner (#715) 
* feat(UploadPage): redirect to login if not connected

* feat: prevent editor to share as owner
 2023-07-20 15:15:32 +02:00
Mamadou DICKO
6c5016c79a
[ShareableBrain]: Feat: require owner role to delete content or remove user access (#711) 
* feat: require owner right to remove content from brain

* feat: require owner right to remove user access from brain
 2023-07-19 17:13:02 +02:00
Mamadou DICKO
aa7bc483c2
feat: check user role on brain before file upload (#709) 2023-07-19 13:41:46 +02:00
Mamadou DICKO
87458d8de1
Feat/brain access rights (#705) 
* refactor(BrainUsers)

* feat: give brain share access to EDITORs

* feat(RBAC): add role enum and supports multiple roles check

* feat: make owner right read only for other permissions
 2023-07-19 13:36:23 +02:00
Mamadou DICKO
7a2450eaf4
[ShareableBrain]: improve UX, fix minors bugs, refactor (#695) 
* feat: add empty access list message

* feat: set default role to viewer

* feat: reset user invitation form after submit

* feat: add removing access indicator

* feat: add brain name on invitation page

* feat: display brain name on chat page

* feat: clear localStorage on logout
 2023-07-18 18:28:44 +02:00
Zineb El Bachiri
1f5443bd24
🚑 fix resend invitation emails (#696) 2023-07-18 18:10:48 +02:00
Stan Girard
8abd979ca2 ci(docker): fixed 2023-07-18 16:54:31 +02:00
Stan Girard
4d3dbc565e feat(ci): fixed 2023-07-18 16:52:55 +02:00
Stan Girard
78fbe378e8 feat(prompt): added continue 2023-07-18 16:48:43 +02:00
Matt
2f5088547a test: fix ci 2023-07-18 15:34:08 +01:00
Stan Girard
d16b419035 feat(prompt): added continue 2023-07-18 16:04:56 +02:00
Mamadou DICKO
81b57c504a
Shareable brain 9 (#677) 
* feat: add 20230717173000_add_get_user_id_by_user_email

* feat(ShareableBrain): add update access endpoint

* feat(sdk): add updateBrainAccess

* feat: add brain access control

* feat: improve ux
 2023-07-18 14:30:19 +02:00
Zineb El Bachiri
0b091bd8c9
Feat/shareable brains open link authenticated (#676) 
* ♻️ use Single Responsibility Principle on brains_subscription

* ✨ new brain subscription endpoints for invited user

* 📝 add documentation to endpoints

* 🎨 add base_frontend_url to send custom url for brain share

* ✏️ brains instead of brain in url

* ✨ use origin in header for frontend url in subscription email

* 🚚 move and remove unused code

* ✨ new subscription API for BE endpoints in frontend

* ✨ new addBrain to  add a shared brain in frontend

* 🥚 new hook for brain invitations

* ✨ new page for brain invitation

* ✨ change frontend url to copy for brain subscription

* ✏️ call RBAC with wrapper function

* ✏️ last typos
 2023-07-18 09:47:59 +02:00
Stan Girard
f63136350b
feat(resend): added env variablers (#673) 
* feat(resend): added env variablers

* fix(resend): made optional
 2023-07-17 15:45:31 +02:00
Mamadou DICKO
430ab54479
Shareable brain 8 (#674) 
* feat(ShareableBrain): add get brain users endpoints

* feat(sdk): add getBrainUsers

* feat(ShareableBrain): display users with access

* feat: rename role to rights

* fix(Brain): fecth brains on auth status change
 2023-07-17 15:45:18 +02:00
Mamadou DICKO
4d00a1ec92
feat(shareBrain): use subscription delete endpoint for brain deletion (#634) 2023-07-17 14:38:34 +02:00
Matt
e61f437ce8
Feat/backend core (#656) 2023-07-17 07:57:27 +01:00
nicksan222
c4c15a497c
Fixed pandocs (#662) 2023-07-15 23:20:47 +02:00
Matt
9f2cd7b7b6
fix: bug if no brain (#635) 2023-07-14 20:02:26 +01:00
Matt
211740b400
fix: defined executable for windows/linux users (#652) 2023-07-14 18:24:09 +02:00
Mamadou DICKO
fc76f9d466
feat(shareBrain): add unsubscribe endpoint (#633) 2023-07-13 18:05:36 +02:00
Mamadou DICKO
f65044e152
fix(RBAC): use dependencies (#629) 2023-07-13 17:54:23 +02:00
Mamadou DICKO
83fe9430d0
feat(RBAC): remove decorators (#627) 2023-07-13 12:12:02 +02:00
Mamadou DICKO
029f7b75df
Fix: explore routes RBAC (#613) 
* fix(explore_routes): update RBAC logic

* test(backend): update test_upload_explore_and_delete_file_pdf
 2023-07-13 11:02:52 +02:00
Matt
cef45ea712
Fix: change vector id to UUID (#609) 
* fix: document upload

* feat: explore fix to use uuid id

* chore: remove prints

* fix: tables.sql
 2023-07-12 11:44:34 +01:00
Stan Girard
f9a04ffbe2 test(backend): added csv and pdf upload 2023-07-12 10:03:07 +02:00
Matt
8fbb4b2d91
fix: gpt4all (#595) 
* fix: gpt4all

* fix: pyright

* Update backend/llm/openai.py

* fix: remove backend tag

* fix: typing

* feat: qa_base class

* fix: pyright

* fix: model_path not found
 2023-07-11 20:15:56 +02:00
Zineb El Bachiri
f837a6e9b9
Feat/shareable brains send link be (#599) 
* 🗃️ new table for invitations to subscribe to brain

* ✨ new BrainSubscription class

* ✨ new subscription router

* 👽️ add RESEND_API_KEY to .env in BE

* 📦 add 'resend' lib to requirements

* ♻️ fix some stanGPT
 2023-07-11 18:20:31 +02:00
Matt
cf37666f02
feat: tests get default brain (#593) 
* feat: tests get default brain

* feat: chains docs

* feat: brains docs

* fix: remove brain_id creation from fe

* fix: docs categories
 2023-07-11 12:09:56 +02:00
Mamadou DICKO
72924b52bc
[Authorization] Check user role before operations (#588) 
* feat(security): add RBAC on /explore/*

* feat(security): add RBAC on /brains/*
 2023-07-11 10:00:06 +02:00
Maria Romanenko
1be71e9e79
docs(backend): add docstrings (#590) 2023-07-10 19:28:38 +02:00
Mamadou DICKO
9e9f531c99
Feat/static analysis (#582) 
* feat: add static analysis

* chore: update Makefile add static analysis script

* chore: add vscode extensions recommandations
 2023-07-10 14:27:49 +02:00
Stan Girard
fa386d7d5c feat(prompt): improved 2023-07-07 16:58:49 +02:00
Matt
5735a8ec8c
feat: email edge function (#545) 2023-07-07 00:54:21 +02:00
Zineb El Bachiri
9e942ba959
✨ add cors security headers (#533) 2023-07-06 19:01:38 +02:00
Matt
f352005dcf
Fix: add missing properties (#530) 2023-07-06 09:52:47 +01:00
Zineb El Bachiri
0edc4f628c
🚑 use multithreading instead of multiprocessing for container in ECS (#525) 2023-07-05 18:15:18 +02:00
Zineb El Bachiri
01ea71aca8
🐛 send select request supabase in batches with multiprocessing (#518) 2023-07-05 14:37:01 +02:00
Matt
f4ba4d9d18
fix: jwt decode to return user object (#513) 2023-07-05 08:27:58 +01:00
Matt
e931d29017
feat: new classes (#499) 
* feat: new classes

* chore: private llm refactor and enable streaming

* fix: validation errors in pytest

* fix: default brain behavior

* Update backend/llm/base.py

* chore: remove 🪦

* fix: remove 🪦
 2023-07-04 16:56:54 +01:00
Stan Girard
dcaa70a947
feat(functions): improved by removing only get_context (#459) 2023-07-03 12:34:37 +02:00
Matt
b33fcc8bf7
feat: base class for llms (#462) 2023-07-03 11:16:36 +02:00
Stan Girard
ce18e033c1
ci(test): added test on endpoints (#454) 2023-07-02 15:37:23 +02:00
Stan Girard
b444761622 fix(sentry): added fix 2023-07-02 10:54:36 +02:00
Stan Girard
3918ad3015 fix(sentry): removed from variables 2023-07-02 10:26:16 +02:00
Stan Girard
e076bbe79f
Feat/testing backend (#446) 
* feat(pytest): added

* feat(brains): added tests

* feat(actions): pytest
 2023-07-02 02:19:30 +02:00
Stan Girard
bab76ba7e5
fix(apikey): fixed with bad casting (#444) 2023-07-01 23:30:14 +02:00
Stan Girard
fbd1e17018
feat(sentry): added sentry (#443) 2023-07-01 21:12:13 +02:00
Stan Girard
b76262c4f0
Fix a bunch of issues (#442) 
* fix(tokens): overflow

* fix(chat): fixed deletion error of the chat

#441

* fix(default-brain): autoset default brain when on upload page

* fix(promptengineering): changed prompt a bit

* feat(growthbook): removed

* docs(readme): updated
 2023-07-01 17:22:28 +02:00
Zineb El Bachiri
81321ea615
🚑 fix on fetch user & csv upload (#438) 
* 🚑 fix on fetch user

* 🚑 fix imports for csv
 2023-06-30 12:15:50 +02:00
Zineb El Bachiri
a13219c73f
🚑 quick fix for brain temporary redirect (#427) 2023-06-30 10:57:17 +02:00
Matt
6f047f4a39
feat: streaming for standard brain picking (#385) 
* feat: streaming for standard brain picking

* fix(bug): private llm

* wip: test

Co-authored-by: Mamadou DICKO <mamadoudicko@users.noreply.github.com>

* wip: almost good

Co-authored-by: Mamadou DICKO <mamadoudicko@users.noreply.github.com>

* feat: useFetch

* chore: remove 💀

* chore: fix linting

* fix: forward the request if not streaming

* feat: streaming for standard brain picking

* fix(bug): private llm

* wip: test

Co-authored-by: Mamadou DICKO <mamadoudicko@users.noreply.github.com>

* wip: almost good

Co-authored-by: Mamadou DICKO <mamadoudicko@users.noreply.github.com>

* feat: useFetch

* chore: remove 💀

* chore: fix linting

* fix: forward the request if not streaming

* fix: 💀 code

* fix: check_user_limit

* feat: brain_id to new chat stream

* fix: missing imports

* feat: message_id created on backend

Co-authored-by: Mamadou DICKO <mamadoudicko@users.noreply.github.com>

* chore: remove dead

* remove: cpython

* remove: dead

---------

Co-authored-by: Mamadou DICKO <mamadoudicko@users.noreply.github.com>
 2023-06-30 10:10:59 +02:00
Zineb El Bachiri
056a68d5ed
Fix/prod (#425) 
* 🐛 update db tables script

* ✏️ fix users routes

* 🐛 add '/' AGAIN in URLs
 2023-06-30 10:09:50 +02:00
Zineb El Bachiri
b016413099
✏️ add / in all explore url (#415) 
* ✏️ add / in all explore url

* 🐛 fix user routes

* ⚰️ remove dead code

* ✨ add name to active brain
 2023-06-30 09:08:40 +02:00
Zineb El Bachiri
4d9bd512ec
Fix/file upload explore (#412) 2023-06-29 18:26:03 +02:00
Stan Girard
3946dbc668 fix(crawler): route 2023-06-29 08:00:34 +02:00
Zineb El Bachiri
ccdc5bb7a7
Feat/multiple brains files (#361) 2023-06-28 19:39:27 +02:00
Michael Tan
0888bc5da2
(fix) temporary audio file not removed. (#377) 
* (fix) temporary audio file not removed.

In audio.py file:
texts = text_splitter.split_text(transcript)

Threw an exception.
TypeError: expected string or bytes-like object, got 'OpenAIObject'

This causes the temporary file to persist.

Signed-off-by: Michael Tan <tan.michael@mac.com>

* Additionally fix the reason why text splitting threw the exception.

Default response format is JSON.

Signed-off-by: Michael Tan <tan.michael@mac.com>

---------

Signed-off-by: Michael Tan <tan.michael@mac.com>
 2023-06-28 09:47:59 +02:00
Stan Girard
00d7ec131c fix(vectorstore): removed userid 2023-06-27 11:44:23 +02:00
Stan Girard
81a3d48fbc
fix(vectorstore): issues with userid (#380) 
* fix(vectorstore): issues with userid

* perf(analytics): added tracking for file upload and chat (#376)

* fix: conditionnaly update functions list (#379)

---------

Co-authored-by: Mamadou DICKO <63923024+mamadoudicko@users.noreply.github.com>
 2023-06-26 19:02:03 +02:00
Mamadou DICKO
e045a55ae7
fix: conditionnaly update functions list (#379) 2023-06-26 17:27:06 +02:00
Stan Girard
5fc837b250
feat(brainpicking): simplified (#371) 
* feat(functions): simplified

* refactor(openai): changed to brainpicking

* feat(functions): made them inherit from brainpicking

* feat(privatebrainpicking): added new class

* feat(history&context): added

* Delete test_brainpicking.py

* Delete __init__.py
 2023-06-26 10:34:03 +02:00
Stan Girard
572fc7e1b0 fix(openai): skip if api key 2023-06-23 17:59:53 +02:00
Matt
d9b2be19d7
feat: start script (#367) 
* feat: start script

* make faster
 2023-06-23 14:20:03 +02:00
Mamadou DICKO
3922d8ca83
Feat/error handling (#366) 
* feat: improve error handling

* docs: explain error handling system
 2023-06-23 10:36:55 +02:00
Mamadou DICKO
59fe7b089b
feat(chat): use openai function for answer (#354) 
* feat(chat): use openai function for answer (backend)

* feat(chat): use openai function for answer (frontend)

* chore: refacto BrainPicking

* feat: update chat creation logic

* feat: simplify chat system logic

* feat: set default method to gpt-3.5-turbo-0613

* feat: use user own openai key

* feat(chat): slightly improve prompts

* feat: add global error interceptor

* feat: remove unused endpoints

* docs: update chat system doc

* chore(linter): add unused import remove config

* feat: improve dx

* feat: improve OpenAiFunctionBasedAnswerGenerator prompt
 2023-06-22 17:50:06 +02:00
Matt
83fde0aeea
feat: private llm (#360) 
* feat: private llm

* Update backend/vectorstore/supabase.py

* Update backend/vectorstore/supabase.py
 2023-06-22 09:45:35 +01:00
Stan Girard
3d11e3fb51
refactor(brain): removed braintoupdate and added brain (#356) 
* refactor(brain): removed braintoupdate and added brain

* feat(brain): updated
 2023-06-21 10:16:44 +02:00
Matt
3e753f2d56
feat: stream router (#353) 
* wip: stream router

* feat: chatai streaming

* chore: add comments

* feat: streaming for chains

* chore: comments
 2023-06-20 21:53:04 +02:00
Zineb El Bachiri
9c8e0aa0e4
Feat/multiple brains frontend (#344) 
* 🗑️ remove date input from fetch_user_id_from_credentials

* ♻️ refactor backend utils by splitting it into files

* 💡 comments for next actions to update /upload

* 🚚 move SupabaseProvider tp libs

* 🚚 move useAxios to hooks

* ♻️ refacto brain routes

* 🚨 update lintermfor paths

* ✨ new brain context provider

* ✨ new brain component in navbar

* 🚨 fix linter and async

* 🇸🇪 add feature flag for multiple-brains
 2023-06-20 16:17:13 +02:00
Mamadou DICKO
e1a740472f
Feat: chat name edit (#343) 
* feat(chat): add name update

* chore(linting): add flake8

* feat: add chat name edit
 2023-06-20 09:54:23 +02:00
Stan Girard
8ed8a2c9ef chore: 🤖 docs 
added docs
 2023-06-20 08:56:17 +02:00
Stan Girard
533446a2b4 chore(refacto): removed unused 2023-06-20 00:14:40 +02:00
Stan Girard
99258790ad refactor(brainpicking): removed one function 2023-06-19 23:14:42 +02:00
Stan Girard
969e0b48a8 refactor(brains): into brainpicking 2023-06-19 22:55:42 +02:00
Stan Girard
c7fdbf6dec feat(refacto): removed commons 2023-06-19 22:54:01 +02:00
Stan Girard
72a6ae3dc0 feat(settings): refactored 2023-06-19 22:46:25 +02:00
Stan Girard
b8ec6fd342 refactor(brainsettings): added moved files 2023-06-19 21:21:26 +02:00
Stan Girard
c222f57b22 refactor(settings): moved to own folder 2023-06-19 21:20:08 +02:00
Stan Girard
dc6f610b26 feat(neurons): added class 2023-06-19 21:15:35 +02:00
Stan Girard
d42f14f431 feat(brainpicking): new class 2023-06-19 20:51:13 +02:00
Stan Girard
17aaf18d61 feat(vectorstore): own folder 2023-06-19 20:15:34 +02:00
Stan Girard
954acf1286 fix(llm): changed to stuff 2023-06-19 17:53:07 +02:00
Stan Girard
225280f2f5 fix(doc): retrieval to 8 because it takes a long time 2023-06-19 11:23:58 +02:00
Stan Girard
c0ff030100 feat(k): increased to 16 2023-06-17 01:32:03 +02:00
Stan Girard
65c44b3fb9 feat(chat): improved chain 2023-06-17 01:16:11 +02:00
Zineb El Bachiri
e53bc6807d
Feat/multiple brains backend (#340) 
* 🗃️ add new tables for multiple brains

* 🗑️ remove date input from fetch_user_id_from_credentials

* ✨ new /brain endpoints

* ♻️ refactor backend utils by splitting it into files

* 💡 comments for next actions to update /upload
 2023-06-16 23:36:53 +02:00
Stan Girard
f833de47bc feat(openai): 16k 2023-06-16 17:52:32 +02:00
Stan Girard
82c74186a8 fix(memory): added memory back 2023-06-15 15:25:12 +02:00
Ruben1701
8fb245fe2a
Improved api docs (#336) 
* auyto removed

* updated the api docs

* fixed missing logic in api_key_route + chat_route

* Update settings.json

---------

Co-authored-by: Ruben170 <35116541+Ruben170@users.noreply.github.com>
 2023-06-15 14:43:40 +02:00
Stan Girard
9af6a250e3 fix(model): broke something yesterday in QA 2023-06-14 22:23:05 +02:00
Stan Girard
f4e85db187 fix(llm): using wrong llm probably because of breaking change in langchain 2023-06-14 22:15:48 +02:00
Cezar Vasconcelos
3b7390dd61
Feat/add new openai llm models (#334) 
* feat(frontend): update OpenAi models and maxToken select's

* fix: update openai model to use the most recent

* fix: adjust max token for gpt-3 turbo

* fix: duplicating models

* fix: openai model type list
 2023-06-14 21:30:52 +02:00
Matt
33f49ee289
feat: user can create api keys (#329) 
* feat: user can create api keys

* fix: linting on build

* Update backend/routes/api_key_routes.py

* chore: rename and refactor AuthBearer

* chore: add types
 2023-06-14 21:21:13 +02:00
Arthur Busser
4546e8bc1d
✨ Improve the prompt sent to the LLM (#318) 
This is a more detailed prompt that explains what Quivr is and what it
does.
 2023-06-13 16:24:07 +02:00
Stan Girard
de8daa04bf fix(prompt): less picky 2023-06-13 14:43:53 +02:00
Stan Girard
6708a7860a fix(prompt): added prompt qa to anthropic 2023-06-13 11:00:15 +02:00
Stan Girard
ab7dfbbaec fix(prompt): condense prompt 2023-06-13 10:59:24 +02:00
Stan Girard
9f0ba8bce7 fix(prompt): fixed qa prompt 2023-06-13 10:35:06 +02:00
Matt
ec29f30f32
Feat: backend refactor (#306) 
* fix: edge cases on migration scripts

* chore: remove unused deps.

* refactor: user_routes

* refactor: chat_routes

* refactor: upload_routes

* refactor: explore_routes

* refactor: crawl_routes

* chore(refactor): get current user

* refactor: more dead dependencies

* bug: wrap email in credentials dict.

---------

Co-authored-by: Stan Girard <girard.stanislas@gmail.com>
 2023-06-12 17:58:05 +02:00
Stan Girard
67530c13f2 fix(google): requirements 2023-06-12 17:37:58 +02:00
Stan Girard
44d067d28d fix(crawl): github 2023-06-11 00:30:19 +02:00
Zineb El Bachiri
4112699db5
Feat/user chat history (#275) 
* ♻️ refactor backend main routes

* 🗃️ new user_id uuid column in users table

* 🗃️ new chats table

* ✨ new chat endpoints

* ✨ change chat routes post to handle undef chat_id

* ♻️ extract components from chat page

* ✨ add chatId to useQuestion

* ✨ new ChatsList

* ✨ new optional dynamic route chat/{chat_id}

* 🩹 add setQuestion to speach utils

* feat: self supplied key (#286)

* feat(brain): increased size if api key and more

* fix(key): not displayed

* feat(apikey): now password input

* fix(twitter): handle wrong

* feat(chat): basic source documents support (#289)

* ♻️ refactor backend main routes

* 🗃️ new user_id uuid column in users table

* 🗃️ new chats table

* ✨ new chat endpoints

* ✨ change chat routes post to handle undef chat_id

* ♻️ extract components from chat page

* ✨ add chatId to useQuestion

* ✨ new ChatsList

* ✨ new optional dynamic route chat/{chat_id}

* 🩹 add setQuestion to speach utils

* 🎨 separate creation and update endpoints for chat

* 🩹 add feat(chat): basic source documents support

* ✨ add chatName upon creation and for chats list

* 💄 improve chatsList style

* User chat history and multiple chats (#290)

* ♻️ refactor backend main routes

* 🗃️ new user_id uuid column in users table

* 🗃️ new chats table

* ✨ new chat endpoints

* ✨ change chat routes post to handle undef chat_id

* ♻️ extract components from chat page

* ✨ add chatId to useQuestion

* ✨ new ChatsList

* ✨ new optional dynamic route chat/{chat_id}

* refactor(chat): use layout to avoid refetching all chats on every chat

* refactor(chat): useChats hook instead of useQuestion

* fix(chat): fix errors

* refactor(chat): better folder structure

* feat: self supplied key (#286)

* feat(brain): increased size if api key and more

* fix(key): not displayed

* feat(apikey): now password input

* fix(twitter): handle wrong

* feat(chat): basic source documents support (#289)

* style(chat): better looking sidebar

* resume merge

* fix(backend): add os and logger imports

* small fixes

* chore(chat): remove empty interface

* chore(chat): use const

* fix(chat): merge errors

* fix(chat): remove useSpeech args

* chore(chat): remove unused args

* fix(chat): remove duplicate components

---------

Co-authored-by: gozineb <zinebe@theodo.fr>
Co-authored-by: Matt <77928207+mattzcarey@users.noreply.github.com>
Co-authored-by: Stan Girard <girard.stanislas@gmail.com>
Co-authored-by: xleven <xleven@outlook.com>

* fix and refactor errors

* fix(fresh): installation issues

* chore(conflict): merged old code

* fix(multi-chat): use update endpoint

* feat(embeddings): now using users api key

---------

Co-authored-by: Matt <77928207+mattzcarey@users.noreply.github.com>
Co-authored-by: Stan Girard <girard.stanislas@gmail.com>
Co-authored-by: xleven <xleven@outlook.com>
Co-authored-by: Aditya Nandan <61308761+iMADi-ARCH@users.noreply.github.com>
Co-authored-by: iMADi-ARCH <nandanaditya985@gmail.com>
Co-authored-by: Mamadou DICKO <mamadoudicko100@gmail.com>
 2023-06-10 23:59:16 +02:00
xleven
5a0f8326df
feat(chat): basic source documents support (#289) 2023-06-10 10:43:44 +02:00
Stan Girard
399a3fe437 feat(brain): increased size if api key and more 2023-06-09 23:20:51 +02:00
Matt
a64df04eab
feat: self supplied key (#286) 2023-06-09 18:49:47 +02:00
Stan Girard
e0cf37791b feat(pdf): added new pdf miner that works 2023-06-06 11:18:33 +02:00
Stan Girard
963fb05682
feat(github): now github loader (#264) 2023-06-06 00:38:15 +02:00
Stan Girard
90b2393b26 fix(zineb): she broke everything 2023-06-05 18:26:23 +02:00
Zineb El Bachiri
a5c71be731
Feat/user page (#255) 
* ✨ new backend get /user endpoint

* ✨ new user page for statistics

* 📦 add @mui/material & its dependencies
📦 add prettyBytes package

* 🌱 new UserStatistics component for user page

* 🏷️ use RequestStat type for requests_stats

* ✨ new brainConsumption component

* ✨ new Date component

* 📦 add  date-fns, victory

* ✨ requests per day chart

* ✨ full UserStatistics component

* ✨ put UserStatistics in Card in user's page
 2023-06-05 17:58:59 +02:00
Stan Girard
5a686a42cb fix(crawl): added error handling 2023-06-05 17:49:19 +02:00
Lecky Lao
fbae9016e2
fix(parser): process audio wrong argument (#243) 2023-06-04 11:53:49 +02:00
Stan Girard
a3ca7ecb37
Back/refacto files (#240) 
* feat(docker): added docker for prod

* feat(refacto): moved to modules
 2023-06-03 23:12:42 +02:00
Stan Girard
f2164db12c feat(cors): fix 2023-06-03 19:17:48 +02:00
Stan Girard
5323d93dc1 feat(api): cors 2023-06-03 19:07:50 +02:00
Stan Girard
ce11a9d54a feat(cors): changed to production value 2023-06-02 11:15:12 +02:00
Zineb El Bachiri
c3bb0257f0
Feat/rate limiter (#222) 
* 🔧 new use rate limiting env variables

* 🗃️ new Users table for rate limiting

* ✨limit user's requests number per day

* ✨ limit user's brain size

* docs(readme): updated with new migration

---------

Co-authored-by: Stan Girard <girard.stanislas@gmail.com>
 2023-06-01 22:51:39 +02:00
Stan Girard
72c92b1a54
VertexAI Google Cloud Palm2 Support (#226) 
* feat(bard): added

* docs(readme): update

* chore(print): removed
 2023-06-01 16:01:27 +02:00
Stan Girard
e0de23e24d
Feat/single brain (#215) 
* feat(llm): update

* feat(singlebrain): added new table with user-id

* feat(user): get user from email

* feat(user_id): added search

* ✨ add user_id to most endpoints

* docs(readme): new script

---------

Co-authored-by: gozineb <zinebe@theodo.fr>
 2023-05-31 13:51:23 +02:00
Zineb El Bachiri
cc6ec8fee9
✨ new ODT document loader (#211) 2023-05-30 13:02:48 +02:00
Stan Girard
e11183ed07 chore(binary): removed 2023-05-26 08:11:34 +02:00
Shan
7adcdd3d51
Support for capitalized file extensions such as PDF (#146) 2023-05-25 23:58:38 +02:00
Stan Girard
327074c5d4
feat(auth): now application has authentication (#144) 
* feat(auth): backend authentification verification

* feat(auth): added to all endpoints

* feat(auth): added to all endpoints

* feat(auth): redirect if not connected

* chore(print): removed

* feat(login): redirect

* feat(icon): added

* chore(yarn): removed lock

* chore(gitignore): removed
 2023-05-24 22:21:22 +02:00
Stan Girard
4bdba525ca docs(readme): updated frontend 2023-05-24 16:43:26 +02:00
Stan Girard
f7e237905f
Feat auth (#142) 
* feat(supabase): added auth

* feat(auth): working for front

* fix(auth): now front can build

* feat(auth): nice screen
 2023-05-24 16:17:08 +02:00
Stan Girard
363528d2ce fix(cors): allow all origins 2023-05-22 15:18:25 +02:00
shaun
ec6f7332b9 make it optional 2023-05-22 00:12:46 -07:00
shaun
a52fcbe684 add missing column 2023-05-21 23:59:06 -07:00
shaun
389a70199a fix message 2023-05-21 23:56:11 -07:00
shaun
c38265a5f5 add summarization backend 2023-05-21 23:39:55 -07:00
shaun
a1693d94b2 Better envs 2023-05-21 21:18:55 -07:00
Murtaza
514f3b352a Add support for jupyter notebooks. 2023-05-22 00:42:41 +05:30
Stan Girard
70a70d0c17
Merge pull request #109 from dheerapat/fix-dockerfile-pip 
set pip timeout to 100 second (default 15)
 2023-05-21 18:54:51 +02:00
Stan Girard
56ba503cdf
url upload (#108) 
* feat(url): input area

* feat(upload): added cards next to each other

* feat(typing): typed urlinput
 2023-05-21 18:50:58 +02:00
Dheerapat Tookkane
020c41b986 set pip timeout to 100 second (default 15) 2023-05-21 23:11:00 +07:00
Stan Girard
d869f642c6
Merge branch 'main' into main 2023-05-21 16:21:34 +02:00
Valerie
a7089ae5bc support other prompt languages in new backend 2023-05-21 16:01:53 +02:00
Stan Girard
97bf4464ad
Merge branch 'main' into main 2023-05-21 09:34:31 +02:00
Evan Carlson
6dab1259ef
add docx2txt package for uploading word docs (#93) 2023-05-21 09:21:51 +02:00
Murtaza
1706538343 Add epub loader to parse epub uploads. 2023-05-21 11:45:31 +05:30
Stan Girard
f952d7a269
New Webapp migration (#56) 
* feat(v2): loaders added

* feature: Add scroll animations

* feature: upload ui

* feature: upload multiple files

* fix: Same file name and size remove

* feat(crawler): added

* feat(parsers): v2 added more

* feat(v2): audio now working

* feat(v2): all loaders

* feat(v2): explorer

* chore: add links

* feat(api): added status in return message

* refactor(website): remove old code

* feat(upload): return type for messages

* feature: redirect to upload if ENV=local

* fix(chat): fixed some issues

* feature: respect response type

* loading state

* feature: Loading stat

* feat(v2): added explore and chat pages

* feature: modal settings

* style: Chat UI

* feature: scroll to bottom when chatting

* feature: smooth scroll in chat

* feature(anim): Slide chat in

* feature: markdown chat

* feat(explorer): list

* feat(doc): added document item

* feat(explore): added modal

* Add clarification on Project API keys and web interface for migration scripts to Readme (#58)

* fix(demo): changed link

* add support to uploading zip file (#62)

* Catch UnicodeEncodeError exception (#64)

* feature: fixed chatbar

* fix(loaders): missing argument

* fix: layout

* fix: One whole chatbox

* fix: Scroll into view

* fix(build): vercel issues

* chore(streamlit): moved to own file

* refactor(api): moved to backend folder

* feat(docker): added docker compose

* Fix a bug where langchain memories were not being cleaned (#71)

* Update README.md (#70)

* chore(streamlit): moved to own file

* refactor(api): moved to backend folder

* docs(readme): updated for new version

* docs(readme): added old readme

* docs(readme): update copy dot env file

* docs(readme): cleanup

---------

Co-authored-by: iMADi-ARCH <nandanaditya985@gmail.com>
Co-authored-by: Matt LeBel <github@lebel.io>
Co-authored-by: Evan Carlson <45178375+EvanCarlson@users.noreply.github.com>
Co-authored-by: Mustafa Hasan Khan <65130881+mustafahasankhan@users.noreply.github.com>
Co-authored-by: zhulixi <48713110+zlxxlz1026@users.noreply.github.com>
Co-authored-by: Stanisław Tuszyński <stanislaw@tuszynski.me>
 2023-05-21 01:20:55 +02:00