StanGirard/quivr
1
1
Fork 0
mirror of https://github.com/StanGirard/quivr.git synced 2024-12-25 04:12:44 +03:00
Code Issues Packages Projects Releases Wiki Activity
939 Commits 837 Branches 349 Tags 203 MiB
db5a6e4b9b
Commit Graph

280 Commits

Author SHA1 Message Date
Stan Girard
54a34c2143
feat(aws): improved size (#1188) 2023-09-16 16:49:31 +02:00
Stan Girard
83286cc9a3
fix(notifications): greater than 5 minutes ago not less (#1187) 
* fix(notifications): dead notifications that are still present long after

* fix(notifications): greater than 5 minutes ago not less
 2023-09-16 16:42:50 +02:00
Stan Girard
f847ee84a9
fix(notifications): dead notifications that are still present long after (#1186) 2023-09-16 16:09:38 +02:00
Stan Girard
f362269600
feat(notificatins): higher refresh rate (#1184) 
* fix

* feat(notificatins): higher refresh rate
 2023-09-16 13:51:08 +02:00
Stan Girard
4fb8c34efd
fix(usersettings): Fix bugs with user settings in the back always gpt-3.5-turbo (#1183) 
* fix

* feat(gpt4): added support for gpt4
 2023-09-16 00:22:42 +02:00
Stan Girard
4d41901106
feat(perf): increased perf embedding and search for files (#1182) 
* feat(upload): changed to task

* feat(sha1): added column for better speed
 2023-09-15 23:39:29 +02:00
Gustavo J. Maciel
cdf587cfde
fix(qa_base): asign max_token to llm (#1179) 2023-09-15 18:51:53 +02:00
Stan Girard
980a704002
feat(upload): changed to task (#1178) 2023-09-15 15:52:06 +02:00
Stan Girard
2ad2788046 feat(aws): increase cpu and memory task 2023-09-14 17:43:28 +02:00
Stan Girard
f46ad68aa4 feat(broker): fixed url 2023-09-14 17:23:30 +02:00
Stan Girard
dda296a95d feat(error): added better upload error display 2023-09-14 16:46:55 +02:00
Stan Girard
1d33fbd3eb
feat(file-system): added queue and filesystem (#1159) 
* feat(queue): added

* feat(crawling): added queue

* fix(crawler): fixed github

* feat(docker): simplified docker compose

* feat(celery): added worker

* feat(files): now uploaded

* feat(files): missing routes

* feat(delete): added

* feat(storage): added policy and migrations

* feat(sqs): implemented

* feat(redis): added queue name variable

* fix(task): updated

* style(env): emoved unused env

* ci(tests): removed broken tests
 2023-09-14 11:56:59 +02:00
Stan Girard
322ee318be
feat: the good user management (#1158) 
* feat(user_management): added user management

* feat(user_management): added user management

* feat(user_management): removed print

* feat: use tanstack query for user data fecthing

* feat: add getUser to sdk

* feat: improve user page ux use tanstack query

* feat: fetch models from backend on brains settings page

* feat: update model selection on chat page

* feat: update tests

---------

Co-authored-by: mamadoudicko <mamadoudicko100@gmail.com>
 2023-09-13 13:47:12 +02:00
Mamadou DICKO
7cc90ef258
feat: add polling for pending notifications (#1152) 
* feat: add notification controller

* feat: add polling logic on pending notifications

* feat: refecth notifications on Feed
 2023-09-12 18:00:46 +02:00
Mamadou DICKO
10af0c949a
feat: add notifications components (#1148) 
* feat: rename ChatMessages to MessagesDialog

* feat: rename history to messages

* feat: add notifications to ChatContext

* feat: add getNotificationsFromChatHistory

* feat: add getMergedChatHistoryWithReducedNotifications

* refactor: update useActionBar

* refactor: update <ChatMessage />-n

* feat: update crawler and endpoint notifications content

* feat: display notifications

* test: update <MessageDialog /> tests

* feat: rename ChatMessage to QADisplay

* feat: rename ChatHistory to ChatMessage

* feat(upload): throw error when file missing

* feat: rename getMergedChatHistoryWithReducedNotifications to getMergedChatMessagesWithReducedNotifications

* feat: change history wording to message

* feat: move getFileIcon to lib

* refactor(NotificationDisplayer): move types to types.ts

* chore: improve ux

* feat: rename MessagesDialog to ChatDialogue
 2023-09-12 17:44:15 +02:00
Mamadou DICKO
72659709a8
fix: update crawl and upload endpoints (#1142) 2023-09-08 11:03:14 +02:00
Mamadou DICKO
9464707d40
feat: merge chat history with chat notifications (#1127) 
* feat: add chat_id to upload and crawl payload

* feat(chat): return chat_history_with_notifications

* feat: explicit notification status on create

* feat: handle notifications in frontend

* feat: delete chat notifications on chat delete request
 2023-09-07 17:23:31 +02:00
Mamadou DICKO
575d9886c5
feat: add notifications table, and push notification on upload and crawl (#1125) 
* feat: add notifications table

* feat: add Notification model

* feat: add notification repositories

* feat: add upload and crawl notifications

* feat: update notification message
 2023-09-07 13:22:06 +02:00
Ishaan Jaff
02964c4077
feat(liteLLM): Add support for Azure OpenAI, Palm, Claude-2, Llama2, CodeLlama (100+LLMs) (#1097) 
* v0 litellm

* bump versions
 2023-09-05 17:38:19 +02:00
Mamadou DICKO
bd2358f425
fix: fix minor bugs (#1070) 
* fix: update get_user_id_by_user_email type

* fix: update getEditorText logic

* feat(mentionSuggestions): render empty popover
 2023-08-31 10:41:02 +02:00
Mamadou DICKO
ba123fe716
feat(shareBrain): prevent re inviting users with access (#1063) 
* fix(shareBrain): prevent access duplication in db

* feat: remove unused function

* fix(linter): update types
 2023-08-30 15:55:06 +02:00
Joey Wang
30cb91531f
feat(brain): add endpoint to return context to question (#1044) 2023-08-27 09:38:41 +02:00
Mamadou DICKO
f1d6b7892c
test: unskip qa_headless.py linter tests (#1041) 2023-08-25 14:03:57 +02:00
Mamadou DICKO
252b1cf964
test: skip failing linter tests (#1036) 2023-08-25 12:03:13 +02:00
Pat Tran
43a00b06ec
fix(dockerfile): backend Dockerfile exit code 1 (#1032) 2023-08-25 11:05:24 +02:00
Zineb El Bachiri
0ca25e2af5
🚑 fix quivr api key types (#1022) 2023-08-22 18:10:17 +02:00
Mamadou DICKO
b967c2d2d6
feat: add brain prompt overwritting from chat (#1012) 2023-08-22 14:23:27 +02:00
joelsprunger
a626b84b96
feat(backend): adds python code parsing (#1003) 
Adds a python code parser and invokes it in processors.py
 2023-08-22 12:05:31 +02:00
Zineb El Bachiri
9aaedcff51
Fix/requests limiting (#980) 
* 🗃️ Rename users table into user_daily_usage

* 💥 replace User model with UserIdentity model

* 🗃️ New UserDailyUsage class for database interaction

* 🐛 fix daily requests rate limiting per user

* 🐛 fix user stats and properties update

* ✏️ add typing and linting

* 🚚 rename user_dialy_usage Class  into user_usage & requests_count into daily_requests_count

* 🚑 fix some rebase errors
 2023-08-21 14:05:13 +02:00
Mamadou DICKO
f61b70a34f
fix: remove conflicts (#998) 2023-08-21 12:58:22 +02:00
Zineb El Bachiri
3821502c6d
✨ add xlsx and xls parser (#997) 2023-08-21 12:56:48 +02:00
Mamadou DICKO
5a3a6fe370
fix: update backend tests (#992) 
* fix: update backend tests

* fix(pytest): update types
 2023-08-21 12:45:32 +02:00
MaHDi
8af6d61e76
improve (importing): reorganization of the import structure (#964) 
* reorganize import level

* add __init__, reorganize import from __init__

* reorganize import level

* reorganize import level

* fix circular import error by keep the import deep as "from models.settings"

* fix the relative import

* restor unwanted staged files

* add backend/venv and backend/.env to gitignore

* clean importing
 2023-08-21 12:25:16 +02:00
Stan Girard
ae7852ec3f
feat(health): added endpoint for services (#989) 2023-08-20 23:20:57 +02:00
Stan Girard
d0370ab499
feat(refacto): changed a bit of things to make better dx (#984) 2023-08-19 13:32:16 +02:00
Stan Girard
cbc8ac4946
Revert "fix: update backend tests (#975)" (#983) 
This reverts commit c746eb1830.
 2023-08-19 12:31:15 +02:00
Stan Girard
ce6b45e1ac
feat(preview): added service upload (#979) 
* feat(docker): improved size image

* feat(preview): added upload service

* ci(aws): using matrix
 2023-08-18 14:49:48 +02:00
Mamadou DICKO
c746eb1830
fix: update backend tests (#975) 
* fix: update backend tests

* fix(pytest): update types
 2023-08-18 14:01:14 +02:00
Stan Girard
aa623c4039
feat(docker): improved size image (#978) 2023-08-18 12:57:16 +02:00
Stan Girard
b894f8e7e5
ci(preview): fixed name service (#977) 
* feat(chat-service): added task definition

* feat(ci): added ci

* feat(ci): added preview chat
 2023-08-18 12:20:47 +02:00
Stan Girard
d1dfd6c7ec
Feat/microservice aws (#976) 
* feat(chat-service): added task definition

* feat(ci): added ci
 2023-08-18 12:02:57 +02:00
Zineb El Bachiri
74da7dde2d
Crawl/update env variables (#931) 
* 📝 add env variable for crawl

* 🐛 make CharQuestion with optional attributes

* 💡 make chatQustion config optional
 2023-08-18 11:59:20 +02:00
Stepan Lebedev
600ff1ede0
feat(Unplug): chatting without brain streaming (#970) 
* feat(Unplug): Adds new basic headless llm

* feat(Unplug): adds chatting without brain option when no streaming

* feat(Unplug): adds chatting without brain option when streaming
 2023-08-18 10:32:22 +02:00
Stan Girard
7281fd905a
feat(microservices): split into 4 quivr to better handle long services (#972) 
* feat(verbose): removed

* feat(microservices): split quivr into micro services

* test(main): fixed
 2023-08-18 10:18:29 +02:00
Stan Girard
ad7ac1516d
fix(windows): removed unused start script (#962) 2023-08-17 19:10:30 +02:00
Zineb El Bachiri
30060944d7
🐛 delete brain with custom prompt (#959) 2023-08-16 13:59:19 +02:00
Stan Girard
d7c5c79043
fix(crawler): using newspaper and fixed recursive by merging content (#955) 
* fix(crawler): using newspaper and fixed recursive by merging content

* feat(code-review): added feedback from code review
 2023-08-15 18:09:09 +02:00
Joey Wang
1fef9b0783
fix(chatMessages): Fix error on answering question (#953) 2023-08-15 15:59:30 +02:00
joelsprunger
59ddfb4882
fix(chat routes): use brain model, temp, and token (#902) 2023-08-11 20:44:48 +02:00
Tural Sadık
d4d19bbf0b
feat (926): make crawl recursively navigate linked pages (#927) 2023-08-11 08:20:12 +02:00
Mamadou DICKO
21db719796
fix(chat): update data keys (#923) 2023-08-10 17:35:30 +01:00
Mamadou DICKO
6e777327aa
feat(chatMessages): add brain_id and prompt_id columns (#912) 
* feat: add prompt_id and brain_id to chat history)

* feat: add prompt_id and brain_id to chat routes
 2023-08-10 10:25:08 +02:00
Stan Girard
9ba724168e
fix(sentry): some unhandled errors (#894) 
* fix(answers): fixed with self.qa not existing anymore

* fix(crawling): fixed when it bugs
 2023-08-08 16:15:43 +01:00
joelsprunger
b31924f71d
debug by running main.py in the backend (#888) 
Co-authored-by: Joel Sprunger <joel@lendertoolkit.com>
 2023-08-08 16:01:31 +01:00
Stan Girard
7028505571
feat(qa): improve code (#886) 
* feat(qa): improve code

* feat: 🎸 customprompt

now in system
 2023-08-07 19:53:04 +02:00
ChloeMouret
e9ebeef72a
feat: gpt4 is not available for brains if there is no given openAiKey (#850) 
* rename defineMaxToken

* use gpt-3.5-turbo instead of gpt-3.5-turbo-0613

* gpt4 not available if no open ai key
 2023-08-07 16:35:23 +02:00
Mamadou DICKO
61cd0a6bde
feat(backend): add custom prompt (#885) 2023-08-07 16:31:00 +02:00
Stan Girard
6d74aea729
Revert "Revert "feat: remove private prompts on related brain delete (#842)" (#870)" (#876) 
This reverts commit 6184dfadfe.
 2023-08-07 10:15:38 +02:00
renovate[bot]
8eb564f5bc
chore(deps): update dependency openai to v0.27.8 (#867) 
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
 2023-08-07 10:11:48 +02:00
renovate[bot]
43597b53d8
chore(deps): update dependency strenum to v0.4.15 (#875) 
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
 2023-08-07 10:09:42 +02:00
renovate[bot]
9f02d9480d
chore(deps): update dependency sqlalchemy to v2.0.19 (#874) 
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
 2023-08-07 10:09:18 +02:00
renovate[bot]
d3bf1bb1a5
chore(deps): update dependency markdown to v3.4.4 (#866) 
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
 2023-08-07 10:08:09 +02:00
Stan Girard
10d4d65c1e
perf(deps): removed (#873) 2023-08-07 10:06:58 +02:00
Stan Girard
6184dfadfe
Revert "feat: remove private prompts on related brain delete (#842)" (#870) 
This reverts commit 4c15fe2bfd.
 2023-08-07 09:16:43 +02:00
Mamadou DICKO
4c15fe2bfd
feat: remove private prompts on related brain delete (#842) 
* feat: remove private prompts on brain deletion

* ui: improve personnality picker
 2023-08-07 08:13:03 +02:00
renovate[bot]
3ebfc6868f
chore(deps): update dependency pypdf to v3.9.0 [security] (#861) 
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
 2023-08-07 00:55:43 +02:00
Mamadou DICKO
b92d058fbe
test(backend): fix failing tests (#851) 2023-08-04 20:25:09 +02:00
Stan Girard
69e2c289e5
fix(pg-database): by default variable is not implemented (#848) 2023-08-04 10:17:26 +02:00
ChloeMouret
eaa574dd19
fix upload (#847) 2023-08-04 09:58:43 +02:00
ChloeMouret
711e9fb8c9
refactor: delete common_dependencies function (#843) 
* use function for get_documents_vector_store

* use function for get_embeddings

* use function for get_supabase_client

* use function for get_supabase_db

* delete lasts common_dependencies
 2023-08-03 20:24:42 +02:00
ChloeMouret
fdcdf581a8
refactor: clean backend, add typing, delete dead code (#835) 
* type common dependencies

* delete unused code
 2023-08-03 16:20:36 +02:00
Laurens Van Keer
659e585145
fix(chats): delete chats from correct table (#834) 2023-08-03 12:15:33 +02:00
Laurens Van Keer
68f03b2416
fix(chat routes): HTTPException import correction (#833) 
* fix(chat routes): HTTPException import correction

Solves "TypeError: HTTPException() takes no keyword arguments"

* fix(chat routes): error in user id comparison in metadata endpoint

current_user.id is a UUID and must be converted to a string before comparison with the chat.user_id string
 2023-08-03 12:11:25 +02:00
Mamadou DICKO
4ca6c667da
feat(backend): implement brain-prompt link (#831) 
* feat: add prompt_id field to brain

* feat(Prompt controller): update prompt routes

* feat: remove unused private prompts

* refactor: add BrainEntity and repo and service

* tests: partially type main Repository

* feat: add PromptStatusEnum enum

* feat: change delete prompt repository return type
 2023-08-03 10:37:13 +02:00
Mamadou DICKO
e3b6114248
feat(prompt): add prompt table, entity and repository (#823) 
* feat: add prompts table

* feat: add Prompt entity

* feat: add prompt router

* refactor(promptRepository): use common reposority
 2023-08-03 09:53:38 +02:00
Stan Girard
c21797905d
feat(prompt): added instructions in standalone question & a bit more things (#826) 
* fix(backend): temperature signature

* feat(qa): fixed temperature question generator and added doc

* feat(prompt): added instructions to standalone question

* fix(brains-rights): fixed function missing
 2023-08-02 10:31:42 +02:00
Noe
303ba72028
feat: Introduce repository pattern to prepare adding other database providers (#646) 
* add sqlalchemy models

* add neon settings

* add insert brain

* abstract supabase from Brain class

* abstract supabase from Brain class

* abstract supabase from /models

* update Database to Repository

* update neon_tables to pg_tables

* update chat, api-key and message

* update vector class

* update settings

* update env vars for test

* Update backend-tests.yml

* fix test

* fix fetch_user_requests_count()

* fix fetch_user_requests_count()

* fix increment_user_request_count

* fix increment_user_request_count

* fix asset upload_response message

* fix pyright

* fix brain_subscription

* fix brain_subscription

* fix brain_subscription

* fix get user request stat

* update create_brain_user

* add delete brain vector and user

* add delete brain vector and user

* correctly call function

---------

Co-authored-by: Noé Pion <noe.pion@onfido.com>
Co-authored-by: raoufchebri <raouf@chebri.com>
Co-authored-by: Stan Girard <girard.stanislas@gmail.com>
 2023-08-01 23:03:47 +02:00
Mamadou DICKO
edcbb30e97
fix: bugs (#818) 
* feat: add chat config modal

* feat: save chat config in localStorage

* feat: remove <ConfigPage/>

* feat: overwrite chat config with brain

* fix(SettingsPage): upload payload keys

* fix: update default brain marker logic

* feat: set new created brain as current selected brain
 2023-08-01 16:25:02 +02:00
Stan Girard
c01433c841 fix(openai): user key now used for llm model 2023-08-01 13:46:53 +02:00
Mamadou DICKO
7532b558c7
feat: add user level open ai key management (#805) 
* feat: add user user identity table

* feat: add user openai api key input

* feat: add encryption missing message

* chore: log more details about 422 errors

* docs(API): update api creation path

* feat: use user openai key if defined
 2023-08-01 09:24:57 +02:00
Stan Girard
3166d089ee
feat(chat): added streaming (#808) 
* feat(tmp): added streaming

* feat(streaming): implemented by changing order
 2023-07-31 21:34:34 +02:00
Stan Girard
db40f3cccd
feat(llm): removing all llms to prepare for genoss (#804) 
* feat(routes): removed all except openai

* feat(deadcode): removed some deadcode and summarization feature that wasn't used

* feat(streaming): removed privateGPT from it

* chore(requirements): increased version
 2023-07-31 16:01:34 +02:00
Mamadou DICKO
3529222b95
Brain management 4 (#762) 
* feat: add <ApiKeyConfig/>

* feat(SDK): add update brain

* feat: add removeUndefined helper

* feat: remove unnecessary autofocus flag

* add brain settings tab

* ui: add tab delimitor

* feat: improve ux
 2023-07-25 23:12:46 +02:00
Zineb El Bachiri
e05f25b025
Feat/update brain fields (#756) 
* 🗃️ update and add fields in brains table

* ✨ update endpoints for more brain attribute

* ✨ new set as default brain endpoint

* 🔥 remove update brain with file

* ✏️ fix wrong  auto imports

* 🐛 fix max tokens for model in front

* 🚑 post instead of put to set default brain

* 🚑 update brain creation endpoint with new fields
 2023-07-25 15:22:17 +02:00
Mamadou DICKO
25f859a31e
fix: make max_brain_size overwritable (#751) 2023-07-24 14:17:33 +02:00
Braden Mars
914689957d
fix(backend): FastAPI docs broken due to missing schema for NullableUUID (#739) 
Just updates `NullableUUID` to extend from `UUID`. This lets FastAPI infer the correct field schema and resolves the /docs and /redocs endpoint failing to render

Signed-off-by: Braden Mars <bradenmars@bradenmars.me>
 2023-07-23 11:14:42 +02:00
Mamadou DICKO
d27504f735
Feat/rate limiting (#719) 
* feat: add max brain count

* fix: prevent page cashing when invitation is invalid

* feat: rename rights to role in frontend
 2023-07-20 18:17:55 +02:00
Braden Mars
b3455d3686
fix(parsers.github): AttributeError when crawling github repository. (#667) 
When attempting to add new files from a crawled github repository, quivr will always fail due to a reference to a non-existant `File.add_files_to_brain` method.

Signed-off-by: Braden Mars <bradenmars@bradenmars.me>
 2023-07-20 16:13:25 +02:00
Mamadou DICKO
eb779f9e58
Minor UX improvments (#717) 
* feat: display user rights on invitation page

* feat: add brain name in invitation email
 2023-07-20 15:15:43 +02:00
Mamadou DICKO
d7ca11f5d1
feat: prevent editor to share as owner (#715) 
* feat(UploadPage): redirect to login if not connected

* feat: prevent editor to share as owner
 2023-07-20 15:15:32 +02:00
Mamadou DICKO
6c5016c79a
[ShareableBrain]: Feat: require owner role to delete content or remove user access (#711) 
* feat: require owner right to remove content from brain

* feat: require owner right to remove user access from brain
 2023-07-19 17:13:02 +02:00
Mamadou DICKO
aa7bc483c2
feat: check user role on brain before file upload (#709) 2023-07-19 13:41:46 +02:00
Mamadou DICKO
87458d8de1
Feat/brain access rights (#705) 
* refactor(BrainUsers)

* feat: give brain share access to EDITORs

* feat(RBAC): add role enum and supports multiple roles check

* feat: make owner right read only for other permissions
 2023-07-19 13:36:23 +02:00
Mamadou DICKO
7a2450eaf4
[ShareableBrain]: improve UX, fix minors bugs, refactor (#695) 
* feat: add empty access list message

* feat: set default role to viewer

* feat: reset user invitation form after submit

* feat: add removing access indicator

* feat: add brain name on invitation page

* feat: display brain name on chat page

* feat: clear localStorage on logout
 2023-07-18 18:28:44 +02:00
Zineb El Bachiri
1f5443bd24
🚑 fix resend invitation emails (#696) 2023-07-18 18:10:48 +02:00
Stan Girard
8abd979ca2 ci(docker): fixed 2023-07-18 16:54:31 +02:00
Stan Girard
4d3dbc565e feat(ci): fixed 2023-07-18 16:52:55 +02:00
Stan Girard
78fbe378e8 feat(prompt): added continue 2023-07-18 16:48:43 +02:00
Matt
2f5088547a test: fix ci 2023-07-18 15:34:08 +01:00
Stan Girard
d16b419035 feat(prompt): added continue 2023-07-18 16:04:56 +02:00