Commit Graph

51 Commits

Author SHA1 Message Date
Zineb El Bachiri
be7acf052b
feat: get files from storage (#1205)
* 🌱 list files in storage & generate signed URL

*  add knowledge router

* 🗃️ add knowledge tables

*  add knowledge during upload

* 🚧 add knowledge a brain_knowledge models and repo

* 🔥 remove brain_knowledge

*  add upload to knowledge table

*  add crawl to knowledge table

* ✏️ fixes
2023-09-20 09:35:37 +02:00
Stan Girard
8914c7c357
fix(chats): now in order and with a little bonus ;) (#1200)
* fix(chats): now in order and with a little bonus ;)

* style(eslint): fixed
2023-09-18 18:25:28 +02:00
Stan Girard
2e4fdc80ec
feat(concurrency): added concurrency for increased performance (#1189) 2023-09-17 22:36:42 +02:00
Stan Girard
83286cc9a3
fix(notifications): greater than 5 minutes ago not less (#1187)
* fix(notifications): dead notifications that are still present long after

* fix(notifications): greater than 5 minutes ago not less
2023-09-16 16:42:50 +02:00
Stan Girard
f847ee84a9
fix(notifications): dead notifications that are still present long after (#1186) 2023-09-16 16:09:38 +02:00
Stan Girard
4fb8c34efd
fix(usersettings): Fix bugs with user settings in the back always gpt-3.5-turbo (#1183)
* fix

* feat(gpt4): added support for gpt4
2023-09-16 00:22:42 +02:00
Stan Girard
4d41901106
feat(perf): increased perf embedding and search for files (#1182)
* feat(upload): changed to task

* feat(sha1): added column for better speed
2023-09-15 23:39:29 +02:00
Stan Girard
2ad2788046 feat(aws): increase cpu and memory task 2023-09-14 17:43:28 +02:00
Stan Girard
1d33fbd3eb
feat(file-system): added queue and filesystem (#1159)
* feat(queue): added

* feat(crawling): added queue

* fix(crawler): fixed github

* feat(docker): simplified docker compose

* feat(celery): added worker

* feat(files): now uploaded

* feat(files): missing routes

* feat(delete): added

* feat(storage): added policy and migrations

* feat(sqs): implemented

* feat(redis): added queue name variable

* fix(task): updated

* style(env): emoved unused env

* ci(tests): removed broken tests
2023-09-14 11:56:59 +02:00
Stan Girard
322ee318be
feat: the good user management (#1158)
* feat(user_management): added user management

* feat(user_management): added user management

* feat(user_management): removed print

* feat: use tanstack query for user data fecthing

* feat: add getUser to sdk

* feat: improve user page ux use tanstack query

* feat: fetch models from backend on brains settings page

* feat: update model selection on chat page

* feat: update tests

---------

Co-authored-by: mamadoudicko <mamadoudicko100@gmail.com>
2023-09-13 13:47:12 +02:00
Mamadou DICKO
9464707d40
feat: merge chat history with chat notifications (#1127)
* feat: add chat_id to upload and crawl payload

* feat(chat): return chat_history_with_notifications

* feat: explicit notification status on create

* feat: handle notifications in frontend

* feat: delete chat notifications on chat delete request
2023-09-07 17:23:31 +02:00
Mamadou DICKO
575d9886c5
feat: add notifications table, and push notification on upload and crawl (#1125)
* feat: add notifications table

* feat: add Notification model

* feat: add notification repositories

* feat: add upload and crawl notifications

* feat: update notification message
2023-09-07 13:22:06 +02:00
Mamadou DICKO
ba123fe716
feat(shareBrain): prevent re inviting users with access (#1063)
* fix(shareBrain): prevent access duplication in db

* feat: remove unused function

* fix(linter): update types
2023-08-30 15:55:06 +02:00
Joey Wang
30cb91531f
feat(brain): add endpoint to return context to question (#1044) 2023-08-27 09:38:41 +02:00
Mamadou DICKO
252b1cf964
test: skip failing linter tests (#1036) 2023-08-25 12:03:13 +02:00
Zineb El Bachiri
0ca25e2af5
🚑 fix quivr api key types (#1022) 2023-08-22 18:10:17 +02:00
Mamadou DICKO
b967c2d2d6
feat: add brain prompt overwritting from chat (#1012) 2023-08-22 14:23:27 +02:00
Zineb El Bachiri
9aaedcff51
Fix/requests limiting (#980)
* 🗃️ Rename users table into user_daily_usage

* 💥 replace User model with UserIdentity model

* 🗃️ New UserDailyUsage class for database interaction

* 🐛 fix daily requests rate limiting per user

* 🐛 fix user stats and properties update

* ✏️ add typing and linting

* 🚚 rename user_dialy_usage Class  into user_usage & requests_count into daily_requests_count

* 🚑 fix some rebase errors
2023-08-21 14:05:13 +02:00
MaHDi
8af6d61e76
improve (importing): reorganization of the import structure (#964)
* reorganize import level

* add __init__, reorganize import from __init__

* reorganize import level

* reorganize import level

* fix circular import error by keep the import deep as "from models.settings"

* fix the relative import

* restor unwanted staged files

* add backend/venv and backend/.env to gitignore

* clean importing
2023-08-21 12:25:16 +02:00
Stan Girard
d0370ab499
feat(refacto): changed a bit of things to make better dx (#984) 2023-08-19 13:32:16 +02:00
Matt
e61f437ce8
Feat/backend core (#656) 2023-07-17 07:57:27 +01:00
Matt
9f2cd7b7b6
fix: bug if no brain (#635) 2023-07-14 20:02:26 +01:00
Mamadou DICKO
fc76f9d466
feat(shareBrain): add unsubscribe endpoint (#633) 2023-07-13 18:05:36 +02:00
Matt
cef45ea712
Fix: change vector id to UUID (#609)
* fix: document upload

* feat: explore fix to use uuid id

* chore: remove prints

* fix: tables.sql
2023-07-12 11:44:34 +01:00
Stan Girard
f9a04ffbe2 test(backend): added csv and pdf upload 2023-07-12 10:03:07 +02:00
Matt
8fbb4b2d91
fix: gpt4all (#595)
* fix: gpt4all

* fix: pyright

* Update backend/llm/openai.py

* fix: remove backend tag

* fix: typing

* feat: qa_base class

* fix: pyright

* fix: model_path not found
2023-07-11 20:15:56 +02:00
Zineb El Bachiri
f837a6e9b9
Feat/shareable brains send link be (#599)
* 🗃️ new table for invitations to subscribe to brain

*  new BrainSubscription class

*  new subscription router

* 👽️ add RESEND_API_KEY to .env in BE

* 📦 add 'resend' lib to requirements

* ♻️ fix some stanGPT
2023-07-11 18:20:31 +02:00
Matt
cf37666f02
feat: tests get default brain (#593)
* feat: tests get default brain

* feat: chains docs

* feat: brains docs

* fix: remove brain_id creation from fe

* fix: docs categories
2023-07-11 12:09:56 +02:00
Mamadou DICKO
72924b52bc
[Authorization] Check user role before operations (#588)
* feat(security): add RBAC on /explore/*

* feat(security): add RBAC on /brains/*
2023-07-11 10:00:06 +02:00
Maria Romanenko
1be71e9e79
docs(backend): add docstrings (#590) 2023-07-10 19:28:38 +02:00
Mamadou DICKO
9e9f531c99
Feat/static analysis (#582)
* feat: add static analysis

* chore: update Makefile add static analysis script

* chore: add vscode extensions recommandations
2023-07-10 14:27:49 +02:00
Zineb El Bachiri
01ea71aca8
🐛 send select request supabase in batches with multiprocessing (#518) 2023-07-05 14:37:01 +02:00
Matt
e931d29017
feat: new classes (#499)
* feat: new classes

* chore: private llm refactor and enable streaming

* fix: validation errors in pytest

* fix: default brain behavior

* Update backend/llm/base.py

* chore: remove 🪦

* fix: remove 🪦
2023-07-04 16:56:54 +01:00
Stan Girard
e076bbe79f
Feat/testing backend (#446)
* feat(pytest): added

* feat(brains): added tests

* feat(actions): pytest
2023-07-02 02:19:30 +02:00
Matt
6f047f4a39
feat: streaming for standard brain picking (#385)
* feat: streaming for standard brain picking

* fix(bug): private llm

* wip: test

Co-authored-by: Mamadou DICKO <mamadoudicko@users.noreply.github.com>

* wip: almost good

Co-authored-by: Mamadou DICKO <mamadoudicko@users.noreply.github.com>

* feat: useFetch

* chore: remove 💀

* chore: fix linting

* fix: forward the request if not streaming

* feat: streaming for standard brain picking

* fix(bug): private llm

* wip: test

Co-authored-by: Mamadou DICKO <mamadoudicko@users.noreply.github.com>

* wip: almost good

Co-authored-by: Mamadou DICKO <mamadoudicko@users.noreply.github.com>

* feat: useFetch

* chore: remove 💀

* chore: fix linting

* fix: forward the request if not streaming

* fix: 💀 code

* fix: check_user_limit

* feat: brain_id to new chat stream

* fix: missing imports

* feat: message_id created on backend

Co-authored-by: Mamadou DICKO <mamadoudicko@users.noreply.github.com>

* chore: remove dead

* remove: cpython

* remove: dead

---------

Co-authored-by: Mamadou DICKO <mamadoudicko@users.noreply.github.com>
2023-06-30 10:10:59 +02:00
Zineb El Bachiri
4d9bd512ec
Fix/file upload explore (#412) 2023-06-29 18:26:03 +02:00
Zineb El Bachiri
ccdc5bb7a7
Feat/multiple brains files (#361) 2023-06-28 19:39:27 +02:00
Stan Girard
5fc837b250
feat(brainpicking): simplified (#371)
* feat(functions): simplified

* refactor(openai): changed to brainpicking

* feat(functions): made them inherit from brainpicking

* feat(privatebrainpicking): added new class

* feat(history&context): added

* Delete test_brainpicking.py

* Delete __init__.py
2023-06-26 10:34:03 +02:00
Mamadou DICKO
3922d8ca83
Feat/error handling (#366)
* feat: improve error handling

* docs: explain error handling system
2023-06-23 10:36:55 +02:00
Mamadou DICKO
59fe7b089b
feat(chat): use openai function for answer (#354)
* feat(chat): use openai function for answer (backend)

* feat(chat): use openai function for answer (frontend)

* chore: refacto BrainPicking

* feat: update chat creation logic

* feat: simplify chat system logic

* feat: set default method to gpt-3.5-turbo-0613

* feat: use user own openai key

* feat(chat): slightly improve prompts

* feat: add global error interceptor

* feat: remove unused endpoints

* docs: update chat system doc

* chore(linter): add unused import remove config

* feat: improve dx

* feat: improve OpenAiFunctionBasedAnswerGenerator prompt
2023-06-22 17:50:06 +02:00
Matt
83fde0aeea
feat: private llm (#360)
* feat: private llm

* Update backend/vectorstore/supabase.py

* Update backend/vectorstore/supabase.py
2023-06-22 09:45:35 +01:00
Stan Girard
3d11e3fb51
refactor(brain): removed braintoupdate and added brain (#356)
* refactor(brain): removed braintoupdate and added brain

* feat(brain): updated
2023-06-21 10:16:44 +02:00
Zineb El Bachiri
9c8e0aa0e4
Feat/multiple brains frontend (#344)
* 🗑️ remove date input from fetch_user_id_from_credentials

* ♻️ refactor backend utils by splitting it into files

* 💡 comments for next actions to update /upload

* 🚚 move SupabaseProvider tp libs

* 🚚 move useAxios to hooks

* ♻️ refacto brain routes

* 🚨 update lintermfor paths

*  new brain context provider

*  new brain component in navbar

* 🚨 fix linter and async

* 🇸🇪 add feature flag for multiple-brains
2023-06-20 16:17:13 +02:00
Mamadou DICKO
e1a740472f
Feat: chat name edit (#343)
* feat(chat): add name update

* chore(linting): add flake8

* feat: add chat name edit
2023-06-20 09:54:23 +02:00
Stan Girard
72a6ae3dc0 feat(settings): refactored 2023-06-19 22:46:25 +02:00
Stan Girard
c222f57b22 refactor(settings): moved to own folder 2023-06-19 21:20:08 +02:00
Zineb El Bachiri
e53bc6807d
Feat/multiple brains backend (#340)
* 🗃️ add new tables for multiple brains

* 🗑️ remove date input from fetch_user_id_from_credentials

*  new /brain endpoints

* ♻️ refactor backend utils by splitting it into files

* 💡 comments for next actions to update /upload
2023-06-16 23:36:53 +02:00
Stan Girard
f833de47bc feat(openai): 16k 2023-06-16 17:52:32 +02:00
Cezar Vasconcelos
3b7390dd61
Feat/add new openai llm models (#334)
* feat(frontend): update OpenAi models and maxToken select's

* fix: update openai model to use the most recent

* fix: adjust max token for gpt-3 turbo

* fix: duplicating models

* fix: openai model type list
2023-06-14 21:30:52 +02:00
Zineb El Bachiri
4112699db5
Feat/user chat history (#275)
* ♻️ refactor backend main routes

* 🗃️ new user_id uuid column in users table

* 🗃️ new chats table

*  new chat endpoints

*  change chat routes post to handle undef chat_id

* ♻️ extract components from chat page

*  add chatId to useQuestion

*  new ChatsList

*  new optional dynamic route chat/{chat_id}

* 🩹 add setQuestion to speach utils

* feat: self supplied key (#286)

* feat(brain): increased size if api key and more

* fix(key): not displayed

* feat(apikey): now password input

* fix(twitter): handle wrong

* feat(chat): basic source documents support (#289)

* ♻️ refactor backend main routes

* 🗃️ new user_id uuid column in users table

* 🗃️ new chats table

*  new chat endpoints

*  change chat routes post to handle undef chat_id

* ♻️ extract components from chat page

*  add chatId to useQuestion

*  new ChatsList

*  new optional dynamic route chat/{chat_id}

* 🩹 add setQuestion to speach utils

* 🎨 separate creation and update endpoints for chat

* 🩹 add feat(chat): basic source documents support

*  add chatName upon creation and for chats list

* 💄 improve chatsList style

* User chat history and multiple chats (#290)

* ♻️ refactor backend main routes

* 🗃️ new user_id uuid column in users table

* 🗃️ new chats table

*  new chat endpoints

*  change chat routes post to handle undef chat_id

* ♻️ extract components from chat page

*  add chatId to useQuestion

*  new ChatsList

*  new optional dynamic route chat/{chat_id}

* refactor(chat): use layout to avoid refetching all chats on every chat

* refactor(chat): useChats hook instead of useQuestion

* fix(chat): fix errors

* refactor(chat): better folder structure

* feat: self supplied key (#286)

* feat(brain): increased size if api key and more

* fix(key): not displayed

* feat(apikey): now password input

* fix(twitter): handle wrong

* feat(chat): basic source documents support (#289)

* style(chat): better looking sidebar

* resume merge

* fix(backend): add os and logger imports

* small fixes

* chore(chat): remove empty interface

* chore(chat): use const

* fix(chat): merge errors

* fix(chat): remove useSpeech args

* chore(chat): remove unused args

* fix(chat): remove duplicate components

---------

Co-authored-by: gozineb <zinebe@theodo.fr>
Co-authored-by: Matt <77928207+mattzcarey@users.noreply.github.com>
Co-authored-by: Stan Girard <girard.stanislas@gmail.com>
Co-authored-by: xleven <xleven@outlook.com>

* fix and refactor errors

* fix(fresh): installation issues

* chore(conflict): merged old code

* fix(multi-chat): use update endpoint

* feat(embeddings): now using users api key

---------

Co-authored-by: Matt <77928207+mattzcarey@users.noreply.github.com>
Co-authored-by: Stan Girard <girard.stanislas@gmail.com>
Co-authored-by: xleven <xleven@outlook.com>
Co-authored-by: Aditya Nandan <61308761+iMADi-ARCH@users.noreply.github.com>
Co-authored-by: iMADi-ARCH <nandanaditya985@gmail.com>
Co-authored-by: Mamadou DICKO <mamadoudicko100@gmail.com>
2023-06-10 23:59:16 +02:00