Commit Graph

42 Commits

Author SHA1 Message Date
Stan Girard
e6cbbbcd37
fix(brain_size): increased size (#1235) 2023-09-20 17:43:46 +02:00
Zineb El Bachiri
be7acf052b
feat: get files from storage (#1205)
* 🌱 list files in storage & generate signed URL

*  add knowledge router

* 🗃️ add knowledge tables

*  add knowledge during upload

* 🚧 add knowledge a brain_knowledge models and repo

* 🔥 remove brain_knowledge

*  add upload to knowledge table

*  add crawl to knowledge table

* ✏️ fixes
2023-09-20 09:35:37 +02:00
Stan Girard
22fa1d4525
Feat/theodo gpt4 (#1198)
* chore(theodo): added trigger function for theodo

* chore(theodo): increased brain to 100MB
2023-09-18 15:25:38 +02:00
Stan Girard
80ba9f2616
chore(theodo): added trigger function for theodo (#1195) 2023-09-18 14:22:36 +02:00
Stan Girard
4d41901106
feat(perf): increased perf embedding and search for files (#1182)
* feat(upload): changed to task

* feat(sha1): added column for better speed
2023-09-15 23:39:29 +02:00
Stan Girard
4b806db3c0
fix(migration): removed (#1170) 2023-09-14 16:45:15 +02:00
Stan Girard
1d33fbd3eb
feat(file-system): added queue and filesystem (#1159)
* feat(queue): added

* feat(crawling): added queue

* fix(crawler): fixed github

* feat(docker): simplified docker compose

* feat(celery): added worker

* feat(files): now uploaded

* feat(files): missing routes

* feat(delete): added

* feat(storage): added policy and migrations

* feat(sqs): implemented

* feat(redis): added queue name variable

* fix(task): updated

* style(env): emoved unused env

* ci(tests): removed broken tests
2023-09-14 11:56:59 +02:00
Stan Girard
322ee318be
feat: the good user management (#1158)
* feat(user_management): added user management

* feat(user_management): added user management

* feat(user_management): removed print

* feat: use tanstack query for user data fecthing

* feat: add getUser to sdk

* feat: improve user page ux use tanstack query

* feat: fetch models from backend on brains settings page

* feat: update model selection on chat page

* feat: update tests

---------

Co-authored-by: mamadoudicko <mamadoudicko100@gmail.com>
2023-09-13 13:47:12 +02:00
Mamadou DICKO
575d9886c5
feat: add notifications table, and push notification on upload and crawl (#1125)
* feat: add notifications table

* feat: add Notification model

* feat: add notification repositories

* feat: add upload and crawl notifications

* feat: update notification message
2023-09-07 13:22:06 +02:00
Zineb El Bachiri
4b1f4b1412
feat(prompts): add public prompts to SQL db (#1014)
*  add public prompts to SQL scripts

* 🚨 fix: missing zh-cn translation key
2023-08-22 13:14:31 +02:00
Zineb El Bachiri
9aaedcff51
Fix/requests limiting (#980)
* 🗃️ Rename users table into user_daily_usage

* 💥 replace User model with UserIdentity model

* 🗃️ New UserDailyUsage class for database interaction

* 🐛 fix daily requests rate limiting per user

* 🐛 fix user stats and properties update

* ✏️ add typing and linting

* 🚚 rename user_dialy_usage Class  into user_usage & requests_count into daily_requests_count

* 🚑 fix some rebase errors
2023-08-21 14:05:13 +02:00
Mamadou DICKO
6e777327aa
feat(chatMessages): add brain_id and prompt_id columns (#912)
* feat: add prompt_id and brain_id to chat history)

* feat: add prompt_id and brain_id to chat routes
2023-08-10 10:25:08 +02:00
Mamadou DICKO
99a3fa9b29
feat: add custom prompt fields on brain setting pages (#837)
* feat(sdk): add prompt apis to sdk

* feat: implement prompt creation-n

* feat: add brain custom prompt fields

* fix: change tables creation order
2023-08-03 15:41:24 +02:00
Mamadou DICKO
4ca6c667da
feat(backend): implement brain-prompt link (#831)
* feat: add prompt_id field to brain

* feat(Prompt controller): update prompt routes

* feat: remove unused private prompts

* refactor: add BrainEntity and repo and service

* tests: partially type main Repository

* feat: add PromptStatusEnum enum

* feat: change delete prompt repository return type
2023-08-03 10:37:13 +02:00
Mamadou DICKO
e3b6114248
feat(prompt): add prompt table, entity and repository (#823)
* feat: add prompts table

* feat: add Prompt entity

* feat: add prompt router

* refactor(promptRepository): use common reposority
2023-08-03 09:53:38 +02:00
Noe
303ba72028
feat: Introduce repository pattern to prepare adding other database providers (#646)
* add sqlalchemy models

* add neon settings

* add insert brain

* abstract supabase from Brain class

* abstract supabase from Brain class

* abstract supabase from /models

* update Database to Repository

* update neon_tables to pg_tables

* update chat, api-key and message

* update vector class

* update settings

* update env vars for test

* Update backend-tests.yml

* fix test

* fix fetch_user_requests_count()

* fix fetch_user_requests_count()

* fix increment_user_request_count

* fix increment_user_request_count

* fix asset upload_response message

* fix pyright

* fix brain_subscription

* fix brain_subscription

* fix brain_subscription

* fix get user request stat

* update create_brain_user

* add delete brain vector and user

* add delete brain vector and user

* correctly call function

---------

Co-authored-by: Noé Pion <noe.pion@onfido.com>
Co-authored-by: raoufchebri <raouf@chebri.com>
Co-authored-by: Stan Girard <girard.stanislas@gmail.com>
2023-08-01 23:03:47 +02:00
Mamadou DICKO
7532b558c7
feat: add user level open ai key management (#805)
* feat: add user user identity table

* feat: add user openai api key input

* feat: add encryption missing message

* chore: log more details about 422 errors

* docs(API): update api creation path

* feat: use user openai key if defined
2023-08-01 09:24:57 +02:00
Zineb El Bachiri
e05f25b025
Feat/update brain fields (#756)
* 🗃️ update and add fields in brains table

*  update endpoints for more brain attribute

*  new set as default brain endpoint

* 🔥 remove update brain with file

* ✏️ fix wrong  auto imports

* 🐛 fix max tokens for model in front

* 🚑 post instead of put to set default brain

* 🚑 update brain creation endpoint with new fields
2023-07-25 15:22:17 +02:00
Mamadou DICKO
81b57c504a
Shareable brain 9 (#677)
* feat: add 20230717173000_add_get_user_id_by_user_email

* feat(ShareableBrain): add update access endpoint

* feat(sdk): add updateBrainAccess

* feat: add brain access control

* feat: improve ux
2023-07-18 14:30:19 +02:00
Mamadou DICKO
f8fce33191
feat: add 20230717164900_add_get_user_email_by_user_id (#675) 2023-07-17 17:24:31 +02:00
Matt
cef45ea712
Fix: change vector id to UUID (#609)
* fix: document upload

* feat: explore fix to use uuid id

* chore: remove prints

* fix: tables.sql
2023-07-12 11:44:34 +01:00
Zineb El Bachiri
f837a6e9b9
Feat/shareable brains send link be (#599)
* 🗃️ new table for invitations to subscribe to brain

*  new BrainSubscription class

*  new subscription router

* 👽️ add RESEND_API_KEY to .env in BE

* 📦 add 'resend' lib to requirements

* ♻️ fix some stanGPT
2023-07-11 18:20:31 +02:00
Michael Tan
1c32eb54be
(fix) syntax error at or near "WHERE" (#548) 2023-07-07 09:10:22 +02:00
Stan Girard
f0a30b8452 docs(migration): updated table 2023-07-03 14:41:33 +02:00
Stan Girard
76156fdad3 feat: 🎸 migration
added migration script
2023-07-03 11:11:29 +02:00
Zineb El Bachiri
056a68d5ed
Fix/prod (#425)
* 🐛 update db tables script

* ✏️ fix users routes

* 🐛 add '/' AGAIN in URLs
2023-06-30 10:09:50 +02:00
Zineb El Bachiri
4d9bd512ec
Fix/file upload explore (#412) 2023-06-29 18:26:03 +02:00
Zineb El Bachiri
ccdc5bb7a7
Feat/multiple brains files (#361) 2023-06-28 19:39:27 +02:00
Mamadou DICKO
59fe7b089b
feat(chat): use openai function for answer (#354)
* feat(chat): use openai function for answer (backend)

* feat(chat): use openai function for answer (frontend)

* chore: refacto BrainPicking

* feat: update chat creation logic

* feat: simplify chat system logic

* feat: set default method to gpt-3.5-turbo-0613

* feat: use user own openai key

* feat(chat): slightly improve prompts

* feat: add global error interceptor

* feat: remove unused endpoints

* docs: update chat system doc

* chore(linter): add unused import remove config

* feat: improve dx

* feat: improve OpenAiFunctionBasedAnswerGenerator prompt
2023-06-22 17:50:06 +02:00
Zineb El Bachiri
e53bc6807d
Feat/multiple brains backend (#340)
* 🗃️ add new tables for multiple brains

* 🗑️ remove date input from fetch_user_id_from_credentials

*  new /brain endpoints

* ♻️ refactor backend utils by splitting it into files

* 💡 comments for next actions to update /upload
2023-06-16 23:36:53 +02:00
Matt
33f49ee289
feat: user can create api keys (#329)
* feat: user can create api keys

* fix: linting on build

* Update backend/routes/api_key_routes.py

* chore: rename and refactor AuthBearer

* chore: add types
2023-06-14 21:21:13 +02:00
Matt
ec29f30f32
Feat: backend refactor (#306)
* fix: edge cases on migration scripts

* chore: remove unused deps.

* refactor: user_routes

* refactor: chat_routes

* refactor: upload_routes

* refactor: explore_routes

* refactor: crawl_routes

* chore(refactor): get current user

* refactor: more dead dependencies

* bug: wrap email in credentials dict.

---------

Co-authored-by: Stan Girard <girard.stanislas@gmail.com>
2023-06-12 17:58:05 +02:00
Stan Girard
537efc834d feat(sql): merged all into one and made migration fault tolerant 2023-06-11 21:11:43 +02:00
Stan Girard
583e4d6378 docs(updated): new scripts 2023-06-11 19:57:54 +02:00
Zineb El Bachiri
4112699db5
Feat/user chat history (#275)
* ♻️ refactor backend main routes

* 🗃️ new user_id uuid column in users table

* 🗃️ new chats table

*  new chat endpoints

*  change chat routes post to handle undef chat_id

* ♻️ extract components from chat page

*  add chatId to useQuestion

*  new ChatsList

*  new optional dynamic route chat/{chat_id}

* 🩹 add setQuestion to speach utils

* feat: self supplied key (#286)

* feat(brain): increased size if api key and more

* fix(key): not displayed

* feat(apikey): now password input

* fix(twitter): handle wrong

* feat(chat): basic source documents support (#289)

* ♻️ refactor backend main routes

* 🗃️ new user_id uuid column in users table

* 🗃️ new chats table

*  new chat endpoints

*  change chat routes post to handle undef chat_id

* ♻️ extract components from chat page

*  add chatId to useQuestion

*  new ChatsList

*  new optional dynamic route chat/{chat_id}

* 🩹 add setQuestion to speach utils

* 🎨 separate creation and update endpoints for chat

* 🩹 add feat(chat): basic source documents support

*  add chatName upon creation and for chats list

* 💄 improve chatsList style

* User chat history and multiple chats (#290)

* ♻️ refactor backend main routes

* 🗃️ new user_id uuid column in users table

* 🗃️ new chats table

*  new chat endpoints

*  change chat routes post to handle undef chat_id

* ♻️ extract components from chat page

*  add chatId to useQuestion

*  new ChatsList

*  new optional dynamic route chat/{chat_id}

* refactor(chat): use layout to avoid refetching all chats on every chat

* refactor(chat): useChats hook instead of useQuestion

* fix(chat): fix errors

* refactor(chat): better folder structure

* feat: self supplied key (#286)

* feat(brain): increased size if api key and more

* fix(key): not displayed

* feat(apikey): now password input

* fix(twitter): handle wrong

* feat(chat): basic source documents support (#289)

* style(chat): better looking sidebar

* resume merge

* fix(backend): add os and logger imports

* small fixes

* chore(chat): remove empty interface

* chore(chat): use const

* fix(chat): merge errors

* fix(chat): remove useSpeech args

* chore(chat): remove unused args

* fix(chat): remove duplicate components

---------

Co-authored-by: gozineb <zinebe@theodo.fr>
Co-authored-by: Matt <77928207+mattzcarey@users.noreply.github.com>
Co-authored-by: Stan Girard <girard.stanislas@gmail.com>
Co-authored-by: xleven <xleven@outlook.com>

* fix and refactor errors

* fix(fresh): installation issues

* chore(conflict): merged old code

* fix(multi-chat): use update endpoint

* feat(embeddings): now using users api key

---------

Co-authored-by: Matt <77928207+mattzcarey@users.noreply.github.com>
Co-authored-by: Stan Girard <girard.stanislas@gmail.com>
Co-authored-by: xleven <xleven@outlook.com>
Co-authored-by: Aditya Nandan <61308761+iMADi-ARCH@users.noreply.github.com>
Co-authored-by: iMADi-ARCH <nandanaditya985@gmail.com>
Co-authored-by: Mamadou DICKO <mamadoudicko100@gmail.com>
2023-06-10 23:59:16 +02:00
Zineb El Bachiri
c3bb0257f0
Feat/rate limiter (#222)
* 🔧 new use rate limiting env variables

* 🗃️ new Users table for rate limiting

* limit user's requests number per day

*  limit user's brain size

* docs(readme): updated with new migration

---------

Co-authored-by: Stan Girard <girard.stanislas@gmail.com>
2023-06-01 22:51:39 +02:00
!MAD!
f4327660e5
fix(script): rename documents to vectors (#221) 2023-05-31 18:01:01 +02:00
Stan Girard
e0de23e24d
Feat/single brain (#215)
* feat(llm): update

* feat(singlebrain): added new table with user-id

* feat(user): get user from email

* feat(user_id): added search

*  add user_id to most endpoints

* docs(readme): new script

---------

Co-authored-by: gozineb <zinebe@theodo.fr>
2023-05-31 13:51:23 +02:00
shaun
fcb108808a Add missing sql 2023-05-21 23:47:49 -07:00
shaun
c1345561ab remove not ready updates 2023-05-21 21:21:27 -07:00
shaun
a1693d94b2 Better envs 2023-05-21 21:18:55 -07:00
Stan Girard
02f58c84b8
docs(readme): making it nice (#102)
* docs(readme): making it nice

* docs(logo): made round
2023-05-21 14:53:47 +02:00