Commit Graph

36 Commits

Author SHA1 Message Date
Stan Girard
6c5496f797
feat: 🎸 sources (#2092)
added metadata object a bit bigger

# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2024-01-25 18:56:54 -08:00
Stan Girard
ae81afbef5
fix: 🐛 crawl (#2076)
notification id issue when not in chat fixed

# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2024-01-23 23:39:45 -08:00
Zineb El Bachiri
f48dab4a7d
refactor: to modules (#1754)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2023-11-30 22:29:28 +01:00
Zineb El Bachiri
9766befb53
refactor: knowledge module (#1743)
# Description

- Refactor knowledge to a module

- This PR breaks the Github Processor function -> need to comment brain
and file imports as it creates a circular dependency issue. Should be
fixed and reverted in next PR.
2023-11-29 09:04:03 +01:00
Zineb El Bachiri
0782df5e37
refactor: Notification module (#1740)
# Description

- Notification module

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2023-11-28 14:27:39 +01:00
Stan Girard
6a041b6f6d
feat: 🎸 openai (#1658)
cleaning old code to introduce better patern

# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2023-11-20 01:22:03 +01:00
Stan Girard
d955e31f50
feat: 🎸 marketplace (#1657)
added explore button and removed unused feature openai key

# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2023-11-19 18:46:12 +01:00
Zineb El Bachiri
375f50356c
refactor: add modules folder (#1633)
# Description

New Modules folder with "user" module:
- controller: contains the current route
- entity: contains the current Models (TO be renamed DTO)
- repository: contains the current repo
- service: methods used by other modules 

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2023-11-15 13:17:51 +01:00
Zineb El Bachiri
1f21c6d7f3
refactor: packages folder be 2 (#1628)
# Description

Move some folders to packages
Remove unused code
2023-11-14 14:31:02 +01:00
Zineb El Bachiri
8f693bc92a
refactor: create "files" package (#1626)
# Description

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context.

## Checklist before requesting a review

Please delete options that are not relevant.

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my code
- [ ] I have commented hard-to-understand areas
- [ ] I have ideally added tests that prove my fix is effective or that
my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged

## Screenshots (if appropriate):
2023-11-14 09:52:44 +01:00
Zineb El Bachiri
be7acf052b
feat: get files from storage (#1205)
* 🌱 list files in storage & generate signed URL

*  add knowledge router

* 🗃️ add knowledge tables

*  add knowledge during upload

* 🚧 add knowledge a brain_knowledge models and repo

* 🔥 remove brain_knowledge

*  add upload to knowledge table

*  add crawl to knowledge table

* ✏️ fixes
2023-09-20 09:35:37 +02:00
Stan Girard
1d33fbd3eb
feat(file-system): added queue and filesystem (#1159)
* feat(queue): added

* feat(crawling): added queue

* fix(crawler): fixed github

* feat(docker): simplified docker compose

* feat(celery): added worker

* feat(files): now uploaded

* feat(files): missing routes

* feat(delete): added

* feat(storage): added policy and migrations

* feat(sqs): implemented

* feat(redis): added queue name variable

* fix(task): updated

* style(env): emoved unused env

* ci(tests): removed broken tests
2023-09-14 11:56:59 +02:00
Stan Girard
322ee318be
feat: the good user management (#1158)
* feat(user_management): added user management

* feat(user_management): added user management

* feat(user_management): removed print

* feat: use tanstack query for user data fecthing

* feat: add getUser to sdk

* feat: improve user page ux use tanstack query

* feat: fetch models from backend on brains settings page

* feat: update model selection on chat page

* feat: update tests

---------

Co-authored-by: mamadoudicko <mamadoudicko100@gmail.com>
2023-09-13 13:47:12 +02:00
Mamadou DICKO
10af0c949a
feat: add notifications components (#1148)
* feat: rename ChatMessages to MessagesDialog

* feat: rename history to messages

* feat: add notifications to ChatContext

* feat: add getNotificationsFromChatHistory

* feat: add getMergedChatHistoryWithReducedNotifications

* refactor: update useActionBar

* refactor: update <ChatMessage />-n

* feat: update crawler and endpoint notifications content

* feat: display notifications

* test: update <MessageDialog /> tests

* feat: rename ChatMessage to QADisplay

* feat: rename ChatHistory to ChatMessage

* feat(upload): throw error when file missing

* feat: rename getMergedChatHistoryWithReducedNotifications to getMergedChatMessagesWithReducedNotifications

* feat: change history wording to message

* feat: move getFileIcon to lib

* refactor(NotificationDisplayer): move types to types.ts

* chore: improve ux

* feat: rename MessagesDialog to ChatDialogue
2023-09-12 17:44:15 +02:00
Mamadou DICKO
72659709a8
fix: update crawl and upload endpoints (#1142) 2023-09-08 11:03:14 +02:00
Mamadou DICKO
9464707d40
feat: merge chat history with chat notifications (#1127)
* feat: add chat_id to upload and crawl payload

* feat(chat): return chat_history_with_notifications

* feat: explicit notification status on create

* feat: handle notifications in frontend

* feat: delete chat notifications on chat delete request
2023-09-07 17:23:31 +02:00
Mamadou DICKO
575d9886c5
feat: add notifications table, and push notification on upload and crawl (#1125)
* feat: add notifications table

* feat: add Notification model

* feat: add notification repositories

* feat: add upload and crawl notifications

* feat: update notification message
2023-09-07 13:22:06 +02:00
Zineb El Bachiri
9aaedcff51
Fix/requests limiting (#980)
* 🗃️ Rename users table into user_daily_usage

* 💥 replace User model with UserIdentity model

* 🗃️ New UserDailyUsage class for database interaction

* 🐛 fix daily requests rate limiting per user

* 🐛 fix user stats and properties update

* ✏️ add typing and linting

* 🚚 rename user_dialy_usage Class  into user_usage & requests_count into daily_requests_count

* 🚑 fix some rebase errors
2023-08-21 14:05:13 +02:00
Mamadou DICKO
5a3a6fe370
fix: update backend tests (#992)
* fix: update backend tests

* fix(pytest): update types
2023-08-21 12:45:32 +02:00
MaHDi
8af6d61e76
improve (importing): reorganization of the import structure (#964)
* reorganize import level

* add __init__, reorganize import from __init__

* reorganize import level

* reorganize import level

* fix circular import error by keep the import deep as "from models.settings"

* fix the relative import

* restor unwanted staged files

* add backend/venv and backend/.env to gitignore

* clean importing
2023-08-21 12:25:16 +02:00
Stan Girard
ae7852ec3f
feat(health): added endpoint for services (#989) 2023-08-20 23:20:57 +02:00
Stan Girard
d0370ab499
feat(refacto): changed a bit of things to make better dx (#984) 2023-08-19 13:32:16 +02:00
Matt
e61f437ce8
Feat/backend core (#656) 2023-07-17 07:57:27 +01:00
Mamadou DICKO
9e9f531c99
Feat/static analysis (#582)
* feat: add static analysis

* chore: update Makefile add static analysis script

* chore: add vscode extensions recommandations
2023-07-10 14:27:49 +02:00
Matt
e931d29017
feat: new classes (#499)
* feat: new classes

* chore: private llm refactor and enable streaming

* fix: validation errors in pytest

* fix: default brain behavior

* Update backend/llm/base.py

* chore: remove 🪦

* fix: remove 🪦
2023-07-04 16:56:54 +01:00
Stan Girard
3946dbc668 fix(crawler): route 2023-06-29 08:00:34 +02:00
Zineb El Bachiri
ccdc5bb7a7
Feat/multiple brains files (#361) 2023-06-28 19:39:27 +02:00
Stan Girard
c7fdbf6dec feat(refacto): removed commons 2023-06-19 22:54:01 +02:00
Stan Girard
72a6ae3dc0 feat(settings): refactored 2023-06-19 22:46:25 +02:00
Stan Girard
225280f2f5 fix(doc): retrieval to 8 because it takes a long time 2023-06-19 11:23:58 +02:00
Zineb El Bachiri
e53bc6807d
Feat/multiple brains backend (#340)
* 🗃️ add new tables for multiple brains

* 🗑️ remove date input from fetch_user_id_from_credentials

*  new /brain endpoints

* ♻️ refactor backend utils by splitting it into files

* 💡 comments for next actions to update /upload
2023-06-16 23:36:53 +02:00
Ruben1701
8fb245fe2a
Improved api docs (#336)
* auyto removed

* updated the api docs

* fixed missing logic in api_key_route + chat_route

* Update settings.json

---------

Co-authored-by: Ruben170 <35116541+Ruben170@users.noreply.github.com>
2023-06-15 14:43:40 +02:00
Matt
33f49ee289
feat: user can create api keys (#329)
* feat: user can create api keys

* fix: linting on build

* Update backend/routes/api_key_routes.py

* chore: rename and refactor AuthBearer

* chore: add types
2023-06-14 21:21:13 +02:00
Matt
ec29f30f32
Feat: backend refactor (#306)
* fix: edge cases on migration scripts

* chore: remove unused deps.

* refactor: user_routes

* refactor: chat_routes

* refactor: upload_routes

* refactor: explore_routes

* refactor: crawl_routes

* chore(refactor): get current user

* refactor: more dead dependencies

* bug: wrap email in credentials dict.

---------

Co-authored-by: Stan Girard <girard.stanislas@gmail.com>
2023-06-12 17:58:05 +02:00
Stan Girard
44d067d28d fix(crawl): github 2023-06-11 00:30:19 +02:00
Zineb El Bachiri
4112699db5
Feat/user chat history (#275)
* ♻️ refactor backend main routes

* 🗃️ new user_id uuid column in users table

* 🗃️ new chats table

*  new chat endpoints

*  change chat routes post to handle undef chat_id

* ♻️ extract components from chat page

*  add chatId to useQuestion

*  new ChatsList

*  new optional dynamic route chat/{chat_id}

* 🩹 add setQuestion to speach utils

* feat: self supplied key (#286)

* feat(brain): increased size if api key and more

* fix(key): not displayed

* feat(apikey): now password input

* fix(twitter): handle wrong

* feat(chat): basic source documents support (#289)

* ♻️ refactor backend main routes

* 🗃️ new user_id uuid column in users table

* 🗃️ new chats table

*  new chat endpoints

*  change chat routes post to handle undef chat_id

* ♻️ extract components from chat page

*  add chatId to useQuestion

*  new ChatsList

*  new optional dynamic route chat/{chat_id}

* 🩹 add setQuestion to speach utils

* 🎨 separate creation and update endpoints for chat

* 🩹 add feat(chat): basic source documents support

*  add chatName upon creation and for chats list

* 💄 improve chatsList style

* User chat history and multiple chats (#290)

* ♻️ refactor backend main routes

* 🗃️ new user_id uuid column in users table

* 🗃️ new chats table

*  new chat endpoints

*  change chat routes post to handle undef chat_id

* ♻️ extract components from chat page

*  add chatId to useQuestion

*  new ChatsList

*  new optional dynamic route chat/{chat_id}

* refactor(chat): use layout to avoid refetching all chats on every chat

* refactor(chat): useChats hook instead of useQuestion

* fix(chat): fix errors

* refactor(chat): better folder structure

* feat: self supplied key (#286)

* feat(brain): increased size if api key and more

* fix(key): not displayed

* feat(apikey): now password input

* fix(twitter): handle wrong

* feat(chat): basic source documents support (#289)

* style(chat): better looking sidebar

* resume merge

* fix(backend): add os and logger imports

* small fixes

* chore(chat): remove empty interface

* chore(chat): use const

* fix(chat): merge errors

* fix(chat): remove useSpeech args

* chore(chat): remove unused args

* fix(chat): remove duplicate components

---------

Co-authored-by: gozineb <zinebe@theodo.fr>
Co-authored-by: Matt <77928207+mattzcarey@users.noreply.github.com>
Co-authored-by: Stan Girard <girard.stanislas@gmail.com>
Co-authored-by: xleven <xleven@outlook.com>

* fix and refactor errors

* fix(fresh): installation issues

* chore(conflict): merged old code

* fix(multi-chat): use update endpoint

* feat(embeddings): now using users api key

---------

Co-authored-by: Matt <77928207+mattzcarey@users.noreply.github.com>
Co-authored-by: Stan Girard <girard.stanislas@gmail.com>
Co-authored-by: xleven <xleven@outlook.com>
Co-authored-by: Aditya Nandan <61308761+iMADi-ARCH@users.noreply.github.com>
Co-authored-by: iMADi-ARCH <nandanaditya985@gmail.com>
Co-authored-by: Mamadou DICKO <mamadoudicko100@gmail.com>
2023-06-10 23:59:16 +02:00