quivr/README.md

145 lines
5.3 KiB
Markdown
Raw Normal View History

# Quivr - Your Second Brain, Empowered by Generative AI
2023-05-13 03:02:45 +03:00
<div align="center">
<img src="./logo.png" alt="Quivr-logo" width="30%" style="border-radius: 50%; padding-bottom: 20px"/>
</div>
2023-05-13 03:02:45 +03:00
<div align="center">
2023-05-17 01:14:13 +03:00
<a href="https://discord.gg/HUpRgp2HG8">
<img src="https://img.shields.io/badge/discord-join%20chat-blue.svg" alt="Join our Discord" height="40">
</a>
</div>
2023-05-15 10:28:25 +03:00
Quivr, your second brain, utilizes the power of GenerativeAI to store and retrieve unstructured information. Think of it as Obsidian, but turbocharged with AI capabilities.
New Webapp migration (#56) * feat(v2): loaders added * feature: Add scroll animations * feature: upload ui * feature: upload multiple files * fix: Same file name and size remove * feat(crawler): added * feat(parsers): v2 added more * feat(v2): audio now working * feat(v2): all loaders * feat(v2): explorer * chore: add links * feat(api): added status in return message * refactor(website): remove old code * feat(upload): return type for messages * feature: redirect to upload if ENV=local * fix(chat): fixed some issues * feature: respect response type * loading state * feature: Loading stat * feat(v2): added explore and chat pages * feature: modal settings * style: Chat UI * feature: scroll to bottom when chatting * feature: smooth scroll in chat * feature(anim): Slide chat in * feature: markdown chat * feat(explorer): list * feat(doc): added document item * feat(explore): added modal * Add clarification on Project API keys and web interface for migration scripts to Readme (#58) * fix(demo): changed link * add support to uploading zip file (#62) * Catch UnicodeEncodeError exception (#64) * feature: fixed chatbar * fix(loaders): missing argument * fix: layout * fix: One whole chatbox * fix: Scroll into view * fix(build): vercel issues * chore(streamlit): moved to own file * refactor(api): moved to backend folder * feat(docker): added docker compose * Fix a bug where langchain memories were not being cleaned (#71) * Update README.md (#70) * chore(streamlit): moved to own file * refactor(api): moved to backend folder * docs(readme): updated for new version * docs(readme): added old readme * docs(readme): update copy dot env file * docs(readme): cleanup --------- Co-authored-by: iMADi-ARCH <nandanaditya985@gmail.com> Co-authored-by: Matt LeBel <github@lebel.io> Co-authored-by: Evan Carlson <45178375+EvanCarlson@users.noreply.github.com> Co-authored-by: Mustafa Hasan Khan <65130881+mustafahasankhan@users.noreply.github.com> Co-authored-by: zhulixi <48713110+zlxxlz1026@users.noreply.github.com> Co-authored-by: Stanisław Tuszyński <stanislaw@tuszynski.me>
2023-05-21 02:20:55 +03:00
## Key Features 🎯
2023-05-13 03:02:45 +03:00
- **Universal Data Acceptance**: Quivr can handle almost any type of data you throw at it. Text, images, code snippets, we've got you covered.
- **Generative AI**: Quivr employs advanced AI to assist you in generating and retrieving information.
- **Fast and Efficient**: Designed with speed and efficiency at its core. Quivr ensures rapid access to your data.
- **Secure**: Your data, your control. Always.
- **File Compatibility**:
- Text
- Markdown
- PDF
- Powerpoint
- Excel
- Word
- Audio
- Video
- **Open Source**: Freedom is beautiful, so is Quivr. Open source and free to use.
2023-05-13 03:02:45 +03:00
## Demo Highlights 🎥
2023-05-13 03:16:41 +03:00
> **Please note: The DEMO WITH STREAMLIT IS USING AN OLD VERSION**
> The new version showcases a fresh UI, however, it is not yet deployed as it lacks some features of the old version. Expected to go live before 25/05/23
2023-05-13 20:56:54 +03:00
### **Demo with GPT3.5**:
2023-05-17 00:18:58 +03:00
https://github.com/StanGirard/quivr/assets/19614572/80721777-2313-468f-b75e-09379f694653
2023-05-13 20:56:54 +03:00
### **Demo with Claude 100k context**: https://github.com/StanGirard/quivr/assets/5101573/9dba918c-9032-4c8d-9eea-94336d2c8bd4
2023-05-13 20:56:54 +03:00
### **Demo of the new version (Work in progress)**:
2023-05-21 03:08:21 +03:00
https://github.com/StanGirard/quivr/assets/19614572/a6463b73-76c7-4bc0-978d-70562dca71f5
## Getting Started: New Version 🚀
2023-05-13 03:02:45 +03:00
Follow these instructions to get a copy of the project up and running on your local machine for development and testing purposes.
New Webapp migration (#56) * feat(v2): loaders added * feature: Add scroll animations * feature: upload ui * feature: upload multiple files * fix: Same file name and size remove * feat(crawler): added * feat(parsers): v2 added more * feat(v2): audio now working * feat(v2): all loaders * feat(v2): explorer * chore: add links * feat(api): added status in return message * refactor(website): remove old code * feat(upload): return type for messages * feature: redirect to upload if ENV=local * fix(chat): fixed some issues * feature: respect response type * loading state * feature: Loading stat * feat(v2): added explore and chat pages * feature: modal settings * style: Chat UI * feature: scroll to bottom when chatting * feature: smooth scroll in chat * feature(anim): Slide chat in * feature: markdown chat * feat(explorer): list * feat(doc): added document item * feat(explore): added modal * Add clarification on Project API keys and web interface for migration scripts to Readme (#58) * fix(demo): changed link * add support to uploading zip file (#62) * Catch UnicodeEncodeError exception (#64) * feature: fixed chatbar * fix(loaders): missing argument * fix: layout * fix: One whole chatbox * fix: Scroll into view * fix(build): vercel issues * chore(streamlit): moved to own file * refactor(api): moved to backend folder * feat(docker): added docker compose * Fix a bug where langchain memories were not being cleaned (#71) * Update README.md (#70) * chore(streamlit): moved to own file * refactor(api): moved to backend folder * docs(readme): updated for new version * docs(readme): added old readme * docs(readme): update copy dot env file * docs(readme): cleanup --------- Co-authored-by: iMADi-ARCH <nandanaditya985@gmail.com> Co-authored-by: Matt LeBel <github@lebel.io> Co-authored-by: Evan Carlson <45178375+EvanCarlson@users.noreply.github.com> Co-authored-by: Mustafa Hasan Khan <65130881+mustafahasankhan@users.noreply.github.com> Co-authored-by: zhulixi <48713110+zlxxlz1026@users.noreply.github.com> Co-authored-by: Stanisław Tuszyński <stanislaw@tuszynski.me>
2023-05-21 02:20:55 +03:00
**Old version's readme** is located in the streamlit-demo folder [here](streamlit-demo/README.md)
2023-05-13 03:02:45 +03:00
### Prerequisites 📋
2023-05-13 03:02:45 +03:00
Before you proceed, ensure you have the following installed:
2023-05-13 03:02:45 +03:00
New Webapp migration (#56) * feat(v2): loaders added * feature: Add scroll animations * feature: upload ui * feature: upload multiple files * fix: Same file name and size remove * feat(crawler): added * feat(parsers): v2 added more * feat(v2): audio now working * feat(v2): all loaders * feat(v2): explorer * chore: add links * feat(api): added status in return message * refactor(website): remove old code * feat(upload): return type for messages * feature: redirect to upload if ENV=local * fix(chat): fixed some issues * feature: respect response type * loading state * feature: Loading stat * feat(v2): added explore and chat pages * feature: modal settings * style: Chat UI * feature: scroll to bottom when chatting * feature: smooth scroll in chat * feature(anim): Slide chat in * feature: markdown chat * feat(explorer): list * feat(doc): added document item * feat(explore): added modal * Add clarification on Project API keys and web interface for migration scripts to Readme (#58) * fix(demo): changed link * add support to uploading zip file (#62) * Catch UnicodeEncodeError exception (#64) * feature: fixed chatbar * fix(loaders): missing argument * fix: layout * fix: One whole chatbox * fix: Scroll into view * fix(build): vercel issues * chore(streamlit): moved to own file * refactor(api): moved to backend folder * feat(docker): added docker compose * Fix a bug where langchain memories were not being cleaned (#71) * Update README.md (#70) * chore(streamlit): moved to own file * refactor(api): moved to backend folder * docs(readme): updated for new version * docs(readme): added old readme * docs(readme): update copy dot env file * docs(readme): cleanup --------- Co-authored-by: iMADi-ARCH <nandanaditya985@gmail.com> Co-authored-by: Matt LeBel <github@lebel.io> Co-authored-by: Evan Carlson <45178375+EvanCarlson@users.noreply.github.com> Co-authored-by: Mustafa Hasan Khan <65130881+mustafahasankhan@users.noreply.github.com> Co-authored-by: zhulixi <48713110+zlxxlz1026@users.noreply.github.com> Co-authored-by: Stanisław Tuszyński <stanislaw@tuszynski.me>
2023-05-21 02:20:55 +03:00
- Docker
- Docker Compose
Additionally, you'll need a [Supabase](https://supabase.com/) account for:
- Creating a new Supabase project
- Supabase Project API key
- Supabase Project URL
2023-05-13 03:02:45 +03:00
### Installation Steps 💽
2023-05-13 03:02:45 +03:00
2023-05-21 22:44:32 +03:00
- **Step 0**: If needed, here is the installation explained on Youtube [here](https://youtu.be/rC-s4QdfY80)
- **Step 1**: Clone the repository using **one** of these commands:
2023-05-13 03:02:45 +03:00
- If you don't have an SSH key set up:
```bash
git clone https://github.com/StanGirard/Quivr.git && cd Quivr
```
- If you have an SSH key set up or want to add it ([guide here](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/adding-a-new-ssh-key-to-your-github-account))
```bash
git clone git@github.com:StanGirard/Quivr.git && cd Quivr
```
2023-05-13 03:02:45 +03:00
- **Step 2**: Copy the `.XXXXX_env` files
2023-05-13 03:13:56 +03:00
```bash
2023-05-21 09:32:22 +03:00
cp .backend_env.example backend/.env
cp .frontend_env.example frontend/.env
2023-05-13 03:13:56 +03:00
```
2023-05-24 17:43:26 +03:00
- **Step 3**: Update the `backend/.env` and `frontend/.env` file
2023-05-13 03:13:56 +03:00
> _Your `supabase_service_key` can be found in your Supabase dashboard under Project Settings -> API. Use the `anon` `public` key found in the `Project API keys` section._
> _Your `JWT_SECRET_KEY`can be found in your supabase settings under Project Settings -> JWT Settings -> JWT Secret_
2023-05-17 21:36:57 +03:00
> _To activate vertexAI with PaLM from GCP follow the instructions [here](https://python.langchain.com/en/latest/modules/models/llms/integrations/google_vertex_ai_palm.html) and update `bacend/.env`- It is an advanced feature, please be expert in GCP before trying to use it_
- [ ] Change variables in `backend/.env`
- [ ] Change variables in `frontend/.env`
- **Step 4**: Run the following migration scripts on the Supabase database via the web interface (SQL Editor -> `New query`)
[Migration Script 1](scripts/supabase_new_store_documents.sql)
[Migration Script 2](scripts/supabase_usage_table.sql)
2023-05-22 09:47:49 +03:00
[Migration Script 3](scripts/supabase_vector_store_summary.sql)
[Migrations Script 4](scripts/supabase_users_table.sql)
- **Step 5**: Launch the app
2023-05-13 03:13:56 +03:00
```bash
docker compose -f docker-compose.yml up --build
2023-05-13 03:13:56 +03:00
```
2023-05-13 03:02:45 +03:00
- **Step 6**: Navigate to `localhost:3000` in your browser
- ** Step 7**: Want to contribute to the project?
```
docker compose -f docker-compose.dev.yml up --build
```
## Contributors ✨
Thanks goes to these wonderful people:
<a href="https://github.com/stangirard/quivr/graphs/contributors">
<img src="https://contrib.rocks/image?repo=stangirard/quivr" />
</a>
## Contribute 🤝
2023-05-13 03:02:45 +03:00
Got a pull request? Open it, and we'll review it as soon as possible. Check out our project board [here](https://github.com/users/StanGirard/projects/5) to see what we're currently focused on, and feel free to bring your fresh ideas to the table!
2023-05-13 03:13:56 +03:00
- [Roadmap](https://github.com/users/StanGirard/projects/5)
- [Open Issues](https://github.com/StanGirard/quivr/issues)
- [Open Pull Requests](https://github.com/StanGirard/quivr/pulls)
- [Good First Issues](https://github.com/StanGirard/quivr/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22)
- [Frontend Issues](https://github.com/StanGirard/quivr/issues?q=is%3Aopen+is%3Aissue+label%3Afrontend)
- [Backend Issues](https://github.com/StanGirard/quivr/issues?q=is%3Aopen+is%3Aissue+label%3Abackend)
2023-05-21 03:07:33 +03:00