quivr/README.md

195 lines
7.3 KiB
Markdown
Raw Normal View History

# Quivr - Your Second Brain, Empowered by Generative AI
2023-05-13 03:02:45 +03:00
<div align="center">
<img src="./logo.png" alt="Quivr-logo" width="30%" style="border-radius: 50%; padding-bottom: 20px"/>
</div>
2023-05-13 03:02:45 +03:00
2023-08-06 01:33:48 +03:00
[![Discord Follow](https://dcbadge.vercel.app/api/server/HUpRgp2HG8?style=flat)](https://discord.gg/HUpRgp2HG8)
[![GitHub Repo stars](https://img.shields.io/github/stars/stangirard/quivr?style=social)](https://github.com/stangirard/quivr)
[![Twitter Follow](https://img.shields.io/twitter/follow/StanGirard?style=social)](https://twitter.com/_StanGirard)
2023-05-15 10:28:25 +03:00
Quivr, your second brain, utilizes the power of GenerativeAI to store and retrieve unstructured information. Think of it as Obsidian, but turbocharged with AI capabilities.
New Webapp migration (#56) * feat(v2): loaders added * feature: Add scroll animations * feature: upload ui * feature: upload multiple files * fix: Same file name and size remove * feat(crawler): added * feat(parsers): v2 added more * feat(v2): audio now working * feat(v2): all loaders * feat(v2): explorer * chore: add links * feat(api): added status in return message * refactor(website): remove old code * feat(upload): return type for messages * feature: redirect to upload if ENV=local * fix(chat): fixed some issues * feature: respect response type * loading state * feature: Loading stat * feat(v2): added explore and chat pages * feature: modal settings * style: Chat UI * feature: scroll to bottom when chatting * feature: smooth scroll in chat * feature(anim): Slide chat in * feature: markdown chat * feat(explorer): list * feat(doc): added document item * feat(explore): added modal * Add clarification on Project API keys and web interface for migration scripts to Readme (#58) * fix(demo): changed link * add support to uploading zip file (#62) * Catch UnicodeEncodeError exception (#64) * feature: fixed chatbar * fix(loaders): missing argument * fix: layout * fix: One whole chatbox * fix: Scroll into view * fix(build): vercel issues * chore(streamlit): moved to own file * refactor(api): moved to backend folder * feat(docker): added docker compose * Fix a bug where langchain memories were not being cleaned (#71) * Update README.md (#70) * chore(streamlit): moved to own file * refactor(api): moved to backend folder * docs(readme): updated for new version * docs(readme): added old readme * docs(readme): update copy dot env file * docs(readme): cleanup --------- Co-authored-by: iMADi-ARCH <nandanaditya985@gmail.com> Co-authored-by: Matt LeBel <github@lebel.io> Co-authored-by: Evan Carlson <45178375+EvanCarlson@users.noreply.github.com> Co-authored-by: Mustafa Hasan Khan <65130881+mustafahasankhan@users.noreply.github.com> Co-authored-by: zhulixi <48713110+zlxxlz1026@users.noreply.github.com> Co-authored-by: Stanisław Tuszyński <stanislaw@tuszynski.me>
2023-05-21 02:20:55 +03:00
[Roadmap here](https://brain.quivr.app/docs/roadmap)
2023-06-05 17:15:34 +03:00
## Key Features 🎯
2023-05-13 03:02:45 +03:00
- **Universal Data Acceptance**: Quivr can handle almost any type of data you throw at it. Text, images, code snippets, we've got you covered.
- **Generative AI**: Quivr employs advanced AI to assist you in generating and retrieving information.
- **Fast and Efficient**: Designed with speed and efficiency at its core. Quivr ensures rapid access to your data.
- **Secure**: Your data, your control. Always.
- **OS Compatible**: Ubuntu 22 or upper.
- **File Compatibility**:
- Text
- Markdown
- PDF
- Powerpoint
- Excel (Not Yet)
2023-07-12 10:24:38 +03:00
- CSV
- Word
- Audio
- Video
2023-07-12 10:24:38 +03:00
- **Open Source**: Freedom is beautiful, and so is Quivr. Open source and free to use.
2023-05-13 03:02:45 +03:00
## Demo Highlights 🎥
2023-05-13 03:16:41 +03:00
2023-05-21 03:08:21 +03:00
https://github.com/StanGirard/quivr/assets/19614572/a6463b73-76c7-4bc0-978d-70562dca71f5
2023-08-01 15:48:20 +03:00
## Disclaimer ⚠️
For a little while, Quivr will be only compatible with OpenAI API.
2023-08-01 15:48:20 +03:00
If you want to use a Local LLM please refer to [v0.0.46](https://github.com/StanGirard/quivr/releases/tag/v0.0.46).
This is due to us preparing a big feature and needing to clean the code a bit.
2023-07-12 10:24:38 +03:00
## Getting Started 🚀
2023-05-13 03:02:45 +03:00
Follow these instructions to get a copy of the project up and running on your local machine for development and testing purposes.
New Webapp migration (#56) * feat(v2): loaders added * feature: Add scroll animations * feature: upload ui * feature: upload multiple files * fix: Same file name and size remove * feat(crawler): added * feat(parsers): v2 added more * feat(v2): audio now working * feat(v2): all loaders * feat(v2): explorer * chore: add links * feat(api): added status in return message * refactor(website): remove old code * feat(upload): return type for messages * feature: redirect to upload if ENV=local * fix(chat): fixed some issues * feature: respect response type * loading state * feature: Loading stat * feat(v2): added explore and chat pages * feature: modal settings * style: Chat UI * feature: scroll to bottom when chatting * feature: smooth scroll in chat * feature(anim): Slide chat in * feature: markdown chat * feat(explorer): list * feat(doc): added document item * feat(explore): added modal * Add clarification on Project API keys and web interface for migration scripts to Readme (#58) * fix(demo): changed link * add support to uploading zip file (#62) * Catch UnicodeEncodeError exception (#64) * feature: fixed chatbar * fix(loaders): missing argument * fix: layout * fix: One whole chatbox * fix: Scroll into view * fix(build): vercel issues * chore(streamlit): moved to own file * refactor(api): moved to backend folder * feat(docker): added docker compose * Fix a bug where langchain memories were not being cleaned (#71) * Update README.md (#70) * chore(streamlit): moved to own file * refactor(api): moved to backend folder * docs(readme): updated for new version * docs(readme): added old readme * docs(readme): update copy dot env file * docs(readme): cleanup --------- Co-authored-by: iMADi-ARCH <nandanaditya985@gmail.com> Co-authored-by: Matt LeBel <github@lebel.io> Co-authored-by: Evan Carlson <45178375+EvanCarlson@users.noreply.github.com> Co-authored-by: Mustafa Hasan Khan <65130881+mustafahasankhan@users.noreply.github.com> Co-authored-by: zhulixi <48713110+zlxxlz1026@users.noreply.github.com> Co-authored-by: Stanisław Tuszyński <stanislaw@tuszynski.me>
2023-05-21 02:20:55 +03:00
2023-07-12 10:24:38 +03:00
You can find everything on the [documentation](https://brain.quivr.app/).
2023-05-13 03:02:45 +03:00
### Prerequisites 📋
2023-05-13 03:02:45 +03:00
2023-07-12 10:24:38 +03:00
Ensure you have the following installed:
2023-05-13 03:02:45 +03:00
New Webapp migration (#56) * feat(v2): loaders added * feature: Add scroll animations * feature: upload ui * feature: upload multiple files * fix: Same file name and size remove * feat(crawler): added * feat(parsers): v2 added more * feat(v2): audio now working * feat(v2): all loaders * feat(v2): explorer * chore: add links * feat(api): added status in return message * refactor(website): remove old code * feat(upload): return type for messages * feature: redirect to upload if ENV=local * fix(chat): fixed some issues * feature: respect response type * loading state * feature: Loading stat * feat(v2): added explore and chat pages * feature: modal settings * style: Chat UI * feature: scroll to bottom when chatting * feature: smooth scroll in chat * feature(anim): Slide chat in * feature: markdown chat * feat(explorer): list * feat(doc): added document item * feat(explore): added modal * Add clarification on Project API keys and web interface for migration scripts to Readme (#58) * fix(demo): changed link * add support to uploading zip file (#62) * Catch UnicodeEncodeError exception (#64) * feature: fixed chatbar * fix(loaders): missing argument * fix: layout * fix: One whole chatbox * fix: Scroll into view * fix(build): vercel issues * chore(streamlit): moved to own file * refactor(api): moved to backend folder * feat(docker): added docker compose * Fix a bug where langchain memories were not being cleaned (#71) * Update README.md (#70) * chore(streamlit): moved to own file * refactor(api): moved to backend folder * docs(readme): updated for new version * docs(readme): added old readme * docs(readme): update copy dot env file * docs(readme): cleanup --------- Co-authored-by: iMADi-ARCH <nandanaditya985@gmail.com> Co-authored-by: Matt LeBel <github@lebel.io> Co-authored-by: Evan Carlson <45178375+EvanCarlson@users.noreply.github.com> Co-authored-by: Mustafa Hasan Khan <65130881+mustafahasankhan@users.noreply.github.com> Co-authored-by: zhulixi <48713110+zlxxlz1026@users.noreply.github.com> Co-authored-by: Stanisław Tuszyński <stanislaw@tuszynski.me>
2023-05-21 02:20:55 +03:00
- Docker
- Docker Compose
Additionally, you'll need a [Supabase](https://supabase.com/) account for:
- Creating a new Supabase project
- Supabase Project API key
- Supabase Project URL
2023-05-13 03:02:45 +03:00
### Installation Steps 💽
2023-05-13 03:02:45 +03:00
2023-07-08 00:44:56 +03:00
- **Step 0**: If needed, the installation is explained on Youtube [here](https://youtu.be/rC-s4QdfY80)
2023-05-21 22:44:32 +03:00
- **Step 1**: Clone the repository using **one** of these commands:
2023-05-13 03:02:45 +03:00
- If you don't have an SSH key set up:
```bash
git clone https://github.com/StanGirard/Quivr.git && cd Quivr
```
- If you have an SSH key set up or want to add it ([guide here](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/adding-a-new-ssh-key-to-your-github-account))
```bash
git clone git@github.com:StanGirard/Quivr.git && cd Quivr
```
2023-05-13 03:02:45 +03:00
- **Step 2**: Use the install helper script to automate subsequent steps.
You can use the install_helper.sh script to setup your env files and execute the migrations.
2023-07-03 10:50:39 +03:00
prerequisites:
```bash
brew install gum # Windows (via Scoop) scoop install charm-gum
brew install postgresql # Windows (via Scoop) scoop install postgresql
```
```bash
chmod +x install_helper.sh
./install_helper.sh
```
2023-07-03 10:50:39 +03:00
If you want to manually set up the environment, follow the steps below, otherwise skip to Step 6.
2023-07-03 10:50:39 +03:00
- **Step 2 - Bis**: Copy the `.XXXXX_env` files
2023-05-13 03:13:56 +03:00
```bash
cp .backend_env.example backend/.env
cp .frontend_env.example frontend/.env
```
- **Step 3**: Update the `backend/.env` and `frontend/.env` file
> _Your `supabase_service_key` can be found in your Supabase dashboard under Project Settings -> API. Use the `anon` `public` key found in the `Project API keys` section._
2023-05-13 03:13:56 +03:00
> _Your `JWT_SECRET_KEY`can be found in your supabase settings under Project Settings -> API -> JWT Settings -> JWT Secret_
2023-05-13 03:13:56 +03:00
> _The `NEXT_PUBLIC_BACKEND_URL` is set to localhost:5050 for the docker. Update it if you are running the backend on a different machine._
> _To activate vertexAI with PaLM from GCP follow the instructions [here](https://python.langchain.com/en/latest/modules/models/llms/integrations/google_vertex_ai_palm.html) and update `backend/.env`- It is an advanced feature, please be expert in GCP before trying to use it_
- [ ] Change variables in `backend/.env`
- [ ] Change variables in `frontend/.env`
- **Step 4**: Use the `migration.sh` script to run the migration scripts
```bash
chmod +x migration.sh
./migration.sh
```
Choose either `Create all tables` if it's your first time or `Run migrations`
if you are updating your database.
Alternatively you can run the script on the Supabase database via the web
interface (SQL Editor -> `New query` -> paste the script -> `Run`)
All the scripts can be found in the [scripts](scripts/) folder
> _If you come from an old version of Quivr, run the scripts in [migration script](scripts/) to migrate your data to the new version in the order of date_
2023-06-11 20:57:54 +03:00
- **Step 5**: Launch the app
2023-05-13 03:13:56 +03:00
```bash
docker compose up --build
```
2023-05-13 03:02:45 +03:00
- **Step 6**: Navigate to `localhost:3000` in your browser
- **Step 7**: Want to contribute to the project?
```
docker compose -f docker-compose.dev.yml up --build
```
## Contributors ✨
2023-07-12 10:24:38 +03:00
Thanks go to these wonderful people:
2023-07-22 14:01:15 +03:00
<a href="https://github.com/stangirard/quivr/graphs/contributors">
<img src="https://contrib.rocks/image?repo=stangirard/quivr" />
</a>
## Contribute 🤝
2023-05-13 03:02:45 +03:00
Got a pull request? Open it, and we'll review it as soon as possible. Check out our project board [here](https://github.com/users/StanGirard/projects/5) to see what we're currently focused on, and feel free to bring your fresh ideas to the table!
2023-05-13 03:13:56 +03:00
- [Open Issues](https://github.com/StanGirard/quivr/issues)
- [Open Pull Requests](https://github.com/StanGirard/quivr/pulls)
- [Good First Issues](https://github.com/StanGirard/quivr/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22)
- [Frontend Issues](https://github.com/StanGirard/quivr/issues?q=is%3Aopen+is%3Aissue+label%3Afrontend)
- [Backend Issues](https://github.com/StanGirard/quivr/issues?q=is%3Aopen+is%3Aissue+label%3Abackend)
2023-06-30 10:40:21 +03:00
## Sponsors ❤️
This project could not be possible without the support of our sponsors. Thank you for your support!
<a href="https://www.theodo.fr/">
2023-07-01 19:50:02 +03:00
<img src="https://avatars.githubusercontent.com/u/332041?s=200&v=4" alt="Theodo" style="padding: 10px" width="70px">
</a>
<a href="https://www.padok.fr/">
2023-07-01 19:50:02 +03:00
<img src="https://avatars.githubusercontent.com/u/46325765?s=200&v=4" alt="Padok" style="padding: 10px" width="70px">
</a>
<a href="https://www.aleios.com/">
2023-07-01 19:50:02 +03:00
<img src="https://avatars.githubusercontent.com/u/97908131?s=200&v=4" alt="Aleios" style="padding: 10px" width="70px">
</a>
2023-07-01 19:52:01 +03:00
<a href="https://www.bam.tech/">
<img src="https://avatars.githubusercontent.com/u/9597329?s=200&v=4" alt="BAM" style="padding: 10px" width="70px">
</a>
2023-07-11 15:41:28 +03:00
<a href="https://www.sicara.fr/">
<img src="https://avatars.githubusercontent.com/u/23194788?s=200&v=4" alt="Sicara" style="padding: 10px" width="70px">
</a>
## License 📄
This project is licensed under the Apache 2.0 License - see the [LICENSE](LICENSE) file for details
2023-06-30 10:40:21 +03:00
## Stars History 📈
[![Star History Chart](https://api.star-history.com/svg?repos=StanGirard/quivr&type=Timeline)](https://star-history.com/#StanGirard/quivr&Timeline)