mirror of
https://github.com/StanGirard/quivr.git
synced 2024-12-18 08:02:03 +03:00
15d43c492d
# Description Please include a summary of the changes and the related issue. Please also include relevant motivation and context. ## Checklist before requesting a review Please delete options that are not relevant. - [ ] My code follows the style guidelines of this project - [ ] I have performed a self-review of my code - [ ] I have commented hard-to-understand areas - [ ] I have ideally added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [ ] Any dependent changes have been merged ## Screenshots (if appropriate):
43 lines
1.4 KiB
Plaintext
43 lines
1.4 KiB
Plaintext
---
|
|
title: Run Quivr locally with Ollama
|
|
---
|
|
The guide was put together in collaboration with members of the Quivr Discord, **Using Quivr fully locally** thread. That is a good place to discuss it. https://discord.com/invite/HUpRgp2HG8
|
|
|
|
## Ollama
|
|
|
|
Ollama is a tool that allows you to run LLMs locally. We are using it to run Llama2, MistralAI and others locally.
|
|
|
|
### Install Ollama
|
|
|
|
Install Ollama from their [website](https://ollama.ai/).
|
|
|
|
Then run the following command to run Ollama in the background:
|
|
|
|
```bash
|
|
ollama run llama2
|
|
```
|
|
|
|
|
|
## Add Ollama Model to Quivr
|
|
|
|
Now that you have your model running locally, you need to add it to Quivr.
|
|
|
|
In order to allow the user to choose between the Ollama, we need to add a new model to the Quivr backend.
|
|
|
|
Go to supabase and in the table `user_settings` either add by default or to your user the following value to the `models` column:
|
|
|
|
```json
|
|
[
|
|
"ollama/llama2",
|
|
"ollama/mistral",
|
|
]
|
|
```
|
|
|
|
This will add the Ollama model to the list of models that the user can choose from.
|
|
|
|
By adding this as default, it means that all new users will have this model by default. If you want to add it to your user only, you can add it to the `models` column in the `user_settings` table. In order for the change to take effect if you put as default your need to drop the entire table with the following command:
|
|
|
|
```sql
|
|
DELETE TABLE user_settings;
|
|
```
|