juspay/services-flake
1
1
Fork 0
mirror of https://github.com/juspay/services-flake.git synced 2024-10-05 16:37:38 +03:00
Code Issues Packages Projects Releases Wiki Activity

docs(ollama): Document how to allow unfree packages (in this case, for proprietary nvidia packages)

Browse Source
This commit is contained in:
shivaraj-bh 2024-06-24 17:29:13 +05:30
parent b291b6f35d
commit fb6de01eb6
1 changed files with 18 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

18
doc/ollama.md
View File

@ -23,6 +23,24 @@ By default Ollama uses the CPU for inference. To enable GPU acceleration:
For NVIDIA GPUs.
Firstly, allow unfree packages:
```nix
# Inside perSystem = { system, ... }: { ...
{
imports = [
"${inputs.nixpkgs}/nixos/modules/misc/nixpkgs.nix"
];
nixpkgs = {
hostPlatform = system;
# Required for CUDA
config.allowUnfree = true;
};
}
```
And then enable CUDA acceleration:
```nix
# In `perSystem.process-compose.<name>`
{