haskell-nix/hnix-store
1
1
Fork 0
mirror of https://github.com/haskell-nix/hnix-store.git synced 2025-01-08 12:28:46 +03:00
Code Issues Projects Releases Wiki Activity

Merge pull request #230 from sorki/srk/safety

Browse Source 
newtypes, Arbitrary instances, chores
This commit is contained in:
Richard Marko 2023-11-15 09:39:47 +01:00 committed by GitHub
commit 4734067134
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
21 changed files with 377 additions and 385 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

132
hnix-store-core/CHANGELOG.md Normal file
View File

@ -0,0 +1,132 @@
# Next
* Changes:
* Constructors of `StorePathName` and `StorePathHashPart` are no longer
exported. Use respective `mkStorePath..` functions. [#230](https://github.com/haskell-nix/hnix-store/pull/230)
* `StorePathSet` type alias is no more, use `HashSet StorePath` [#230](https://github.com/haskell-nix/hnix-store/pull/230)
* Additions:
* Added `Arbitrary` instances for (exported by default) [#230](https://github.com/haskell-nix/hnix-store/pull/230)
* `StorePath`
* `StorePathName`
* `StorePathHashPart`
* `StoreDir`
# [0.7.0.0](https://github.com/haskell-nix/hnix-store/compare/core-0.6.1.0...core-0.7.0.0) 2023-11-15
* Changes:
* `StorePath` no longer carries `storePathRoot` field and we
have a stand-alone `StoreDir` type instead to be used instead of `FilePath`
when store root directory is needed as a context [#216](https://github.com/haskell-nix/hnix-store/pull/216)
* Fixes:
* NAR encoding and decoding now supports case-insensitive filesystems [#218](https://github.com/haskell-nix/hnix-store/pull/218)
* The "case hack" replicates the behavior of the `use-case-hack` option in Nix, which adds a suffix to conflicting filenames.
This feature is enabled by default on macOS (darwin).
* Additions:
* `data NarOptions` has been added to configure NAR encoding and decoding. The `optUseCaseHack` field can be used to enable or disable the case hack [#218](https://github.com/haskell-nix/hnix-store/pull/218)
* New `streamNarIOWithOptions` and `runParserWithOptions` functions have been added to `System.Nix.Nar` to support the new configurable options [#218](https://github.com/haskell-nix/hnix-store/pull/218)
# [0.6.1.0](https://github.com/haskell-nix/hnix-store/compare/core-0.6.0.0...core-0.6.1.0) 2023-01-02
* Fixes:
* NAR serialization compatibility (symlinks, directory symlinks, UTF-8 handling) [#201](https://github.com/haskell-nix/hnix-store/pull/201) [#203](https://github.com/haskell-nix/hnix-store/pull/203)
# [0.6.0.0](https://github.com/haskell-nix/hnix-store/compare/core-0.5.0.0...core-0.6.0.0) 2022-06-06
* Breaking:
* `streamNarIO` changes type and returns `NarSource m` [#177](https://github.com/haskell-nix/hnix-store/pull/177)
* `FilePath` can turn to `NarSource m` using `dumpPath`
* `ByteString` can turn to `NarSource m` using `dumpString`
# [0.5.0.0](https://github.com/haskell-nix/hnix-store/compare/0.4.3.0...core-0.5.0.0) 2021-06-10
* Breaking:
* `System.Nix.Hash`:
* Migration from packages `cryptohash-` -> `cryptonite` [#157](https://github.com/haskell-nix/hnix-store/pull/157/commits/97146b41cc87327625e02b81971aeb2fd7d66a3f)
* rm `newtype Digest` in favour of `cryptonite: newtype Digest`
* rm `data HashAlgorithm` in favour of `cryptonite: class HashAlgorithm`
* rm `class ValidAlgo` in favour of `cryptonite: class HashAlgorithm`.
* `class NamedAlgo` removed `hashSize` in favour of `cryptonite: class HashAlgorithm: hashDigestSize`. Former became a subclass of the latter.
* rm `hash` in favour of `cryptonite: hash`
* rm `hashLazy` in favour of `cryptonite: hashlazy`
* Base encoding/decoding function for hashes (digests) changed (due to changes in type system & separation of specially truncated Nix Store hasing) [#157](https://github.com/haskell-nix/hnix-store/pull/157/commits/2af74986de8aef1a13dbfc955886f9935ca246a3)
* `encode(InBase -> DigestWith)`
* `decode(Base -> DigestWith)`
* `System.Nix.StorePath` [#157](https://github.com/haskell-nix/hnix-store/pull/157/commits/2af74986de8aef1a13dbfc955886f9935ca246a3)
* rm `type StorePathHashAlgo = 'Truncated 20 'SHA256` in favour of `StorePathHashPart` & `mkStorePathHashPart`.
* rm `unStorePathName`, please use `GHC: coerce` for `StorePathName <-> Text`, `StorePathName` data constructor is provided.
* `Internal` modules now have export lists, if something, please contact.
* Additional:
* Support of GHC 9.0 [#157](https://github.com/haskell-nix/hnix-store/pull/157/commits/97146b41cc87327625e02b81971aeb2fd7d66a3f)
* `System.Nix.StorePath` [#157](https://github.com/haskell-nix/hnix-store/pull/157/commits/2af74986de8aef1a13dbfc955886f9935ca246a3)
* exposed `StorePathName` data constructor to API.
* added `newtype StorePathHashPart = StorePathHashPart ByteString`.
* added builder `mkStorePathHashPart :: ByteString -> StorePathHashPart`
* `System.Nix.Hash` [#157](https://github.com/haskell-nix/hnix-store/pull/157/commits/2af74986de8aef1a13dbfc955886f9935ca246a3)
* Nix store (which are specially truncated) hashes are now handled separately from other hashes:
* add `mkStorePathHash` - a function to create a content into Nix storepath-style hash:
`mkStorePathHash :: HashAlgorithm a => ByteString -> ByteString`
but recommend to at once use `mkStorePathHashPart`.
# [0.4.3.0](https://github.com/haskell-nix/hnix-store/compare/0.4.2.0...0.4.3.0) 2021-05-30
* Additional:
* `System.Nix.ReadonlyStore`: add a readonly `computeStorePathForPath` [b85f7c8](https://github.com/haskell-nix/hnix-store/commit/b85f7c875fe6b0bca939ffbcd8b9bd0ab1598aa0)
* `System.Nix.ReadonlyStore`: `computeStorePathForPath`: force SHA256 as it's the only valid choice [db71ece](https://github.com/haskell-nix/hnix-store/commit/db71ecea3109c0ba270fa98a9041a8556e35217f)
* `makeTextPath`: order the references [5fddf3c](https://github.com/haskell-nix/hnix-store/commit/5fddf3c66ba1bcabb72c4d6b6e09fb41a7acd62c)
# [0.4.2.0](https://github.com/haskell-nix/hnix-store/compare/0.4.1.0...0.4.2.0) 2021-03-12
* Additional:
* Cabal now properly states `tasty-discover` as `build-tool-depends` [5d03ffc](https://github.com/haskell-nix/hnix-store/commit/5d03ffc4cde9448df05e84838ece70cc83b1b6c)
* Added explicit `hie.yml` cradle description for `cabal` to help Haskell Language Server to work with monorepo [5bad385](https://github.com/haskell-nix/hnix-store/commit/b5ad38573d27e0732d0fadfebd98de1f753b4f07)
* Removed vacuous `Setup.hs`, it was throwing-off HLS, and anyway file is vacuous and gets deprecated by Cabal itself [a5b7a61](https://github.com/haskell-nix/hnix-store/commit/a5b7a614c0e0e11147a93b9a197c2a443afa3244)
* Nix dev env: removed GHC 8.6.5 support, afaik it is not even in Nixpkgs anymore [cf04083](https://github.com/haskell-nix/hnix-store/commit/cf04083aba98ad40d183d1e26251101816cc07ae)
* Test suite: fixed nar test for the envs without `/proc` (test suite now works on `macOS`) [2a897ab](https://github.com/haskell-nix/hnix-store/commit/2a897ab581c0501587ce04da6d6e3a6f543b1d72)
# [0.4.1.0](https://github.com/haskell-nix/hnix-store/compare/0.4.0.0...0.4.1.0) 2021-01-16
* Big clean-up of dependencies.
# [0.4.0.0](https://github.com/haskell-nix/hnix-store/compare/0.3.0.0...0.4.0.0) 2020-12-30
* `System.Nix.Hash` no longer exports `encodeBase16, decodeBase16` and their `Base32` counterparts.
These were replaced by `encodeInBase` and `decodeBase` functions
accepting `BaseEncoding` data type [#87](https://github.com/haskell-nix/hnix-store/pull/87)
* Support `base16-bytestring >= 1` [#86](https://github.com/haskell-nix/hnix-store/pull/86) [#100](https://github.com/haskell-nix/hnix-store/pull/100)
# 0.3.0.0 -- 2020-11-29
* `System.Nix.Nar` changes API to support NAR format streaming:
* `buildNarIO :: FilePath -> Handle -> IO ()` - Create a NAR from a regular filesystem object, stream it out on the Handle
* `unpackNarIO :: Handle -> FilePath -> IO ()` - Recreate filesystem object from a NAR file accessed by the Handle
* `StorePath` type changed to simple variant without type level
symbolic store path root.
* Added `makeFixedOutputPath` to `System.Nix.ReadonlyStore`
* Added `decodeBase16` and `decodeBase32` to `System.Nix.Hash`
* `System.Nix.StorePath` module now provides
* `storePathToFilePath` and `storePathToText` helpers
* `storePathToNarInfo` for converting paths to `narinfo` URLs
* `parsePath` function
* `pathParser` Attoparsec parser
* Added `System.Nix.Build` module
* Added `System.Nix.Derivation` module
* Removed `System.Nix.Util` module, moved to `hnix-store-remote`
* Added base64 and SHA512 hash support
# 0.2.0.0 -- 2020-03-12
Removed `System.Nix.Store`. We may reintroduce it later when multiple backends
exist and we can tell what common effects they should share.
# 0.1.0.0 -- 2019-03-18
* First version.

125
hnix-store-core/ChangeLog.md
View File

@ -1,125 +0,0 @@
# ChangeLog
## [0.7.0.0](https://github.com/haskell-nix/hnix-store/compare/core-0.6.1.0...core-0.7.0.0) 2023-11-15
* Breaking:
* [(link)](https://github.com/haskell-nix/hnix-store/pull/216) `StorePath` no longer carries `storePathRoot` field and we
have a stand-alone `StoreDir` type instead to be used instead of `FilePath`
when store root directory is needed as a context.
* Additional:
* [(link)](https://github.com/haskell-nix/hnix-store/pull/218) NAR encoding and decoding now supports case-insensitive filesystems.
* The "case hack" replicates the behavior of the `use-case-hack` option in Nix, which adds a suffix to conflicting filenames.
This feature is enabled by default on macOS (darwin).
* `data NarOptions` has been added to configure NAR encoding and decoding. The `optUseCaseHack` field can be used to enable or disable the case hack.
* New `streamNarIOWithOptions` and `runParserWithOptions` functions have been added to `System.Nix.Nar` to support the new configurable options.
## [0.6.1.0](https://github.com/haskell-nix/hnix-store/compare/core-0.6.0.0...core-0.6.1.0) 2023-01-02
* Fixed:
* [(link)](https://github.com/haskell-nix/hnix-store/pull/201) [(link)](https://github.com/haskell-nix/hnix-store/pull/203) NAR serialization compatibility (symlinks, directory symlinks, UTF-8 handling)
## [0.6.0.0](https://github.com/haskell-nix/hnix-store/compare/core-0.5.0.0...core-0.6.0.0) 2022-06-06
* Breaking:
* [(link)](https://github.com/haskell-nix/hnix-store/pull/177) `streamNarIO` changes type and returns `NarSource m`
* `FilePath` can turn to `NarSource m` using `dumpPath`
* `ByteString` can turn to `NarSource m` using `dumpString`
## [0.5.0.0](https://github.com/haskell-nix/hnix-store/compare/0.4.3.0...core-0.5.0.0) 2021-06-10
* Breaking:
* `System.Nix.Hash`:
* [(link)](https://github.com/haskell-nix/hnix-store/pull/157/commits/97146b41cc87327625e02b81971aeb2fd7d66a3f) Migration from packages `cryptohash-` -> `cryptonite`:
* rm `newtype Digest` in favour of `cryptonite: newtype Digest`
* rm `data HashAlgorithm` in favour of `cryptonite: class HashAlgorithm`
* rm `class ValidAlgo` in favour of `cryptonite: class HashAlgorithm`.
* `class NamedAlgo` removed `hashSize` in favour of `cryptonite: class HashAlgorithm: hashDigestSize`. Former became a subclass of the latter.
* rm `hash` in favour of `cryptonite: hash`
* rm `hashLazy` in favour of `cryptonite: hashlazy`
* [(link)](https://github.com/haskell-nix/hnix-store/pull/157/commits/2af74986de8aef1a13dbfc955886f9935ca246a3) Base encoding/decoding function for hashes (digests) changed (due to changes in type system & separation of specially truncated Nix Store hasing):
* `encode(InBase -> DigestWith)`
* `decode(Base -> DigestWith)`
* [(link)](https://github.com/haskell-nix/hnix-store/pull/157/commits/2af74986de8aef1a13dbfc955886f9935ca246a3) `System.Nix.StorePath`:
* rm `type StorePathHashAlgo = 'Truncated 20 'SHA256` in favour of `StorePathHashPart` & `mkStorePathHashPart`.
* rm `unStorePathName`, please use `GHC: coerce` for `StorePathName <-> Text`, `StorePathName` data constructor is provided.
* `Internal` modules now have export lists, if something, please contact.
* Additional:
* [(link)](https://github.com/haskell-nix/hnix-store/pull/157/commits/97146b41cc87327625e02b81971aeb2fd7d66a3f) Support of GHC 9.0.
* [(link)](https://github.com/haskell-nix/hnix-store/pull/157/commits/2af74986de8aef1a13dbfc955886f9935ca246a3) `System.Nix.StorePath`:
* exposed `StorePathName` data constructor to API.
* added `newtype StorePathHashPart = StorePathHashPart ByteString`.
* added builder `mkStorePathHashPart :: ByteString -> StorePathHashPart`
* [(link)](https://github.com/haskell-nix/hnix-store/pull/157/commits/2af74986de8aef1a13dbfc955886f9935ca246a3) `System.Nix.Hash`:
* Nix store (which are specially truncated) hashes are now handled separately from other hashes:
* add `mkStorePathHash` - a function to create a content into Nix storepath-style hash:
`mkStorePathHash :: HashAlgorithm a => ByteString -> ByteString`
but recommend to at once use `mkStorePathHashPart`.
## [0.4.3.0](https://github.com/haskell-nix/hnix-store/compare/0.4.2.0...0.4.3.0) 2021-05-30
* Additional:
* [(link)](https://github.com/haskell-nix/hnix-store/commit/b85f7c875fe6b0bca939ffbcd8b9bd0ab1598aa0) `System.Nix.ReadonlyStore`: add a readonly `computeStorePathForPath`
* [(link)](https://github.com/haskell-nix/hnix-store/commit/db71ecea3109c0ba270fa98a9041a8556e35217f) `System.Nix.ReadonlyStore`: `computeStorePathForPath`: force SHA256 as it's the only valid choice
* [(link)](https://github.com/haskell-nix/hnix-store/commit/5fddf3c66ba1bcabb72c4d6b6e09fb41a7acd62c): `makeTextPath`: order the references
## [0.4.2.0](https://github.com/haskell-nix/hnix-store/compare/0.4.1.0...0.4.2.0) 2021-03-12
* Additional:
* [(link)](https://github.com/haskell-nix/hnix-store/commit/5d03ffc43cde9448df05e84838ece70cc83b1b6c) Cabal now properly states `tasty-discover` as `build-tool-depends`.
* [(link)](https://github.com/haskell-nix/hnix-store/commit/b5ad38573d27e0732d0fadfebd98de1f753b4f07) added explicit `hie.yml` cradle description for `cabal` to help Haskell Language Server to work with monorepo.
* [(link)](https://github.com/haskell-nix/hnix-store/commit/a5b7a614c0e0e11147a93b9a197c2a443afa3244) rm vacuous `Setup.hs`, it was throwing-off HLS, and anyway file is vacuous and gets deprecated by Cabal itself.
* [(link)](https://github.com/haskell-nix/hnix-store/commit/cf04083aba98ad40d183d1e26251101816cc07ae) Nix dev env: removed GHC 8.6.5 support, afaik it is not even in Nixpkgs anymore.
* [(link)](https://github.com/haskell-nix/hnix-store/commit/2a897ab581c0501587ce04da6d6e3a6f543b1d72) Test suite: fixed nar test for the envs without `/proc` (test suite now works on `macOS`).
## [0.4.1.0](https://github.com/haskell-nix/hnix-store/compare/0.4.0.0...0.4.1.0) 2021-01-16
* Big clean-up of dependencies.
## [0.4.0.0](https://github.com/haskell-nix/hnix-store/compare/0.3.0.0...0.4.0.0) 2020-12-30
* `System.Nix.Hash` no longer exports `encodeBase16, decodeBase16` and their `Base32` counterparts.
These were replaced by `encodeInBase` and `decodeBase` functions
accepting `BaseEncoding` data type [#87](https://github.com/haskell-nix/hnix-store/pull/87)
* Support `base16-bytestring >= 1` [#86](https://github.com/haskell-nix/hnix-store/pull/86) [#100](https://github.com/haskell-nix/hnix-store/pull/100)
## 0.3.0.0 -- 2020-11-29
* `System.Nix.Nar` changes API to support NAR format streaming:
* `buildNarIO :: FilePath -> Handle -> IO ()` - Create a NAR from a regular filesystem object, stream it out on the Handle
* `unpackNarIO :: Handle -> FilePath -> IO ()` - Recreate filesystem object from a NAR file accessed by the Handle
* `StorePath` type changed to simple variant without type level
symbolic store path root.
* Added `makeFixedOutputPath` to `System.Nix.ReadonlyStore`
* Added `decodeBase16` and `decodeBase32` to `System.Nix.Hash`
* `System.Nix.StorePath` module now provides
* `storePathToFilePath` and `storePathToText` helpers
* `storePathToNarInfo` for converting paths to `narinfo` URLs
* `parsePath` function
* `pathParser` Attoparsec parser
* Added `System.Nix.Build` module
* Added `System.Nix.Derivation` module
* Removed `System.Nix.Util` module, moved to `hnix-store-remote`
* Added base64 and SHA512 hash support
## 0.2.0.0 -- 2020-03-12
Removed `System.Nix.Store`. We may reintroduce it later when multiple backends
exist and we can tell what common effects they should share.
## 0.1.0.0 -- 2019-03-18
* First version.

4
hnix-store-core/hnix-store-core.cabal
View File

@ -15,7 +15,7 @@ copyright: 2018 Shea Levy
category: Nix category: Nix
build-type: Simple build-type: Simple
extra-source-files: extra-source-files:
ChangeLog.md CHANGELOG.md
, README.md , README.md
, tests/samples/example0.drv , tests/samples/example0.drv
, tests/samples/example1.drv , tests/samples/example1.drv
@ -70,6 +70,7 @@ library
, monad-control , monad-control
, mtl , mtl
, nix-derivation >= 1.1.1 && <2 , nix-derivation >= 1.1.1 && <2
, QuickCheck
, saltine , saltine
, time , time
, text , text
@ -114,7 +115,6 @@ test-suite format-tests
type: exitcode-stdio-1.0 type: exitcode-stdio-1.0
main-is: Driver.hs main-is: Driver.hs
other-modules: other-modules:
Arbitrary
Derivation Derivation
NarFormat NarFormat
Hash Hash

39
hnix-store-core/src/System/Nix/Internal/StorePath.hs
View File

@ -13,9 +13,8 @@ module System.Nix.Internal.StorePath
StoreDir(..) StoreDir(..)
, StorePath(..) , StorePath(..)
, StorePathName(..) , StorePathName(..)
, StorePathSet
, mkStorePathHashPart
, StorePathHashPart(..) , StorePathHashPart(..)
, mkStorePathHashPart
, ContentAddressableAddress(..) , ContentAddressableAddress(..)
, NarHashMode(..) , NarHashMode(..)
, -- * Manipulating 'StorePathName' , -- * Manipulating 'StorePathName'
@ -47,8 +46,11 @@ import qualified Data.Attoparsec.Text.Lazy as Parser.Text.Lazy
import qualified System.FilePath as FilePath import qualified System.FilePath as FilePath
import Crypto.Hash ( SHA256 import Crypto.Hash ( SHA256
, Digest , Digest
, HashAlgorithm
) )
import Test.QuickCheck
-- | A path in a Nix store. -- | A path in a Nix store.
-- --
-- From the Nix thesis: A store path is the full path of a store -- From the Nix thesis: A store path is the full path of a store
@ -72,6 +74,12 @@ instance Hashable StorePath where
hashWithSalt s StorePath{..} = hashWithSalt s StorePath{..} =
s `hashWithSalt` storePathHash `hashWithSalt` storePathName s `hashWithSalt` storePathHash `hashWithSalt` storePathName
instance Arbitrary StorePath where
arbitrary =
liftA2 StorePath
arbitrary
arbitrary
-- | The name portion of a Nix path. -- | The name portion of a Nix path.
-- --
-- 'unStorePathName' must only contain a-zA-Z0-9+._?=-, can't start -- 'unStorePathName' must only contain a-zA-Z0-9+._?=-, can't start
@ -82,15 +90,29 @@ newtype StorePathName = StorePathName
unStorePathName :: Text unStorePathName :: Text
} deriving (Eq, Hashable, Ord, Show) } deriving (Eq, Hashable, Ord, Show)
instance Arbitrary StorePathName where
arbitrary = StorePathName . toText <$> ((:) <$> s1 <*> listOf sn)
where
alphanum = ['a' .. 'z'] <> ['A' .. 'Z'] <> ['0' .. '9']
s1 = elements $ alphanum <> "+-_?="
sn = elements $ alphanum <> "+-._?="
-- | The hash algorithm used for store path hashes. -- | The hash algorithm used for store path hashes.
newtype StorePathHashPart = StorePathHashPart ByteString newtype StorePathHashPart = StorePathHashPart
{ -- | Extract the contents of the hash.
unStorePathHashPart :: ByteString
}
deriving (Eq, Hashable, Ord, Show) deriving (Eq, Hashable, Ord, Show)
mkStorePathHashPart :: ByteString -> StorePathHashPart instance Arbitrary StorePathHashPart where
mkStorePathHashPart = coerce . mkStorePathHash @SHA256 arbitrary = mkStorePathHashPart @SHA256 . Bytes.Char8.pack <$> arbitrary
-- | A set of 'StorePath's. mkStorePathHashPart
type StorePathSet = HashSet StorePath :: forall hashAlgo
. HashAlgorithm hashAlgo
=> ByteString
-> StorePathHashPart
mkStorePathHashPart = coerce . mkStorePathHash @hashAlgo
-- | An address for a content-addressable store path, i.e. one whose -- | An address for a content-addressable store path, i.e. one whose
-- store path hash is purely a function of its contents (as opposed to -- store path hash is purely a function of its contents (as opposed to
@ -163,6 +185,9 @@ newtype StoreDir = StoreDir {
unStoreDir :: RawFilePath unStoreDir :: RawFilePath
} deriving (Eq, Hashable, Ord, Show) } deriving (Eq, Hashable, Ord, Show)
instance Arbitrary StoreDir where
arbitrary = StoreDir . ("/" <>) . Bytes.Char8.pack <$> arbitrary
-- | Render a 'StorePath' as a 'RawFilePath'. -- | Render a 'StorePath' as a 'RawFilePath'.
storePathToRawFilePath :: StoreDir -> StorePath -> RawFilePath storePathToRawFilePath :: StoreDir -> StorePath -> RawFilePath
storePathToRawFilePath storeDir StorePath{..} = storePathToRawFilePath storeDir StorePath{..} =

18
hnix-store-core/src/System/Nix/ReadonlyStore.hs
View File

@ -22,27 +22,27 @@ import Crypto.Hash ( Context
makeStorePath makeStorePath
:: forall h :: forall hashAlgo
. (NamedAlgo h) . (NamedAlgo hashAlgo)
=> StoreDir => StoreDir
-> ByteString -> ByteString
-> Digest h -> Digest hashAlgo
-> StorePathName -> StorePathName
-> StorePath -> StorePath
makeStorePath storeDir ty h nm = StorePath (coerce storeHash) nm makeStorePath storeDir ty h nm = StorePath storeHash nm
where where
storeHash = mkStorePathHash @h s storeHash = mkStorePathHashPart @hashAlgo s
s = s =
BS.intercalate ":" $ BS.intercalate ":" $
ty:fmap encodeUtf8 ty:fmap encodeUtf8
[ algoName @h [ algoName @hashAlgo
, encodeDigestWith Base16 h , encodeDigestWith Base16 h
, toText . Bytes.Char8.unpack $ unStoreDir storeDir , toText . Bytes.Char8.unpack $ unStoreDir storeDir
, coerce nm , unStorePathName nm
] ]
makeTextPath makeTextPath
:: StoreDir -> StorePathName -> Digest SHA256 -> StorePathSet -> StorePath :: StoreDir -> StorePathName -> Digest SHA256 -> HashSet StorePath -> StorePath
makeTextPath storeDir nm h refs = makeStorePath storeDir ty h nm makeTextPath storeDir nm h refs = makeStorePath storeDir ty h nm
where where
ty = ty =
@ -70,7 +70,7 @@ makeFixedOutputPath storeDir recursive h =
<> ":" <> ":"
computeStorePathForText computeStorePathForText
:: StoreDir -> StorePathName -> ByteString -> (StorePathSet -> StorePath) :: StoreDir -> StorePathName -> ByteString -> (HashSet StorePath -> StorePath)
computeStorePathForText storeDir nm = makeTextPath storeDir nm . hash computeStorePathForText storeDir nm = makeTextPath storeDir nm . hash
computeStorePathForPath computeStorePathForPath

7
hnix-store-core/src/System/Nix/StorePath.hs
View File

@ -5,14 +5,15 @@ module System.Nix.StorePath
( -- * Basic store path types ( -- * Basic store path types
StoreDir(..) StoreDir(..)
, StorePath(..) , StorePath(..)
, StorePathName(..) , StorePathName
, StorePathSet , StorePathHashPart
, mkStorePathHashPart , mkStorePathHashPart
, StorePathHashPart(..) , unStorePathHashPart
, ContentAddressableAddress(..) , ContentAddressableAddress(..)
, NarHashMode(..) , NarHashMode(..)
, -- * Manipulating 'StorePathName' , -- * Manipulating 'StorePathName'
makeStorePathName makeStorePathName
, unStorePathName
, validStorePathName , validStorePathName
, -- * Rendering out 'StorePath's , -- * Rendering out 'StorePath's
storePathToFilePath storePathToFilePath

3
hnix-store-core/src/System/Nix/StorePathMetadata.hs
View File

@ -4,7 +4,6 @@ Description : Metadata about Nix store paths.
module System.Nix.StorePathMetadata where module System.Nix.StorePathMetadata where
import System.Nix.StorePath ( StorePath import System.Nix.StorePath ( StorePath
, StorePathSet
, ContentAddressableAddress , ContentAddressableAddress
) )
import System.Nix.Hash ( SomeNamedDigest ) import System.Nix.Hash ( SomeNamedDigest )
@ -22,7 +21,7 @@ data StorePathMetadata = StorePathMetadata
-- | The hash of the nar serialization of the path. -- | The hash of the nar serialization of the path.
narHash :: !SomeNamedDigest narHash :: !SomeNamedDigest
, -- | The paths that this path directly references , -- | The paths that this path directly references
references :: !StorePathSet references :: !(HashSet StorePath)
, -- | When was this path registered valid in the store? , -- | When was this path registered valid in the store?
registrationTime :: !UTCTime registrationTime :: !UTCTime
, -- | The size of the nar serialization of the path, in bytes. , -- | The size of the nar serialization of the path, in bytes.

58
hnix-store-core/tests/Arbitrary.hs
View File

@ -1,58 +0,0 @@
{-# language DataKinds #-}
{-# OPTIONS_GHC -Wno-orphans #-}
module Arbitrary where
import qualified Data.ByteString.Char8 as BSC
import Test.Tasty.QuickCheck
import System.Nix.Internal.StorePath
import Crypto.Hash ( SHA256
, Digest
, hash
)
genSafeChar :: Gen Char
genSafeChar = choose ('\1', '\127') -- ASCII without \NUL
nonEmptyString :: Gen String
nonEmptyString = listOf1 genSafeChar
dir :: Gen String
dir = ('/':) <$> listOf1 (elements $ '/':['a'..'z'])
instance Arbitrary StorePathName where
arbitrary = StorePathName . toText <$> ((:) <$> s1 <*> listOf sn)
where
alphanum = ['a' .. 'z'] <> ['A' .. 'Z'] <> ['0' .. '9']
s1 = elements $ alphanum <> "+-_?="
sn = elements $ alphanum <> "+-._?="
instance Arbitrary StorePathHashPart where
arbitrary = mkStorePathHashPart . BSC.pack <$> arbitrary
instance Arbitrary (Digest SHA256) where
arbitrary = hash . BSC.pack <$> arbitrary
instance Arbitrary StoreDir where
arbitrary = StoreDir . ("/" <>) . BSC.pack <$> arbitrary
newtype NixLike = NixLike {getNixLike :: StorePath}
deriving (Eq, Ord, Show)
instance Arbitrary NixLike where
arbitrary =
NixLike <$>
liftA2 StorePath
arbitraryTruncatedDigest
arbitrary
where
-- 160-bit hash, 20 bytes, 32 chars in base32
arbitraryTruncatedDigest = coerce . BSC.pack <$> replicateM 20 genSafeChar
instance Arbitrary StorePath where
arbitrary =
liftA2 StorePath
arbitrary
arbitrary

12
hnix-store-core/tests/Hash.hs
View File

@ -13,7 +13,6 @@ import Test.Tasty.QuickCheck
import System.Nix.Hash import System.Nix.Hash
import System.Nix.StorePath import System.Nix.StorePath
import Arbitrary
import System.Nix.Internal.Base import System.Nix.Internal.Base
import Crypto.Hash ( MD5 import Crypto.Hash ( MD5
, SHA1 , SHA1
@ -37,7 +36,7 @@ spec_hash = do
-- The example in question: -- The example in question:
-- https://nixos.org/nixos/nix-pills/nix-store-paths.html -- https://nixos.org/nixos/nix-pills/nix-store-paths.html
it "produces same base32 as nix pill flat file example" $ do it "produces same base32 as nix pill flat file example" $ do
shouldBe (encodeWith NixBase32 $ coerce $ mkStorePathHashPart "source:sha256:2bfef67de873c54551d884fdab3055d84d573e654efa79db3c0d7b98883f9ee3:/nix/store:myfile") shouldBe (encodeWith NixBase32 $ unStorePathHashPart $ mkStorePathHashPart @SHA256 "source:sha256:2bfef67de873c54551d884fdab3055d84d573e654efa79db3c0d7b98883f9ee3:/nix/store:myfile")
"xv2iccirbrvklck36f1g7vldn5v58vck" "xv2iccirbrvklck36f1g7vldn5v58vck"
where where
cmp :: String -> BaseEncoding -> (ByteString -> Digest a) -> ByteString -> Text -> SpecWith () cmp :: String -> BaseEncoding -> (ByteString -> Digest a) -> ByteString -> Text -> SpecWith ()
@ -49,10 +48,17 @@ spec_hash = do
prop_nixBase32Roundtrip :: Property prop_nixBase32Roundtrip :: Property
prop_nixBase32Roundtrip = forAllShrink nonEmptyString genericShrink $ prop_nixBase32Roundtrip = forAllShrink nonEmptyString genericShrink $
\x -> pure (encodeUtf8 x) === (B32.decode . B32.encode . encodeUtf8 $ x) \x -> pure (encodeUtf8 x) === (B32.decode . B32.encode . encodeUtf8 $ x)
where
nonEmptyString :: Gen String
nonEmptyString = listOf1 genSafeChar
genSafeChar :: Gen Char
genSafeChar = choose ('\1', '\127') -- ASCII without \NUL
-- | API variants -- | API variants
prop_nixBase16Roundtrip :: StorePathHashPart -> Property prop_nixBase16Roundtrip :: StorePathHashPart -> Property
prop_nixBase16Roundtrip x = pure (coerce x) === decodeWith Base16 (encodeWith Base16 $ coerce x) prop_nixBase16Roundtrip x =
pure (unStorePathHashPart x) === decodeWith Base16 (encodeWith Base16 $ unStorePathHashPart x)
-- | Hash encoding conversion ground-truth. -- | Hash encoding conversion ground-truth.
-- Similiar to nix/tests/hash.sh -- Similiar to nix/tests/hash.sh

21
hnix-store-core/tests/StorePath.hs
View File

@ -9,22 +9,13 @@ import qualified Data.Attoparsec.Text
import Test.Tasty.QuickCheck import Test.Tasty.QuickCheck
import System.Nix.StorePath import System.Nix.StorePath
import Arbitrary
-- | Test that Nix(OS) like paths roundtrip -- | Test @StorePath@ roundtrips using @parsePath@
prop_storePathRoundtrip :: StoreDir -> NixLike -> NixLike -> Property prop_storePathRoundtrip :: StoreDir -> StorePath -> Property
prop_storePathRoundtrip storeDir (_ :: NixLike) (NixLike x) = prop_storePathRoundtrip storeDir x =
parsePath storeDir (storePathToRawFilePath storeDir x) === pure x parsePath storeDir (storePathToRawFilePath storeDir x) === pure x
-- | Test that any `StorePath` roundtrips -- | Test @StorePath@ roundtrips using @pathParser@
prop_storePathRoundtrip' :: StoreDir -> StorePath -> Property prop_storePathRoundtripParser :: StoreDir -> StorePath -> Property
prop_storePathRoundtrip' storeDir x = prop_storePathRoundtripParser storeDir x =
parsePath storeDir (storePathToRawFilePath storeDir x) === pure x
prop_storePathRoundtripParser :: StoreDir -> NixLike -> NixLike -> Property
prop_storePathRoundtripParser storeDir (_ :: NixLike) (NixLike x) =
Data.Attoparsec.Text.parseOnly (pathParser storeDir) (storePathToText storeDir x) === pure x
prop_storePathRoundtripParser' :: StoreDir -> StorePath -> Property
prop_storePathRoundtripParser' storeDir x =
Data.Attoparsec.Text.parseOnly (pathParser storeDir) (storePathToText storeDir x) === pure x Data.Attoparsec.Text.parseOnly (pathParser storeDir) (storePathToText storeDir x) === pure x

54
hnix-store-remote/CHANGELOG.md Normal file
View File

@ -0,0 +1,54 @@
# Unreleased 202y-mm-dd
* Changes:
* `StorePath` no longer carries `storePathRoot` field and we
have a stand-alone `StoreDir` type instead to be used instead of `FilePath`
when store root directory is needed as a context.
Fore `-remote`, this affects `runStoreOpts` and its variants [#216](https://github.com/haskell-nix/hnix-store/pull/216)
# [0.6.0.0](https://github.com/haskell-nix/hnix-store/compare/remote-0.5.0.0...remote-0.6.0.0) 2021-06-06
* Changes:
* `System.Nix.Store.Remote` [#179](https://github.com/haskell-nix/hnix-store/pull/179)
* `addToStore` no longer accepts `FilePath` as its second argument but uses
more generic `NarSource` [(NarSource PR)](https://github.com/haskell-nix/hnix-store/pull/177)
# [0.5.0.0](https://github.com/haskell-nix/hnix-store/compare/0.4.3.0...remote-0.5.0.0) 2021-06-11
* Changes:
* `System.Nix.Store.Remote` [#161](https://github.com/haskell-nix/hnix-store/pull/161)
* `addToStore`: constraint of `ValidAlgo a` removed in favour of constraint on `cryptonite: HashAlgorithm a` through constraint `NamedAlgo a`.
* `queryPathFromHashPart`: 1st arg changed from `Digest StorePathHashAlgo` to `StorePathHashPart`, for details: [hnix-store-core 0.5.0.0 ChangeLog](https://hackage.haskell.org/package/hnix-store-core-0.5.0.0/changelog).
# [0.4.3.0](https://github.com/haskell-nix/hnix-store/compare/0.4.2.0...0.4.3.0) 2021-05-30
Nothing (it is tandem `hnix-store-core` fix release)
# [0.4.2.0](https://github.com/haskell-nix/hnix-store/compare/0.4.1.0...0.4.2.0) 2021-03-12
* Additions:
* Cabal now properly states `tasty-discover` as `build-tool-depends` [#130](https://github.com/haskell-nix/hnix-store/pull/130)
* added explicit `hie.yml` cradle description for `cabal` to help Haskell Language Server to work with monorepo. [#132](https://github.com/haskell-nix/hnix-store/pull/132)
* Nix dev env: removed GHC 8.6.5 support, afaik it is not even in Nixpkgs anymore [#136](https://github.com/haskell-nix/hnix-store/pull/136)
# [0.4.1.0](https://github.com/haskell-nix/hnix-store/compare/0.4.0.0...0.4.1.0) 2021-01-16
* `System.Nix.Store.Remote`: module API now re-exports `System.Nix.Store.Remote.Types` API
* Big clean-up of dependencies.
# [0.4.0.0](https://github.com/haskell-nix/hnix-store/compare/0.3.0.0...0.4.0.0) 2020-12-30
* `hnix-store-core` compatibility
# 0.3.0.0 -- 2020-11-29
* Restored most store API functions except `addToStoreNar`
* Added `buildDerivation`
# 0.2.0.0 -- skipped
* `hnix-store-core` release only
# 0.1.0.0 -- 2019-03-18
* First version.

60
hnix-store-remote/ChangeLog.md
View File

@ -1,60 +0,0 @@
# Revision history for hnix-store-remote
## Unreleased 202y-mm-dd
* Breaking:
* [(link)](https://github.com/haskell-nix/hnix-store/pull/216) `StorePath` no longer carries `storePathRoot` field and we
have a stand-alone `StoreDir` type instead to be used instead of `FilePath`
when store root directory is needed as a context.
Fore `-remote`, this affects `runStoreOpts` and its variants.
## [0.6.0.0](https://github.com/haskell-nix/hnix-store/compare/remote-0.5.0.0...remote-0.6.0.0) 2021-06-06
* Breaking:
* [(link)](https://github.com/haskell-nix/hnix-store/pull/179) `System.Nix.Store.Remote`:
* `addToStore` no longer accepts `FilePath` as its second argument but uses
more generic `NarSource` [(NarSource PR)](https://github.com/haskell-nix/hnix-store/pull/177)
## [0.5.0.0](https://github.com/haskell-nix/hnix-store/compare/0.4.3.0...remote-0.5.0.0) 2021-06-11
* Breaking:
* [(link)](https://github.com/haskell-nix/hnix-store/commit/3b948d112aa9307b0451258f28c7ee5dc86b24c7) `System.Nix.Store.Remote`:
* `addToStore`: constraint of `ValidAlgo a` removed in favour of constraint on `cryptonite: HashAlgorithm a` through constraint `NamedAlgo a`.
* `queryPathFromHashPart`: 1st arg changed from `Digest StorePathHashAlgo` to `StorePathHashPart`, for details: [hnix-store-core 0.5.0.0 ChangeLog](https://hackage.haskell.org/package/hnix-store-core-0.5.0.0/changelog).
## [0.4.3.0](https://github.com/haskell-nix/hnix-store/compare/0.4.2.0...0.4.3.0) 2021-05-30
Nothing (it is tandem `hnix-store-core` fix release)
## [0.4.2.0](https://github.com/haskell-nix/hnix-store/compare/0.4.1.0...0.4.2.0) 2021-03-12
* Additional:
* [(link)](https://github.com/haskell-nix/hnix-store/commit/5d03ffc43cde9448df05e84838ece70cc83b1b6c) Cabal now properly states `tasty-discover` as `build-tool-depends`.
* [(link)](https://github.com/haskell-nix/hnix-store/commit/b5ad38573d27e0732d0fadfebd98de1f753b4f07) added explicit `hie.yml` cradle description for `cabal` to help Haskell Language Server to work with monorepo.
* [(link)](https://github.com/haskell-nix/hnix-store/commit/cf04083aba98ad40d183d1e26251101816cc07ae) Nix dev env: removed GHC 8.6.5 support, afaik it is not even in Nixpkgs anymore.
## [0.4.1.0](https://github.com/haskell-nix/hnix-store/compare/0.4.0.0...0.4.1.0) 2021-01-16
* `System.Nix.Store.Remote`: module API now re-exports `System.Nix.Store.Remote.Types` API
* Big clean-up of dependencies.
## [0.4.0.0](https://github.com/haskell-nix/hnix-store/compare/0.3.0.0...0.4.0.0) 2020-12-30
* `hnix-store-core` compatibility
## 0.3.0.0 -- 2020-11-29
* Restored most store API functions except `addToStoreNar`
* Added `buildDerivation`
## 0.2.0.0 -- skipped
* `hnix-store-core` release only
## 0.1.0.0 -- 2019-03-18
* First version.

2
hnix-store-remote/README.md
View File

@ -25,6 +25,6 @@ main = do
roots <- findRoots roots <- findRoots
liftIO $ print roots liftIO $ print roots
res <- addTextToStore "hnix-store" "test" mempty False res <- addTextToStore "hnix-store" "test" mempty dontRepair
liftIO $ print res liftIO $ print res
``` ```

75
hnix-store-remote/hnix-store-remote.cabal
View File

@ -11,13 +11,37 @@ maintainer: srk@48.io
copyright: 2018 Richard Marko copyright: 2018 Richard Marko
category: Nix category: Nix
build-type: Simple build-type: Simple
extra-source-files: ChangeLog.md, README.md extra-source-files:
CHANGELOG.md
, README.md
, README.lhs
Common commons common commons
if impl(ghc >= 8.10) if impl(ghc >= 8.10)
ghc-options: -Wall -Wunused-packages ghc-options: -Wall -Wunused-packages
else else
ghc-options: -Wall ghc-options: -Wall
default-extensions:
OverloadedStrings
, DeriveGeneric
, DeriveDataTypeable
, DeriveFunctor
, DeriveFoldable
, DeriveTraversable
, DeriveLift
, FlexibleContexts
, FlexibleInstances
, RecordWildCards
, ScopedTypeVariables
, StandaloneDeriving
, TypeApplications
, TypeSynonymInstances
, InstanceSigs
, MultiParamTypeClasses
, TupleSections
, LambdaCase
, BangPatterns
, ViewPatterns
flag io-testsuite flag io-testsuite
default: default:
@ -64,25 +88,6 @@ library
base hiding (Prelude) base hiding (Prelude)
, relude (Relude as Prelude) , relude (Relude as Prelude)
, relude , relude
default-extensions:
OverloadedStrings
, DeriveGeneric
, DeriveDataTypeable
, DeriveFunctor
, DeriveFoldable
, DeriveTraversable
, DeriveLift
, FlexibleContexts
, FlexibleInstances
, StandaloneDeriving
, TypeApplications
, TypeSynonymInstances
, InstanceSigs
, MultiParamTypeClasses
, TupleSections
, LambdaCase
, BangPatterns
, ViewPatterns
hs-source-dirs: src hs-source-dirs: src
default-language: Haskell2010 default-language: Haskell2010
ghc-options: -Wall ghc-options: -Wall
@ -97,7 +102,7 @@ executable remote-readme
main-is: README.lhs main-is: README.lhs
ghc-options: -pgmL markdown-unlit -Wall ghc-options: -pgmL markdown-unlit -Wall
test-suite hnix-store-remote-tests test-suite remote-io
import: commons import: commons
if !flag(io-testsuite) || os(darwin) if !flag(io-testsuite) || os(darwin)
@ -110,10 +115,7 @@ test-suite hnix-store-remote-tests
other-modules: other-modules:
NixDaemon NixDaemon
, Spec , Spec
, Util hs-source-dirs: tests-io
hs-source-dirs: tests
build-tool-depends:
tasty-discover:tasty-discover
build-depends: build-depends:
base base
, bytestring , bytestring
@ -135,27 +137,10 @@ test-suite hnix-store-remote-tests
, temporary , temporary
, unix , unix
, unordered-containers , unordered-containers
build-tool-depends:
tasty-discover:tasty-discover
mixins: mixins:
base hiding (Prelude) base hiding (Prelude)
, relude (Relude as Prelude) , relude (Relude as Prelude)
, relude , relude
default-extensions:
OverloadedStrings
, DeriveGeneric
, DeriveDataTypeable
, DeriveFunctor
, DeriveFoldable
, DeriveTraversable
, DeriveLift
, FlexibleContexts
, FlexibleInstances
, StandaloneDeriving
, TypeApplications
, TypeSynonymInstances
, InstanceSigs
, MultiParamTypeClasses
, TupleSections
, LambdaCase
, BangPatterns
, ViewPatterns
default-language: Haskell2010 default-language: Haskell2010

67
hnix-store-remote/src/System/Nix/Store/Remote.hs
View File

@ -49,7 +49,6 @@ import System.Nix.Hash ( NamedAlgo(..)
) )
import System.Nix.StorePath ( StorePath import System.Nix.StorePath ( StorePath
, StorePathName , StorePathName
, StorePathSet
, StorePathHashPart , StorePathHashPart
) )
import System.Nix.StorePathMetadata ( StorePathMetadata(..) import System.Nix.StorePathMetadata ( StorePathMetadata(..)
@ -71,9 +70,6 @@ import System.Nix.Store.Remote.Util
import Crypto.Hash ( SHA256 ) import Crypto.Hash ( SHA256 )
import System.Nix.Nar ( NarSource ) import System.Nix.Nar ( NarSource )
type RepairFlag = Bool
type CheckFlag = Bool
type SubstituteFlag = Bool
-- | Pack `Nar` and add it to the store. -- | Pack `Nar` and add it to the store.
addToStore addToStore
@ -81,15 +77,18 @@ addToStore
. (NamedAlgo a) . (NamedAlgo a)
=> StorePathName -- ^ Name part of the newly created `StorePath` => StorePathName -- ^ Name part of the newly created `StorePath`
-> NarSource MonadStore -- ^ provide nar stream -> NarSource MonadStore -- ^ provide nar stream
-> Bool -- ^ Add target directory recursively -> Recursive -- ^ Add target directory recursively
-> RepairFlag -- ^ Only used by local store backend -> RepairFlag -- ^ Only used by local store backend
-> MonadStore StorePath -> MonadStore StorePath
addToStore name source recursive _repair = do addToStore name source recursive repair = do
when (unRepairFlag repair)
$ error "repairing is not supported when building through the Nix daemon"
runOpArgsIO AddToStore $ \yield -> do runOpArgsIO AddToStore $ \yield -> do
yield $ toStrict $ Data.Binary.Put.runPut $ do yield $ toStrict $ Data.Binary.Put.runPut $ do
putText $ System.Nix.StorePath.unStorePathName name putText $ System.Nix.StorePath.unStorePathName name
putBool $ not $ System.Nix.Hash.algoName @a == "sha256" && recursive putBool $ not $ System.Nix.Hash.algoName @a == "sha256" && (unRecursive recursive)
putBool recursive putBool (unRecursive recursive)
putText $ System.Nix.Hash.algoName @a putText $ System.Nix.Hash.algoName @a
source yield source yield
sockGetPath sockGetPath
@ -99,14 +98,15 @@ addToStore name source recursive _repair = do
-- Reference accepts repair but only uses it -- Reference accepts repair but only uses it
-- to throw error in case of remote talking to nix-daemon. -- to throw error in case of remote talking to nix-daemon.
addTextToStore addTextToStore
:: Text -- ^ Name of the text :: Text -- ^ Name of the text
-> Text -- ^ Actual text to add -> Text -- ^ Actual text to add
-> StorePathSet -- ^ Set of `StorePath`s that the added text references -> HashSet StorePath -- ^ Set of `StorePath`s that the added text references
-> RepairFlag -- ^ Repair flag, must be `False` in case of remote backend -> RepairFlag -- ^ Repair flag, must be `False` in case of remote backend
-> MonadStore StorePath -> MonadStore StorePath
addTextToStore name text references' repair = do addTextToStore name text references' repair = do
when repair when (unRepairFlag repair)
$ error "repairing is not supported when building through the Nix daemon" $ error "repairing is not supported when building through the Nix daemon"
storeDir <- getStoreDir storeDir <- getStoreDir
runOpArgs AddTextToStore $ do runOpArgs AddTextToStore $ do
putText name putText name
@ -137,7 +137,7 @@ addTempRoot pn = do
-- | Build paths if they are an actual derivations. -- | Build paths if they are an actual derivations.
-- --
-- If derivation output paths are already valid, do nothing. -- If derivation output paths are already valid, do nothing.
buildPaths :: StorePathSet -> BuildMode -> MonadStore () buildPaths :: HashSet StorePath -> BuildMode -> MonadStore ()
buildPaths ps bm = do buildPaths ps bm = do
storeDir <- getStoreDir storeDir <- getStoreDir
void $ simpleOpArgs BuildPaths $ do void $ simpleOpArgs BuildPaths $ do
@ -197,22 +197,22 @@ isValidPathUncached p = do
-- | Query valid paths from set, optionally try to use substitutes. -- | Query valid paths from set, optionally try to use substitutes.
queryValidPaths queryValidPaths
:: StorePathSet -- ^ Set of `StorePath`s to query :: HashSet StorePath -- ^ Set of `StorePath`s to query
-> SubstituteFlag -- ^ Try substituting missing paths when `True` -> SubstituteFlag -- ^ Try substituting missing paths when `True`
-> MonadStore StorePathSet -> MonadStore (HashSet StorePath)
queryValidPaths ps substitute = do queryValidPaths ps substitute = do
storeDir <- getStoreDir storeDir <- getStoreDir
runOpArgs QueryValidPaths $ do runOpArgs QueryValidPaths $ do
putPaths storeDir ps putPaths storeDir ps
putBool substitute putBool (unSubstituteFlag substitute)
sockGetPaths sockGetPaths
queryAllValidPaths :: MonadStore StorePathSet queryAllValidPaths :: MonadStore (HashSet StorePath)
queryAllValidPaths = do queryAllValidPaths = do
runOp QueryAllValidPaths runOp QueryAllValidPaths
sockGetPaths sockGetPaths
querySubstitutablePaths :: StorePathSet -> MonadStore StorePathSet querySubstitutablePaths :: HashSet StorePath -> MonadStore (HashSet StorePath)
querySubstitutablePaths ps = do querySubstitutablePaths ps = do
storeDir <- getStoreDir storeDir <- getStoreDir
runOpArgs QuerySubstitutablePaths $ putPaths storeDir ps runOpArgs QuerySubstitutablePaths $ putPaths storeDir ps
@ -261,25 +261,25 @@ queryPathInfoUncached path = do
pure $ StorePathMetadata{..} pure $ StorePathMetadata{..}
queryReferrers :: StorePath -> MonadStore StorePathSet queryReferrers :: StorePath -> MonadStore (HashSet StorePath)
queryReferrers p = do queryReferrers p = do
storeDir <- getStoreDir storeDir <- getStoreDir
runOpArgs QueryReferrers $ putPath storeDir p runOpArgs QueryReferrers $ putPath storeDir p
sockGetPaths sockGetPaths
queryValidDerivers :: StorePath -> MonadStore StorePathSet queryValidDerivers :: StorePath -> MonadStore (HashSet StorePath)
queryValidDerivers p = do queryValidDerivers p = do
storeDir <- getStoreDir storeDir <- getStoreDir
runOpArgs QueryValidDerivers $ putPath storeDir p runOpArgs QueryValidDerivers $ putPath storeDir p
sockGetPaths sockGetPaths
queryDerivationOutputs :: StorePath -> MonadStore StorePathSet queryDerivationOutputs :: StorePath -> MonadStore (HashSet StorePath)
queryDerivationOutputs p = do queryDerivationOutputs p = do
storeDir <- getStoreDir storeDir <- getStoreDir
runOpArgs QueryDerivationOutputs $ putPath storeDir p runOpArgs QueryDerivationOutputs $ putPath storeDir p
sockGetPaths sockGetPaths
queryDerivationOutputNames :: StorePath -> MonadStore StorePathSet queryDerivationOutputNames :: StorePath -> MonadStore (HashSet StorePath)
queryDerivationOutputNames p = do queryDerivationOutputNames p = do
storeDir <- getStoreDir storeDir <- getStoreDir
runOpArgs QueryDerivationOutputNames $ putPath storeDir p runOpArgs QueryDerivationOutputNames $ putPath storeDir p
@ -289,17 +289,20 @@ queryPathFromHashPart :: StorePathHashPart -> MonadStore StorePath
queryPathFromHashPart storePathHash = do queryPathFromHashPart storePathHash = do
runOpArgs QueryPathFromHashPart runOpArgs QueryPathFromHashPart
$ putByteStringLen $ putByteStringLen
$ encodeUtf8 (encodeWith NixBase32 $ coerce storePathHash) $ encodeUtf8
$ encodeWith NixBase32
$ System.Nix.StorePath.unStorePathHashPart
storePathHash
sockGetPath sockGetPath
queryMissing queryMissing
:: StorePathSet :: (HashSet StorePath)
-> MonadStore -> MonadStore
( StorePathSet-- Paths that will be built ( HashSet StorePath -- Paths that will be built
, StorePathSet -- Paths that have substitutes , HashSet StorePath -- Paths that have substitutes
, StorePathSet -- Unknown paths , HashSet StorePath -- Unknown paths
, Integer -- Download size , Integer -- Download size
, Integer -- Nar size? , Integer -- Nar size?
) )
queryMissing ps = do queryMissing ps = do
storeDir <- getStoreDir storeDir <- getStoreDir
@ -321,5 +324,5 @@ syncWithGC = void $ simpleOp SyncWithGC
-- returns True on errors -- returns True on errors
verifyStore :: CheckFlag -> RepairFlag -> MonadStore Bool verifyStore :: CheckFlag -> RepairFlag -> MonadStore Bool
verifyStore check repair = simpleOpArgs VerifyStore $ do verifyStore check repair = simpleOpArgs VerifyStore $ do
putBool check putBool $ unCheckFlag check
putBool repair putBool $ unRepairFlag repair

51
hnix-store-remote/src/System/Nix/Store/Remote/Types.hs
View File

@ -4,6 +4,22 @@
module System.Nix.Store.Remote.Types module System.Nix.Store.Remote.Types
( MonadStore ( MonadStore
, StoreConfig(..) , StoreConfig(..)
, CheckFlag
, doCheck
, dontCheck
, unCheckFlag
, RepairFlag
, doRepair
, dontRepair
, unRepairFlag
, SubstituteFlag
, doSubstitute
, dontSubstitute
, unSubstituteFlag
, Recursive
, addRecursive
, addNonRecursive
, unRecursive
, Logger(..) , Logger(..)
, Field(..) , Field(..)
, mapStoreDir , mapStoreDir
@ -29,6 +45,41 @@ data StoreConfig = StoreConfig
, storeSocket :: Socket , storeSocket :: Socket
} }
-- | Check flag, used by @verifyStore@
newtype CheckFlag = CheckFlag { unCheckFlag :: Bool }
deriving (Eq, Ord, Show)
doCheck, dontCheck :: CheckFlag
doCheck = CheckFlag True
dontCheck = CheckFlag False
-- | Repair flag, used by @addToStore@, @addTextToStore@
-- and @verifyStore@
newtype RepairFlag = RepairFlag { unRepairFlag :: Bool }
deriving (Eq, Ord, Show)
doRepair, dontRepair :: RepairFlag
doRepair = RepairFlag True
dontRepair = RepairFlag False
-- | Substitute flag, used by @queryValidPaths@
newtype SubstituteFlag = SubstituteFlag { unSubstituteFlag :: Bool }
deriving (Eq, Ord, Show)
doSubstitute, dontSubstitute :: SubstituteFlag
doSubstitute = SubstituteFlag True
dontSubstitute = SubstituteFlag False
-- | Recursive, used by @addToStore@
newtype Recursive = Recursive { unRecursive :: Bool }
deriving (Eq, Ord, Show)
addRecursive, addNonRecursive :: Recursive
-- | Add target directory recursively
addRecursive = Recursive True
-- | Add target directory non-recursively
addNonRecursive = Recursive False
type MonadStore a type MonadStore a
= ExceptT = ExceptT
String String

6
hnix-store-remote/src/System/Nix/Store/Remote/Util.hs
View File

@ -83,7 +83,7 @@ sockGetPathMay = do
Just Just
pth pth
sockGetPaths :: MonadStore StorePathSet sockGetPaths :: MonadStore (HashSet StorePath)
sockGetPaths = do sockGetPaths = do
sd <- getStoreDir sd <- getStoreDir
getSocketIncremental (getPaths sd) getSocketIncremental (getPaths sd)
@ -109,14 +109,14 @@ putTexts = putByteStrings . fmap textToBSL
getPath :: StoreDir -> Get (Either String StorePath) getPath :: StoreDir -> Get (Either String StorePath)
getPath sd = parsePath sd <$> getByteStringLen getPath sd = parsePath sd <$> getByteStringLen
getPaths :: StoreDir -> Get StorePathSet getPaths :: StoreDir -> Get (HashSet StorePath)
getPaths sd = getPaths sd =
Data.HashSet.fromList . rights . fmap (parsePath sd) <$> getByteStrings Data.HashSet.fromList . rights . fmap (parsePath sd) <$> getByteStrings
putPath :: StoreDir -> StorePath -> Put putPath :: StoreDir -> StorePath -> Put
putPath storeDir = putByteStringLen . fromStrict . storePathToRawFilePath storeDir putPath storeDir = putByteStringLen . fromStrict . storePathToRawFilePath storeDir
putPaths :: StoreDir -> StorePathSet -> Put putPaths :: StoreDir -> HashSet StorePath -> Put
putPaths storeDir = putByteStrings . Data.HashSet.toList . Data.HashSet.map putPaths storeDir = putByteStrings . Data.HashSet.toList . Data.HashSet.map
(fromStrict . storePathToRawFilePath storeDir) (fromStrict . storePathToRawFilePath storeDir)

0
hnix-store-remote/tests/Driver.hs → hnix-store-remote/tests-io/Driver.hs
View File

16
hnix-store-remote/tests/NixDaemon.hs → hnix-store-remote/tests-io/NixDaemon.hs
View File

@ -154,23 +154,23 @@ itLefts name action = it name action isLeft
withPath :: (StorePath -> MonadStore a) -> MonadStore a withPath :: (StorePath -> MonadStore a) -> MonadStore a
withPath action = do withPath action = do
path <- addTextToStore "hnix-store" "test" (HS.fromList []) False path <- addTextToStore "hnix-store" "test" mempty dontRepair
action path action path
-- | dummy path, adds <tmp>/dummpy with "Hello World" contents -- | dummy path, adds <tmp>/dummpy with "Hello World" contents
dummy :: MonadStore StorePath dummy :: MonadStore StorePath
dummy = do dummy = do
let name = Data.Either.fromRight (error "impossible") $ makeStorePathName "dummy" let name = Data.Either.fromRight (error "impossible") $ makeStorePathName "dummy"
addToStore @SHA256 name (dumpPath "dummy") False False addToStore @SHA256 name (dumpPath "dummy") addNonRecursive dontRepair
invalidPath :: StorePath invalidPath :: StorePath
invalidPath = invalidPath =
let name = Data.Either.fromRight (error "impossible") $ makeStorePathName "invalid" let name = Data.Either.fromRight (error "impossible") $ makeStorePathName "invalid"
in StorePath (mkStorePathHashPart "invalid") name in StorePath (mkStorePathHashPart @SHA256 "invalid") name
withBuilder :: (StorePath -> MonadStore a) -> MonadStore a withBuilder :: (StorePath -> MonadStore a) -> MonadStore a
withBuilder action = do withBuilder action = do
path <- addTextToStore "builder" builderSh (HS.fromList []) False path <- addTextToStore "builder" builderSh mempty dontRepair
action path action path
builderSh :: Text builderSh :: Text
@ -186,14 +186,14 @@ spec_protocol = Hspec.around withNixDaemon $
context "verifyStore" $ do context "verifyStore" $ do
itRights "check=False repair=False" $ itRights "check=False repair=False" $
verifyStore False False `shouldReturn` False verifyStore dontCheck dontRepair `shouldReturn` False
itRights "check=True repair=False" $ itRights "check=True repair=False" $
verifyStore True False `shouldReturn` False verifyStore doCheck dontRepair `shouldReturn` False
--privileged --privileged
itRights "check=True repair=True" $ itRights "check=True repair=True" $
verifyStore True True `shouldReturn` False verifyStore doCheck doRepair `shouldReturn` False
context "addTextToStore" $ context "addTextToStore" $
itRights "adds text to store" $ withPath pure itRights "adds text to store" $ withPath pure
@ -252,7 +252,7 @@ spec_protocol = Hspec.around withNixDaemon $
itRights "adds file to store" $ do itRights "adds file to store" $ do
fp <- liftIO $ writeSystemTempFile "addition" "lal" fp <- liftIO $ writeSystemTempFile "addition" "lal"
let name = Data.Either.fromRight (error "impossible") $ makeStorePathName "tmp-addition" let name = Data.Either.fromRight (error "impossible") $ makeStorePathName "tmp-addition"
res <- addToStore @SHA256 name (dumpPath fp) False False res <- addToStore @SHA256 name (dumpPath fp) addNonRecursive dontRepair
liftIO $ print res liftIO $ print res
context "with dummy" $ do context "with dummy" $ do

0
hnix-store-remote/tests/Spec.hs → hnix-store-remote/tests-io/Spec.hs
View File

12
hnix-store-remote/tests/Util.hs
View File

@ -1,12 +0,0 @@
module Util where
import Data.Text.Arbitrary ()
import System.Nix.Store.Remote.Util
import Test.Tasty.QuickCheck
prop_TextToBSLRoundtrip :: Text -> Property
prop_TextToBSLRoundtrip x = bslToText (textToBSL x) === x
prop_TextToBSRoundtrip :: Text -> Property
prop_TextToBSRoundtrip x = bsToText (textToBS x) === x