regenerate haskell nix file (drops hex dependency) apply ormolu to a few files fixup cabal file add polysemy logger interpretation of Output, fix bug in nixEvalSem bug * nixEvalSem was accidentally not stripping the output create test_data and simplify RewriteSpec to use it move Polysemy interpretation stacks closer to where polysemy methods are made eventually these will move more top level but for now this keeps it more contained update to ghc883, remove raw-strings-qq dep |
||
---|---|---|
.github | ||
app | ||
nix | ||
src | ||
test | ||
test_data | ||
.gitattributes | ||
.gitignore | ||
.hindent.yaml | ||
CVENOTES.org | ||
default.nix | ||
LICENSE | ||
nixpkgs-update.nix | ||
package.yaml | ||
README.md |
nixpkgs-update
The future is here; let's evenly distribute it!
Introduction
nixpkgs-update is the code
used by @r-ryantm to semi-automatically
update nixpkgs. It uses package repository information from
Repology.org to
generate a list of outdated packages. A package is considered
"outdated" if a newer version of the package is available at any other
repository tracked by
Repology. nixpkgs-update
tries to update each package in the dumbest way that could work. It
find-replaces the old version number with the new one, uses
nix-build
to try to get the new hash, then tries to build the
package. If it succeeds, it checks the outputs and makes a pull
request. It also uploads the built package to
Cachix, which people can use to
manually test the package without building it themselves.
nixpkgs-update features
Checks
A number of checks are performed to help nixpkgs maintainers gauge the likelihood that an update was successful. All the binaries are run with various flags to see if they have a zero exit code and output the new version number. The outpath directory tree is searched for files containing the new version number. A directory tree and disk usage listing is provided.
Security report
Information from the National Vulnerability Database maintained by NIST is compared against the current and updated package version. The nixpkgs package name is matched with the Common Platform Enumeration vendor, product, edition, software edition, and target software fields to find candidate Common Vulnerabilities and Exposures (CVEs). The CVEs are filtered by the matching the current and updated versions with the CVE version ranges.
The general philosophy of the CVE search is to avoid false negatives, which means we expect to generate many false positives. The false positives can be carefully removed by manually created rules implemented in the filter function in the NVDRules module.
If there are no CVE matches, the report is not shown. The report has three parts: CVEs resolved by this update, CVEs introduced by this update, and CVEs present in both version.
If you would like to report a problem with the security report, please use the nixpkgs-update GitHub issues.
The initial development of the security report was made possible by a partnership with Serokell and the NLNet Foundation through their Next Generation Internet Zero Discovery initiative (NGI0 Discovery). NGI0 Discovery is made possible with financial support from the European Commission.
Rebuild report
The PRs made by nixpkgs-update say what packages need to be rebuilt if the pull request is merged. This uses the same mechanism OfBorg uses to put rebuild labels on PRs. Not limited by labels, it can report the exact number of rebuilds and list some of the attrpaths that would need to be rebuilt.
PRs against staging
If a PR merge would cause more than 100 packages to be rebuilt, the PR is made against staging.
Logs
Logs from r-ryantm's runs are
available online. There are a lot of packages nixpkgs-update
currently has no hope of updating. Please dredge the logs to find out
why your pet package is not receiving updates.
Cachix
By uploading the build outputs to Cachix, nixpkgs-update allows you to test a package with one command.
Instructions
-
Clone this repository and build
nixpkgs-update
:git clone https://github.com/ryantm/nixpkgs-update && cd nixpkgs-update nix-build
-
Setup hub and copy the oauth token to
github_token.txt
in the root of this repository. -
To test your config, try to update a single package:
./result/bin/nixpkgs-update update --dry-run --additional-updates "X oldVer newVer"`,
where
X
is the attribute name, andoldver
andnewVer
are the versions.If this works, you are now setup to hack on
nixpkgs-update
! To send a batch of updates, proceed as follows: -
Get a list of outdated packages and place them in a
packages-to-update.txt
file:
./result/bin/nixpkgs-update fetch-repology > packages-to-update.txt
- Run the tool in batch mode with
update-list
:
./result/bin/nixpkgs-update update-list
Development
Setup a Cabal file (also run this when adding new dependencies):
nix run nixpkgs.haskellPackages.hpack -c hpack && nix run nixpkgs.cabal2nix -c cabal2nix --hpack . > nixpkgs-update.nix
For incremental building, first make a Cabal file with the above command, then use nix-shell
nix run nixpkgs.haskellPackages.hpack -c hpack && nix run nixpkgs.cabal2nix -c cabal2nix --hpack . > nixpkgs-update.nix
nix-shell
cabal new-repl
Run a type checker in the background for quicker type checking feedback:
nix-shell --run ghcid
Source files are formatted with Ormolu.