2019-12-18 17:18:55 +03:00
<!--
2021-07-15 16:16:08 +03:00
- SPDX-FileCopyrightText: 2018-2021 Serokell < https: // serokell . io >
2019-12-18 17:18:55 +03:00
-
- SPDX-License-Identifier: MPL-2.0
-->
2020-01-13 15:58:14 +03:00
# Xrefcheck
2019-03-14 20:21:20 +03:00
2020-01-13 15:58:14 +03:00
[![Build status ](https://badge.buildkite.com/75461331a6058b334383cdfca1071dc1f908b70cf069d857b7.svg?branch=master )](https://buildkite.com/serokell/xrefcheck)
2019-12-05 16:10:15 +03:00
2020-01-13 15:58:14 +03:00
Xrefcheck is a tool for verifying local and external references in repository documentation that is quick, easy to setup, and suitable to be added to CI.
2019-03-14 20:21:20 +03:00
2019-12-05 17:38:25 +03:00
< img src = "https://user-images.githubusercontent.com/5394217/70820564-06b06e00-1dea-11ea-9680-27f661ca2a58.png" alt = "Output sample" width = "600" / >
2019-12-05 18:12:01 +03:00
### Motivation
2020-01-13 15:58:14 +03:00
As the project evolves, links in documentation have a tendency to get broken. This is usually because of:
2019-12-18 12:44:03 +03:00
1. File movements;
2. Markdown header renames;
2019-12-05 18:12:01 +03:00
3. Outer sites ceasing their existence.
This tool will help you to keep references in order.
2019-12-18 15:49:37 +03:00
### Aims
2019-12-05 17:38:25 +03:00
2019-12-18 12:44:03 +03:00
Comparing to alternative solutions, this tool tries to achieve the following points:
2019-12-05 17:38:25 +03:00
2019-12-05 18:12:01 +03:00
* Quickness - local references are verified instantly even for moderately-sized repositories.
* Easy setup - no extra actions required, just run the tool in the repository root.
Both relative and absolute local links are supported out of the box.
* Conservative verifier allows using this tool in CI, no false positives (e.g. on sites which require authentication) should be reported.
### A comparison with other solutions
2020-10-08 13:52:35 +03:00
* [linky ](https://github.com/mattias-p/linky ) - a well-configurable verifier written in Rust, scans one specified file at a time and works good in pair with system utilities like `find` .
2019-12-05 18:12:01 +03:00
This tool requires some configuring before it can be applied to a repository or added to CI.
2019-12-18 12:44:03 +03:00
* [awesome_bot ](https://github.com/dkhamsing/awesome_bot ) - a solution written in Ruby that can be easily included in CI or integrated into GitHub.
Its features include duplicated URLs detection, specifying allowed HTTP error codes and reporting generation.
2021-07-08 16:22:44 +03:00
At the moment of writing, it scans only external references and checking anchors is not possible.
2019-12-05 18:12:01 +03:00
* [remark-validate-links ](https://github.com/remarkjs/remark-validate-links ) and [remark-lint-no-dead-urls ](https://github.com/davidtheclark/remark-lint-no-dead-urls ) - highly configurable Javascript solution for checking local and remote links resp.
2019-12-18 12:44:03 +03:00
It is able to check multiple repositores at once if they are gathered in one folder.
Being written on JavaScript, it is fairly slow on large repositories.
2020-10-08 13:52:35 +03:00
* [markdown-link-check ](https://github.com/tcort/markdown-link-check ) - another checker written in JavaScript, scans one specific file at a time.
2019-12-18 12:44:03 +03:00
Supports `mailto:` link resolution.
2019-12-05 18:12:01 +03:00
* [url-checker ](https://github.com/paramt/url-checker ) - GitHub action which checks links in specified files.
2020-10-08 13:52:35 +03:00
* [linkcheck ](https://github.com/filiph/linkcheck ) - advanced site crawler, checks for `HTML` files. There are other solutions for this particular task which we don't mention here.
2019-03-14 20:21:20 +03:00
2021-07-08 16:22:44 +03:00
At the moment of writing, the listed solutions don't support ftp/ftps links.
2019-03-14 20:21:20 +03:00
2020-06-29 17:48:16 +03:00
## Usage [↑](#xrefcheck)
2019-12-04 23:37:21 +03:00
2020-06-29 17:48:16 +03:00
We provide the following ways for you to use xrefcheck:
2019-12-09 12:38:33 +03:00
2020-09-16 17:51:36 +03:00
- [GitHub action ](https://github.com/marketplace/actions/xrefcheck )
2020-06-29 17:48:16 +03:00
- [statically linked binaries ](https://github.com/serokell/xrefcheck/releases )
- [Docker image ](https://hub.docker.com/r/serokell/xrefcheck )
- [building from source ](#build-instructions- )
2019-12-04 23:37:21 +03:00
2020-06-29 17:48:16 +03:00
If none of those are suitable for you, please open an issue!
2019-03-14 20:21:20 +03:00
2019-12-18 12:44:03 +03:00
To find all broken links in a repository, run from within its folder:
2019-03-14 20:21:20 +03:00
```sh
2020-01-13 15:58:14 +03:00
xrefcheck
2019-03-14 20:21:20 +03:00
```
2019-12-18 12:44:03 +03:00
To also display all found links and anchors:
2019-12-05 18:12:01 +03:00
```sh
2020-01-13 15:58:14 +03:00
xrefcheck -v
2019-12-05 18:12:01 +03:00
```
2019-12-18 12:44:03 +03:00
For description of other options:
2019-03-14 20:21:20 +03:00
```sh
2020-01-13 15:58:14 +03:00
xrefcheck --help
2019-03-14 20:21:20 +03:00
```
2020-12-28 16:37:56 +03:00
### Special functionality
< details >
< summary > Ignoring external links< / summary >
If you want some external links to not be verified, you can use one of the following ways to ignore those links:
2022-08-29 09:53:24 +03:00
1. Add the regular expression that matches the ignoring link to the `ignoreRefs` parameter of your config file.
2020-12-28 16:37:56 +03:00
2021-03-11 14:42:32 +03:00
For example:
2020-12-28 16:37:56 +03:00
```yaml
ignoreRefs:
- https://bad.reference.(org|com)(/?)
```
allows to ignore both `https://bad.reference.org` and `https://bad.reference.com` with or without last "/".
2. Add right in-place annotation using one of the following ignoring modes (each mode is just a comment with a certain syntax).
* Ignore the link:
There are several ways to add this annotation:
* Just add it like a regular text before the ignoring link.
```markdown
Bad ['com' reference ](https://bad.reference.com ) <!-- xrefcheck: ignore link --> and bad ['org' reference ](https://bad.reference.org )
```
* Separate the ignoring link from the annotation and the following text with single new lines.
```markdown
Bad ['com' reference ](https://bad.reference.com ) and bad <!-- xrefcheck: ignore link -->
['org' ](https://bad.reference.org )
reference
```
Therefore only `https://bad.reference.org` will be ignored.
* If the ignoring link is the first in a paragraph, then the annotation can also be added before a paragraph.
```markdown
<!-- xrefcheck: ignore link -->
[Bad 'org' reference ](https://bad.reference.org )
[Bad 'com' reference ](https://bad.reference.com )
```
It is still the same `https://bad.reference.org` will be ignored in this case.
* Ignore the paragraph:
```markdown
<!-- xrefcheck: ignore paragraph -->
Bad ['org' reference ](https://bad.reference.org )
Bad ['com' reference ](https://bad.reference.com )
Bad ['io' reference ](https://bad.reference.io )
```
In this way, `https://bad.reference.org` and `https://bad.reference.com` will be ignored and `https://bad.reference.io` will still be verified.
* Ignore the whole file:
```markdown
<!-- a comment -->
<!-- another comment -->
<!-- xrefcheck: ignore file -->
...the rest of the file...
```
Using this you can ignore the whole file.
< / details >
2019-12-05 18:12:01 +03:00
## Configuring
2019-12-18 12:44:03 +03:00
Configuration template (with all options explained) can be dumped with:
2019-12-05 18:12:01 +03:00
```sh
2021-02-25 18:48:51 +03:00
xrefcheck dump-config -t GitHub
2019-12-05 18:12:01 +03:00
```
Currently supported options include:
* Timeout for checking external references;
2022-09-03 20:25:02 +03:00
* List of ignored files.
2019-12-05 18:12:01 +03:00
2020-06-29 17:48:16 +03:00
## Build instructions [↑](#xrefcheck)
Run `stack install` to build everything and install the executable.
2021-07-15 16:16:08 +03:00
If you want to use cabal, you need to run (`stack2cabal`)[https://hackage.haskell.org/package/stack2cabal] first!
2020-06-29 17:48:16 +03:00
### CI and nix [↑](#xrefcheck)
To build only the executables, run `nix-build` . You can use this line on your CI to use xrefcheck:
```
nix run -f https://github.com/serokell/xrefcheck/archive/master.tar.gz -c xrefcheck
```
Our CI uses `nix-build xrefcheck.nix` to build the whole project, including tests and Haddock.
It is based on the [`haskell.nix` ](https://input-output-hk.github.io/haskell.nix/ ) project.
You can do that too if you wish.
2020-01-13 15:58:14 +03:00
## For further work [↑](#xrefcheck)
2019-12-05 18:12:01 +03:00
- [ ] Support for non-Unix systems.
2019-12-18 12:44:03 +03:00
- [ ] Support link detection in different languages, not only Markdown.
- [ ] Haskell Haddock is first in turn.
2019-12-05 18:12:01 +03:00
2020-01-13 15:58:14 +03:00
## Issue tracker [↑](#xrefcheck)
2019-03-14 20:21:20 +03:00
2019-12-05 19:41:08 +03:00
We use GitHub issues as our issue tracker.
You can login using your GitHub account to leave a comment or create a new issue.
2019-03-14 20:21:20 +03:00
2020-01-13 15:58:14 +03:00
## For Contributors [↑](#xrefcheck)
2019-03-14 20:21:20 +03:00
Please see [CONTRIBUTING.md ](/.github/CONTRIBUTING.md ) for more information.
2020-01-13 15:58:14 +03:00
## About Serokell [↑](#xrefcheck)
2019-03-14 20:21:20 +03:00
2020-01-13 15:58:14 +03:00
Xrefcheck is maintained and funded with ❤️ by [Serokell ](https://serokell.io/ ).
2019-12-05 16:10:25 +03:00
The names and logo for Serokell are trademark of Serokell OÜ.
2019-03-14 20:21:20 +03:00
2019-12-05 16:10:25 +03:00
We love open source software! See [our other projects ](https://serokell.io/community?utm_source=github ) or [hire us ](https://serokell.io/hire-us?utm_source=github ) to design, develop and grow your idea!