Problem: We had hardcoded HTML tag parser, that doesn't work with add valid HTML tags
Solution: Replace it with `tagsoup` library, that care about all parsing stuff
Problem: The current version of xrefcheck handles the HTTP responses
with the 429 status code just like every other error, when it is
possible to try and eliminate the occurrences of such errors within the
program itself.
Solution: Each time the result of performing a request on a given link
is a 429 error, retrieve the Retry-After information, describing the
delay (in seconds), from the headers of the HTTP response, or,
alternatively, use a configurable default value if the Retry-After
header is absent, and rerun the request after an amount of time
described by the said value had passed. Only after the number of retries
had reached its limiting value, which, as of right now, is not
configurable and is hardcoded, is when the 429 error is converted into
becoming 'unfixable', and any further attempts to remove the error are
terminated.
Additionally, the progress bar has been upgraded and the following
elements are supplied:
1. an extra color -- Blue -- indicating the errors that might get
eliminated during the verification;
2. a timer with the number of seconds left to wait for the restart of
the request; if, during the verification, a new 429 error had emerged
with the new Retry-After value being greater than or equal to the
elapsed time, the timer is immediately updated with that value and
begins ticking down each second from scratch.
Problem: We used default ports to test error reports in checking of localhost link, but this port may be in use by other program, so xrefcheck reports another message
Solution: Specify port by value that is likely not to be used by other programs
Problem: --ignored and --root CLI options had misbehave when you use ./ in path
Solution: add path normalisation and path-equality from System.FilePath instead of common functions for strings
Problem: we had a lot redundant dependencies and had no linter for handling obvious errors
Solution: hlint support and enable -Weverything flag, fix all hints from them, add hlint to the CI pipeline
Problem: At new resolver version we recieved obscure error when tried to cross-compile project to Windows on CI. Changing file-embed version to the old one doesn't help us.
Solution: inline content of this file into haskell source, using raw-string-qq library, that helps us to avoid escaping and typing newline characters.
Problem: We had old resolver version, that used ghc 8.10.4 and outdated versions of libraries
Solution: Change resolver to lts 19.13, that uses ghc 9.0.2 and update haskell.nix configuration files to can build project via `nix-build` command
Problem: In
```
import qualified Foo.Bar as Bar
import Foo.Bar (Bar)
```
names of the imported modules are on different
vertical lines, which disables autosorting,
and makes it harder to read.
Solution: Use `ImportQualifiedPost`
Problem: Stack cannot build projects with mixins, and they only
used to splice universum instead of base.
Solution: Use `NoImplicitPreduce` and import `Universum`
everywhere explicitly.
Problem: The results of the repository analysis will always contain
invalid references if the root contains a trailing forward slash.
Solution: Strip the root's trailing slash (if present) when having it be
given as an argument of the System.FilePath.Posix.takeDirectory
function.
Problem: All the duplicate external links get verified independently,
which is wasteful.
Solution: Store the verification results in the map. Verify the link
once if it hasn't been verified already, and insert the results
into the map; alternatively, if the link has been verified before,
return the results by retrieving the respective value from the map.
The traversal of all the filepath-reference pairs is done via
'forConcurrentlyCaching', which is semantically similar to
Control.Concurrent.Async.forConcurrently, except it stores the
action results in a map (in our case, we need to store the
verification results of only the external references).
Problem:
Currently we support only http and https links. If there is an `ftp://`
link, you will get exception.
Solution:
Use `ftp-client` to check connection to ftp, see response statuses and
check file existence. This produces adding new error types and small
refactoring.
Provide a test which is separate executable, where we have to pass CLA -
ftp host.
Co-authored-by: Alexander Bantyev <alexander.bantyev@serokell.io>
Problem:
We have `exec` directory with `Main.hs` which (from semantic) supposed
to only run Xrefcheck.
Also, it's a common practice to keep `Main.hs` as simple as it possible.
But now it have some logic.
Solution:
Move commands related functionality to lib (separate module) and keep
`Main.hs` very simple, only as runner.
Problem:
We do not know what to do with protected links, because we cannot check
them. So we have two options, assume that these links is valid or not.
Solution:
Provide config option for user to decide what to do - assume protected
links valid or not.
Problem:
Almost all the time we can't validate localhost links, so we just skip them.
But to run ftp links tests (#47) we need to refer to localhost.
Solution:
Add config option whether to ignore localhost links and provide bats
tests for this new feature.
Co-authored-by: Alexander Bantyev <alexander.bantyev@serokell.io>
Problem: A header name with its custom anchor appended after the
octothorpe symbol(s) is parsed with the leading spaces present, thus
generating an anchor from it will result in having a leading hyphen
prepended to its beginning.
Solution: Strip the header name to ensure the absence of the leading
hyphen in the later generated anchor.
Problem:
Our style guide requires two spaces indention, but in some places we have four or even more spaces.
Solution:
Delete spaces where it contradicts our style guide.
Problem:
I've met problem with one package version. I use cabal to build project and cabal for some reason decided to use old version of this package. With stack everything fine.
So, I think not only me, but other people prefer cabal over stack.
Solution:
Add note to our README.md that in case of cabal usage it's required to run `stack2cabal` first.
Also ignore all cabal staff so it can't be accidentally added to repo.
