compaREST/benchmarks

OpenApi comparison benchmarks

This is a nix expression that automatically downloads, builds and runs multiple OpenApi comparison tools on the same set of inputs to compare how well they behave.

To run the benchmarks run the following command in this directory:

nix-build

The tools

Tool	Output file name
Typeable OpenApi Diff	typeable.md
Atlassian OpenApi Diff	atlassian.json, atlassian.error.txt
OpenApi Tools OpenApi Diff	openapitools.md, openapitools.error.txt
Tufin OpenApi Diff	oasdiff.txt, oasdiff.error.txt
Bump	bump.txt

Running with Bump.sh

Bump.sh is a closed-source tool that is only available via REST API. To include bump.sh in the benchmark results you have to supply your credentials as arguments:

nix-build --argstr bumpToken <TOKEN> --argstr bumpDocumentation <DOCUMENTATION IDENTIFIER>