benchmarks/README.md · compaREST

git.iko.soy/compaREST

27 lines 1.5 kB history raw

1# OpenApi comparison benchmarks

3This is a nix expression that automatically downloads, builds and runs multiple OpenAPI comparison tools on the same set of inputs to compare how well they behave.

5To run the benchmarks run the following command in this directory:

7```bash

8nix-build

9```

11## The tools

13| Tool | Output file name |

14| -------------------------------------------------------------------------- | ------------------------------------------- |

15| [Typeable CompaREST](https://github.com/typeable/compaREST) | `typeable.md` |

16| [Atlassian OpenApi Diff](https://bitbucket.org/atlassian/openapi-diff) | `atlassian.json`, `atlassian.error.txt` |

17| [OpenApi Tools OpenApi Diff](https://github.com/OpenAPITools/openapi-diff) | `openapitools.md`, `openapitools.error.txt` |

18| [Tufin OpenApi Diff](https://github.com/tufin/oasdiff) | `oasdiff.txt`, `oasdiff.error.txt` |

19| [Bump](https://bump.sh) | `bump.txt` |

21## Running with Bump.sh

23Bump.sh is a closed-source tool that is only available via REST API. To include bump.sh in the benchmark results you have to supply your credentials as arguments:

25```bash

26nix-build --argstr bumpToken <TOKEN> --argstr bumpDocumentation <DOCUMENTATION IDENTIFIER>

27```