Commit Graph

15 Commits

Author SHA1 Message Date
Brandon Simmons
d2b3ec8938 ci/benchmarks: fix copypaste bug affecting regression report highlighting in #5942
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6021
GitOrigin-RevId: eb77bf97a52e533c565bfb12a45c0851f832610d
2022-09-22 16:24:15 +00:00
kodiakhq[bot]
81df74852e ci/benchmarks: tweak what we present in regression report
- Make regression report more compact, remove `min` since it doesn't behave like a limit
- It looks like the low-load variants are probably just noisier with fewer samples, so remove them from the regression view:

![samples](https://user-images.githubusercontent.com/210815/191076137-f5c3a0c6-9586-4ea4-a5a7-66000e7a2540.png)

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5942
GitOrigin-RevId: 8a9ab00c251f04d7d5a542731696cb5f86ad2b78
2022-09-22 13:28:56 +00:00
kodiakhq[bot]
36860c8fb2 ci/benchmarks: fix division-by-zero in report generation, after e09406170e1
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5775
GitOrigin-RevId: 5cd641f23ffe1929c47568a04746ef507c8c5874
2022-09-06 20:29:22 +00:00
Brandon Simmons
6e578f229a server/ci/benchmarks: incorporate ad-hoc queries into benchmark reports
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5704
GitOrigin-RevId: e09406170e1245ceeb3be51f498159f76af4f4bf
2022-09-01 20:48:13 +00:00
Brandon Simmons
6ab128a7e3 server/benchmarks: report response body size changes in PR regression comment
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5705
GitOrigin-RevId: 76c355f5ce6872a2bfd6c9b17a766a3c974445db
2022-09-01 17:25:10 +00:00
Brandon Simmons
da1d580d3d ci: Fix Benchmark regression reporting for empty reports (exposed after #5517)
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5638
GitOrigin-RevId: d2a8e21408919556ebb89fb273f7067e955035fe
2022-08-29 08:59:24 +00:00
Antoine Leblanc
1537bbd442 [ci] add benchmark for deep schema
### Description

This PR adds a new benchmarl set named `deep_schema`, that is made to replicate one very specific edge-case: schemas that have deeply nested remote relationships. Our schema-building code is, in essence, "depth-first", and there are a lot of subtleties in the way we jump across remote relationship boundaries: this set will allows us to better understand the performance implications of technical decisions we make wrt. schema building.

This set, unlike others, does not declare any query: we are, for now, only interested in the schema building, which is tested with an ad-hoc script.

## Remaining work

There are several points worth discussing, wrt. this PR:
- should we make the schema larger, to make measures more consistent?
- should we extend this idea of measuring schema build performance to other sets?
- how do we extend the report to include this new information?

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5517
GitOrigin-RevId: 9d8f4fddb9bbdca5ef85f3d22337b992acf13bce
2022-08-24 15:55:29 +00:00
Brandon Simmons
a3fc1889bb benchmarks: add an initial adhoc operation mode, suitable for benchma…
…rking metadata operations

And add an initial benchmark for replace_metadata, to unblock some
performance improvements to that op in a PR to be merged after this.

This is an MVP just to have something in CI to reference when optimizing
metadata operations. See TODO for roadmap.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3673
Co-authored-by: jkachmar <8461423+jkachmar@users.noreply.github.com>
GitOrigin-RevId: 968d1f92ca79c78ad90b2304d2214069bc739621
2022-02-22 16:59:44 +00:00
Brandon Simmons
681113f1d8 Jberryman/1993 skip just report mode
https://github.com/hasura/graphql-engine-mono/pull/2028

Co-authored-by: David Overton <7734777+dmoverton@users.noreply.github.com>
GitOrigin-RevId: afbdb8d74bede1e78b054f5582cbf144bba20762
2021-08-06 18:41:27 +00:00
Brandon Simmons
bdf0954c9d server benchmarks: Link directly to graphs in regression report, other style tweaks (#1720)
Depends on https://github.com/hasura/graphql-bench/pull/46

https://github.com/hasura/graphql-engine-mono/pull/2016

GitOrigin-RevId: ef7c5909e56cb1a945b20d5e072ee103bbd8a5ea
2021-08-06 15:49:56 +00:00
kodiakhq[bot]
5dbb3eb289 Jberryman/1720 rts stats tweak
https://github.com/hasura/graphql-engine-mono/pull/2030

GitOrigin-RevId: 398da37edd64b6e6f89bd52771aa7a6f7797983f
2021-08-05 23:37:53 +00:00
David Overton
394a1fa83b Skip remote_schema benchmark in CI
https://github.com/hasura/graphql-engine-mono/pull/2017

GitOrigin-RevId: e6c31ab35816a8dc690c9e5f9d64f76469db65ae
2021-08-05 16:20:47 +00:00
awjchen
cd957fb5ea ci: hide old benchmark reports on Github
https://github.com/hasura/graphql-engine-mono/pull/1996

GitOrigin-RevId: 0d2b607de9ea180e269c8f30f4a8b9618129d98f
2021-08-05 05:55:18 +00:00
Antoine Leblanc
3b3fee2fe0 ci: small formatting improvements to the benchmark output
## Description

This PR does a few minor improvements to the formatting of the benchmark results:
- changes the title from `<h1>` to `<h2>`, consistent with other hasura-bot reports AFAICT
- concatenates lines together to allow for more natural line wrapping
- allows for the detailed report to be collapsed (but is open by default)

## Notes

The state of the report (open / closed) isn't persisted across refreshes, and I don't know if it's feasible for it to be. An alternative would be to, when generating a new report, edit the old ones to make them collapsed by default.

https://github.com/hasura/graphql-engine-mono/pull/1867

GitOrigin-RevId: 7021a05aebdebe4cc5738d567a6dcc3aaabd38bf
2021-07-22 21:05:48 +00:00
Brandon Simmons
8306a4d1e4 Add initial benchmark suite, running in CI (closes hasura/graphql-engine-mono#736)
https://github.com/hasura/graphql-engine-mono/pull/1742

GitOrigin-RevId: 17ccd836171f6f39e808f8ac0f28da5cf8ef8f27
2021-07-08 18:19:51 +00:00