graphql-engine/server/benchmarks/benchmark_sets/deep_schema/README.md

This data set is meant to test nested remote relationships: it contains a huge
amount of remote relationships, chosen to make it so that as "depth-first"
traversal will cross from one source to the other 50 times. Building the schema
across remote relationships isn't easy, and the reason this set came to be was
that an approach we considered for how to use different contexts for different
sources actually resulted in a performance degradation with deep schemas such as
this artificially created one.

At time of writing, we have no plan to use this set to benchmark queries: our
main concern is the schema building time, as measured by a call to
`replace_metadata`.

A degradation of performance that is made visible by this set but no other
likely indicates a problem with the implementation of remote relationships in
the schema.
[ci] add benchmark for deep schema ### Description This PR adds a new benchmarl set named `deep_schema`, that is made to replicate one very specific edge-case: schemas that have deeply nested remote relationships. Our schema-building code is, in essence, "depth-first", and there are a lot of subtleties in the way we jump across remote relationship boundaries: this set will allows us to better understand the performance implications of technical decisions we make wrt. schema building. This set, unlike others, does not declare any query: we are, for now, only interested in the schema building, which is tested with an ad-hoc script. ## Remaining work There are several points worth discussing, wrt. this PR: - should we make the schema larger, to make measures more consistent? - should we extend this idea of measuring schema build performance to other sets? - how do we extend the report to include this new information? PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5517 GitOrigin-RevId: 9d8f4fddb9bbdca5ef85f3d22337b992acf13bce 2022-08-24 18:54:16 +03:00			`This data set is meant to test nested remote relationships: it contains a huge`
			`amount of remote relationships, chosen to make it so that as "depth-first"`
			`traversal will cross from one source to the other 50 times. Building the schema`
			`across remote relationships isn't easy, and the reason this set came to be was`
			`that an approach we considered for how to use different contexts for different`
			`sources actually resulted in a performance degradation with deep schemas such as`
			`this artificially created one.`

			`At time of writing, we have no plan to use this set to benchmark queries: our`
			`main concern is the schema building time, as measured by a call to`
			`replace_metadata`.

			`A degradation of performance that is made visible by this set but no other`
			`likely indicates a problem with the implementation of remote relationships in`
			`the schema.`