<!-- Thank you for submitting this PR! :) -->
## Description
The getting starting instructions for Docker don't work because they use
the reference agent sample metadata from `open-dds` crate. I managed to
get the metadata building but for some reason I get connection errors
when running the queries (cannot reach the reference agent).
I would like to investigate this further, but in the meantime, this
updates the readme to use the metadata from the tests, which should at
least get them up and running with a working GraphiQL / Jaeger.
V3_GIT_ORIGIN_REV_ID: d0dc81560a068b960104228f89ee7e7d483a58d9
## Description
1. I've moved the architecture information we had in `CONTRIBUTING.md`
to a separate document `docs/architecture.md` so we can evolve both
separately in the future.
2. I've introduced a couple of sub directories: `utils` and `auth`, for
supporting crates that are not the core functionality of the engine so
it is easier to find the most relevant crates.
New structure:
```
crates
├── auth
│ ├── dev-auth-webhook
│ ├── hasura-authn-core
│ ├── hasura-authn-jwt
│ └── hasura-authn-webhook
├── custom-connector
├── engine
├── lang-graphql
├── metadata-schema-generator
├── open-dds
└── utils
├── opendds-derive
├── recursion_limit_macro
└── tracing-util
```
V3_GIT_ORIGIN_REV_ID: e0e9394da2fcd911f329c48107a76f8492fa304c
This splits out a `debug.Dockerfile` which makes use of out-of-band
caching to speed up builds drastically, at the expense of
reproducibility.
It is used to run tests and auxiliary test services (i.e. the custom
connector).
The new `debug.Dockerfile` marks the Cargo dependency and build caches
as Docker caches, which means they are shared between builds. This is
probably fine for local work and testing. The `Dockerfile` continues to
not use a cache like this, to guarantee that it is not polluted by extra
information, at the expense of build speed.
In addition, we build a `nextest` archive ahead of time to avoid
building tests when attempting to run them.
On my machine, a re-run of `just test` now takes seconds.
I have also sped up the `postgres` container start time by creating a
database called "finished" last, and then waiting for that to show up.
V3_GIT_ORIGIN_REV_ID: 7ef0548361987175b68a0cad44c8f2295110a1fb
Support both W3C and Zipkin B3 headers and connectors will
prefer B3 headers to extract traces because Cloud Run doesn't respect
those headers.
V3_GIT_ORIGIN_REV_ID: 1471b848e71d87420bc75b9adee60951f306c856
Revert the local docker-compose to expose ndc-postgres on port `8080` to
match the metadata. Otherwise running tests locally fails.
V3_GIT_ORIGIN_REV_ID: 6ca104a3041f758f13dfb2a05c2a6ab0b449e7cf
This injects trace context headers into requests to the auth hook,
allowing us to figure out how much time is spent here.
I added a basic tracing setup to the dev-auth-webhook, using
`tracing-util`, allowing me to verify that this works. This required
moving the Dockerfile to the root so the context contains the
`tracing-util` crate too.
I have also fixed the reference agent (by updating it), and patched our
Docker Compose files to correctly set up connectivity to Jaeger.
V3_GIT_ORIGIN_REV_ID: 2ff930bda4147d00dcc73268a814b08c8a07a359
This seems appropriate now that we've stabilized the new configuration.
Of note are the configuration updates and the use of an environment
variable to specify the connection URI. This upgrade also fixes the
health checks.
Regenerating the configuration lost the table descriptions, which seems
to be because they were not present in the Chinook SQL. I have dragged
the Chinook SQL in from ndc-postgres and kept it separate from the
initialization of other tables.
The auto-generated configuration is slightly different from the
manually-created configuration in that the collection names are
singular, not plural. This means that I had to change a lot of test
metadata files too.
V3_GIT_ORIGIN_REV_ID: 2b66fd3049aaf4daeb386915ea3b64a209b1f393
- Use `rust-toolchain.toml` to specify tools.
- Drop the unused version field from the compose files.
- Use a modern Docker base for the dev-auth-webhook.
- Handle signals in the test servers so `docker stop` is quick.
- Upgrade Jaeger.
V3_GIT_ORIGIN_REV_ID: b5fee1d5dc953a1fdb0aabd75ba02df6846b7518
I got frustrated by the lack of useful output when services failed to
start in CI, so I thought I'd mess around.
I made some changes so that:
1. we wait for services with health checks to report as healthy,
2. we do not rebuild the engine or custom connector when they start,
3. the health checks work (apart from ndc-postgres, which will come
later), and
4. we use environment variables rather than command-line arguments where
possible.
I have also renamed the "agent" binary to "custom-connector", matching
its crate, because it was driving me a little crazy.
V3_GIT_ORIGIN_REV_ID: 8d672b0b25438b54d47368ce82cd236cfdd4e554
## Description
The `just dev` command had ceased to work because the path to the auth
hook dockerfile had changed with the move to the `crates/` folder.
V3_GIT_ORIGIN_REV_ID: 52c21e41ff39125daeffb7e1a8e5b77664617d7b
<!-- Thank you for submitting this PR! :) -->
## Description
This moves all the crates into a `/crates` folder. Everything appears to
just work, thanks Cargo!
V3_GIT_ORIGIN_REV_ID: 8e3ef287b1a46cabdb4d919a50e813ab2cddf8b1
