hasura/graphql-engine
1
0
Fork 0
mirror of https://github.com/hasura/graphql-engine.git synced 2024-12-15 17:31:56 +03:00
Code Issues Projects Releases Wiki Activity
6ed9f36125
graphql-engine/server/tests-py/test_query_cache.py

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

57 lines
2.2 KiB
Python
Raw Normal View History

Caching, Rate Limiting, Metrics & Session Variable Improvements (#376) * server: use a leaky bucket algorithm for bytes-per-second cache rate limiting * Use evalsha properly * Adds redis cache limit parameters to PoliciesConfig * Loads Leaky Bucket Script On Server Start * Adds more redis logging and moves cache update into lua script * reverts setex in lua and adds notes * Refactors cacheStore and adds max TTL and cache size limits * Filter session vars in cache key * WIP * parens * cache-clear-hander POC implementation * cache-clear-hander POC implementation * Pro projectId used as cache key * POC working! * prefixing query-response keys in redis * Add cacheClearer to RedisScripts * Partial implementation of cacheClearer from scripts record * updating tests * [automated] stylish-haskell commit * Adds query look with up with metrics script * Adds missing module and lua script from last commit * Changes redis script module structure to match cache clearing branch * minor change to lua script * cleaning up cache clearing * generalising JsonLog * [automated] stylish-haskell commit * Draft Cache Metrics Endpoint * Adds Cache Metrics Handler * Adds hook handler module * Missed HandlerHook module in last commit * glob * Fixes redis mget bug * Removes cache totals and changes dashes to colons in metric cache keys * Adds query param to clear clear endpoint for deleting specific keys * Adds query param to clear clear endpoint for deleting specific keys * Cache Metrics on query families rather then queries * Replace Set with nub * Base16 Redis Hashes * Query Family Redis Keys With Roles * response headers for cache keys * fixing bug in family key by excluding operation name; using hash for response header instead of entire key * Adds query family to redis cache keys and cache clear endpoint * Fixes queryfamily hash bug * Moves cache endpoints to /pro * Moved cache clear to POST * Refactors cache clear function * Fixes query family format bug * Adds query cache tests and optional --redis-url flag to python test suite * Adds session variable cache test * Update pro changelog * adding documentation for additional caching features * more docs * clearing up units of leaky bucket params * Adds comments to leaky bucket script * removes old todo * Fixes session variable filtering to work with new query rootfield * more advanced defaulting behaviour for bucket rate and capacity. * Updates Docs * Moves Role into QueryFamily hash * Use Aeson for Cache Clear endpoint response * Moves trace to bracket the leaky bucket script * Misc review tweaks * Adds sum type for cache clear query params * Hardcodes RegisReplyLog log level * Update docs/graphql/cloud/response-caching.rst Co-authored-by: Phil Freeman <phil@hasura.io> * new prose for rate limiting docs * [automated] stylish-haskell commit * make rootToSessVarPreds total * [automated] stylish-haskell commit * Fixes out of scope error * Renamed _acRedis to _acCacheStore Co-authored-by: Solomon Bothwell <ssbothwell@gmail.com> Co-authored-by: Lyndon Maydwell <lyndon@sordina.net> Co-authored-by: David Overton <david@hasura.io> Co-authored-by: Stylish Haskell Bot <stylish-haskell@users.noreply.github.com> Co-authored-by: Lyndon Maydwell <lyndon@hasura.io> GitOrigin-RevId: dda5c1a3f902967b3d78310f950541a55fabb1b0
2021-02-13 03:05:23 +03:00
import pytest
import redis
Remove the server upgrade/downgrade tests. These tests are intended to catch issues in upgrading HGE. However: * the tests are very convoluted and hard to understand, * we can only run a small subset of Python tests that don't mutate any data or metadata, and * I have never seen them fail for a legitimate reason, but I've seen a lot of flakes. While we do believe it's important to test that upgrades don't break the stored introspection, these tests don't seem to be doing that any more. I humbly request that we delete them now and either (a) figure out how to test this properly, or (b) just wait for v3, which does away with reintrospecting on server startup entirely. [NDAT-259]: https://hasurahq.atlassian.net/browse/NDAT-259?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8844 GitOrigin-RevId: 528bc632fce377b7eff2026b832bd58586ac5a0b
2023-04-24 13:02:40 +03:00
from validate import check_query_f
Caching, Rate Limiting, Metrics & Session Variable Improvements (#376) * server: use a leaky bucket algorithm for bytes-per-second cache rate limiting * Use evalsha properly * Adds redis cache limit parameters to PoliciesConfig * Loads Leaky Bucket Script On Server Start * Adds more redis logging and moves cache update into lua script * reverts setex in lua and adds notes * Refactors cacheStore and adds max TTL and cache size limits * Filter session vars in cache key * WIP * parens * cache-clear-hander POC implementation * cache-clear-hander POC implementation * Pro projectId used as cache key * POC working! * prefixing query-response keys in redis * Add cacheClearer to RedisScripts * Partial implementation of cacheClearer from scripts record * updating tests * [automated] stylish-haskell commit * Adds query look with up with metrics script * Adds missing module and lua script from last commit * Changes redis script module structure to match cache clearing branch * minor change to lua script * cleaning up cache clearing * generalising JsonLog * [automated] stylish-haskell commit * Draft Cache Metrics Endpoint * Adds Cache Metrics Handler * Adds hook handler module * Missed HandlerHook module in last commit * glob * Fixes redis mget bug * Removes cache totals and changes dashes to colons in metric cache keys * Adds query param to clear clear endpoint for deleting specific keys * Adds query param to clear clear endpoint for deleting specific keys * Cache Metrics on query families rather then queries * Replace Set with nub * Base16 Redis Hashes * Query Family Redis Keys With Roles * response headers for cache keys * fixing bug in family key by excluding operation name; using hash for response header instead of entire key * Adds query family to redis cache keys and cache clear endpoint * Fixes queryfamily hash bug * Moves cache endpoints to /pro * Moved cache clear to POST * Refactors cache clear function * Fixes query family format bug * Adds query cache tests and optional --redis-url flag to python test suite * Adds session variable cache test * Update pro changelog * adding documentation for additional caching features * more docs * clearing up units of leaky bucket params * Adds comments to leaky bucket script * removes old todo * Fixes session variable filtering to work with new query rootfield * more advanced defaulting behaviour for bucket rate and capacity. * Updates Docs * Moves Role into QueryFamily hash * Use Aeson for Cache Clear endpoint response * Moves trace to bracket the leaky bucket script * Misc review tweaks * Adds sum type for cache clear query params * Hardcodes RegisReplyLog log level * Update docs/graphql/cloud/response-caching.rst Co-authored-by: Phil Freeman <phil@hasura.io> * new prose for rate limiting docs * [automated] stylish-haskell commit * make rootToSessVarPreds total * [automated] stylish-haskell commit * Fixes out of scope error * Renamed _acRedis to _acCacheStore Co-authored-by: Solomon Bothwell <ssbothwell@gmail.com> Co-authored-by: Lyndon Maydwell <lyndon@sordina.net> Co-authored-by: David Overton <david@hasura.io> Co-authored-by: Stylish Haskell Bot <stylish-haskell@users.noreply.github.com> Co-authored-by: Lyndon Maydwell <lyndon@hasura.io> GitOrigin-RevId: dda5c1a3f902967b3d78310f950541a55fabb1b0
2021-02-13 03:05:23 +03:00
from context import PytestConf
usefixtures = pytest.mark.usefixtures
@pytest.mark.skipif(not PytestConf.config.getoption('--redis-url'), reason="Must enable redis")
@pytest.mark.parametrize("transport", ['http'])
@usefixtures('per_class_tests_db_state')
class TestQueryCache:
def flushRedis(self):
# TODO: Move this into setup/teardown
r = redis.from_url(PytestConf.config.getoption('--redis-url'))
r.flushall()
@classmethod
def dir(cls):
return 'queries/query_cache'
def test_simple_cached_endpoint(self, hge_ctx, transport):
check_query_f(hge_ctx, self.dir() + '/simple_cached.yaml', transport)
self.flushRedis()
def test_metrics_one_endpoint(self, hge_ctx, transport):
check_query_f(hge_ctx, self.dir() + '/test_metrics_one.yaml', transport)
self.flushRedis()
def test_metrics_two_endpoint(self, hge_ctx, transport):
check_query_f(hge_ctx, self.dir() + '/test_metrics_two.yaml', transport)
self.flushRedis()
def test_cache_clear_endpoint(self, hge_ctx, transport):
check_query_f(hge_ctx, self.dir() + '/test_cache_clear.yaml', transport)
self.flushRedis()
def test_cache_clear_endpoint_key(self, hge_ctx, transport):
check_query_f(hge_ctx, self.dir() + '/test_cache_clear_key.yaml', transport)
self.flushRedis()
def test_cache_clear_endpoint_family(self, hge_ctx, transport):
check_query_f(hge_ctx, self.dir() + '/test_cache_clear_family.yaml', transport)
self.flushRedis()
def test_no_variables_in_query_key(self, hge_ctx, transport):
check_query_f(hge_ctx, self.dir() + '/test_no_variables_in_query_key.yaml', transport)
self.flushRedis()
server: disable caching for actions with forward client headers enabled Co-authored-by: Lyndon Maydwell <92299+sordina@users.noreply.github.com> Co-authored-by: Antoine Leblanc <1618949+nicuveo@users.noreply.github.com> Co-authored-by: Abby Sassel <3883855+sassela@users.noreply.github.com> Co-authored-by: hasura-bot <30118761+hasura-bot@users.noreply.github.com> Co-authored-by: Rikin Kachhia <54616969+rikinsk@users.noreply.github.com> Co-authored-by: Ikechukwu Eze <22247592+iykekings@users.noreply.github.com> Co-authored-by: Aleksandra Sikora <9019397+beerose@users.noreply.github.com> Co-authored-by: Rishichandra Wawhal <27274869+wawhal@users.noreply.github.com> Co-authored-by: Naveen Naidu <30195193+Naveenaidu@users.noreply.github.com> Co-authored-by: Vishnu Bharathi <4211715+scriptnull@users.noreply.github.com> GitOrigin-RevId: c9a8be3cb607f7767e9d6791717106adf123e3a8
2021-04-13 10:00:43 +03:00
def test_action_query_with_forward_client_headers_set(self, hge_ctx, transport):
check_query_f(hge_ctx, self.dir() + '/test_action_query_with_forward_client_headers_set.yaml', transport)
self.flushRedis()
def test_action_query_with_forward_client_header_unset(self, hge_ctx, transport):
check_query_f(hge_ctx, self.dir() + '/test_action_query_with_forward_client_headers_unset.yaml', transport)
self.flushRedis()
Reference in New Issue Copy Permalink