GitOrigin-RevId: c3858e4ed2f3d544f797ee159bd03b458d9cf82e
3.6 KiB
Serverless ETL demo using Hasura GraphQL Engine Event Triggers and Algolia Search
Live demo at https://serverless-etl.demo.hasura.app/
This application demonstrates an ETL process using event triggers on Hasura GraphQL Engine.
A searchable book library is shown where user can add title and author for new books. When a new book in inserted into the database using GraphQL Mutation, a Google Cloud Function is triggered which updates an Algolia search index. On the search screen, user can search through this index and results are shown using Algolia APIs. As many users add more books, the search index gets bigger.
Instructions
Pre-requisites
-
A Google Cloud account with billing enabled
-
gcloud
CLI -
Algolia account
-
Hasura GraphQL Engine (HGE) installation
-
Deploy GraphQL Engine on Hasura Cloud and setup PostgreSQL via Heroku:
Step 1: Create table
Go to HGE console and create the following table:
Table name: book
id UUID default: gen_random_uuid()
created_at Timestamp
title Text
author Text
Primary key: id
Step 2: Setup an Algolia index
- Sign-up for Algolia account
- Go to the Dashboard,
- Click on
Indices
tab on the left sidebar - Click on
New Index
button - Name it as
demo_serverless_etl_app
- Goto
API Keys
tab on the sidebar - Copy the
Application ID
(we'll call thisAPP_ID
) - Click on
All API Keys
tab and then clickNew API Key
button - Give description as
server key
- Choose the
demo_serverless_etl_app
index - Select
Add records
under the ACL section - Click
Generate
and copy this API key from the list (we'll call itADMIN_KEY
) - Similarly create one more API key called
search key
for the same index, but ACL as onlySearch
(we'll call this oneSEARCH_KEY
)
Step 3: Deploy the cloud function
- Go to
cloudfunction
directory and deploy the function:
Replacegcloud beta functions deploy serverless-etl \ --runtime nodejs8 --trigger-http \ --set-env-vars="ALGOLIA_APP_ID=<APP_ID>,ALGOLIA_ADMIN_API_KEY=<ADMIN_KEY>"
APP_ID
andADMIN_KEY
with the keys generated in the previous step. - Note down the url for triggering the function from the output (call it
TRIGGER_URL
):httpsTrigger: url: https://us-central1-hasura-test.cloudfunctions.net/serverless-etl
Step 4: Create an Event Trigger
Go to Events
tab in HGE Console and add a new trigger:
Trigger name: book_event
Schema/Table: public/book
Operations: Insert
Webhook URL: <TRIGGER_URL>
Use the TRIGGER_URL
from previous step.
Step 5: Add configuration variables
Edit index.js
and add the following values:
HGE_URL
, your GraphQL Engine URL (ending withv1/graphql
)APP_ID
, the algolia application idSEARCH_KEY
, algolia search api key created in Step 2
Step 6: Run the application
Run any HTTP server locally and visit the URL on a browser:
npx http-server