wasp/web/blog/2022-01-27-waspleau.md
Filip Sodić 2fe7398462
Adapt example apps to project struct change (#809)
* Separate ext code to client and server

* Use skeleton in createNewProject and refactor

* Refactor Lib.hs to use ExceptT

* Fix formatting

* Pop up returns

* Extract liftIO and add a do block

Co-authored-by: Shayne Czyzewski <523636+shayneczyzewski@users.noreply.github.com>

* Address some review comments

* Add skeleton comment

* Extract common CommandError message

* Separate skeleton comment into two rows

* Move server and client dirs into src

* Simplify maybeToEither

* Further refactor Lib.hs

* Further simplify skeleton comment

* Add shared code directory to project structure

* Update e2e test inputs

* Update e2e test outputs

* Fix formatting

* Fix bug in compile function

Co-authored-by: Martin Šošić <Martinsos@users.noreply.github.com>

* Change map to fmap in compile function

* Fix formatting

* Force git to include empty directories

* Remove extra empty line from .gitkeep files

* Add .jsconfig to enable go-to-definition

* Watch shared directory for changes

* Add final newline to jsconfigs

* Fix regular and e2e tests

* Update e2e tests

* Fix cli template packaging and update todoApp

* Add a shared function demo to todoApp

* Update waspc and e2e tests

* Fix compiler warnings and rename function

* Rename mkError to mkParserError

* Remove redundant empty line

* Fix test warnings

* Fix formatting

* Update waspc.cabal with jsconfigs

* Minimize jsconfig.json files

* Add jsconfigs to waspc todoApp

* Update e2e tests

* Update e2e tests

* Update docs for new project structure

* Update todoApp example to new structure

* Update ItWaspsOnMyMachine

* Update thoughts

* Update Waspello to new structure

* Update Waspleau to new structure

* Update Realworld to new structure, fix warnings

* Fix directory tree watching on wasp start

* Implement review feedback

* Fix typo in docs

Co-authored-by: Shayne Czyzewski <523636+shayneczyzewski@users.noreply.github.com>

* Fix typo in docs

Co-authored-by: Shayne Czyzewski <523636+shayneczyzewski@users.noreply.github.com>

* Fix another typo in docs

Co-authored-by: Shayne Czyzewski <523636+shayneczyzewski@users.noreply.github.com>

* Add src prefix to file path in docs

Co-authored-by: Shayne Czyzewski <523636+shayneczyzewski@users.noreply.github.com>

* Remove env server file

* Apply suggestions from code review

Co-authored-by: Shayne Czyzewski <523636+shayneczyzewski@users.noreply.github.com>

* Address code review comments

* Fix incorrect path in docs

* Fix references to ext in docs

* Edit changelog and add migration guide

* Further update docs to new structure

* Update blogs about example apps to new structure

* Fix typo in docs

Co-authored-by: Shayne Czyzewski <523636+shayneczyzewski@users.noreply.github.com>

* Update typo on frontpage

Co-authored-by: Shayne Czyzewski <523636+shayneczyzewski@users.noreply.github.com>

* Add missing src in docs

Co-authored-by: Shayne Czyzewski <523636+shayneczyzewski@users.noreply.github.com>

Co-authored-by: Shayne Czyzewski <523636+shayneczyzewski@users.noreply.github.com>
Co-authored-by: Martin Šošić <Martinsos@users.noreply.github.com>
2022-11-11 18:58:29 +01:00

6.7 KiB
Raw Blame History

title authors image tags
Build a metrics dashboard with background jobs in Wasp - Say hello to Waspleau!
shayneczyzewski
/img/waspleau-screenshot.png
webdev
wasp

import Link from '@docusaurus/Link'; import useBaseUrl from '@docusaurus/useBaseUrl'; import InBlogCta from './components/InBlogCta'; import WaspIntro from './_wasp-intro.md'; import ImgWithCaption from './components/ImgWithCaption'

Hello, Waspleau

See Waspleau here! | See the code

We've built a dashboard powered by a job queue using Wasp!

Measure all the metrics!

Hello, Waspleau!

Lets face it - metrics are all around us. Wouldnt it be great if there was a quick and easy way to build a nice-looking metrics dashboard from data pulled in by HTTP calls to many different sources, cache the data in-memory, and periodically update it via background jobs? Why yes, yes it would... so we made an example Wasp app called Waspleau that does just that!

Here is what it looks like live: https://waspleau.netlify.app/ There is also a screenshot at the top of this post for those who refrain from clicking on any unknown web links for fear of being Rickrolled. Respect.

“Show me the code”

So, what do we need to get started? First, we need a way to schedule and run jobs; for this, we decided to use Bull. Ok, lets wire it up. This should be easy, right? We can add external NPM dependencies in our Wasp files like so:

app waspleau {
  title: "Waspleau",

  dependencies: [
    ("bull", "4.1.1"),
    ("axios", "^0.21.1")
  ]
}

But where do we declare our queue and processing callback functions in Wasp? Uh oh...

Sad

server.setupFn for queue setup

Thankfully, Waspleau can leverage a powerful and flexible hook supplied by Wasp called server.setupFn. This declares a JavaScript function that will be executed on server start. Yahoo! This means we can do things like the following:

app waspleau {
  ...

  server: {
    setupFn: import serverSetup from "@server/serverSetup.js"
  }
}
import Queue from 'bull'

const queue = new Queue('waspleau', process.env.REDIS_URL || 'redis://127.0.0.1:6379',
  { defaultJobOptions: { removeOnComplete: true } }
)

queue.process('*', async (job) => { ... })

export default async () => {
  // To initially populate the queue, we can do:
  await queue.add({ ... }) // first run, one-off job
  await queue.add({ ... }, { repeat: { cron: '*/10 * * * *' } }) // recurring job
}

Abstracting workers and job processing

Awesome, we can now enqueue and process background jobs, but how can we make it easy to create many different kinds of jobs and schedule them to run at different intervals? For Waspleau, we created our own type of worker object convention to help standardize and simplify adding more:

const workerFunction = async (opts) => {
  return [
    { name: 'Metric 1 name', value: 'foo', updatedAt: ... },
    { name: 'Metric 2 name', value: 'bar', updatedAt: ... },
  ]
}

export const workerTemplate = { name: 'Job Name', fn: workerFunction, schedule: '*/10 * * * *' }

With this workerFunction setup, we can return one or more metrics per worker type. Waspleau can easily use any module that exports this shape. Here is a real example from the demo that makes HTTP calls to GitHubs API with Axios:

import axios from 'axios'

const workerFunction = async (opts) => {
  console.log('github.js workerFunction')

  const now = Date.now()

  try {
    const response = await axios.get('https://api.github.com/repos/wasp-lang/wasp')

    return [
      { name: 'Wasp GitHub Stars', value: response.data.stargazers_count, updatedAt: now },
      { name: 'Wasp GitHub Language', value: response.data.language, updatedAt: now },
      { name: 'Wasp GitHub Forks', value: response.data.forks, updatedAt: now },
      { name: 'Wasp GitHub Open Issues', value: response.data.open_issues, updatedAt: now },
    ]
  } catch (error) {
    console.error(error)
    return []
  }
}

export const githubWorker = { name: 'GitHub API', fn: workerFunction, schedule: '*/10 * * * *' }

Note: Please see the actual serverSetup.js file for how we use this abstraction in practice.

Server → client

We now have jobs running and data updating at regular intervals, nice, but we still need a way to send that data down the wire. Here, we expose the in-memory data from our server.setupFn module so our queries can also use it:

...

query dashboard {
  fn: import { refreshDashboardData } from "@server/dashboard.js"
}
import { getDashboardData } from './serverSetup.js'

export const refreshDashboardData = async (_args, _context) => {
  return getDashboardData()
}
...

const dashboardData = {} // This is updated in the queue process callback
export const getDashboardData = () => Object.values(dashboardData).flat()

From there, we can request it on the frontend in React components as usual and also set a one-minute client-side refetch interval just for good measure:

...

const { data: dashboardData, isFetching, error } = useQuery(refreshDashboardData, null, { refetchInterval: 60 * 1000 })

...

Congratulations, lets dance!

Whew, we did it! If youd like to deploy your own customized version of this dashboard, please clone our repo and check out the Waspleau example README.md for tips on getting started. You can also check out our docs to dive deeper into anything.

Rickroll

Still got ya! :D

2022 is going to be exciting 🚀

While this functionality currently exists outside of Wasp, keep an eye on our roadmap as we head toward 1.0. We will be busy adding lots of great features to our Wasp DSL in the coming months that will supercharge your web development experience! Thanks for reading, and please feel free to connect with us in Discord about using Wasp on your next project.