Made examples/todoApp more approachable (+ documented what it should be used for) (#2094)

* Made waspc/examples/todoApp easier to run (docs + env files).

* Added .env.server management via dotenv-vault.

* Documented how to test compilation and building of todoApp.

* Updated PR template to mention example todo apps.

* fix

* Apply suggestions from code review

* Update waspc/README.md

* fix

* Update waspc/examples/todoApp/.gitignore

.github/pull_request_template.md

@@ -3,8 +3,7 @@
 > Describe your PR! If it fixes specific issue, mention it with "Fixes # (issue)".
 
 ### Select what type of change this PR introduces:
-
-1. [ ] **Just code/docs improvement** (no functional change). 
+1. [ ] **Just code/docs improvement** (no functional change).
 2. [ ] **Bug fix** (non-breaking change which fixes an issue).
 3. [ ] **New feature** (non-breaking change which adds functionality).
 4. [ ] **Breaking change** (fix or feature that would cause existing functionality to not work as expected).
@@ -13,3 +12,8 @@
 If you did a bug fix, new feature, or breaking change, that affects waspc, make sure you satisfy the following:
 1. [ ] I updated [ChangeLog.md](https://github.com/wasp-lang/wasp/blob/main/waspc/ChangeLog.md) with description of the change this PR introduces.
 2. [ ] I bumped waspc version in [waspc.cabal](https://github.com/wasp-lang/wasp/blob/main/waspc/waspc.cabal) to reflect changes I introduced, with regards to the version of the latest wasp release, if the bump was needed.
+
+### Update example apps if needed
+If you did code changes and added a new feature or modified an existing feature, make sure you satisfy the following:
+1. [ ] I updated `waspc/examples/todoApp` as needed (updated modified feature or added new feature) and manually checked it works correctly.
+2. [ ] I updated `waspc/headless-test/examples/todoApp` and its e2e tests as needed (updated modified feature and its tests or added new feature and new tests for it).

.github/workflows/waspc-ci.yaml

@@ -158,9 +158,9 @@ jobs:
       - name: Run unit tests
         run: cabal test cli-test waspc-test waspls-test
 
-      - name: Ensure todoApp works
+      - name: Ensure examples/todoApp compiles and builds
         if: matrix.os == 'ubuntu-20.04' || matrix.os == 'macos-latest'
-        run: ./tools/ensure_todoapp_works.sh
+        run: ./examples/todoApp/ensure_app_compiles_and_builds.sh
 
       - name: Headless - Cache Node Modules
         id: headless-cache-node-modules

waspc/README.md

@@ -121,20 +121,19 @@ NOTE: Reload page if blank.
 3. Do a change in the codebase (most often in `src/` or `cli/src/` or `data/`) (together with tests if that makes sense: see "Tests").
    Fix any errors shown by HLS/`ghcid`.
    Rinse and repeat.
-4. If you did a bug fix, added new feature or did a breaking change, add short info about it to Changelog.md. Also, bump version in waspc.cabal and ChangeLog.md if needed -> check the version of latest release when doing it. If you are not sure how to decide which version to go with, check later in this file instructions on it.
-5. Once close to done, run `cabal test` to confirm that the project's tests are passing (both new and old).
-6. If needed, confirm that `examples/todoApp/` is working correctly by running `cabal build` first, to build the wasp executable, and then by running that executable with `cabal run wasp-cli start` from the `examples/todoApp/` dir -> this will run the web app in development mode with the current version of your Wasp code.
-   Manually inspect that app behaves ok: In the future we will add automatic integration tests, but for now testing is manual.
-7. Squash all the commits into a single commit (or a few in case it makes more sense) and create a PR.
+4. Run `cabal test` to confirm that the project's unit and integration tests are passing (both new and old).
+5. For easily testing the changes you did on a Wasp app, you have `examples/todoApp`, which we always keep updated. Also, if you added a new feature, add it to this app. Check its README for more details (including how to run it).
+6. There is also `headless-test/examples/todoApp`, that comes equiped with Playwright tests. This one is not so much for experimenting as `examples/todoApp` is, but more for running Playwright tests in the CI. Make sure this one is also passing, and add a new feature to it (+ tests) if needed. We will be merging this one with `examples/todoApp` in the future so it is a single app.
+7. If you did a bug fix, added new feature or did a breaking change, add short info about it to Changelog.md. Also, bump version in waspc.cabal and ChangeLog.md if needed -> check the version of latest release when doing it. If you are not sure how to decide which version to go with, check later in this file instructions on it.
+8. Squash all the commits into a single commit (or a few in case it makes more sense) and create a PR.
    Keep an eye on CI tests -> they should all be passing, if not, look into it.
-8. If your PR changes how users(Waspers) use Wasp, make sure to also update the documentation, which is in this same repo, but under `/web/docs`.
-9. Work with reviewer(s) to get the PR approved.
+9. If your PR changes how users(Waspers) use Wasp, make sure to also update the documentation, which is in this same repo, but under `/web/docs`.
+10. Work with reviewer(s) to get the PR approved.
    Keep adding "fix" commits until PR is approved, then again squash them all into one commit.
-10. Reviewer will merge the branch into `main`. Yay!
+11. Reviewer will merge the branch into `main`. Yay!
 
-NOTE: Why don't you use a cabal freeze file?
-In order to better support a wider range of developer operating systems, we have decided against using a cabal freeze file and instead
-use cabal's `index-state` feature to get package version pinning from hackage. See this question for more: https://github.com/haskell/cabal/issues/8059
+NOTE: Why don't we use a cabal freeze file to lock our dependencies?
+In order to better support a wider range of developer operating systems, we have decided against using a cabal freeze file and instead use cabal's `index-state` feature to get package version pinning from hackage. See this question for more: https://github.com/haskell/cabal/issues/8059
 
 ## Design docs (aka RFCs)
 

waspc/examples/todoApp/.env.server.example

@@ -0,0 +1,20 @@
+TEST_ENV_VAR="I am test"
+
+# Dummy values here will allow app to run, but you will need real values to get Google Auth to work.
+GOOGLE_CLIENT_SECRET='dummy-g-client-secret'
+GOOGLE_CLIENT_ID='dummy-g-client-id.apps.googleusercontent.com'
+
+# Dummy values here will allow app to run, but you will need real values to get GitHub Auth to work.
+GITHUB_CLIENT_ID='dummy-gh-client-id'
+GITHUB_CLIENT_SECRET='dummy-gh-client-secret'
+
+# Uncomment lines below and set them to real values if you want to use smtp email sender.
+# SMTP_HOST='smtp.dummy.com'
+# SMTP_USERNAME='dummy@dummy.com'
+# SMTP_PASSWORD='dummy_pass'
+# SMTP_PORT='587'
+
+# Uncomment lines below and set them to real values if you want to use Keycloak Auth.
+# KEYCLOAK_CLIENT_ID='dummy-id'
+# KEYCLOAK_CLIENT_SECRET='dummy-secret'
+# KEYCLOAK_REALM_URL='dummy-url'

waspc/examples/todoApp/.env.vault

@@ -0,0 +1,25 @@
+#/-------------------.env.vault---------------------/
+#/         cloud-agnostic vaulting standard         /
+#/   [how it works](https://dotenv.org/env-vault)   /
+#/--------------------------------------------------/
+
+# development
+DOTENV_VAULT_DEVELOPMENT="Afog8ZPUwdbDvWg46qIdxHqzMghXgcbkYViEFXRj8gi2s2/AH/yiyNomjO/lmf2CjOF3f5wWr9ArQ+1BIbmmTI2c/Pgc1LNKp3NCzfeFv5MGlgVvUBIjJ9qsbdE5Iz8U8xNReUdb9s1u7Urye671rX3RhiL7UGvEl6OcO9vpzcqbMbGY6v3AyGR+DaKNjEY68iTyqjeC6flbPHIJ1GXEjK+k+R62jZEwcNGYBedo6vM9GQW6CWmWP9h4eVOxg+EUbdrUPFPnJ5gJisGgTP+/mjT9VQNbzkd+VKaXVsVunU9EJ1Ppnpvqno+u1GgBTFMxmX2HYgMOOAlvHuaJN6zWOohF2kKSIWmyzx7ratLuYx0wL7aCCG/IWuoul5OUBCFQcWaUyehJie7l0w4iIPKhchmPylQp+dawtl6QCeGO0f4OdnMQPQ8XqHi0knQtaC9UVniKJLQlop+8px54VBYBPOhFbjqVGmQrggwY5wF26p/sD63rmjFRGj11ZpqAo7XCqLM8OPrZAF2091XE7p+BCWHIyKfCHWNtPit4RpqKQkcOfZBYBRPnuwen1yB+sL+I4MSyZ4Sc+HQ2cEqY3y5nSWJ/UkkiH+5ghsBu53CbnwrLa/qysTAlvmS4UW+O8gleWFP+hm10Dta5VgbA6Mj6907/XqlFbizfOMhBSEfZJX7k4ujmOJXJ9A3ZMEeerfWyHubZPW5HGRt58MfLYTHZhSqjxrhOfc/0vQB1aRKqGM/qUO8U98uD6IplMBpXA8Lf8jrL3eqHfL5VscYVkJSgsZxA"
+DOTENV_VAULT_DEVELOPMENT_VERSION=3
+
+# ci
+DOTENV_VAULT_CI="/ca9KordiK0CT57pAqdyAYbiVpTJP0xYWWxw3yvQHhBDORDg//wUByl4pfBWuihLNsoDi9zTe3oU+h4I5Fu49eKj1I8Y6F0K8m5NSOwfn/V4f+l3p4Ca5MSo/bacCLhRp7OoDYl6A7pHPXurkQlxfBdzprE4kkOH1kYfJ9/jyB7HXca4jILIe4eQ2p36Q7yDPfmQjyxcAUuyoezub0ss+ItyCQpNPE/erpqzcQ9JaMUm21V/nd7HZZnHt3J6gXZjuqI40wk6FRLMBLgJgc239KyCM8EwWDON3XqEEfMzbe0U8IEqCCWI6Cl+rAStzrZnjyMllKsrZqGiqG4jYoI="
+DOTENV_VAULT_CI_VERSION=2
+
+# staging
+DOTENV_VAULT_STAGING="LeqyS8zV/1zoEaktIZFgEWm8dw8NH2EtZ1jdslHI0Kc+CDOWcb7V5nu5Pzhv1PzElF6Uy2WtszE9v+urynqp0F9xaVUylnU0QG9bpMHXoM0m2QhHOb2WVfJe25fqJ9kBKWslVV8BA1R5ncg+8jGi9reJP8C0BFnOHzJdpn9450EIdfIP3vKb/cdxagLvrl4ZJvopkbwblvIPhsUImCUuhOY+pzGVw2wDm7VMk1/a+qztRyEf3mU8FS7vK+zOSM48IQ9a+JJKvpYz9tymnaVuqgwmA0tZP7lgf5F2cP0RQnTFx2rkHxtA7iGSbYXqVaIMrUXrIC8YDkE1FPHQNkc="
+DOTENV_VAULT_STAGING_VERSION=2
+
+# production
+DOTENV_VAULT_PRODUCTION="JFZnsB7IZ3dsPqBkdbbQxJXBoBZaZLZ4aMTQ9YV0/leH0FUjLEzXsIXMNPL5qpnP0LrUKQUypsSR1INST972nNbcZYdXz6qU2j6XX02cIhM3hYN4L2zRZGayI7VhDmdkJ3f+ocsuDJ+qmK71RoFoAP8WOyi/HQTEIVNdZn9Cyv8y2IuQUlqFuyojNzbWbcMhhHLBX8IM8+YYOGhV1Mf7s8J2QaRruA+vNfRyFuhNsBvGPg1fvot9GTj0G7hJnffsC4sIUqBYJ5I/GZNaLzt9HReb6MBezhxEeGWOwtzwWIk/Mt9eCwSgeAG+vx5MZRbcJFNRZTVz/ZJD7F7/keY="
+DOTENV_VAULT_PRODUCTION_VERSION=2
+
+#/----------------settings/metadata-----------------/
+DOTENV_VAULT="vlt_42e44a97f39aa7e0e5ca2c68a664443384c3307ba60fe8aa6e1e30a21662b2ff"
+DOTENV_API_URL="https://vault.dotenv.org"
+DOTENV_CLI="npx dotenv-vault@latest"

waspc/examples/todoApp/.gitignore

@@ -6,6 +6,15 @@ node_modules/
 .env
 .env.*
 
+# These two we added only because dotenv-vault keeps adding them if it doesn't find them,
+# even though we don't need them. Remove them once dotenv-vault stops doing that.
+.env*
+.flaskenv*
+
 # Don't ignore example dotenv files.
 !.env.example
 !.env.*.example
+
+# These are config files for dotenv-vault, so we don't want to ignore them.
+!.env.project
+!.env.vault

waspc/examples/todoApp/README.md

@@ -0,0 +1,58 @@
+# TodoApp
+
+The main purpose of this app is for contributors to easily test the version of the `wasp` CLI that they are working on right now and therefore aid them during development.
+
+That is why it is a bit crowded with features and somewhat incoherent: it is not an app to be used as a showcase, but an app that we use as a "workbench" / "playground" while working on `wasp`.
+
+This app should always be "up to date", in the sense that we keep updating it to always work with the latest changes to `wasp`. This means you can count on it being up to date, but also that you should make sure to keep it up to date as you do changes. You should also be adding new features to it if you implement them, so they can easily be tested / tried out.
+
+## Set up
+
+### Env vars
+
+To do the minimum to get the app running, do:
+```sh
+cp .env.server.example .env.server
+```
+This will create a `.env.server` file with some basic env vars set to dummy values. This will allow your app to run and work, but not all features will work, e.g. Google Auth won't work because it needs you to provide real API keys via env vars.
+
+To obtain all the API keys to get all the features of the app working, if you do need that, you will have to obtain them on your own, normally from the corresponding third-party services. Check Wasp docs for instructions on how to do this for Google Auth, GitHub Auth, and so on.
+
+#### Obtaining API keys as a Wasp Team member
+If you are a member of the Wasp Team, you don't have to obtain all the API keys on your own.
+
+Instead, you can run `npm run env:pull`: this will obtain `.env.server` file that we share on the team level.  
+We are using https://vault.dotenv.org to power this and have an account/organization up there.  
+If you modify `.env.server` and want to persist the changes (for yourself and for the other team members), do `npm run env:push`.
+
+## Running
+
+Start the database in a separate terminal session:
+```
+cabal run wasp-cli start db
+```
+
+Migrate the database if needed:
+```
+cabal run wasp-cli db migrate-dev
+```
+
+Run the app!
+```
+cabal run wasp-cli start
+```
+
+Open `localhost:3000` in the browser to see the app!  
+NOTE: Normally wasp apps open automatically in browser, to help new users find their way, but that can become annoying after some time so here we turned that off.
+
+## Testing
+
+### Compiling and building
+
+Run `./ensure_app_compiles_and_builds.sh` to verify the app correctly compiles (TS) and builds (both client and server).
+
+This is also run in CI.
+
+### E2E tests
+
+We don't have any yet, but there is `headless-test/examples/todoApp` that is a fork from this app but with Playwright tests. We plan to merge these two into one app, to get the best of both worlds.

waspc/examples/todoApp/ensure_app_compiles_and_builds.sh

@@ -6,9 +6,9 @@
 
 # Gets the directory of where this script lives.
 dir=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
-echo "Running ensure_todoapp_works.sh from $dir"
+echo "Running ensure_app_compiles_and_builds.sh from $dir"
 
-cd "$dir/../examples/todoApp"
+cd "$dir"
 
 # Compile example app.
 cabal run wasp-cli build

waspc/examples/todoApp/package.json

@@ -1,5 +1,9 @@
 {
   "name": "todoApp",
+  "scripts": {
+    "env:pull": "npx dotenv-vault@latest pull development .env.server",
+    "env:push": "npx dotenv-vault@latest push development .env.server"
+  },
   "dependencies": {
     "@tailwindcss/forms": "^0.5.3",
     "@tailwindcss/typography": "^0.5.7",

waspc/examples/todoApp/sample.env.server

@@ -1,3 +0,0 @@
-TEST_ENV_VAR="I am test"
-DATABASE_URL=postgresql://postgres:postgres@localhost/todoapp
-#PORT=3001

waspc/headless-test/examples/todoApp/.gitignore

@@ -1,3 +1,11 @@
-/.wasp/
-/.env.server
-/.env.client
+.wasp/
+node_modules/
+
+# Ignore all dotenv files by default to prevent accidentally committing any secrets.
+# To include specific dotenv files, use the `!` operator or adjust these rules.
+.env
+.env.*
+
+# Don't ignore example dotenv files.
+!.env.example
+!.env.*.example

waspc/headless-test/package.json

@@ -8,7 +8,7 @@
     "example-app:start-db": "npm run example-app:cleanup-db && cd ./examples/todoApp && cabal run wasp-cli start db",
     "example-app:start-app": "npm run example-app:wait-for-db && npm run example-app:prepare-env && cd ./examples/todoApp && cabal run wasp-cli db migrate-dev && cabal run wasp-cli start",
     "example-app:wait-for-db": "sleep 5 && ./node_modules/.bin/wait-port 5432",
-    "example-app:prepare-env": "cp ./examples/todoApp/sample.env.server ./examples/todoApp/.env.server",
+    "example-app:prepare-env": "cp ./examples/todoApp/.env.server.example ./examples/todoApp/.env.server",
     "example-app:cleanup-db": "(docker container rm $(docker container ls -f name=^wasp-dev-db-todoApp- -q) -f || true) && docker volume rm $(docker volume ls -f name=^wasp-dev-db-todoApp- -q) -f || true"
   },
   "keywords": [],