daml/infra
Bernhard Elsner cda93db944
Daml case and logo (#8433)
* Replace many occurrences of DAML with Daml

* Update docs logo

* A few more CLI occurrences

CHANGELOG_BEGIN
- Change DAML capitalization and docs logo
CHANGELOG_END

* Fix some over-eager replacements

* A few mor occurrences in md files

* Address comments in *.proto files

* Change case in comments and strings in .ts files

* Revert changes to frozen proto files

* Also revert LF 1.11

* Update get-daml.sh

* Update windows installer

* Include .py files

* Include comments in .daml files

* More instances in the assistant CLI

* some more help texts
2021-01-08 12:50:15 +00:00
..
macos update copyright notices for 2021 (#8257) 2021-01-01 19:49:51 +01:00
modules/gcp_cdn_bucket update copyright notices for 2021 (#8257) 2021-01-01 19:49:51 +01:00
.gitignore open-sourcing daml 2019-04-04 09:33:38 +01:00
apply infra: add a VSTS windows agents (#368) 2019-04-18 11:20:57 +00:00
bazel_cache.tf update copyright notices for 2021 (#8257) 2021-01-01 19:49:51 +01:00
binaries.tf update copyright notices for 2021 (#8257) 2021-01-01 19:49:51 +01:00
data_bucket.tf update copyright notices for 2021 (#8257) 2021-01-01 19:49:51 +01:00
dumps_bucket.tf update copyright notices for 2021 (#8257) 2021-01-01 19:49:51 +01:00
hoogle_server.tf update copyright notices for 2021 (#8257) 2021-01-01 19:49:51 +01:00
main.tf update copyright notices for 2021 (#8257) 2021-01-01 19:49:51 +01:00
nix_cache.tf update copyright notices for 2021 (#8257) 2021-01-01 19:49:51 +01:00
periodic_killer.tf update copyright notices for 2021 (#8257) 2021-01-01 19:49:51 +01:00
README.md Daml case and logo (#8433) 2021-01-08 12:50:15 +00:00
vsts_agent_linux_startup.sh Include new Nix signing key in static nix config on CI nodes (#8407) 2021-01-06 13:24:34 +01:00
vsts_agent_linux.tf update copyright notices for 2021 (#8257) 2021-01-01 19:49:51 +01:00
vsts_agent_windows.tf update copyright notices for 2021 (#8257) 2021-01-01 19:49:51 +01:00
writer.tf update copyright notices for 2021 (#8257) 2021-01-01 19:49:51 +01:00

Daml

This is the terraform code used by the Daml repository to deploy supporting infrastructure such as the Bazel caches, Nix caches and Azure Pipeline (VSTS) Agents.

Setup

To deploy the infrastructure changes, you will to get access to the da-dev-gcp-daml-language Google project from DA IT. Then run gcloud auth login to configure the local credentials.

Deployment

All the infrastructure is currently deployed using Terraform. For convenience we have a little wrapper script that you can run to apply the latest changes:

$ ./apply

Writer service-account key

To avoid holding the secret key into the store, creating the key has to be done through the UI.

This can be done here: https://console.cloud.google.com/iam-admin/serviceaccounts/details/104272946446260011088?project=da-dev-gcp-daml-language

Setting up credentials

In order to interact with these Terraform files, you will need security to give you access to the relevant GCP project (da-dev-gcp-daml-language), and login via gcloud by running:

gcloud auth application-default login --account your.name@gcloud-domain.com

Resetting build nodes

Permissions to reset build nodes are defined in periodic-killer.tf using the killCiNodes role. CI nodes are managed so killed nodes will be immediately replaced by a new one with the exact same configuration (but starting its initialization from scratch); we can therefore see killing a node and resetting a node as the same operation.

Nodes can be listed with

gcloud compute instances list --project=da-dev-gcp-daml-language

and individual nodes can be killed with

gcloud compute instances --project=da-dev-gcp-daml-language delete --zone=us-east4-a vsts-agent-linux-dhw4

where zone and name have to match.

As a reference, here are a couple zsh functions I have added to my shell to make my life easier:

refresh_machines() {
    machines=$(gcloud compute instances list --format=json --project=da-dev-gcp-daml-language | jq -c '[.[] | select (.name | startswith("vsts-")) | {key: .name, value: .zone | sub (".*/"; "")}] | from_entries')
}

kill_machine() {
    if [ -z "$machines" ]; then
        refresh_machines
    fi
    for machine in $@; do
        gcloud -q compute instances --project=da-dev-gcp-daml-language delete --zone=$(echo $machines | jq -r ".[\"$machine\"]") $machine
    done
}
_kill_machine() {
    local machine_names
    if [ -z "$machines" ]; then
        refresh_machines
    fi
    machine_names=$(echo $machines | jq -r "keys - $(echo -n $words | jq -sRc 'split(" ")') | .[]")
    _arguments "*: :($machine_names)"
}
compdef _kill_machine kill_machine