.github/workflows | ||
bin | ||
examples | ||
fixtures | ||
imgs | ||
src | ||
.editorconfig | ||
.gitignore | ||
default.nix | ||
flake.lock | ||
flake.nix | ||
go.mod | ||
go.sum | ||
LICENSE | ||
Makefile | ||
process-compose-win.yaml | ||
process-compose.yaml | ||
README.md | ||
test_loop.bash | ||
test_loop.ps1 |
Process Compose
Process Compose is a simple and flexible scheduler and orchestrator to manage non-containerized applications.
Why? Because sometimes you just don't want to deal with docker files, volume definitions, networks and docker registries.
Main use cases would be:
- Processes execution (in parallel or serially)
- Defining processes dependencies and startup order
- Defining recovery policies (restart
on-failure
,always
,no
). Manual recovery is also supported. - Declaring processes arguments
- Declaring processes environment variables
It is heavily inspired by docker-compose, but without the need for containers. The configuration syntax tries to follow the docker-compose specifications, with a few minor additions and lots of subtractions.
![TUI](/F1bonacc1/process-compose/media/commit/639b3fbe305b347d57bd467a9a58aa386c1f7404/imgs/tui.png)
Quick Start
Imaginary system diagram:
process-compose.yaml
definitions for the system above:
version: "0.5"
environment:
- "GLOBAL_ENV_VAR=1"
log_location: /path/to/combined/output/logfile.log
log_level: debug
processes:
Manager:
command: "/path/to/manager"
availability:
restart: "always"
depends_on:
ClientA:
condition: process_started
ClientB:
condition: process_started
ClientA:
command: "/path/to/ClientA"
availability:
restart: "always"
depends_on:
Server_1A:
condition: process_started
Server_2A:
condition: process_started
environment:
- "LOCAL_ENV_VAR=1"
ClientB:
command: "/path/to/ClientB -some -arg"
availability:
restart: "always"
depends_on:
Server_1B:
condition: process_started
Server_2B:
condition: process_started
environment:
- "LOCAL_ENV_VAR=2"
Server_1A:
command: "/path/to/Server_1A"
availability:
restart: "always"
Server_2A:
command: "/path/to/Server_2A"
availability:
restart: "always"
Server_1B:
command: "/path/to/Server_1B"
availability:
restart: "always"
Server_2B:
command: "/path/to/Server_2B"
availability:
restart: "always"
Finally, run process-compose
in the process-compose.yaml
directory. Or give it a direct path:
process-compose -f /path/to/process-compose.yaml
Installation
- Go to the releases, download the package for your OS, and copy the binary to somewhere on your PATH.
- If you have the Nix package manager installed with Flake support, just run:
nix run github:F1bonacc1/process-compose
Documentation
- See examples of workflows for best practices
- See below
List of Features and Planned Features
✅ Mostly implemented
❌ Implementation not started (Your feedback and ⭐ will motivate further development 😃)
✅ Launcher
✅ Parallel
process1:
command: "sleep 3"
process2:
command: "sleep 3"
✅ Serial
process1:
command: "sleep 3"
depends_on:
process2:
condition: process_completed_successfully # or "process_completed" if you don't care about errors
process2:
command: "sleep 3"
depends_on:
process3:
condition: process_completed_successfully # or "process_completed" if you don't care about errors
❌ Instance Number
✅ Define process dependencies
process2:
depends_on:
process3:
condition: process_completed_successfully # or "process_started" (default)
process3:
condition: process_completed_successfully
✅ Termination Parameters
process1:
command: "pg_ctl start"
shutdown:
command: "pg_ctl stop"
timeout_seconds: 10 # default 10
signal: 15 # default 15, but only if command is not defined or empty
shutdown
is optional and can be omitted. The default behaviour in this case: SIGTERM
is issued to the running process.
In case only shutdown.signal
is defined [1..31]
the running process will be terminated with its value.
In case the the shutdown.command
is defined:
- The
shutdown.command
is executed with all the Environment Variables of the main process - Wait
shutdown.timeout_seconds
for its completion (if not defined wait for 10 seconds) - In case of timeout the process will receive the
SIGKILL
signal
✅ Output Handling
✅ Show process name
✅ Different colors per process
✅ StdErr is printed in Red
![output](/F1bonacc1/process-compose/media/commit/639b3fbe305b347d57bd467a9a58aa386c1f7404/imgs/output.png)
❌ Silence specific processes
✅ TUI (Terminal User Interface)
✅ Review processes status
✅ Start processes (only completed or disabled)
✅ Stop processes
✅ Review logs
TUI is the default run mode, but it's possible to disable it:
./process-compose -t=false
Control the UI log buffer size:
log_level: info
log_length: 1200 #default: 1000
processes:
process2:
command: "ls -R /"
Note: Using a too large buffer will put a significant penalty on your CPU.
✅ Logger
✅ Per Process Log Collection
process2:
log_location: ./pc.process2.log #if undefined or empty no logs will be saved
✅ Capture StdOut output
✅ Capture StdErr output
✅ Merge into a single file
processes:
process2:
command: "chmod 666 /path/to/file"
environment:
- "ABC=42"
log_location: ./pc.global.log #if undefined or empty no logs will be saved (if also not defined per process)
❌ Silence specific processes
✅ Process compose console log level
log_level: info # other options: "trace", "debug", "info", "warn", "error", "fatal", "panic"
processes:
process2:
command: "chmod 666 /path/to/file"
This setting controls the process-compose
log level. The processes log level should be defined inside the process. It is recommended to support its definition with an environment variable that can be defined in process-compose.yaml
✅ Health Checks
Many applications running for long periods of time eventually transition to broken states, and cannot recover except by being restarted. Process Compose provides liveness and readiness probes to detect and remedy such situations.
Probes configuration and functionality are designed to work similarly to Kubernetes liveness and readiness probes.
✅ Liveness Probe
nginx:
command: "docker run -d --rm -p80:80 --name nginx_test nginx"
is_daemon: true
shutdown:
command: "docker stop nginx_test"
signal: 15
timeout_seconds: 5
liveness_probe:
exec:
command: "[ $(docker inspect -f '{{.State.Running}}' nginx_test) = 'true' ]"
initial_delay_seconds: 5
period_seconds: 2
timeout_seconds: 5
success_threshold: 1
failure_threshold: 3
✅ Readiness Probe
nginx:
command: "docker run -d --rm -p80:80 --name nginx_test nginx"
is_daemon: true
shutdown:
command: "docker stop nginx_test"
readiness_probe:
http_get:
host: 127.0.0.1
scheme: http
path: "/"
port: 80
initial_delay_seconds: 5
period_seconds: 10
timeout_seconds: 5
success_threshold: 1
failure_threshold: 3
Each probe type (liveness_probe
or readiness_probe
) can be configured in to use one of the 2 mutually exclusive modes:
exec
: Will run a configuredcommand
and based on theexit code
decide if the process is in a correct state. 0 indicates success. Any other value indicates failure.http_get
: For an HTTP probe, the Process Compose sends an HTTP request to the specified path and port to perform the check. Response code 200 indicates success. Any other value indicates failure.host
: Host name to connect to.scheme
: Scheme to use for connecting to the host (HTTP or HTTPS). Defaults to HTTP.path
: Path to access on the HTTP server. Defaults to /.port
: Number of the port to access on the process. Number must be in the range 1 to 65535.
Configure Probes
Probes have a number of fields that you can use to more precisely control the behavior of liveness and readiness checks:
initial_delay_seconds
: Number of seconds after the container has started before liveness or readiness probes are initiated. Defaults to 0 seconds. Minimum value is 0.period_seconds
: How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1.timeout_seconds
: Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1.success_threshold
: Minimum consecutive successes for the probe to be considered successful after having failed. Defaults to 1. Must be 1 for liveness and startup Probes. Minimum value is 1. Note: this value is not respected and was added as a placeholder for future implementation.failure_threshold
: When a probe fails, Process Compose will tryfailure_threshold
times before giving up. Giving up in case of liveness probe means restarting the process. In case of readiness probe the Pod will be marked Unready. Defaults to 3. Minimum value is 1.
✅ Auto Restart if not Healthy
In order to insure that the process is restarted (and not transitioned to completed state) in case of readiness check fail, please make sure to define the availability
configuration. For background (is_daemon=true
) processes, the restart
policy should be always
.
✅ Auto Restart on Exit
process2:
availability:
restart: on-failure # other options: "always", "no" (default)
backoff_seconds: 2 # default: 1
max_restarts: 5 # default: 0 (unlimited)
✅ Environment Variables
✅ Per Process
process2:
environment:
- "I_AM_LOCAL_EV=42"
✅ Global
processes:
process2:
command: "chmod 666 /path/to/file"
environment:
- "I_AM_LOCAL_EV=42"
environment:
- "I_AM_GLOBAL_EV=42"
Default environment variables:
PC_PROC_NAME
- Defines the process name as defined in the process-compose.yaml
file.
PC_REPLICA_NUM
- Defines the process replica number. Useful for port collision avoidance for processes with multiple replicas.
✅ REST API
A convenient Swagger API is provided: http://localhost:8080/swagger/index.html
![Swagger](/F1bonacc1/process-compose/media/commit/639b3fbe305b347d57bd467a9a58aa386c1f7404/imgs/swagger.png)
✅ Configuration
✅ Support .env file
✅ Override ${var} and $var from environment variables or .env values
❌ Merge 2 or more configuration files with override values
✅ Specify which configuration files to use
process-compose -f "path/to/process-compose-file.yaml"
✅ Auto discover configuration files
The following discovery order is used: compose.yml, compose.yaml, process-compose.yml, process-compose.yaml
. If multiple files are present the first one will be used.
✅ Multi-platform
✅ Linux
The default backend is bash
. You can define a different backend with a SHELL
environment variable.
✅ Windows
The default backend is cmd
. You can define a different backend with a SHELL
environment variable.
process1:
command: "python -c print(str(40+2))"
#note that the same command for bash/zsh would look like: "python -c 'print(str(40+2))'"
Using powershell
backend had some funky behaviour (like missing command1 && command2
functionality in older versions). If you need to run powershell scripts, use the following syntax:
process2:
command: "powershell.exe ./test.ps1 arg1 arg2 argN"