Worker Labels
Worker labels allow you to tag workers with specific capabilities and route tasks to appropriate workers based on their requirements.
TIP
Worker labels are a key component of Dagu's Distributed Execution feature. Make sure to understand the distributed execution architecture before implementing worker labels.
Configuring Worker Labels
Command Line
Specify labels when starting a worker using the --worker-labels
flag:
bash
# GPU-enabled worker
dagu worker --worker.labels gpu=true,memory=64G,region=us-east-1
# CPU-optimized worker
dagu worker --worker.labels cpu-arch=amd64,cpu-cores=16,instance-type=m5.large
# Region-specific worker
dagu worker --worker.labels region=eu-west-1,compliance=gdpr
Configuration File
Set labels in the configuration file:
yaml
# config.yaml
worker:
labels:
gpu: "true"
memory: "64G"
region: "us-east-1"
Environment Variable
Set labels via environment variable:
bash
export DAGU_WORKER_LABELS="gpu=true,memory=64G"
dagu worker
Using Worker Selectors in DAGs
Specify workerSelector
on any step to route it to workers with matching labels:
yaml
steps:
# This task will only run on workers with gpu=true label
- name: train-model
run: train
---
name: train
workerSelector:
gpu: "true"
steps:
- name: train
command: python train.py
Label Matching Rules
- All labels must match: A worker must have ALL labels specified in the
workerSelector
to be eligible - Empty selector: Tasks without
workerSelector
can run on any worker - Exact match: Label values must match exactly (case-sensitive)
Example Use Cases
GPU/CPU Task Routing
yaml
# GPU worker
dagu worker --worker.labels gpu=true
# CPU worker
dagu worker --worker.labels cpu=true
# DAG
steps:
- name: gpu-task
run: gpu-task
- name: cpu-task
run: cpu-task
---
# Run on a worker with gpu
name: gpu-task
workerSelector:
gpu-task: "true"
steps:
- name: gpu task
command: python gpu-task.py
---
# Run on a worker with faster cpu
name: cpu-task
workerSelector:
cpu-task: "true"
steps:
- name: cpu task
command: python cpu-task.py
See Also
- Distributed Execution - Complete guide to distributed execution
- YAML Reference - workerSelector configuration
- CLI Reference - Worker command options
- Configuration Reference - Worker configuration