Skip to main content

Local

The local executor submits pipeline tasks as operating system processes on the computer where Nextflow is launched. It is the default executor. Use it to develop and test a pipeline on your computer before scaling up to production workloads on a cluster or in the cloud.

The local executor supports two types of tasks:

  • Script tasks (processes with a script block) -- executed via a Bash wrapper script.
  • Native tasks (processes with an exec block) -- executed directly in the JVM.

Use the following process directives to control resource requests and other job characteristics:

Nextflow runs tasks in parallel based on the available resources (CPUs and memory). For example, on a machine with 8 CPUs and 16 GB of memory, a pipeline submitting tasks with 2 CPUs and 4 GB of memory can run up to 3 tasks in parallel, because Nextflow reserves one CPU for itself. If a task requests more than 7 CPUs or 16 GB, the run fails.

note

Because the operating system does not enforce resource limits, a local task can use more CPUs and memory than it requested and starve other tasks. To avoid this problem, use a container runtime such as Docker or Podman that can enforce resource limits.

Accelerators​

Added in version 26.10

The local executor allocates accelerators, such as GPUs, to tasks that request them with the accelerator directive. Set the environment variable for your accelerator type:

  • CUDA_VISIBLE_DEVICES for NVIDIA CUDA applications

  • HIP_VISIBLE_DEVICES for HIP applications

  • ROCR_VISIBLE_DEVICES for AMD ROCm applications

Set the variable to a comma-separated list of device IDs that Nextflow can use. The list must not be empty. If no devices are listed, no accelerators are available and any task that requests one fails.

For example, to use all GPUs on a node with four NVIDIA GPUs, set CUDA_VISIBLE_DEVICES to 0,1,2,3. If four tasks each request one GPU, they run with CUDA_VISIBLE_DEVICES set to 0, 1, 2, and 3, respectively.

note

To run tasks in containers, propagate the device environment variable into the container and configure the container runtime for GPU access. For example, with Docker, add CUDA_VISIBLE_DEVICES to docker.envWhitelist and set GPU options such as --gpus via docker.runOptions. See your container runtime's documentation for GPU support.

On this Page