Nextflow integration

You can use Wave directly from your Nextflow pipelines, with full support for private repositories, container freezing, and conda packages.

This feature requires Nextflow 22.10.0 or later.

You can use Wave can in any Nextflow pipeline by adding the following snippet to your nextflow.config file:

wave {
    enabled = true
}

tower {
    accessToken = '<your_access_token>'
}

The use of the Seqera access token is not mandatory. However, it grants the following additional capabilities:

Access to private repositories
More API requests than is permitted for anonymous users

For the entire list of configuration options, see Configuration options.

Use Wave with Nextflow

The following sections describe several common usage cases. To get started by creating an example pipeline that uses Wave, see Get started.

Access private container repositories

Wave allows the use of private repositories in your Nextflow pipelines. The repository access keys must be provided in the form of Seqera Platform credentials.

Access private container repositories

After creating the credentials, specify your personal access token in your pipeline configuration file. If the credentials were created in a Seqera Platform organization workspace, specify the workspace ID as well in the config file as shown below:

tower {
  accessToken = '<your access token>'
  workspaceId = '<your workspace id>'
}

Containers built by Wave are uploaded to the Wave default repository hosted on AWS ECR with name 195996028523.dkr.ecr.eu-west-1.amazonaws.com/wave/build. The images in this repository are automatically deleted 1 week from the date of their push.

If you want to store Wave containers in your own container repository use the following settings in the Nextflow configuration file:

wave.build.repository = 'example.com/your/build-repo'
wave.build.cacheRepository = 'example.com/your/cache-repo'

The first repository is used to store the built container images. The second one is used to store the individual image layers for caching purposes.

When launching the pipeline execution, Wave allows Nextflow to access the private container repositories defined in your pipeline configuration, by using the credentials stored in the Seqera Platform credentials manager.

Build Nextflow modules containers

Wave can build and provision container images on-demand for your Nextflow pipelines.

Build Nextflow modules containers

To enable this feature, add the Dockerfile of the container to be built in the module directory where the pipeline process is defined. When Wave is enabled, it automatically uses the Dockerfile to build the required container, upload to the registry, and it uses the container to execute the script defined in the process.

Make sure the process does not declare a container directive, otherwise it will take precedence over the Dockerfile definition.

If a process uses a container directive and you still want to build the container using the Dockerfile provided in the module directory, add the following setting to the pipeline config file:

wave.strategy = ['dockerfile','container']

This setting instructs Wave to prioritize the module Dockerfile over process container directives.

When building containers, Wave currently does not support ADD, COPY, or any other Dockerfile commands that access files in the host file system.

Build Conda-based containers

Wave allows the provisioning of containers based on the process-conda directive used by the processes in your pipeline. This is a quick alternative to building Conda packages in the local computer. Moreover, this enables the use of Conda packages in your pipeline when deploying in cloud-native platforms such as AWS Batch and Kubernetes, which do not allow the convenient use of the Conda package manager.

Build Conda-based containers

With Wave enabled in your pipeline, simply define the conda requirements in the pipeline processes, provided the same process does not also specify a container directive or a Dockerfile.

In the latter case, add the following setting to your pipeline configuration:

wave.strategy = ['conda']

The above setting instructs Wave to use the conda directive to provision the pipeline containers and ignore the container directive and any Dockerfile(s).

For versions of Nextflow 23.10.x or newer, when a container is provisioned, the conda-forge::procps-ng package is included automatically. This package includes the ps command.

Some configuration options in the conda scope are used when Wave is used to build Conda-based containers. For example, the Conda channels and their priority can be set with conda.channels:

wave.strategy = ['conda']
conda.channels = 'seqera,conda-forge,bioconda,defaults'

Build Singularity containers

Nextflow can build Singularity native images on-demand using Singularityfile, Conda packages. The Singularity images are automatically uploaded in a container registry OCI compliant of your choice and stored as a ORAS artifact.

Available as of Nextflow version 23.09.0-edge.

Build Singularity containers

This feature requires a version of Singularity (or Apptainer) that supports pulling images using the oras: pseudo-protocol.

For example to enable the provisioning of Singularity images in your pipeline use the following configuration snippet:

singularity.enabled = true
wave.enabled = true
wave.freeze = true
wave.strategy = ['conda']
wave.build.repository = 'docker.io/user/repo'

In the above configuration replace docker.io/user/repo with a repository of your choice where Singularity image files should be uploaded.

When using a private repository, the repository access keys must be provided via the Seqera Platform credentials manager. For more information, see Authenticate private repositories.

Moreover the access to the repository must be granted in the compute nodes by using the command singularity remote login <registry>. Please see Singularity documentation for further details.

In order to build Singularity native images, both singularity.ociAutoPull and singularity.ociMode must be disabled in the configuration. For more information, see the Nextflow configuration documentation.

Use Wave with Fusion

Wave containers allows you to run your containerized workflow with the Fusion file system.

Use Wave with Fusion

This enables the use of an object storage bucket such as AWS S3 or Google Cloud Storage as your pipeline work directory, simplifying and speeding up many operations on local, AWS Batch, Google Batch or Kubernetes executions.

For more information, refer to the following documentation pages:

Configuration options

The following configuration options are available:

Method	Description
`wave.enabled`	Enable/disable the execution of Wave containers
`wave.endpoint`	The Wave service endpoint (default: `https://wave.seqera.io`)
`wave.build.repository`	The container repository where image built by Wave needs to be uploaded (note: the corresponding credentials need to be provided in your Seqera Platform account).
`wave.build.cacheRepository`	The container repository used to cache image layers build by the Wave service (note: the corresponding credentials need to be provided in your Seqera Platform account).
`wave.conda.mambaImage`	The Mamba container image is used to build Conda based container. This is expected to be micromamba-docker image.
`wave.conda.commands`	One or more commands to be added to the Dockerfile used by build a Conda based image.
`wave.strategy`	The strategy to be used when resolving ambiguous Wave container requirement (default: `'container,dockerfile,conda'`)
`wave.freeze`	When `freeze` mode is enabled containers provisioned by Wave are stored permanently in the repository specified via the setting `wave.build.repository`.

Limitations

Use of sha256 digest in the image name

The Wave does not support the use of sha256 digest in the image name, e.g. ubuntu@sha256:3235...ce8f, when using the augmentation process to extend container images.

In order to reference a container via sha256 digest in the image name with Wave you will need to freeze image mode that will force the creation of a new container image using the container you have specified as base image.

In your pipeline configuration, ensure that you specify the following settings:

wave.enabled = true
wave.freeze = true
wave.strategy = ['dockerfile']
wave.build.repository = 'docker.io/<user>/<repository>'

Use Wave with Nextflow​

Access private container repositories​

Build Nextflow modules containers​

Build Conda-based containers​

Build Singularity containers​

Use Wave with Fusion​

Configuration options​

Limitations​

Use of sha256 digest in the image name​