Release notes for version 22.1.x

Nextflow Tower 22.1.x brings the following changes:

• Added: Pipeline Secrets feature
• Added: Custom workflows run name
• Added: CSV and TSV reports rendering
• Added: Support for AWS agent and logging
• Added: Support for Moab platform
• Added: GetLogsEvents perm to AWS Batch instance role
• Added: Support for local pipelines
• Added: Navigate back button to second level screens
• Added: Validation for SSH hostname and username
• Added: Credentials view page
• Added: ECS pull strategy in user-data template
• Added: Root users environment when TOWER_ROOT_USERS variable is provided
• Added: Tower system message
• Added: Config option to disable user private workspace
• Added: Support for JSON formatted logs
• Stability improvements
• Security improvements as for penetration test performed on March 2022

New features and usability improvements

We're pleased to announce the 22.1 release of Tower. This release brings long-awaited features including a customizable run name for workflows, enhanced secrets handling, better reports, and improved HPC scheduler support.

!!! tip "" The release naming convention is changed and it reflects the year and the quarter, so our customers know to expect 4 enterprise releases per year.

Customizable workflow run name

From this release, Tower allows users to specify a custom name for each workflow run. This name will override the default runName parameter for the Nextflow run name. This feature allows to identify each run at a glance, facilitating the retrieval of a specific run from the list presented in the Runs page from the application.

More than 60% of new workflow runs in tower.nf already make use of the custom run name.

Pipeline secrets - improved

Starting from 21.12, Tower uses the concept of secrets to store the keys and tokens used by workflow tasks to interact with external systems e.g., a password to connect to an external database or an API token. In version 22.1 we have improved the secrets management by:

• Introducing the possibility for users to create user-level secrets.
• Implementing a selection mechanism so users can dynamically select which secrets will be used by each pipeline launch.
• Enable Tower to capture specific OIDC attributes as secrets.

Full information about secrets is available here

Improved web reports

Most Nextflow pipelines will generate reports or output files which are useful to inspect at the end of the pipeline execution. Since 21.12, Tower has a Reports feature that allows to directly visualise supported file types or to download them directly via the user interface.

The key improvements in 22.1

• Reports are configurable and are no longer limited to workdir subdirectories.
• Tower enables the rendering of CSV and TSV files.

Comprehensive documentation on setup and constraints is available here.

Enhanced HPC support

Tower 22.1 allows working with HPC MOAB platform, expanding the supported alternatives for HPC computing.

We've also implemented a first version of a much requested feature from HPC users: Launch local repositories. Using this feature, users can instruct Tower to source the pipeline code stored as a Git bare repository and launch the workflows.

Notes

• As of version 22.1.x, Nextflow Tower Enterprise will follow a three month release cadence, using the following version number scheme: YY.Q.PATCH, where YY represents the year, Q represents the quarter and PATCH the incremental patch number.
• As of version 21.02.x, a license key must be provided to enable the Tower deployment feature. The license key should be specified using the configuration variable TOWER_LICENSE. If you don't have a license key, contact sales@seqera.io.

Warnings

• This version now expects the use of HTTPS by default for all browser client connections. If your Tower installation requires the use of unsecured HTTP, set the following environment variable in the infrastructure hosting the Tower application: TOWER_ENABLE_UNSAFE_MODE=true.
• If you're upgrading from a version of Tower prior to 21.04.x, please update your implementation to 21.04.x before installing this release.

Database Schema

This Tower version requires a database schema update. Follow these steps to update your DB instance and the Tower installation.

1. Make a backup of the Tower database.

2. Download and update your container versions to:

   • 195996028523.dkr.ecr.eu-west-1.amazonaws.com/nf-tower-enterprise/backend:v22.1.5
   • 195996028523.dkr.ecr.eu-west-1.amazonaws.com/nf-tower-enterprise/frontend:v22.1.5

3. Redeploy the Tower application:

   docker-compose:

   • Restart the application with docker-compose restart. This will automatically migrate the database schema.

   kubernetes:

   • Update the cron service with kubectl apply -f tower-cron.yml. This will automatically migrate the database schema.
   • Update the frontend and backend services with kubectl apply -f tower-srv.yml.

   custom deployment

   • Run the /migrate-db.sh script provided in the backend container. This will migrate the database schema.
   • Deploy Tower following your usual procedures.

Nextflow launcher image

If you must host your container images on a private image registry, copy the nf-launcher image to your private registry. Then update your tower.env with the following environment variable:

TOWER_LAUNCH_CONTAINER=<FULL_PATH_TO_YOUR_PRIVATE_IMAGE>

!!! warning If you're using AWS Batch, you will need to configure a custom job-definition and populate the TOWER_LAUNCH_CONTAINER with the job-definition name instead.

Compute environments

The Tower compute environments to be made available to users must be specified in the Tower configuration.

The following ids options are available:

• awsbatch-platform: AWS Batch cloud compute service
• gls-platform: Google LifeSciences cloud compute service
• azbatch-platform: Azure Batch cloud compute service
• lsf-platform: IBM LSF batch scheduler
• slurm-platform: Slurm batch scheduler
• altair-platform: Altair PBS Pro batch scheduler
• univa-platform: (Univa/Sun) GridEngine
• moab-platform: Adaptive Computing Moab batch scheduler
• k8s-platform: Kubernetes compute platform
• eks-platform: AWS EKS compute platform
• gke-platform: Google GKE compute platform

Choose one or more of these platform ids specify them via the environment variable TOWER_ENABLE_PLATFORMS, separating them via a comma.

Changelog

For a detailed list of all changes, please consult the Nextflow Tower Changelog.

Questions/Feedback

Contact us at support@seqera.io.