Skip to main content

Seqera Enterprise v23.3_major

Seqera Enterprise

We're excited to announce that Tower is now Seqera Platform. This name change underscores our vision to evolve Seqera as a single platform for the scientific data analysis lifecycle.

The Seqera platform​

While the underlying platform remains the same, over time you can expect Seqera to become even more scalable, flexible and capable. In the coming weeks and months, references to Tower will be replaced across our product documentation and communications.

We're pleased to announce the availability of Seqera Enterprise 23.3, an important first step in delivering on this revamped product vision and roadmap. Seqera 23.3 includes significant new functionality, including a new Data Explorer, enhanced support for Google Cloud Batch and Google Life Sciences, and much more.

New features​

Data Explorer​

Data Explorer is a powerful new feature of the Seqera platform that lets you easily visualize, search for, and manage data across different cloud providers. This enables you to easily link data to pipelines, troubleshoot runs, and examine outputs - all without switching context. Actions such as file preview, download and upload, as well as custom bucket creation and deletion are logged and details can be accessed in the admin panel.

Data Explorer addresses the scientific community's need to streamline data management for pipelines, from arrival in cloud storage, to diving into the different outputs of a pipeline, and passing data to downstream analysis. We started simplifying this process with datasets, a convenient metadata layer to organize versioned, structured data. Data Explorer is the next big step to enable users to manage their data and analyses in one simple workflow.

Data Explorer simplifies data management across multiple cloud object stores, including Amazon S3, Azure Blob Storage, and Google Cloud Storage. With Data Explorer, organizations can:

  • Browse, search for, preview, or upload data to cloud object stores prior to pipeline submission.
  • Navigate workflow and tasks work directories.
  • Link data to pipelines with a single click.
  • Easily view pipeline outputs or dive into task and working directory data.
  • Pagination of buckets listing and content browsing/listing.
  • Access and view audit logs, and download files.

Data Explorer is accessible via the new Data Explorer tab in Seqera Platform. You can also access the interface to upload files or select datasets and destination storage buckets for pipeline runs.

Other feature improvements​

  • Data Explorer: Workspace/global feature toggle
  • Data Explorer: Support uploading files to bucket
  • Data Explorer: Use in launch form path fields
  • Data Explorer: Addition of file select via Data Explorer modal in pipeline launch
  • Data Explorer: Preview text files up to a certain number of lines only

Improvements​

Enhanced Google Cloud support​

Seqera uses secrets to store the keys and tokens used by workflow tasks to interact with external systems, e.g., a password to connect to an external database or an API token. Seqera relies on third-party secret manager services to maintain security between the workflow execution context and the secret container. This means that no secure data is transmitted from Seqera to the compute environment.

In Seqera 23.3, you can now take advantage of secrets in Google Cloud Batch or Google Life Sciences compute environments by using Google Secrets Manager as the underlying user secrets store.

Pipeline resource optimization​

Pipeline resource optimization allows you to minimize the resources used in your pipeline runs based on the resource use of previous runs.

When a run completes successfully, Seqera automatically creates an optimized profile for it. This profile consists of Nextflow configuration settings for each process and each of the following resource directives (where applicable): cpus, memory, and time. The optimized setting for a given process and resource directive is based on the maximum use of that resource across all tasks in that process.

Other improvements​

  • Implement live events endpoint with WebSockets
  • Permission checker for pipeline launch with simple labels
  • Add support for nf-cloudcache
  • Add Fusion support to Azure Batch
  • Add Fusion support for EKS and GKE platform providers
  • Add support for service account, VPC, and subnet for Google Cloud Batch

Breaking changes​

Login redirection logic update

Login redirection logic has changed in version 23.3. Seqera now prepends the TOWER_SERVER_URL (or tower.serverUrl in tower.yml configuration) to the authentication redirect URL during the login flow. This is useful when your server URL contains a contextual path.

If you specify a DNS name as your TOWER_SERVER_URL, but access your Seqera instance using a different address (such as using an IP address that resolves to the server URL asynchronously), user login will not resolve.

Revert default Tower name changes in documentation

A previous iteration of the rebranded Seqera documentation noted seqera as the default and example value for certain variables (such as default database names). The rebranding from Nextflow Tower to Seqera Platform is an ongoing, incremental process and as such, legacy tower values and naming conventions used by the Seqera backend will remain in place until a future release. Updates to configuration variables and values will be communicated well in advance to prepare users for any breaking changes.

Upgrade steps​

This version requires a database schema update. Follow these steps to update your DB instance and the Seqera installation.

To ensure no data loss, the database volume must be persistent on the local machine. Use the volumes key in the db or redis section of your docker-compose.yml file to specify a local path to the DB or Redis instance.

  1. Make a backup of the Seqera Platform database.
  2. Download and update your container versions.
  3. Redeploy the application:

Docker Compose:

  • To migrate the database schema, restart the application with docker compose down, then docker compose up.

Kubernetes:

  • Update the cron service with kubectl apply -f tower-cron.yml. This will automatically migrate the database schema.
  • Update the frontend and backend services with kubectl apply -f tower-srv.yml.

Custom deployment:

  • Run the /migrate-db.sh script provided in the backend container. This will migrate the database schema.
  • Deploy Seqera following your usual procedures.

Nextflow launcher image​

If you must host your nf-launcher container image on a private image registry, copy the nf-launcher image to your private registry. Then update your tower.env with the launch container environment variable:

TOWER_LAUNCH_CONTAINER=<FULL_PATH_TO_YOUR_PRIVATE_IMAGE>

If you're using AWS Batch, you will need to configure a custom job definition and populate the TOWER_LAUNCH_CONTAINER with the job definition name instead.