38 posts tagged with "seqera enterprise"

• No changes in this release.

Seqera Enterprise version 24.2 introduces new Data Studios features, global Nextflow configuration at the compute environment level, Azure service principal and managed identity authentication support, and a number of bug fixes and performance enhancements.

Feature updates and improvements​

Data Studios​

• Data Studios now supports custom environments:
  • Create custom analysis environments, or link to public or private ECR containers.
  • Audit log now includes custom environment build events. The events data_studio_session_build_started and data_studio_session_build_failed appear in the audit log events table.
• Data Studios dashboard:
  • A new dashboard provides information about Data Studios usage to help you manage your resources.
• In a GPU instance type compute environment, both CPU and GPU resources are supported and can be added to your Data Studio sessions.
• Added NVMe support.
• EFS volume mounting now supported.
• Browse a studio session's mounted data directly from the studio details page using Data Explorer.

Nextflow configuration updates​

• Added a Global Nextflow config field to all compute environments. This field allows you to define Nextflow configuration values in the compute environment that are pre-filled in the pipeline Nextflow config file field during launch.
• Implemented custom launch container logic in the workflow/launch API endpoint. This allows you to specify a custom container using the launchContainer key in your request body when submitting a workflow execution. For example:

  {
    "launch": {
        "id": "T3xxxxxxxxxxig",
        "launchContainer": "quay.io/seqeralabs/nf-launcher:j17-23.10.1-up1",
        "computeEnvId": "6Uxxxxxxxxxxxhl",
        "workDir": "s3://your-bucket",
        "pipeline": "https://github.com/nextflow-io/hello",
        ...
    }
  }
  

Azure Batch: Entra service principal and managed identity authentication support​

• Manual Azure Batch compute environments now support authentication using Microsoft Entra service principals. This provides a secure alternative to long-lived access keys, as service principals enable role-based access control with more precise permissions.
• You can now also use a managed identity in conjunction with your credentials (access keys or service principals) to authenticate Nextflow to Azure services. The combination of an Entra service principal and user-assigned managed identity offers enhanced security over access keys. Note that both authentication methods are only supported in manual Azure Batch compute environments.

Launchpad​

• The new Launch Form is enabled by default. See Core feature configuration for options to disable the launch form per workspace or globally.

Compute environments​

• AWS Batch Forge compute environments now support Amazon Linux 2023. Previously, Batch Forge only created compute environments with Amazon Linux2. Seqera now supports specifying Amazon Linux 2023 ECS-optimized AMIs when you create AWS Batch compute environments. AWS-recommended Amazon Linux 2023 AMI names start with al2023-.
• The Google Life Sciences API will be deprecated in June 2025. Seqera Platform UI now displays a Google Life Sciences API deprecation notice.
• AWS Batch launch templates created by Batch Forge now include tag propagation, which propagates resource labels to storage volumes (as well as instances which are the default).

Credentials​

• GitLab credentials no longer require the Password field, as public repositories only require an access token. Note that private repositories will require a credential with both a password and access token.

Settings​

• Added flexibility for pipeline names in workspaces. Pipeline names must be unique at the workspace level for private workspaces, while shared workspaces require names to be unique at the organization level.

General​

Workflows​

• Refactored now reports file streaming over Agent connections directly to the browser.

Launchpad​

• Input forms now support JSON schema draft 2020-12 features. Includes: Nested parameters, "anyOf", "allOf", and "oneOf" input form validation support, "dependantSchemas" initial support, and "if-then-else" support. This improves support of JSON schema and prevents form errors for unrecognized validations.

Compute environments​

• Added ECS agent configuration variables to set the timeout for several container actions:
  • ECS_CONTAINER_CREATE_TIMEOUT=10m: Timeout for creating a container.
  • ECS_CONTAINER_STOP_TIMEOUT=10m: Timeout for stopping a container. Increase to prevent forcibly killing Nextflow and leave running jobs.
  • ECS_MANIFEST_PULL_TIMEOUT=10m: Timeout for pulling a container manifest. This may be needed when pulling a Wave container with a long build time.
• Disabled AWS and Google Cloud Batch spot auto-retry: This disables the automatic retry of Batch jobs submitted by Nextflow in favour of using the classic Nextflow retry mechanism. This is because the Batch automatic retry is completely invisble to Seqera and the user. Nextflow retry gives better feedback and allows the user to have more control.
  note

  This requires a change to pipeline configurations that previously used Spot auto-retry for AWS and Google Cloud Batch.

Infrastructure​

• Bumped nf-launcher:j17-24.10.2.
• Upgraded to Angular 16.
• Security fixes: All security fixes for previous versions.

Bug fixes​

Studios​

• Display initial mounted data when starting an existing Data Studio session.
• Updated lastUsed on compute environment when creating a job for a Data Studios session.

Launchpad​

• Fixed input form view for private GitHub repositories using pipeline Quicklaunch.
• Fixed an issue where workspace secrets were not selected by default when using pipeline Quicklaunch.
• Fixed incorrect role access in new launch form which allowed users with "Maintain" role and lower to edit resource labels when launching.
• Send fetch schema request in quicklaunchmode, even if revision is empty.
• Allow changing the working directory when launching a pipeline.
• Fixed NF schema nested object values lost when switching to input form view. Removed processing of unsupported JSON schema types inside pipeline input form components.

Pipelines​

• Shared pipelines without an associated compute environment now default to using the workspace primary compute environment.
• Avoid using master as the fallback pipeline revision, so that pipeline repositories without a master version don't return errors on Quicklaunch.

Workflows​

• Fixed issue where searching for task tags using underscores did not return the expected values.
• Fixed report access issue for Nextflow CLI runs with TOWER_CONTENT_URL set.

Compute environments​

• Allow resume with compatible compute environments when using Tower Agent and $TW_AGENT_WORKDIR as working directory.
• Added managedIdentityId to the API response payload when describing a compute environment.

Data Explorer​

• Enabled task working directory navigation using Data Explorer in personal workspaces.
• Encode file names when requesting a download URL in Data Explorer.
• Show CORS modal when detecting an incorrect eTag configuration for Data Explorer, and update link to Seqera docs.

General​

• Check if a workspace is disabled before fetching data links.
• Fixed Entra and javax.mail transitive dependency problem.

Infrastructure​

• Improved performance of queries reported as intensive on the RDS database. Added an index to the tw_workflow(complete) column, which improves the performance of the query that finds workflows with missing progress. Added an index to the tw_workflow(status) column conditionally; the index already exists in the cloud.seqera.io database, but did not previously appear in the database migration scripts.

Breaking changes and warnings​

Seqera AWS ECR repository customer access ends June 1, 2025​

Customers will no longer be able to pull Seqera Enterprise container images from the legacy Seqera AWS ECR repository after June 1, 2025. All Seqera Enterprise images must be retrieved via the cr.seqera.io container registry after this cutoff date. The installation and configuration templates provided for both Docker Compose and Kubernetes installations already reference the cr.seqera.io container image URLs. If you have not yet transitioned to this registry, contact Support to request credentials and for any further assistance.

See Legacy Seqera container image registries for more information on the AWS ECR and other deprecated Seqera container registries.

Redis version change​

From this version of Seqera Platform:

• Redis version 6.2 or greater is required.
• Redis version 7 is officially supported.

Redisson properties deprecated​

From this version of Seqera Platform, redisson.* configuration properties are deprecated. If you have set redisson.* properties directly previously, do the following:

• Replaced /redisson/* references in AWS Parameter Store entries with TOWER_REDIS_*.
• Replaced redisson.* references in tower.yml with TOWER_REDIS_*.

MariaDB driver: New MySQL connection parameter required​

MariaDB driver 3.x requires a special parameter in the connection URL to connect to a MySQL database:

jdbc:mysql://<domain>:<port>/tower?permitMysqlScheme=true

All deployments using a MySQL database (regardless of version: 5.6, 5.7, or 8) should be updated accordingly when upgrading to Platform version 24.1 or later.

MariaDB driver: No truncation support for MySQL 5.6​

The MariaDB driver has dropped support for the jdbcCompliantTruncation parameter, which was true by default and set the STRICT_TRANS_TABLES SQL mode. The STRICT_TRANS_TABLES mode produces an error when the value of a VARCHAR column exceeds its limit, instead of truncating it to fit. Most common installations of MySQL 5.7 and 8 already include this mode at the server level, but the Docker container version of MySQL 5.6 does not.

The SQL mode must be set explicitly through the connection URL for deployments still using MySQL 5.6:

jdbc:mysql://<domain>:<port>/tower?permitMysqlScheme=true&sessionVariables=sql_mode='STRICT_TRANS_TABLES,NO_ENGINE_SUBSTITUTION'

Micronaut property key changes​

The property that determines the expiration time of the JWT access token (used for authenticating web sessions and Nextflow-Platform interactions) has changed since version 24.1:

Enterprise deployments that have customized this value previously will need to adopt the new format.

Upgrade steps​

1. This version includes an update to the Platform Enterprise H8 cache. Do not start the upgrade while any pipelines are in a running state as active run data may be lost.
2. This version requires a database schema update. Make a backup of your Platform database prior to upgrade.
3. If you are upgrading from a version older than 23.4.1, update your installation to version 23.4.4 first, before updating to 24.2 with the steps below.
4. For recommended Platform memory settings, add the following environment variable to your Platform configuration values (tower.env, configmap.yml, etc.):

   JAVA_OPTS: -Xms1000M -Xmx2000M -XX:MaxDirectMemorySize=800m -Dio.netty.maxDirectMemory=0 -Djdk.nio.maxCachedBufferSize=262144
   

5. See Upgrade installation for installation upgrade guidance.

Bug fixes​

Workflows​

• Search task names in run details.

Bug fixes​

General​

• Mailer dependency issue causing email send failures.

Security updates​

Infrastructure​

• Fixed critical frontend vulnerabilities.

Feature updates and improvements​

Compute environments​

• Added tag propagation for storage volumes to AWS launch template.

Infrastructure​

• Implemented custom H8 2nd-level cache to replace Redisson cache.
• Extended determineTextFormat helper function.

Bug fixes​

Launchpad​

• Disabled launch form submit button during form validation.
• Allow workDir edit during pipeline launch.

Data Explorer​

• Data Explorer preview for Azure Blob Storage files.

Compute environments​

• GCP resource leak.

Pipelines​

• Remote config read issue.

Infrastructure​

• Dependency vulnerabilities.

Version bumps​

Infrastructure​

• Bumped nf-launcher:j17-24.04.4

Bug fixes​

Infrastructure​

• Redis TLS connection issue.

Security updates​

Infrastructure​

• Security patches.

Feature updates and improvements​

General​

• The standard email login can be disabled via tower.yml or an environment variable, provided an alternative OIDC provider is set up first.

Seqera Platform Enterprise version 24.1 introduces three new features: Data Studios (in public preview), Data Explorer, and managed identities. A number of bug fixes and performance enhancements are included in this major release.

Feature updates and improvements​

Data Studios​

Data Studios closes the loop from development to deployment and insights, allowing you to create, manage, and share notebook environments in Seqera with the click of a button. Data Studios makes it seamless to work across teams with multi-user support, built-in authentication, and automatic snapshots as you work.

• Host a combination of container images and compute environments for interactive analysis using your preferred tools, like Jupyter notebooks, RStudio, and Visual Studio Code IDEs.
• Checkpoints provide a stable point-in-time snapshot.
• Collaborate and share data in real time.

Data Studios is available as a public preview. Contact support to request access to this feature.

Data Explorer​

Data Explorer allows you to browse and interact with remote data repositories from organization workspaces in Seqera Platform. It supports AWS S3, Azure Blob Storage, and Google Cloud Storage repositories.

Data Explorer is now Generally Available (GA) and supports multi-file and multi-folder download. This allows users to download entire datasets of pipeline results to share with their team.

Extended data roles​

User roles provide flexibility for admins to provide users with the permissions they need, without compromising security. The Connect user role has been added to the existing user roles with functionality related to Data Studios:

• Connect: Open a connection to a running data studio session and interact with the contents.
• Maintain: Create, update, start, stop, configure, and delete data studio sessions.
• View: List, search, and view the status, configuration, and details of data studios.
• Launch: Open a connection to a running data studio session and interact with the contents.

See User roles for more information.

Managed identities​

Managed identities offer significant advantages for High-Performance Computing (HPC) environments by enabling granular access control for individual workspace users. Unlike traditional SSH credentials that grant all workspace users access to HPC clusters with the same service account, managed identities ensure each user’s activity is logged under their own credentials.

See Managed identities for more information.

Other feature improvements​

• A redesigned pipeline launch form provides an improved workflow and an interface to create and edit run parameters as raw JSON/YAML. Enable the new launch form per workspace via environment variable.
• Retrieve reports from primary compute environment on Nextflow CLI runs. See Configure reports for Nextflow CLI runs for more information.
• The standard email login can be disabled via tower.yml or an environment variable, provided an alternative OIDC provider is set up first. See Basic configuration for more information.

Breaking changes​

MariaDB driver: New MySQL connection parameter required

MariaDB driver 3.x requires a special parameter in the connection URL to connect to a MySQL database:

jdbc:mysql://<domain>:<port>/tower?permitMysqlScheme=true

All deployments using a MySQL database (regardless of version: 5.6, 5.7, or 8) should be updated accordingly when upgrading to Platform version 24.1 or later.

MariaDB driver: No truncation support for MySQL 5.6

The MariaDB driver has dropped support for the jdbcCompliantTruncation parameter, which was true by default and set the STRICT_TRANS_TABLES SQL mode. The STRICT_TRANS_TABLES mode produces an error when the value of a VARCHAR column exceeds its limit, instead of truncating it to fit. Most common installations of MySQL 5.7 and 8 already include this mode at the server level, but the Docker container version of MySQL 5.6 does not.

The SQL mode must be set explicitly through the connection URL for deployments still using MySQL 5.6:

jdbc:mysql://<domain>:<port>/tower?permitMysqlScheme=true&sessionVariables=sql_mode='STRICT_TRANS_TABLES,NO_ENGINE_SUBSTITUTION'

Micronaut property key changes

The property that determines the expiration time of the JWT access token (used for authenticating web sessions and Nextflow-Platform interactions) has changed:

Enterprise deployments that have customized this value previously will need to adopt the new format.

Upgrade steps​

1. This version includes an update to the Platform Enterprise H8 cache. Do not start the upgrade while any pipelines are in a running state as active run data may be lost.
2. This version requires a database schema update. Make a backup of your Platform database prior to upgrade.
3. If you are upgrading from a version older than 23.4.1, update your installation to version 23.4.4 first, before updating to 24.1 with the steps below.
4. For recommended Platform memory settings, add the following environment variable to your Platform configuration values (tower.env, configmap.yml, etc.):

   JAVA_OPTS: -Xms1000M -Xmx2000M -XX:MaxDirectMemorySize=800m -Dio.netty.maxDirectMemory=0 -Djdk.nio.maxCachedBufferSize=262144
   

5. See Upgrade installation for database backup and installation upgrade guidance.

Feature updates and improvements​

Data Studios​

• Added the ability to rename data studio checkpoints.

Data Explorer​

• Added OpenAPI support for Data Explorer.
• Show loading in Workflow/Task Data Explorer tab while waiting for data link cache refresh.

General​

• Datasets can now be used in personal workspaces.
• Pipeline secrets can now be used in personal workspaces.

Compute environments​

• Added Graviton3 EC2 instance family as valid NVMe instance types.
• Added new G6 EC2 instance family as valid NVMe instance types.
• Deprecated EBS autoscale in the Seqera Platform user interface.

Workflows​

• Retrieve reports from primary compute environment on NF CLI runs.
• At nf-launcher, propagate a curl failures and exit.

Security updates​

Infrastructure​

• Upgraded base Docker images to get upstream updates.

Version bump​

Infrastructure​

• Bumped nf-launcher:j17-23.10.1-up1