Tower Enterprise v23.1_major
New features
Launchpad redesign and pipeline enhancements
To enhance pipeline search and navigation capabilities, we now support a new list view to complement the existing card view. The list view allows users to efficiently search for and navigate to their pipeline of choice, while also ensuring that the most relevant information is visible and the relationships between pipelines are clear. With this new feature, users can access their pipelines in either card or list view, making them easier to manage.
We've also introduced a new pipeline detail view that shows in-depth information about each pipeline without needing to access the edit screen.
Enhanced support for Fusion file system
Tower 23.1 introduces support for the Fusion file system in Google Cloud Batch environments. Fusion is a distributed, lightweight file system for cloud-native pipelines that has been shown to improve performance by up to ~2.2x compared to cloud native object storage.
With this new integration, Google Cloud Batch users can enjoy a faster, more efficient, and cheaper processing experience. Fusion offers many benefits, including faster real-time data processing, batch processing, and ETL operations, making it a valuable tool for managing complex data pipelines. By using Fusion with Google Cloud Batch, users can run their data integration workflows directly against data residing in Google Cloud Storage. This integration will allow Google users to streamline their data processing workflows, increase productivity, reduce cloud spending, and achieve better outcomes.
Wave WebSockets support
We have added a new secure way to connect two elements, Tower and Wave, using WebSockets. This is an important addition for our enterprise customers as it ensures connection safety, improved efficiency, and better control over traffic sent between Tower and Wave. This connection will help facilitate the adoption of Fusion by enterprise customers, as it provides a more secure and reliable way to manage their data integration workflows. With WebSockets, users can easily connect their Tower and Wave instances and take advantage of the many benefits that Fusion has to offer.
Other enhancements
- Save executed runs as pipelines
- Improved all runs list view and filtering
- Filter runs by label
- Admin panel enhancements: team and workspace management
- Additional dashboard enhancements:
- Export dashboard data to CSV
- Improved date filtering
- Default resource labels for compute environments per workspace
- Fusion log download
- Upgrade Micronaut to 3.8.5
- Tower Agent connection sharing
- Customizable log format
- AWS Parameter store support (distributed config values)
- Azure Repos credential support
- Fusion v2 EBS disk optimized configuration
Breaking changes and warnings
Breaking changes and instructions listed here apply when updating from Tower version 22.4. If you are updating from an earlier version, see the release notes of previous versions for a complete picture of changes that may affect you.
Updated AWS permissions policies
Several new Tower features over the last few releases require updated AWS IAM permissions policies. Retrieve and apply the latest policy files here.
Wave requires container registry credentials
The Wave containers service uses container registry credentials in Tower to authenticate to your (public or private) container registries. This is separate from your existing cloud provider credentials stored in Tower.
This means that, for example, AWS ECR (Elastic Container Registry) authentication requires an ECR container registry credential if you are running a compute environment with Wave enabled, even if your existing AWS credential in Tower has IAM access to your ECR.
See the relevant container registry credentials page for provider-specific instructions.
Upgrade steps
This Tower version requires a database schema update. Follow these steps to update your DB instance and the Tower installation.
!!! warning ""
To ensure no data loss, the database volume must be persistent on the local machine. Use the volumes
key in the db
or redis
section of your docker-compose.yml
file to specify a local path to the DB or Redis instance.
-
Make a backup of the Tower database.
-
Download and update your container versions.
-
Redeploy the Tower application:
docker compose:
- To migrate the database schema, restart the application with
docker compose down
, thendocker compose up
.
kubernetes:
- Update the cron service with
kubectl apply -f tower-cron.yml
. This will automatically migrate the database schema. - Update the frontend and backend services with
kubectl apply -f tower-srv.yml
.
custom deployment:
- Run the
/migrate-db.sh
script provided in thebackend
container. This will migrate the database schema. - Deploy Tower following your usual procedures.
- To migrate the database schema, restart the application with
Nextflow launcher image
If you must host your nf-launcher container image on a private image registry, copy the nf-launcher image to your private registry. Then update your tower.env
with the launch container environment variable:
TOWER_LAUNCH_CONTAINER=<FULL_PATH_TO_YOUR_PRIVATE_IMAGE>
!!! warning ""
If you're using AWS Batch, you will need to configure a custom job definition and populate the TOWER_LAUNCH_CONTAINER
with the job definition name instead.
Sharing feedback
Share your feedback via support.seqera.io.