Intelligent Compute on Google Cloud
Seqera Intelligent Compute is in preview. Seqera must enable it on a per-workspace basis before you can use it. Contact your account manager to request access for one or more workspaces.
Intelligent Compute doesn't support Google Cloud credentials that use Workload Identity Federation yet. Use credentials with a service account key.
On Google Cloud, Intelligent Compute runs on a Google Cloud compute environment. The scheduler launches your pipeline tasks on Compute Engine VMs in your project, as Spot or standard (On-Demand) VMs according to the provisioning model. For how scheduling, retries, and the Spot to On-Demand fallback work, see Intelligent Compute. For the features that aren't available on Google Cloud, see Feature availability by cloud.
Seqera requests Google Cloud's 120-second preemption notice on every Spot VM, followed by Google Cloud's shutdown window of up to 30 seconds, so a reclaimed VM gets up to 150 seconds to write a Fusion snapshot.
Fast instance storage uses local SSDs, available on the n1, n2, n2d, c2, c2d, and g2 machine series.
Created resources
Intelligent Compute creates Compute Engine VMs named sched-vm-<id>, with their boot disks and, with Fast instance storage, local SSDs. Intelligent Compute labels the VMs managed-by=seqera-sched, tags them seqera-sched-vm, and deletes them when the cluster scales down.
Apart from what the Google Cloud compute environment itself creates (a towerforge-* service account with its IAM bindings, and a head VM for each run), Intelligent Compute doesn't create service accounts, firewall rules, instance templates, or IAM bindings in your project.
Required permissions
Intelligent Compute uses two identities in your project:
- The compute environment service account: the service account in your Seqera credentials, used to launch and manage VMs.
- The worker VM service account: the project's default Compute Engine service account,
<PROJECT_NUMBER>-compute@developer.gserviceaccount.com. Task VMs run as this account.
Compute environment service account
Grant the roles in Google Cloud required permissions, plus:
| Role | Required |
|---|---|
Logs Viewer (roles/logging.viewer), to read task logs | Yes |
BigQuery Job User (roles/bigquery.jobUser) on the credentials' project, and BigQuery Data Viewer (roles/bigquery.dataViewer) on the billing export dataset | Only for cost reporting |
Compute Network User (roles/compute.networkUser) on the host project | Only for Shared VPC subnets |
Worker VM service account
Seqera doesn't grant or change roles on this account. In older Google Cloud organizations, Google grants it the Editor role when the project is created, which covers everything Intelligent Compute needs. Organizations created on or after May 3, 2024 enforce the iam.automaticIamGrantsForDefaultServiceAccounts organization policy by default, so the account starts with no roles. In that case, grant it:
| Role | Required |
|---|---|
Storage Object Admin (roles/storage.objectAdmin) on the work directory bucket and every bucket your pipelines write | Yes |
Storage Object Viewer (roles/storage.objectViewer) and Storage Legacy Bucket Reader (roles/storage.legacyBucketReader) on every bucket your pipelines only read | Yes |
Logs Writer (roles/logging.logWriter) | Yes |
Secret Manager Secret Accessor (roles/secretmanager.secretAccessor) | Only for pipeline secrets |
Artifact Registry Reader (roles/artifactregistry.reader) | Only for private Artifact Registry images |
Fusion also needs storage.buckets.get on every bucket it reads. Storage Object Admin doesn't include it, so also grant Storage Legacy Bucket Reader on those buckets if the account doesn't hold Editor on the bucket's project. This applies to buckets in other projects even when the default account keeps Editor.
This service account controls which buckets tasks can reach. To restrict access, grant its storage roles on individual buckets instead of the project.
Create the compute environment
You need the following:
- Seqera has enabled Intelligent Compute for your workspace.
- Google Cloud credentials in Seqera using a service account key. Workload Identity Federation isn't supported yet.
- The required permissions granted to both service accounts.
- A Cloud Storage bucket for the work directory, in the credentials' project.
- In your Seqera workspace, select Compute Environments, then select Add compute environment.
- Enter a name and select Google Cloud as the platform.
- Select your Google Cloud credentials.
- Select the Region where Seqera launches VMs.
- Enter a Work directory (for example,
gs://my-bucket/work). - Under Compute mode, select Seqera Intelligent Compute.
- Under Intelligent Compute options, review the defaults. See Configuration options.
- Optional: Enter a BigQuery billing export table. See Cost reporting.
- Optional: Under Advanced options, configure VPC, Subnets, Network tags, and Use private address. See Networking.
- Select Add.
Seqera creates task VMs when you launch the first pipeline.
Networking
Task VMs need no ingress firewall rules. The scheduler agent on each VM connects out to Seqera over HTTPS, so the network needs outbound access to Seqera, Cloud Storage, Wave, and your container registries.
- No VPC selected: VMs launch in the project's default network, with an ephemeral external IP address.
- VPC with Subnets: the scheduler spreads VMs across all listed subnets and the region's zones, which gives it more places to find Spot capacity. All subnets must belong to the same network. With no subnets listed, it uses the network's first subnet in the region.
- Use private address: VMs launch without an external IP address. The subnets need Cloud NAT and Private Google Access.
- Network tags: applied to task VMs along with
seqera-sched-vm, for your firewall rules to target.
Cost reporting
Seqera shows each run's cost on the run details page, from your Cloud Billing export to BigQuery, in your billing account's currency, 24 hours or more after the run. Seqera sums the cost and credits of the billing export rows that carry the run's seqera-io-platform-workflowid label, which Intelligent Compute applies to every VM and boot disk it launches for the run.
Labels on task VMs
Intelligent Compute labels each task VM and its boot disk with the following keys. Google Cloud label keys and values allow only lowercase letters, digits, -, and _, up to 63 characters, so Seqera lowercases each key and value and replaces every other character with -. For example, the project name nf-core/rnaseq becomes nf-core-rnaseq.
| Label key | Value |
|---|---|
seqera-io-platform-workflowid | Seqera run ID |
seqera-io-platform-workspaceid | Seqera workspace ID. Nextflow 26.10 or later. |
seqera-io-platform-computeenvid | Seqera compute environment ID. Nextflow 26.10 or later. |
nextflow-io-projectname | Pipeline project name |
nextflow-io-runname | Nextflow run name |
nextflow-io-sessionid | Nextflow session ID |
nextflow-io-username | Seqera user who launched the run. Before Nextflow 26.10, the operating system user that runs Nextflow. |
nextflow-io-revision | Pipeline revision |
nextflow-io-commitid | Pipeline commit ID |
nextflow-io-repository | Pipeline repository URL |
nextflow-io-manifestname | Pipeline manifest name |
nextflow-io-runtimeversion | Nextflow version |
nextflow-io-resume | true if the run was resumed, otherwise false |
Task VMs also carry labels set with a fixed value at the top level of your Nextflow configuration, for example process.resourceLabels = [team: 'genomics']. One VM runs tasks from many processes, so it can't carry a label whose value differs between tasks. Labels set with a closure, such as process.resourceLabels = { [step: task.process] }, apply to tasks only, not to VMs.
Unlike on AWS and Azure, Seqera doesn't apply the run's resource labels to task VMs on Google Cloud: it applies them to the head job VM only.
Set up cost reporting
- Enable Cloud Billing export to BigQuery for your project's billing account, with the Standard usage cost export type.
- Enter the export table in BigQuery billing export table, in the form
project.dataset.table. The standard usage cost export table is namedgcp_billing_export_v1_<BILLING_ACCOUNT_ID>, for examplemy-project.billing.gcp_billing_export_v1_01A2B3_C4D5E6_F7G8H9. - Grant the compute environment service account the BigQuery roles. Without them, Seqera can't read run cost.
What run cost includes
Run cost includes the task VMs and their boot disks. It doesn't include:
- The head job VM, which runs Nextflow. Seqera doesn't currently capture the head job's cost for Intelligent Compute runs.
- Other services, such as Cloud Storage and Cloud Logging.