Intelligent Compute on Google Cloud
Seqera Intelligent Compute is in preview. Seqera must enable it on a per-workspace basis before you can use it. Contact your account manager to request access for one or more workspaces.
Intelligent Compute doesn't support Google Cloud credentials that use Workload Identity Federation yet. Use credentials with a service account key.
On Google Cloud, Intelligent Compute runs on a Google Cloud compute environment. The scheduler launches your pipeline tasks on Compute Engine VMs in your project, as Spot or standard (On-Demand) VMs according to the provisioning model. For how scheduling, retries, and the Spot to On-Demand fallback work, see Intelligent Compute. For the features that aren't available on Google Cloud, see Feature availability by cloud.
Seqera requests Google Cloud's 120-second preemption notice on every Spot VM, followed by Google Cloud's shutdown window of up to 30 seconds, so a reclaimed VM gets up to 150 seconds to write a Fusion snapshot.
Fast instance storage uses local SSDs, available on the n1, n2, n2d, c2, c2d, and g2 machine series.
Created resources
Intelligent Compute creates Compute Engine VMs named sched-vm-<id>, with their boot disks and, with Fast instance storage, local SSDs. Intelligent Compute labels the VMs managed-by=seqera-sched, tags them seqera-sched-vm, and deletes them when the cluster scales down.
Apart from what the Google Cloud compute environment itself creates (a towerforge-* service account with its IAM bindings, and a head VM for each run), Intelligent Compute doesn't create service accounts, firewall rules, instance templates, or IAM bindings in your project.
Required permissions
Intelligent Compute uses two identities in your project:
- The compute environment service account: the service account in your Seqera credentials, used to launch and manage VMs.
- The worker VM service account: the project's default Compute Engine service account,
<PROJECT_NUMBER>-compute@developer.gserviceaccount.com. Task VMs run as this account.
Compute environment service account
Grant the roles in Google Cloud required permissions, plus:
| Role | Required |
|---|---|
Logs Viewer (roles/logging.viewer), to read task logs | Yes |
BigQuery Job User (roles/bigquery.jobUser) on the credentials' project, and BigQuery Data Viewer (roles/bigquery.dataViewer) on the billing export dataset | Only for cost reporting |
Compute Network User (roles/compute.networkUser) on the host project | Only for Shared VPC subnets |
Worker VM service account
Seqera doesn't grant or change roles on this account. In older Google Cloud organizations, Google grants it the Editor role when the project is created, which covers everything Intelligent Compute needs. Organizations created on or after May 3, 2024 enforce the iam.automaticIamGrantsForDefaultServiceAccounts organization policy by default, so the account starts with no roles. In that case, grant it:
| Role | Required |
|---|---|
Storage Object Admin (roles/storage.objectAdmin) on the work directory bucket and every bucket your pipelines write | Yes |
Storage Object Viewer (roles/storage.objectViewer) and Storage Legacy Bucket Reader (roles/storage.legacyBucketReader) on every bucket your pipelines only read | Yes |
Logs Writer (roles/logging.logWriter) | Yes |
Secret Manager Secret Accessor (roles/secretmanager.secretAccessor) | Only for pipeline secrets |
Artifact Registry Reader (roles/artifactregistry.reader) | Only for private Artifact Registry images |
Fusion also needs storage.buckets.get on every bucket it reads. Storage Object Admin doesn't include it, so also grant Storage Legacy Bucket Reader on those buckets if the account doesn't hold Editor on the bucket's project. This applies to buckets in other projects even when the default account keeps Editor.
This service account controls which buckets tasks can reach. To restrict access, grant its storage roles on individual buckets instead of the project.
Create the compute environment
You need the following:
- Seqera has enabled Intelligent Compute for your workspace.
- Google Cloud credentials in Seqera using a service account key. Workload Identity Federation isn't supported yet.
- The required permissions granted to both service accounts.
- A Cloud Storage bucket for the work directory, in the credentials' project.
- In your Seqera workspace, select Compute Environments, then select Add compute environment.
- Enter a name and select Google Cloud as the platform.
- Select your Google Cloud credentials.
- Select the Region where Seqera launches VMs.
- Enter a Work directory (for example,
gs://my-bucket/work). - Under Compute mode, select Seqera Intelligent Compute.
- Under Intelligent Compute options, review the defaults. See Configuration options.
- Optional: Enter a BigQuery billing export table. See Cost reporting.
- Optional: Under Advanced options, configure VPC, Subnets, Network tags, and Use private address. See Networking.
- Select Add.
Seqera creates task VMs when you launch the first pipeline.
Networking
Task VMs need no ingress firewall rules. The scheduler agent on each VM connects out to Seqera over HTTPS, so the network needs outbound access to Seqera, Cloud Storage, Wave, and your container registries.
- No VPC selected: VMs launch in the project's default network, with an ephemeral external IP address.
- VPC with Subnets: the scheduler spreads VMs across all listed subnets and the region's zones, which gives it more places to find Spot capacity. All subnets must belong to the same network. With no subnets listed, it uses the network's first subnet in the region.
- Use private address: VMs launch without an external IP address. The subnets need Cloud NAT and Private Google Access.
- Network tags: applied to task VMs along with
seqera-sched-vm, for your firewall rules to target.
Cost reporting
Seqera shows run cost from your Cloud Billing export to BigQuery:
- Set up Cloud Billing export to BigQuery for your project's billing account.
- Enter the export table in BigQuery billing export table, in the form
project.dataset.table. - Grant the compute environment service account the BigQuery roles.
Seqera sums cost and credits for the run's seqera-io-platform-workflowid label. Cost appears 24 hours or more after the run.