Skip to main content

Intelligent Compute on Azure

Preview

Seqera Intelligent Compute is in preview. Seqera must enable it on a per-workspace basis before you can use it. Contact your account manager to request access for one or more workspaces.

On Azure, Intelligent Compute runs on an Azure Cloud compute environment. The scheduler launches your pipeline tasks on Azure VMs in your subscription, as Spot or regular (On-Demand) VMs according to the provisioning model. For how scheduling, retries, and the Spot to On-Demand fallback work, see Intelligent Compute. For the features that aren't available on Azure, see Feature availability by cloud.

Azure doesn't support Fusion snapshots: a task interrupted on a Spot VM restarts from the beginning on its new VM.

Created resources​

In addition to the resources the Azure Cloud compute environment creates, Intelligent Compute creates:

  • VMs sched-vm-<id>, with their network interfaces and OS disks. Deleted when the cluster scales down.
  • A user-assigned managed identity seqera-sched-vm-<cluster-id> per cluster, granted Storage Blob Data Contributor on the credentials' storage account, and Key Vault Secrets User on the Key Vault when you set a Key Vault URL. Deleted with the cluster.
  • An empty network security group seqera-sched-vm-<vnet-name> per virtual network.
  • Blob containers sched-logs-* for task logs, in the credentials' storage account. Deleted with the cluster.

Required permissions​

Assign the service principal in your Azure Cloud credentials the following roles at subscription scope:

  1. Azure Cloud role: the seqera-azure-cloud custom role in Azure Cloud required permissions. If you already use an Azure Cloud compute environment, it's already in place.
  2. Storage Blob Data Contributor: the built-in role. Seqera uses it to read and write the work directory, and to issue the short-lived shared access signature (SAS) tokens Fusion uses on the task VMs.
  3. Intelligent Compute role: a custom role with the following additional permissions.
{
    "properties": {
        "roleName": "seqera-intelligent-compute",
        "description": "Additional permissions for Seqera Intelligent Compute on Azure Cloud compute environments",
        "assignableScopes": [
            "/subscriptions/<SUBSCRIPTION-ID>"
        ],
        "permissions": [
            {
                "actions": [
                    "Microsoft.Network/networkSecurityGroups/read",
                    "Microsoft.Network/networkSecurityGroups/write",
                    "Microsoft.Network/networkSecurityGroups/delete",
                    "Microsoft.Storage/storageAccounts/read",
                    "Microsoft.Storage/storageAccounts/blobServices/containers/write",
                    "Microsoft.Compute/skus/read",
                    "Microsoft.Resources/subscriptions/resources/read",
                    "Microsoft.CostManagement/query/action"
                ],
                "notActions": [],
                "dataActions": [],
                "notDataActions": []
            }
        ]
    }
}

You can omit Microsoft.Resources/subscriptions/resources/read if you don't use pipeline secrets, and Microsoft.CostManagement/query/action if you don't need run cost. Instead of Microsoft.CostManagement/query/action, you can assign the built-in Cost Management Reader role. To use pipeline secrets, also grant the service principal Key Vault Secrets Officer on the vault.

To create and assign the role, follow Create a custom role in Microsoft Entra and Assign the custom role to the service principal with the JSON above.

Spot quota​

Azure counts all Spot VMs in a region against one Total Regional Spot vCPUs quota. Request a quota high enough for the tasks you run in parallel. When it's exhausted, Spot first falls back to On-Demand, which uses the per-family quotas.

Create the compute environment​

Prerequisites

You need the following:

  • Seqera has enabled Intelligent Compute for your workspace.
  • Azure Cloud credentials in Seqera that include the Subscription ID, which Intelligent Compute requires.
  • The Azure Cloud and Intelligent Compute roles assigned to the service principal.
  • A Blob Storage container for the work directory, in the credentials' storage account.
  1. In your Seqera workspace, select Compute Environments, then select Add compute environment.
  2. Enter a name and select Azure Cloud as the platform.
  3. Select your Azure Cloud credentials.
  4. Select the Location where Seqera launches VMs.
  5. Enter a Work directory (for example, az://my-container/work).
  6. Under Compute mode, select Seqera Intelligent Compute.
  7. Under Intelligent Compute options, review the defaults. See Configuration options.
  8. Optional: Under Advanced options, select a Virtual network and Subnets, and enter a Key Vault URL to use pipeline secrets.
  9. Select Add.

Seqera creates task VMs and managed identities when you launch the first pipeline.

Networking​

Task VMs have no public IP address and no inbound rules. The scheduler agent on each VM connects out to Seqera over HTTPS.

If you don't select a virtual network, Seqera creates one, as described in Azure Cloud networking. If you select your own, it must be in the same region as the compute environment, and its subnets need outbound access to Seqera, Azure Storage, Microsoft Entra ID, Wave, and your container registries. Seqera doesn't modify your virtual network.

Storage access​

Fusion on the task VMs gets a SAS token for the work directory's container only. Tasks can't read data from other containers or storage accounts directly. Copy input data into the work directory's container, for example under an inputs/ prefix, and point your pipeline at it there.

Cost reporting​

Seqera shows run cost from Azure Cost Management, filtered by the run's seqera.io:platform:workflowId tag, once Azure publishes the data, usually 24–48 hours after the run. OS disk cost is included only with OS disk cost attribution.

On this Page