Skip to main content

Fusion v2.6.4

Fusion v2.6.4 adds retryable and non-retryable exit codes for startup failures in NFS service mode, and new health and liveness metrics. It also fixes renamed directories and very large files on Google Cloud Storage.

How to pin this version​

fusion {
  enabled = true
  containerConfigUrl = 'https://fusionfs.seqera.io/releases/v2.6.4-amd64.json'
}
info

For ARM64 CPU architecture, use containerConfigUrl = 'https://fusionfs.seqera.io/releases/v2.6.4-arm64.json'.

Feature updates and improvements​

NFS service mode​

  • Added distinct exit codes for startup failures when Fusion runs as the NFS daemon and cannot complete its startup handshake with the control node. Fusion exits with 172 when the control node rejects the request, which is a configuration or permission problem that a retry cannot fix. Fusion exits with 173 for anything else, such as an unreachable or slow control node, and for ambiguous failures. Intelligent Compute relocates the task on 173 instead of aborting the run. See Exit codes 172 and 173.
  • Added FUSION_NFS_READINESS_TIMEOUT to set how long Fusion waits for its work directory to become ready before it classifies the failure and exits. The default is 60 seconds.

Metrics​

  • Added the fusion_health_* metric family to the per-task metrics file. It reports uptime, memory use split into non-overlapping categories, open file handles, cache device disk usage, mount initialization time, and shutdown flush time. Resource exhaustion is now visible after the fact.
  • Added two liveness gauges that report how many file system requests are outstanding and the age of the oldest one. The age stays near zero when nothing is calling Fusion and climbs when Fusion has taken a request and not answered it. Use them to tell a stalled mount from an idle one.
  • Published a schema and triage runbook for the metrics file. See Fusion task metrics.

Bug fixes​

General​

  • Fixed a renamed directory dropping files and symlinks that were not yet uploaded.
  • Fixed a "no such file" error on NFS daemon clusters for an input produced by another task.
  • Fixed a retried task starting on leftovers from the previous attempt.
  • Fixed .tbi, .csi, and .bai index files appearing older than the data they index.
  • Fixed symlink targets longer than 2048 bytes being truncated on read.
  • Fixed declared outputs with adjacent brace patterns being evicted at shutdown instead of persisted.
  • Fixed a truncate or seek-write to an implausible offset exhausting memory and killing the mount, which is now rejected with a clear error.
  • Fixed temporary files being uploaded while the task was still writing them, except on runs with snapshots enabled.

Google Cloud​

  • Fixed corrupt results for very large files on Google Cloud Storage, roughly 128 GiB and above at the default chunk size.
  • Added a check that rejects an object over the 5 TiB Google Cloud Storage per-object limit before uploading.

Dependencies​

  • Reorganized the metrics collector, added a streaming upload endpoint on the control interface, and moved fusion-tracer onto it.
  • Updated CI, build pipeline, and dependencies.

Upgrade notes​

  • Drop-in replacement for v2.6.3. No configuration changes required.
  • In NFS mode, fusion-tracer now sends its trace through the control interface and no longer falls back to writing it through the mount. A trace produced while the control node is unreachable is dropped. Tracing is unchanged when the control node is reachable.
  • FUSION_NFS_READINESS_TIMEOUT is new and defaults to 60 seconds.
On this Page