Issue Station
Sign In
Tools hereToolbox

ollama - ✅(Solved) Fix x/imagegen/transfer: disk write errors silently ignored in downloader copy(), causing silent data corruption [1 pull requests]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…

In x/imagegen/transfer/download.go, the copy() function ignores the error returned by dst.Write(). This causes silent data corruption when downloading blobs: if a disk-write fails (e.g. disk full, I/O error, permission error), the function continues as if nothing happened, the SHA256 hash still validates against the received bytes (not written bytes), and the download is reported as successful — but the on-disk file is corrupt or incomplete.

Error Message

In x/imagegen/transfer/download.go, the copy() function ignores the error returned by dst.Write(). This causes silent data corruption when downloading blobs: if a disk-write fails (e.g. disk full, I/O error, permission error), the function continues as if nothing happened, the SHA256 hash still validates against the received bytes (not written bytes), and the download is reported as successful — but the on-disk file is corrupt or incomplete. dst.Write(buf[:nr]) // ← error return value is discarded!

  • I/O errors: Any transient or persistent disk I/O error is dropped; the download reports success. The error from dst.Write() should be checked and returned immediately, causing the download to fail and retry (or report an error to the user).

Root Cause

  • Disk full: If the disk runs out of space mid-download, writes silently fail. The SHA256 hash is computed over bytes received from the network (not bytes on disk), so the hash check passes. The resulting file is silently truncated/corrupt.
  • I/O errors: Any transient or persistent disk I/O error is dropped; the download reports success.
  • This is especially dangerous because the hash verification if got := fmt.Sprintf("sha256:%x", h.Sum(nil)); got != blob.Digest is meant to be the final correctness check, but it can falsely pass when writes fail.

Fix Action

Fix

if nw, werr := dst.Write(buf[:nr]); werr != nil {
    return n, werr
} else if nw != nr {
    return n, io.ErrShortWrite
}

PR fix notes

PR #15469: fix: check dst.Write error in transfer downloader copy() to prevent silent data corruption

Description (problem / solution / changelog)

Problem

In x/imagegen/transfer/download.go, the copy() function ignores the error returned by dst.Write():

nr, err := src.Read(buf)
if nr > 0 {
    lastRead.Store(time.Now().UnixNano())
    dst.Write(buf[:nr])   // ← error silently discarded!
    h.Write(buf[:nr])
    d.progress.add(int64(nr))
    n += int64(nr)
}

This causes silent data corruption:

  1. If the disk is full or an I/O error occurs during download, writes fail silently
  2. The SHA256 hash is computed over bytes received from the network (not bytes written to disk)
  3. The final hash check passes — the download reports success
  4. But the file on disk is incomplete or corrupt

Fix

Check the error and byte count from dst.Write() and return immediately on failure:

nw, werr := dst.Write(buf[:nr])
if werr != nil {
    return n, werr
}
if nw != nr {
    return n, io.ErrShortWrite
}

This follows the standard Go pattern for detecting short writes (as used in io.Copy in the standard library).

Testing

The fix ensures:

  • Disk-full scenarios cause download failure (not silent corruption)
  • I/O errors propagate up and trigger the retry logic in download()
  • Existing tests continue to pass (no behavior change under normal conditions)

Closes #15468

Changed files

  • x/imagegen/transfer/download.go (modified, +7/-1)

Code Example

nr, err := src.Read(buf)
if nr > 0 {
    lastRead.Store(time.Now().UnixNano())
    dst.Write(buf[:nr])   // ← error return value is discarded!
    h.Write(buf[:nr])     // hash counts bytes regardless of whether write succeeded
    d.progress.add(int64(nr))
    n += int64(nr)
}

---

if nw, werr := dst.Write(buf[:nr]); werr != nil {
    return n, werr
} else if nw != nr {
    return n, io.ErrShortWrite
}
RAW_BUFFERClick to expand / collapse

Bug Report

Summary

In x/imagegen/transfer/download.go, the copy() function ignores the error returned by dst.Write(). This causes silent data corruption when downloading blobs: if a disk-write fails (e.g. disk full, I/O error, permission error), the function continues as if nothing happened, the SHA256 hash still validates against the received bytes (not written bytes), and the download is reported as successful — but the on-disk file is corrupt or incomplete.

Affected File

x/imagegen/transfer/download.go, function copy()

Code

nr, err := src.Read(buf)
if nr > 0 {
    lastRead.Store(time.Now().UnixNano())
    dst.Write(buf[:nr])   // ← error return value is discarded!
    h.Write(buf[:nr])     // hash counts bytes regardless of whether write succeeded
    d.progress.add(int64(nr))
    n += int64(nr)
}

Impact

  • Disk full: If the disk runs out of space mid-download, writes silently fail. The SHA256 hash is computed over bytes received from the network (not bytes on disk), so the hash check passes. The resulting file is silently truncated/corrupt.
  • I/O errors: Any transient or persistent disk I/O error is dropped; the download reports success.
  • This is especially dangerous because the hash verification if got := fmt.Sprintf("sha256:%x", h.Sum(nil)); got != blob.Digest is meant to be the final correctness check, but it can falsely pass when writes fail.

Expected Behavior

The error from dst.Write() should be checked and returned immediately, causing the download to fail and retry (or report an error to the user).

Fix

if nw, werr := dst.Write(buf[:nr]); werr != nil {
    return n, werr
} else if nw != nr {
    return n, io.ErrShortWrite
}

Steps to Reproduce

  1. Start a large blob download via the imagegen transfer package
  2. Fill the disk to capacity mid-download
  3. Observe that the download reports "success" but the file is corrupt (truncated)

Environment

  • Introduced in the recent x/imagegen/transfer package (new code, not yet in production stable releases but shipping in latest main)

extent analysis

TL;DR

  • Check the error return value of dst.Write() in the copy() function to prevent silent data corruption during blob downloads.

Guidance

  • Modify the copy() function to handle the error returned by dst.Write() as shown in the provided fix: if nw, werr := dst.Write(buf[:nr]); werr != nil { return n, werr }.
  • Verify that the download fails and reports an error when a disk-write fails, such as when the disk is full or an I/O error occurs.
  • Test the modified function with the steps to reproduce provided in the issue to ensure that it correctly handles errors and prevents silent data corruption.
  • Consider adding additional error handling or logging to ensure that any errors that occur during the download process are properly reported and handled.

Example

if nw, werr := dst.Write(buf[:nr]); werr != nil {
    return n, werr
} else if nw != nr {
    return n, io.ErrShortWrite
}

Notes

  • This fix assumes

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#vector store#embedding generation#cache error#pipeline error#permission error

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING