Atomic file replacement needs a durability check

Python added os.replace in version 3.3, whose first release arrived on 29 September 2012. The operation provides an atomic rename when it succeeds, with the usual filesystem boundary constraints. SQLite's atomic-commit explanation goes further into the hard part: operating systems and storage can buffer and reorder writes, so surviving a crash requires more than a convincing filename.

Atomic file replacement needs a durability check when a result is supposed to survive power loss. My narrower starting rule is to separate complete-file visibility from persistence. A reader seeing either the old file or the new one is useful. It does not establish what a reboot will reveal.

Replace the complete file in the same directory

The Python operating-system interface supplies the replacement primitive. Write a temporary sibling, finish it, then replace the destination. Keeping both paths in the same directory avoids deliberately crossing filesystems.

Here is a small helper for a trusted local result path. It synchronises the temporary file before replacement and removes an unused temporary file on failure. It does not claim to be a complete crash-durable storage protocol.

import os
import tempfile
from pathlib import Path

def replace_text(destination, text):
    destination = Path(destination)
    fd, temporary = tempfile.mkstemp(
        dir=destination.parent, prefix=".result-"
    )
    try:
        with os.fdopen(fd, "w", encoding="utf-8") as output:
            output.write(text)
            output.flush()
            os.fsync(output.fileno())
        os.replace(temporary, destination)
    finally:
        if os.path.exists(temporary):
            os.unlink(temporary)

The destination must already have an existing parent directory. This helper is not a permission-preserving editor: the replacement inherits the temporary file's mode rather than the old file's metadata. Decide whether that is appropriate for your artifact before using it.

Atomic visibility is only one storage promise

The SQLite atomic-commit document is worth reading even if you are not building a database. Its explanation distinguishes application writes from what storage has durably accepted and describes the assumptions needed for recovery.

In this helper, synchronising file contents does not by itself prove that the directory-entry replacement will survive a sudden outage. Directory synchronisation and platform-specific guarantees need separate consideration. Network filesystems introduce another contract to inspect rather than inherit by assumption.

There is also a multi-writer issue: two workers can each produce a complete file and replace the same destination. The last replacement wins. Atomic replacement is not a compare-and-swap operation, a lock or a version conflict detector.

The synthetic replacement check

A local Python 3.14.6 fixture first wrote an old result, then used the helper to install complete new text. A second check injected an OSError at the replacement step. The accepted destination remained unchanged, the error propagated and the unused temporary file was removed.

These checks exercise successful replacement and a failure before the rename. No power-loss test, concurrent-reader experiment, network-filesystem test or cross-platform certification was performed. The sample demonstrates a useful boundary, not a storage durability guarantee.

Choose storage from the cost of loss

For a hypothetical agent that writes a JSON report, I would serialise and validate the report before calling this helper. A complete invalid report is still invalid. I would then decide whether the artifact is reproducible scratch output or an authoritative record whose loss would matter.

That distinction changes the storage choice. A regenerable report may tolerate a narrowly documented local-file contract. A record of an irreversible external action deserves transactional storage, an operation identity and recovery behaviour tested against the actual failure model.

What's in it for you

• Test failure before replacement without damaging the last accepted result.
• Specify visibility, concurrent-writer behaviour and crash persistence separately.

An atomic rename closes the partial-file window; durability needs its own evidence.