RDI File - Final State

Since our configuration is set to "Every Minute," a new RDI output file is created in the delivery path, and Safer Payments continues appending statement data to that same file for up to one minute before it is closed and finalized.

This means:

- A file's presence in the output directory does not indicate it is complete.

- Up to a minute (or more, depending on configuration) may elapse between when a file first appears and when it is actually finalized and safe to consume.

- Any downstream process that picks up files based solely on their existence in the directory risks reading a partial/incomplete file.

This creates a race condition for our downstream consumers, since we currently have no reliable way to distinguish an in-progress file from a completed one.

As a workaround, our process currently waits 2 minutes after the last file was created before attempting to retrieve files, on the assumption that this is enough time for a file to be finalized. However, this is not a reliable solution either — it only reduces the odds of collision, it doesn't eliminate them. Depending on system load or timing, a new file could start well within that 2-minute window, and we'd have no way of knowing whether the file we're retrieving is truly complete or still being written to. This approach is a stopgap, not a fix.

Requested Enhancement

We'd like to request that Safer Payments change its file-writing behavior so that incomplete files are never visible in the final output directory. Specifically, one of the following approaches would resolve this:

1. Write-then-move pattern: Write each file to a temporary/staging location (e.g., a .tmp or hidden staging directory) while it is being appended to, and only move/rename it into the delivery output directory once the file is closed and finalized. A rename within the same filesystem is atomic, so consumers would never see a partial file.

2. Completion marker: If the write-then-move approach isn't feasible, an alternative would be to emit a companion "done" or "complete" marker file (e.g., filename.done) once the statement file is finalized, so downstream processes can safely poll for that marker before consuming the file.