NATS JetStream under the same faults, with and without a sync on every write.
Jepsen showed in December 2025 that JetStream’s default settings can lose writes it has
already acknowledged. We reproduced that as a comparison: the same cluster, the same
workload and the same mix of faults, a hundred runs with the default and a hundred with
sync_interval: always.
Default fsync every 2 min
31of 100 runs lost acknowledged writes
One power loss, replayed: 5 of 5 lose writes
Changed sync_interval: always
0of 100 runs lost an acknowledged write
One power loss, replayed: 0 of 5 lose writes
A known issue, used as a test
The behaviour was first published in Jepsen’s analysis of NATS 2.12.1 (8 December 2025) and is tracked upstream as nats-server issue #7564. By default JetStream acknowledges a write straight away and flushes data to disk every two minutes. If the nodes lose power together inside that window, writes the cluster said it had accepted are gone.
We ran it for a different reason. A tool that compares two configurations should be able to show a difference that is already known. If it cannot, its answer on an unknown one is not worth much.
The comparison
Both columns are the same three-node cluster and the same workload, under the same mix of faults. The counts below show that mix: the same kinds of fault in similar numbers, not identical schedules. Only one setting differs.
| Default | Always sync | |
|---|---|---|
| Runs that lost acknowledged writes | 31 | 0 |
| Writes acknowledged | 1,155 of 1,190 | 1,167 of 1,190 |
| Power losses | 117 | 119 |
| Operating-system crashes | 67 | 58 |
| Process kills | 80 | 81 |
| Pauses | 74 | 80 |
| Network partitions | 88 | 88 |
| Disk corruptions | 23 | 29 |
The smallest schedule that fails
After the hundred runs, Afterfault cut a failing schedule down to the faults it needs.
One is enough: a power loss on the nodes at the same moment. Replayed five times
against the default settings, it lost writes five times. Replayed five times with
sync_interval: always, it lost none.
The setting
jetstream {
sync_interval: always
}
It is not free. Jepsen’s report quotes the NATS documentation warning that this setting slows a server to a few hundred messages a second. We did not measure throughput in this comparison, so we have no number of our own for the cost.
What to take from it
- Find out what “acknowledged” means in your configuration. For many systems it means accepted in memory, not written to disk. The default is often the fast one.
- Test the fault that hits every node at once. Killing one process at a time rarely loses data in a replicated system. Losing power everywhere is the case replication does not cover.
- Compare settings under the same faults. Changing a durability setting and watching for trouble tells you little. Running both settings through the same faults and counting gives a number you can show someone.