Indexing and translog replication

The engine factory replaces the translog config with a callback that serializes operations into TranslogReplicator.add. It also disables filesystem fsync for that translog because object store replication is the durability path.

(operation, seqNo, location) ->
    replicator.add(translogConfig.getShardId(), operation, seqNo, location),
false // translog is replicated to the object store, no need fsync that

TranslogReplicator.add writes serialized operations into a node-level buffer. Size and sync pressure create upload tasks. sync and syncAll wait until the relevant locations are included in uploaded files.

if (nodeTranslogBuffer.writeToBuffer(shardSyncState, operation, seqNo, location)) {
    if (nodeTranslogBuffer.shouldFlushBufferDueToSize()) {
        executor.execute(new FlushTask(nodeTranslogBuffer));
    }
}

Upload tasks retain the byte reference, call ObjectStoreService.uploadTranslogFile, and retry on failure. The max uploaded translog generation is then used by commits to record a safe recovery start point.

objectStoreService.uploadTranslogFile(
    translog.metadata().name(),
    translog.bytes().data(),
    ActionListener.releaseAfter(...));

Before committing Lucene, IndexEngine records maxUploadedFile + 1. That value becomes metadata on the stateless compound commit, allowing recovery to know which translog files may be needed after the Lucene commit.

translogStartFileForNextCommit =
    translogReplicator.getMaxUploadedFile() + 1;
super.commitIndexWriter(writer, translog);