Let’s see how I/O is performed on a distributed object in VSAN.
Let’s look at the write operations first.
When the Guest OS performs a write to a virtual SCSI disk, that write goes to the local VSAN module on that host and from there it is sent concurrently to all the replicas of that .vmdk. The write operation is persisted on the flash(SSD) that fronts the disk of each replica and only after that it is acknowledged back to the VM. All data from the moment a write operation completes have the required degree of replication per specified policy.
Later and asynchronously, VSAN de-stages the written data to the magnetic disk. This may take seconds or even many minutes depending on the circumstances. A certain logical block of a .vmdk may be overwritten multiple times on the Flash, but only the latest copy will be finally de-staged. Also, VSAN de-stages data in batches that are going to the same physical proximity on the HDD, so that we essentially get sequential performance out of the HDD.
image
And now lets look at the read operations.
When a read operation is issued from the Guest OS, it again goes through the local VSAN module. VSAN decides which replica to read the data from and if it is remote, it sends it to that node. In the vast majority of the cases, the read will be satisfied from the Flash (when the blocks are part of the active data set of the .vmdk that is cached in the Flash). If it is not there, then it is fetched from the HDD and placed in the Read Cache in the Flash for future accesses.
VSAN is intelligent in how it forwards reads to replicas. First it load balances across them, but it also makes sure that reads for certain ranges of logical .vmdk addresses go always to the same replica. This way, we never have the same block cached in more than one place and therefore we make a much better use of Flash capacity.
image