cache - flush and invalidate operation

Question

I have some questions on cache synchronization operations.

Invalidate: Before cpu tries to read a portion of memory updated by a device, the corresponding memory needs to be invalidated.

Flush: Before the device read a portion of memory updated by CPU, CPU must flush (write back is also correct?) the contents from cache to memory, so that device reads the contents from memory with updated contents.

If flush is not carried out it may read junk data present in memory as the memory is not still updated with contents written to cache.

Please confirm whether my above understanding is correct?

When do you want to combine both flush and invalidate? I heard that while playing with device control descriptors we need to synchronize by combining flush and invalidate. Why so?

Do we need to follow a sequence like flush followed by invalidate?

Is there a scenario in which invalidate followed by flush will be useful?

Solution 1

Flush does write back the contents of cache to main memory, and invalidate does mark cache lines as invalid so that future reads go to main memory.

I think you would combine flush and invalidate if the device was updating a block of memory: the flush would ensure that the device had the latest contents, and the invalidate would then ensure that when the device had finished that the CPU would read the new contents from memory.

Solution 2

Please confirm whether my above understanding is correct?

Generally you are totally right but there is some stones that might cause to stumble. You did not specify the HW platform. If we are not talking about small embedded controllers packed w/ SRAM, consider the following. Processors packed with MMU support different memory attributes for normal DDR memory and Driver (HW related) memory. The last one is non cacheable so there is no worries about flushing/invalidating.

When do you want to combine both flush and invalidate? I heard that while playing with device control descriptors we need to synchronize by combining flush and invalidate. Why so?

Since DMA is mentioned in a tags there are a few scenarios (assuming that HW buffer is non-cacheable device memory):

1. DMA transfer data from DDR memory to HW buffer.
2. DMA transfer data from HW buffer to DDR (HW received data and want to make it available for CPU)
3. DMA transfer from DDR into another region of DDR.

1. DDR buffer should be flushed before DMA. Driver buffer is non cacheable so no need for invalidation.
2. DDR buffer should be invalidated before or after (look NOTE below for details) DMA transfering to prevent CPU to use 'old' data from cache. Flushing of HW buffer is redundant.
3. 'Source' buffer should be flushed, 'Destination' buffer should be invalidated. So valid data is in memory for DMA before transfer, and CPU doesn't take a 'dirt' from cache after DMA did it's job.

NOTE: It's obvious that 'source' should be flushed before DMAing. Still there is the question when to invalidate. Technically it's before CPU will try to access 'Destination' data and could be before or after DMA (we should ensure that DMA has finished the job). IRL invalidating after DMAing might lead to a problem. Refer to Flush/Invalidate range by virtual address; ARMv8; Cache;

As you could see invalidate for this particular platform should be done before DMAing. Also rambling th/ the BSP code for ARMv7 device I found recommendation to invalidate destination buffer before a DMA transfer.
But that's not all. You might want to invalidate destination buffer after a DMA transfer again (that's right, second time). If chip in question has prefetch it could fetch data back to cache while DMA is working. So sequence might be: after first invalidate prefetch put data in cache again -> DMA override data in memory -> cache has data different than memory and cache is marked as having valid data. Second invalidate would ensure that data would be retrieved into cache again, therefore cache and memory finished to be in sync :)

Do we need to follow a sequence like flush followed by invalidate?

Assuming that source and destination buffers do not intersect each other, there is not dependancies. You could flush-invalidate or invalidate-flush.

Is there a scenario in which invalidate followed by flush will be useful?

Don't think so.

Comments

[fazkan]

Thank you for such explanation, couldnt find an explanation better than that on the internet....

[kumar]

Adding for clarity, This is carried out mainly during initialization stage. The initialized descriptors are both Flushed(to update in memory) and Invalidated(So that next read by CPU is with useful contents) during initialization stage.