Cache
A small amount of memory between the processor and the main memory which is really fast.
Cache is partitioned into lines or blocks. Basically each of these is a block of words. And each block has a tag.
Parameters of cache memory
- Hit and miss: When data is found and not found, respectively.
- Hit rate and miss rate: Percentage of times memory is found or not found, respectively.
- Hit time is the time taken for the memory to be retrieved.
- Miss penalty is the total time taken to process the miss.
- Latency is the time required to retrieve the first word of the block.
- Bandwidth is the time take to retrieve the rest of the block.
Sources of cache misses
- Compulsory misses or cold misses is when the cache is empty and the first reference to a memory address is made.
- Capacity misses happen when the cache doesn't contain the necessary block that is needed.
- Conflict misses occur when multiple blocks of memory are mapped to the same cache line. This is common in direct mapping and set associative mapping.
Organization
Split cache
Separate caches are used for instructions and data. I cache for instructions and D cache for data. There is lesser contention due to split cache. It is used in pipelining.
Unified cache
The same cache is used for storing both data and instructions. Therefore, has a higher hit rate.
Multilevel caches
- If a cache miss happens, then the penalty is that the data has to be accessed from the main memory.
- To reduce this penalty, we have multilevel caching. So, we have primary, secondary caches and so on.
- A cache miss in primary cache will result in checking the secondary cache, and so on. In this way, we won't have to directly check the main memory in case of miss, and just use other caches.
- It is not as fast as primary cache hits, but it will still be faster than accessing from main memory.
- Primary cache is called L1. It is located on the same chip as the CPU. The secondary cache is L2 and so on. L2, L3 caches are located off chip.
Management techniques
- Cache block placement
- Cache block replacement is not needed by direct mapping as we just replace whatever entry the mapping function gives us. For associative and set associative, we need to know which cache line to replace the newer one with.