In computer science, a cache ( kash ) is a component that transparently stores data so that future requests for that data can be served faster. The data that is stored within a cache might be values that have been computed earlier or duplicates of original values that are stored elsewhere. If requested data is contained in the cache (cache hit), this request can be served by simply reading the cache, which is comparatively faster. Otherwise (cache miss), the data has to be recomputed or fetched from its original storage location, which is comparatively slower. Hence, the greater the number of requests that can be served from the cache, the faster the overall system performance becomes.
To be cost efficient and to enable an efficient use of data, caches are relatively small. Nevertheless, caches have proven themselves in many areas of computing because access patterns in typical computer applications have locality of reference. References exhibit temporal locality if data is requested again that has been recently requested already. References exhibit spatial locality if data is requested that is physically stored close to data that has been requested already.
The percentage of accesses that result in cache hits is known as the hit rate or hit ratio of the cache.
cache hit rate = Num. of cache hit / Num. of data accesses
The percentage of accesses that result in cache misses is known as the miss rate or miss ratio of the cache.
cache miss rate = Num. of cache miss / Num. of data accesses
so for a specific cache, cache hit rate + cache miss rate = 1.
Cache algorithms(replacement algorithms, replacement policies): each replacement policy is a compromise between hit rate and latency.
(1) Belady's Algorithm is optimal, but impossible to implement.
(2) LRU replaces the least recently used entry
(3) MRU replaces the most recently used entry. It is suitable for looping sequential reference pattern. The older an item is, the more likely it is to be accessed.
(4) Random Replacement
(5) Segmented LRU
(6) LFU replaces least frequently used entry.
(7) Adaptive Replacement Cache.
The cache idea has been applied in many areas.
(1) CPU cache uses small memories on or close to the CPU as cache.
(2) Translation lookaside buffer: for more information, please see this page.
(3) Disk cache: The page cache in main memory is an example of disk cache. It is managed by OS kernel.
(4) Web cache: Web browsers and web proxy servers employ web caches to store previous responses from web servers, such as web pages and images. Web caches reduce the amount of information that needs to be transmitted across the network, as information previously stored in the cache can often be re-used. This reduces bandwidth and processing requirements of the web server, and helps to improve responsiveness for users of the web.
Difference between cache and buffer:
A buffer is a region of a physical memory storage used to temporarily hold data while it is being moved from one place to another. Buffers are typically used when there is a difference between the rate at which data is received and the rate at which it can be processed, or in the case that these rates are variable. The intent to use buffer is to increase transfer performance and reduce the variation of the transfer's latency.
A cache's sole purpose is to reduce accesses to the underlying slower storage and thus reduce latency.
Useful articles:
1) Distributed Cache with Centralized Control by S. Paul and Z. Fei
2) Functional Principles of Cache Memory by Paul V. Bolotoff
