CACHE

10/11/2003

Cache: A high speed store for frequently used data. Typically a cache will use a fairly small amount of expensive higher speed storage to speed up access to a larger, cheaper storage medium. 386 computers often used a small Static RAM cache to improve the performance of slower DRAM memories. 486 and later CPUs have on chip cache to speed up access to DRAM. Most modern Operating systems cache disk data in faster DRAM memory. Modems and CDROM drivers may cache data on faster hard drives. Web browsers cache recently viewed pages with mixed results. Viewing is faster, but updates to the pages may not be seen unless they are manually requested.

Caching is complex. There must be some technique for determining which information to place in the cache. Information must be selected and loaded into the cache(s) without significantly slowing down operations. Writes to cached data must be reflected both in the cache(s) and eventually (or sometimes immediately) in the main storage. There are two approaches to handling writing -- write through and write back. Write back is faster, but can be complex, so much so that reliability can be an issue. The algorithm used to find cached data must be fast and simple and should minimize flushing information at different, but similar, addresses. The latter is accomplished by a technique called "association". Association deals with two or more cached bytes mapping to the same cache address.

Caching can speed up reads. It is likely to slow write operations somewhat overall although individual programs may perceive their write operations as being faster.. Fortunately most applications do many more reads than writes. While memory caching is fast enough that write delays usually do not affect user perceptions, slower caches such as disk caches can have problems with cache flushing delays -- pauses while unwritten data is moved out of the cache and back to longer term storage. These, in turn can lead to a need for orderly OS shutdown if data is not to be lost. In some portable, industrial, or specialized applications, orderly shutdown can not be guaranteed.

Caches can exhibit some pathological problems including "thrashing" where two sets of data compete for cache space and the constant flushing of one by the other slows data access dramatically -- sometimes below the speed of an uncached system.

Overall one level of caching usually speeds up computer operation considerably. Multiple levels of caching may be less effective or even counterproductive as each level adds write delays but lower levels may not produce much read improvement if a higher level is caching most read information.