Maintaining thousands of in-flight instructions

Abstract

Superscalar processors tolerate long-latency memory operations by maintaining a high number of in-flight instructions. Since the gap between processor and memory speed continues increasing every year, the number of in-flight instructions needed to support the large memory access latencies expected in the future should be higher and higher. However, scaling-up the structures required by current processors to support such a high number of in-flight instructions is impractical due to area, power consumption, and cycle time constraints. The kilo-instruction processor is an affordable architecture able to tolerate the memory access latency by supporting thousands of in-flight instructions. Instead of simply up-sizing the processor structures, the kilo-instruction architecture relies on an efficient multi-checkpointing mechanism. Multi-checkpointing leverages a set of techniques like multi-level instruction queues, late register allocation, and early register release. These techniques emphasize the intelligent use of the available resources, avoiding scalability problems in the design of the critical processor structures. Furthermore, the kilo-instruction architecture is orthogonal to other architectures, like multi-processors and vector processors, which can be combined to boost the overall processor performance.