CDA-4101 Lecture 3 Notes

• There are limits to how fast you can get the full fetch-decode-execute cycle to run.
• Doing multiple things at the same time (parallelism) is an alternative to increasing the execution speed.

• pipelining
• superscalar architectures

• array processors
• vector processors
• multiprocessors
• multicomputers

Pipelining?

Monte-Carlo Assembly Line

Henry Ford meets CPUs

• Instruction latency is 5 cycles
• Ability to "issue" an instruction every cycle
• After time 't4', an instruction finished every single cycle
• Processor bandwidth is one instruction per cycle

There is an apparent factor of 5 speed-up in a 5 stage pipeline.

Problems:

• All stages must execute in the same amount of time, so "cycle" time has to be as long as the stage that takes the maximal amount of time.
• Care must be put into decomposing the design into stages.
• Taking a 100 ns hardware operation and dividing into 5 parts that execute in exactly the same time is difficult.
• Idealistic calculations assume pipeline to always be "full".
• Hazards!

Data Hazards - when an instruction needs data that is not yet available. Structural Hazards - when the same hardware is needed by more than one instruction in the pipeline. Control Hazards (Branch Hazards) - when changes to the program counter affect the pipeline execution.

• Suppose the IF (instruction fetch) stage and the OF (operand fetch) stages both need to access main memory for their data.
• Suppose we only have one data path to main memory.
• We will not be able to execute an IF and an OF at the same time.

• One solution is to add redundant hardware to avoid these hazards.
• Decision depends on the design costs, transitor constraints and the performance increase it will yield.

• Another solution is to stall the pipeline, but this will reduce the overall processor performance.

• In this case, stalling for one stage results in another conflict so we must stall for two stages.

• Obviously a prefetch buffer would be useful here.

    I1: R3 := R1 + R2
    I2: R4 := R3 + 10

• Instruction I2 needs the data from R3 at time 't4' (operand fetch).
• However, I1 hasn't finished yet, so R3 value has not been written back and will not be available until after time 't5'.

• slows down execution

• requires extra hardware/more complex hardware design

• Who reorders the instructions: hardware or compiler?
• Reordering instructions and preserving proper program semantics can be tricky
• Reordering instructions can cause its own conflicts

• Previous was an example of a read-after-write (RAW) data hazard
• There are also write-after-write (WAW) and write-after-read (WAR) data hazards that occur, especially when instructions are reordered.

• Consider this code fragment:

    I1: R2 = R1 + 5
    I2: IF R2 > 10 GOTO I4
    I3: R2 = R2 + 9
    I4: R3 = R2 - 9

• We will not know whether or not to execute I3 or I4 until we have executed I2 at time 't5'.
• For efficient use of the pipeline we need to start the instruction after I2 at time 't3'

• Stalling after each branch

  	Time
  	t1	t2	t3	t4	t5	t6	t7	t8
  I1	IF	ID	OF	EX	WB
  I2		IF	ID	OF	EX	WB
  I3 | I4			--	--	--	IF	ID	OF

• Discarding pipeline results.
• Guessing which branch will be taken (branch prediction)

• Duplicating pipeline stage hardware can prevent stalls and get more instructions executed,
• The extreme case, which is now commonly used, it to duplicate the entire pipeling to get more instructions executing in parallel.
• Procesors with multiple pipelines are termed superscalar (this also is instruction level parallelism)

• There are many ways to organize parallel pipelines.
• Common prefetch stage:
• Multiple functional units, (e.g., one stage takes longer than the others)

• There are limits to what can be achieved with instruction level parallelism
• greater speed-up factors are possible by using multiple processors
• At some level, a pipelined processor could be viewed as consisting of multiple processors where each processor is specialized and sequenced.

Images of parallel machines borrowed from: http://www.cems.uwe.ac.uk/teaching/notes/PARALLEL/ARCHITEC

• von Neumann architecture is SISD
• with pipeling and superscalar architectures, modern processor do not fit so neatly into this classification

Illiac IV

array processors - same instruction executes all all processor, only data varies vector processor - similar to array processor, only a single instruction executes by passing many data words through a heavily pipelined ALU

Cray 1

Cray SV1

Motherboard (2 processors, shared-memory)

MIMD can use either: shared memory (a.k.a., multi-processor machine) - communicate through shared memory distributed memory (a.k.a., multi-computer machine) - commuicate through message passing

Cray Y-MP (8 processors, shared-memory)

(a) True shared memory
(b) Hybrid shared/distributed memory

Rely on message passing for communication and coordination Too many connections is costly and unmanageable Which processors can talk to which other processors?

Distributed Memory MIMD

• No machines really fit this description
• At best, you have to be creative to fit any current machines into this classification.

A typical PC is sort of a fully connected multi-computer Multiple processors: CPU video card bus controllers disk controllers etc. Control signals are the messages Common communication bus gives full connectivity