chapter 3 & appendix c pipelining - cs.sjtu.edu.cnshen-yy/cs359/slides/4_pipelining_a.pdf ·...
TRANSCRIPT
1
CS359: Computer Architecture
Chapter 3 & Appendix CPipeliningPart A: Basic and Intermediate Concepts
Yanyan ShenDepartment of Computer Science and EngineeringShanghai Jiao Tong University
2
Chapter 3: Pipelining
Parallel Processing
3
Chapter 3: Pipelining
Outline
C.1 Introduction to PipeliningHow Pipeline is ImplementedPipeline HazardsExceptionsHandling Multicycle Operations
4
Chapter 3: Pipelining
Processor Performance Performance of single-cycle processor is
limited by the long critical path delay The critical path limits the operating clock
frequency
Can we do better? New semiconductor technology will reduce the
critical path delay by manufacturing small-sized transistors Core 2 Duo is manufactured with 65nm technology Core i7 is manufactured with 45nm technology Next semiconductor technology is 32nm technology
Can we increase the processor performance with a different microarchitecture? Yes! Pipelining
5
Chapter 3: Pipelining
Revisiting Performance
• Laundry Example Ann, Brian, Cathy, Dave
each has one load of clothes to wash, dry, and fold
Washer takes 30 minutes Dryer takes 40 minutes Folder takes 20 minutes
A B C D
6
Chapter 3: Pipelining
Sequential Laundry
• Response time:• Throughput:
A
B
C
D
30 40 20 30 40 20 30 40 20 30 40 20
6 PM 7 8 9 10 11 Midnight
Task
Order
Time
90 mins0.67 tasks / hr (= 90mins/task, 6 hours for 4 loads)
7
Chapter 3: Pipelining
Pipelined Laundry
A
B
C
D
6 PM 7 8 9 10 11 Midnight
Task
Order
Time
30 40 40 40 40 20
90 mins1.14 tasks / hr (= 52.5 mins/task, 3.5 hours for 4 loads)
• Response time:• Throughput:
• Pipelining doesn’t help latency (response time) of a single task
• Pipelining helps throughput of entire workload
• Multiple tasks operating simultaneously
Potential speedup = # of pipeline stages
Unbalanced lengths of pipeline stages reduce speedup
Notes
8
Chapter 3: Pipelining
PipeliningImprove performance by increasing instruction throughput
Instruction Fetch
Register File Access (Read)
ALU Operation Data Access Register
Access (Write)
2ns 1ns 2ns 2ns 1ns
Instructionfetch Reg ALU Data
access Reg
8 ns Instructionfetch Reg ALU Data
access Reg
8 ns Instructionfetch
8 ns
Time
lw $1, 100($0)
lw $2, 200($0)
lw $3, 300($0)
2 4 6 8 10 12 14 16 18
...
Programexecutionorder(in instructions)
2 4 6 8 10 12 14
Instructionfetch Reg ALU Data
access Reg
Time
lw $1, 100($0)
lw $2, 200($0)
lw $3, 300($0)
2 ns Instructionfetch Reg ALU Data
access Reg
2 ns Instructionfetch Reg ALU Data
access Reg
2 ns 2 ns 2 ns 2 ns 2 ns
Programexecutionorder(in instructions)
SequentialExecution
PipelinedExecution
9
Chapter 3: Pipelining
Pipelining (Cont.)
2 4 6 8 10 12 14
Instructionfetch Reg ALU Data
access Reg
Time
lw $1, 100($0)
lw $2, 200($0)
lw $3, 300($0)
2 ns Instructionfetch Reg ALU Data
access Reg
2 ns Instructionfetch Reg ALU Data
access Reg
Programexecutionorder(in instructions)
Instructionfetch Reg ALU Data
access Reg
Instructionfetch Reg ALU Data
access Reg
Instructionfetch Reg ALU Data
access Reg
Instructionfetch Reg ALU Data
access Reg
Instructionfetch Reg ALU Data
access Reg
Instructionfetch Reg ALU Data
access Reg
…
Multiple instructions are being executed simultaneously
10
Chapter 3: Pipelining
Pipelining (Cont.)
Pipeline Speedup
= Time to execute an instructionsequential
Number of stagesTime to execute an instructionpipeline
• If not balanced, speedup is less
• Speedup comes from increased throughput
• the latency of instruction does not decrease
11
Chapter 3: Pipelining
Outline
Introduction to PipeliningC.2 How Pipeline is ImplementedPipeline HazardsExceptionsHandling Multicycle Operations
12
Chapter 3: Pipelining
Basic Idea
What do we have to add to actually split the datapath into stages?
Instruction memory
Address
4
32
0
Add Add result
Shift left 2
Instruction
M u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read dataAddress
Data memory
1
ALU result
M u x
ALUZero
IF: Instruction fetch ID: Instruction decode/ register file read
EX: Execute/ address calculation
MEM: Memory access WB: Write back
13
Chapter 3: Pipelining
Basic Idea
Instruction memory
Address
4
32
0
Add Add result
Shift left 2
Instruction
M u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read dataAddress
Data memory
1
ALU result
M u x
ALUZero
IF: Instruction fetch ID: Instruction decode/ register file read
EX: Execute/ address calculation
MEM: Memory access WB: Write back
D QQF/F
D QQF/F
D QQF/F
D QQF/F
clock
14
Chapter 3: Pipelining
Graphically Representing Pipelines
Shading indicates the unit is being used by the instruction Shading on the right half of the register file (ID or WB) or memory
means the element is being read in that stage Shading on the left half means the element is being written in that
stage
IF ID MEM WBEX
2 4 6 8 10Time
lw
IF ID MEM WBEXadd
15
Chapter 3: Pipelining
Pipelined Datapath
Instruction memory
Address
4
32
0
Add Add result
Shift left 2
Inst
ruct
ion
IF/ID EX/MEM MEM/WB
M u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read data
1
ALU result
M u x
ALUZero
ID/EX
Data memory
Address
16
Chapter 3: Pipelining
lw: Instruction Fetch (IF)
Instructionmemory
Address
4
32
0
Add Addresult
Shiftleft 2
Inst
r uct
ion
IF/ID EX/MEM MEM/WB
Mux
0
1
Add
PC
0Writedata
Mux
1Registers
Readdata 1
Readdata 2
Readregister 1
Readregister 2
16Sign
extend
Writeregister
Writedata
Readdata
1
ALUresultM
ux
ALUZero
ID/EX
Datamemory
Address
Instruction fetch
17
Chapter 3: Pipelining
lw: Instruction Decode (ID)
Instructionmemory
Address
4
32
0
Add Addresult
Shiftleft 2
Inst
r uct
ion
IF/ID EX/MEM MEM/WB
Mux
0
1
Add
PC
0Writedata
Mux
1Registers
Readdata 1
Readdata 2
Readregister 1
Readregister 2
16Sign
extend
Writeregister
Writedata
Readdata
1
ALUresultM
ux
ALUZero
ID/EX
Datamemory
Address
Instruction decode
18
Chapter 3: Pipelining
lw: Execution (EX)
Instructionmemory
Address
4
32
0
Add Addresult
Shiftleft 2
Inst
r uct
ion
IF/ID EX/MEM MEM/WB
Mux
0
1
Add
PC
0Writedata
Mux
1Registers
Readdata 1
Readdata 2
Readregister 1
Readregister 2
16Sign
extend
Writeregister
Writedata
Readdata
1
ALUresultM
ux
ALUZero
ID/EX
Datamemory
Address
Execution
19
Chapter 3: Pipelining
lw: Memory (MEM)
Instructionmemory
Address
4
32
0
Add Addresult
Shiftleft 2
Inst
r uct
ion
IF/ID EX/MEM MEM/WB
Mux
0
1
Add
PC
0Writedata
Mux
1Registers
Readdata 1
Readdata 2
Readregister 1
Readregister 2
16Sign
extend
Writeregister
Writedata
Readdata
1
ALUresultM
ux
ALUZero
ID/EX
Datamemory
Address
Memory
20
Chapter 3: Pipelining
lw: Writeback (WB)
Instructionmemory
Address
4
32
0
Add Addresult
Shiftleft 2
Inst
r uct
ion
IF/ID EX/MEM MEM/WB
Mux
0
1
Add
PC
0Writedata
Mux
1Registers
Readdata 1
Readdata 2
Readregister 1
Readregister 2
16Sign
extend
Writeregister
Writedata
Readdata
1
ALUresultM
ux
ALUZero
ID/EX
Datamemory
Address
Writeback
21
Chapter 3: Pipelining
sw: Memory (MEM)
Instructionmemory
Address
4
32
0
Add Addresult
Shiftleft 2
Inst
r uct
ion
IF/ID EX/MEM MEM/WB
Mux
0
1
Add
PC
0Writedata
Mux
1Registers
Readdata 1
Readdata 2
Readregister 1
Readregister 2
16Sign
extend
Writeregister
Writedata
Readdata
1
ALUresultM
ux
ALUZero
ID/EX
Datamemory
Address
Memory
22
Chapter 3: Pipelining
sw: Writeback (WB): do nothing
Instructionmemory
Address
4
32
0
Add Addresult
Shiftleft 2
Inst
r uct
ion
IF/ID EX/MEM MEM/WB
Mux
0
1
Add
PC
0Writedata
Mux
1Registers
Readdata 1
Readdata 2
Readregister 1
Readregister 2
16Sign
extend
Writeregister
Writedata
Readdata
1
ALUresultM
ux
ALUZero
ID/EX
Datamemory
Address
Writeback
23
Chapter 3: Pipelining
Corrected Datapath (for lw)
Instructionmemory
Address
4
32
0
Add Addresult
Shiftleft 2
Inst
ruct
ion
IF/ID EX/MEM MEM/WB
Mux
0
1
Add
PC
0
Address
Writedata
Mux
1Registers
Readdata1
Readdata2
Readregister 1
Readregister 2
16Sign
extend
Writeregister
Writedata
Readdata
Datamemory
1
ALUresult
Mux
ALUZero
ID/EX
24
Chapter 3: Pipelining
Pipelining Example
Instructionmemory
Address
4
32
0
Add Addresult
Shiftleft 2
Inst
r uct
ion
IF/ID EX/MEM MEM/WB
Mux
0
1
Add
PC
0Writedata
Mux
1Registers
Readdata 1
Readdata 2
Readregister 1
Readregister 2
16Sign
extend
Writeregister
Writedata
Readdata
1
ALUresultM
ux
ALUZero
ID/EX
Datamemory
Address
add $14, $5, $6 lw $13, 24($1) add $12, $3, $4 sub $11, $2, $3 lw $10, 20($1)
25
Chapter 3: Pipelining
Pipeline Control
PC
Instruction memory
Address
Inst
ruct
ion
Instruction [20– 16]
MemtoReg
ALUOp
Branch
RegDst
ALUSrc
4
16 32Instruction [15– 0]
0
0Registers
Write register
Write data
Read data 1
Read data 2
Read register 1
Read register 2
Sign extend
M u x
1Write data
Read data M
u x
1
ALU control
RegWrite
MemRead
Instruction [15– 11]
6
IF/ID ID/EX EX/MEM MEM/WB
MemWrite
Address
Data memory
PCSrc
Zero
Add Add result
Shift left 2
ALU result
ALUZero
Add
0
1
M u x
0
1
M u x
Note that in this implementation, the branch is resolved in the MEM stage
26
Chapter 3: Pipelining
Pipeline Control• What needs to be controlled in each stage (IF, ID, EX,
MEM, WB)? IF: Instruction fetch and PC increment ID: Instruction decode and operand fetch from register file and/or
immediate EX: Execution stage
RegDst ALUop[1:0] ALUSrc
MA: Memory stage Branch MemRead MemWrite
WB: Writeback MemtoReg RegWrite (note that this signal is in ID stage)
27
Chapter 3: Pipelining
Pipeline Control Extend pipeline registers to include control information created in ID stage Pass control signals along just like the data
Execution/Address Calculation stage control
linesMemory access stage
control lines
Write-back stage control
lines
InstructionReg Dst
ALU Op1
ALU Op0
ALU Src Branch
Mem Read
Mem Write
Reg write
Mem to Reg
R-format 1 1 0 0 0 0 0 1 0lw 0 0 0 1 0 1 0 1 1sw X 0 0 1 0 0 1 0 Xbeq X 0 1 0 1 0 0 0 X
Control
EX
M
WB
M
WB
WB
IF/ID ID/EX EX/MEM MEM/WB
Instruction
28
Chapter 3: Pipelining
Datapath with Control
PC
Instruction memory
Inst
ruct
ion
Add
Instruction [20– 16]
Mem
toR
eg
ALUOp
Branch
RegDst
ALUSrc
4
16 32Instruction [15– 0]
0
0
M u x
0
1
Add Add result
RegistersWrite register
Write data
Read data 1
Read data 2
Read register 1
Read register 2
Sign extend
M u x
1
ALU result
Zero
Write data
Read data
M u x
1
ALU control
Shift left 2
Reg
Writ
e
MemRead
Control
ALU
Instruction [15– 11]
6
EX
M
WB
M
WB
WBIF/ID
PCSrc
ID/EX
EX/MEM
MEM/WB
M u x
0
1
Mem
Writ
e
AddressData
memory
Address
29
Chapter 3: Pipelining
Datapath with Control
PC
Instructionmemory
Inst
ruct
ion
Add
Instruction[20– 16]
Mem
toR
eg
ALUOp
Branch
RegDst
ALUSrc
4
16 32Instruction[15– 0]
0
0
Mux
0
1
Add Addresult
RegistersWriteregister
Writedata
Readdata 1
Readdata 2
Readregister 1
Readregister 2
Signextend
Mux1
ALUresult
Zero
Writedata
Readdata M
ux
1
ALUcontrol
Shiftleft 2R
egW
r ite
MemRead
Control
ALU
Instruction[15– 11]
6
EX
M
WB
M
WB
WBIF/ID
PCSrc
ID/EX
EX/MEM
MEM/WB
Mux
0
1
Mem
Wr it
e
AddressData
memory
Address
IF: lw $10, 9($1)
30
Chapter 3: Pipelining
Datapath with Control
PC
Instructionmemory
Inst
ruct
ion
Add
Instruction[20– 16]
Mem
toR
eg
ALUOp
Branch
RegDst
ALUSrc
4
16 32Instruction[15– 0]
0
0
Mux
0
1
Add Addresult
RegistersWriteregister
Writedata
Readdata 1
Readdata 2
Readregister 1
Readregister 2
Signextend
Mux1
ALUresult
Zero
Writedata
Readdata M
ux
1
ALUcontrol
Shiftleft 2R
egW
r ite
MemRead
Control
ALU
Instruction[15– 11]
6
X
M
WB
M
WB
WBIF/ID
PCSrc
ID/EX
EX/MEM
MEM/WB
Mux
0
1
Mem
Wr it
e
AddressData
memory
Address
IF: sub $11, $2, $3 ID: lw $10, 9($1)
11
0100001
E
“lw”
31
Chapter 3: Pipelining
Datapath with Control
PC
Instructionmemory
Inst
ruct
ion
Add
Instruction[20– 16]
Mem
toR
eg
ALUOp
Branch
RegDst
ALUSrc
4
16 32Instruction[15– 0]
0
0
Mux
0
1
Add Addresult
RegistersWriteregister
Writedata
Readdata 1
Readdata 2
Readregister 1
Readregister 2
Signextend
Mux1
ALUresult
Zero
Writedata
Readdata M
ux
1
ALUcontrol
Shiftleft 2R
egW
r ite
MemRead
Control
ALU
Instruction[15– 11]
6
X
M
WB
M
WB
WBIF/ID
PCSrc
ID/EX
EX/MEM
MEM/WB
Mux
0
1
Mem
Wr it
e
AddressData
memory
Address
11
010
00E
ID: sub $11, $2, $3 EX: lw $10, 9($1)IF: and $12, $4, $5
1
0
10
000
1100
“sub”
32
Chapter 3: Pipelining
Datapath with Control
PC
Instructionmemory
Inst
ruct
ion
Add
Instruction[20– 16]
Mem
toR
eg
ALUOp
Branch
RegDst
ALUSrc
4
16 32Instruction[15– 0]
0
0
Mux
0
1
Add Addresult
RegistersWriteregister
Writedata
Readdata 1
Readdata 2
Readregister 1
Readregister 2
Signextend
Mux1
ALUresult
Zero
Writedata
Readdata M
ux
1
ALUcontrol
Shiftleft 2R
egW
r ite
MemRead
Control
ALU
Instruction[15– 11]
6
X
M
WB
M
WB
WBIF/ID
PCSrc
ID/EX
EX/MEM
MEM/WB
Mux
0
1
Mem
Wr it
e
AddressData
memory
Address
10
000
10E
EX: sub $11, $2, $3 MEM: lw $10, 9($1)ID: and $12, $4, $5
0
1
10
000
1100
IF: or $13, $6, $7
110
10
“and”
33
Chapter 3: Pipelining
Datapath with Control
PC
Instructionmemory
Inst
ruct
ion
Add
Instruction[20– 16]
Mem
toR
eg
ALUOp
Branch
RegDst
ALUSrc
4
16 32Instruction[15– 0]
0
0
Mux
0
1
Add Addresult
RegistersWriteregister
Writedata
Readdata 1
Readdata 2
Readregister 1
Readregister 2
Signextend
Mux1
ALUresult
Zero
Writedata
Readdata M
ux
1
ALUcontrol
Shiftleft 2R
egW
r ite
MemRead
Control
ALU
Instruction[15– 11]
6
X
M
WB
M
WB
WBIF/ID
PCSrc
ID/EX
EX/MEM
MEM/WB
Mux
0
1
Mem
Wr it
e
AddressData
memory
Address
10
000
10E
MEM: sub $11, .. WB: lw $10, 9($1)
EX: and $12, $4, $5
0
1
10
000
1100
ID: or $13, $6, $7
100
00
“or”
IF: add $14, $8, $9
1
1
34
Chapter 3: Pipelining
Datapath with Control
PC
Instructionmemory
Inst
ruct
ion
Add
Instruction[20– 16]
Mem
toR
eg
ALUOp
Branch
RegDst
ALUSrc
4
16 32Instruction[15– 0]
0
0
Mux
0
1
Add Addresult
RegistersWriteregister
Writedata
Readdata 1
Readdata 2
Readregister 1
Readregister 2
Signextend
Mux1
ALUresult
Zero
Writedata
Readdata M
ux
1
ALUcontrol
Shiftleft 2R
egW
r ite
MemRead
Control
ALU
Instruction[15– 11]
6
X
M
WB
M
WB
WBIF/ID
PCSrc
ID/EX
EX/MEM
MEM/WB
Mux
0
1
Mem
Wr it
e
AddressData
memory
Address
10
000
10E
WB: sub $11, ..MEM: and $12…
0
1
10
000
1100
EX: or $13, $6, $7
100
00
“add”
ID: add $14, $8, $9
1
0
IF: xxxx
35
Chapter 3: Pipelining
Datapath with Control
PC
Instructionmemory
Inst
ruct
ion
Add
Instruction[20– 16]
Mem
toR
eg
ALUOp
Branch
RegDst
ALUSrc
4
16 32Instruction[15– 0]
0
0
Mux
0
1
Add Addresult
RegistersWriteregister
Writedata
Readdata 1
Readdata 2
Readregister 1
Readregister 2
Signextend
Mux1
ALUresult
Zero
Writedata
Readdata M
ux
1
ALUcontrol
Shiftleft 2R
egW
r ite
MemRead
Control
ALU
Instruction[15– 11]
6
M
WB
WBIF/ID
PCSrc
EX/MEM
MEM/WB
Mux
0
1
Mem
Wr it
e
AddressData
memory
Address
10
000
10
WB: and $12…
0
1
MEM: or $13, ..
100
00
EX: add $14, $8, $9
1
0
IF: xxxx ID: xxxx
X
M
WB
ID/EX
E
36
Chapter 3: Pipelining
Datapath with ControlWB: or $13…
PC
Instructionmemory
Inst
ruct
ion
Add
Instruction[20– 16]
Mem
toR
eg
ALUOp
Branch
RegDst
ALUSrc
4
16 32Instruction[15– 0]
0
0
Mux
0
1
Add Addresult
RegistersWriteregister
Writedata
Readdata 1
Readdata 2
Readregister 1
Readregister 2
Signextend
Mux1
ALUresult
Zero
Writedata
Readdata M
ux
1
ALUcontrol
Shiftleft 2R
egW
r ite
MemRead
Control
ALU
Instruction[15– 11]
6
M
WB
WBIF/ID
PCSrc
EX/MEM
MEM/WB
Mux
0
1
Mem
Wr it
e
AddressData
memory
Address
MEM: add $14, ..
1000
0
EX: xxxx
1
0
IF: xxxx ID: xxxx
X
M
WB
ID/EX
E
37
Chapter 3: Pipelining
Datapath with Control
PC
Instructionmemory
Inst
ruct
ion
Add
Instruction[20– 16]
Mem
toR
eg
ALUOp
Branch
RegDst
ALUSrc
4
16 32Instruction[15– 0]
0
0
Mux
0
1
Add Addresult
RegistersWriteregister
Writedata
Readdata 1
Readdata 2
Readregister 1
Readregister 2
Signextend
Mux1
ALUresult
Zero
Writedata
Readdata M
ux
1
ALUcontrol
Shiftleft 2R
egW
r ite
MemRead
Control
ALU
Instruction[15– 11]
6
M
WB
WBIF/ID
PCSrc
EX/MEM
MEM/WB
Mux
0
1
Mem
Wr it
e
AddressData
memory
Address
WB: add $14..MEM: xxxxEX: xxxx
1
0
IF: xxxx ID: xxxx
X
M
WB
ID/EX
E