simd image processor eric liskay andrew northy neraj kumar 1
TRANSCRIPT
![Page 1: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/1.jpg)
SIMD Image Processor
Eric LiskayAndrew NorthyNeraj Kumar
1
![Page 2: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/2.jpg)
Features
•SIMD architecture• Large array of cell processors operating in parallel• Each processor performs the same function on a kernel
- Kernel size is a 3x3 array of pixels
• Number of cell processors equal to (Tile Size in kernels)2
- For example, if the tile size is 10 kernels, each tile will be 30x30 pixels
•Convolution Mode• 3x3 pixel matrix read into the cell, only center pixel is written out• Used for minimum, maximum, average, and edge detection
operations
•User Inputs• User can specify parameters for different functions• User can specify a custom kernel for convolution 2
![Page 3: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/3.jpg)
Top Level Diagram
A
B
CellProcessor
x10
x10
C
Pixel Buffers A & B
Pixel Buffer C (Output)
900x24-bits
900x24-bits
A - Pixel Data
B - Pixel Data 900x24-bits
Opcode
X•Y Pixels
User Inputs
24-bit Color RAW Image
3
![Page 4: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/4.jpg)
Data Structures
Pixel
3x3 KernelTile
Red Channel
Green Channel
Blue Channel8 bits
4
![Page 5: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/5.jpg)
Image Processor Controller
•4 Processes
•Load images from files into pixel buffers
•Positive Edge of Clock: Send data to cell processors
•Negative Edge of Clock: Read image from cell processors into Pixel Buffer C
•Write Output to file when finished processing the image
![Page 6: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/6.jpg)
Cell Processor
•Each Cell processor operates on 3x3 pixel array, each pixel has 3 color channels
- Two image inputs
- Two user inputs
- Opcode input
- Kernel input (user’s filter)
- One image output
6
![Page 7: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/7.jpg)
Cell Functions
•Arithmetic and logical operations
- Add, subtract, multiply divide
- AND, OR, NOR, XOR
- Operands may be two images, or one image and user input
- Operation can be limited to one color channel or all channels
7
![Page 8: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/8.jpg)
Cell Functions
•Darken Highlights
- Compare sum of color channels with threshold value
- Decrease value from all channels by set amount if sum is larger than threshold
•Brighten Shadows
- Compare sum of color channels with threshold value
- Increases value from all channels by set amount if sum is smaller than threshold
8
![Page 9: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/9.jpg)
Cell Functions
•Convolution
- Sobel, Prewitt, and Robinson Edge Detection
- User specified Kernel
- 3x3 Minimum, Maximum, and Averaging Filters
9
![Page 10: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/10.jpg)
Edge Detection Using the Sobel Operator
• The Sobel operator uses two 3x3 kernels which are convolved with a 3x3 array of pixels from the image
• A is the 3x3 array of pixels from the image
• Gx and Gy are the derivatives of the convolutions
• is the convolution operator
• G is the gradient magnitude10
![Page 11: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/11.jpg)
The Opcodes
Regular Processing Mode(9 pixels in, 9 pixels out)
Mneumonic
Opcode
Description
add 0x00 add pixel channel values in array A to array B
addi 0x01 add userInputA to values in all channels from array A
addir 0x02 add userInputA to value in red channel from array A
addig 0x03 add userInputA to value in green channel from array A
addib 0x04 add userInputA to value in blue channel from array A
sub 0x05 subtract pixel channel values in array B from array A
subi 0x06 subtract userInputA from values in all channels from array A
subir 0x07 subtract userInputA from value in red channel from array A
subig 0x08 subtract userInputA from value in green channel from array A
subib 0x09 subtract userInputA from value in blue channel from array A
mult 0x0A multiply pixel channel values in array A by array B
mult2 0x0B multiply pixel channel values in array A by 2
multi 0x0C multiply pixel channel values in array A by shifting left by userInputA
div2 0x0D divide pixel channel values in array A by 2
divi 0x0E divide pixel channel values in array A by shifting right by userInputA
inv 0x0F Invert pixel channel values in array A
and 0x10 AND pixel channel values in Array A with pixels in array B
or 0x11 OR pixel channel values in Array A with pixels in array B
nor 0x12 NOR pixel channel values in Array A with pixels in array B
xor 0x13 XOR pixel channel values in Array A with pixels in array B
drkn 0x14 Darken Highlights: If sum of pixel channel values in array A are above userInputA, subtract userInputB
brtn 0x15 Brighten Shadows: If sum of pixel channel values in array A are below userInputA, add userInputB
grayr 0x16 Grayscale based on Red Channel
grayg 0x17 Grayscale based on Green Channel
grayb 0x18 Grayscale based on Blue Channel
pixflip 0x19 Pixels mirrored over middle pixel
11
![Page 12: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/12.jpg)
The Opcodes
Convolution Mode(9 pixels in, 1 pixels out)
Mneumonic
Opcode
Description
sobel 0x1A Sobel
sobel2 0x1B Sobel Method 2
prewitt 0x1C Prewitt
robn 0x1D Robinson North
robnw 0x1E Robinson Northwest
robne 0x1F Robinson Northeast
robe 0x20 Robinson East
robw 0x21 Robinson West
robs 0x22 Robinson South
robse 0x23 Robinson Southeast
robsw 0x24 Robinson Southwest
avggry 0x25 3x3 pixel average in grayscale
avgclr 0x26 3x3 pixel average in color
ukernel 0x27 User programmable filter
min 0x28 3x3 pixel minimum
max 0x29 3x3 pixel maximum
12
![Page 13: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/13.jpg)
Color Codes
![Page 14: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/14.jpg)
Regular Processing Mode
14
![Page 15: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/15.jpg)
Red ImageoutputC(x,y)(0) <= B"11111111”;outputC(x,y)(1) <= inputA(x,y)(1);outputC(x,y)(2) <= inputA(x,y)(2);
Original Processed
15
![Page 16: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/16.jpg)
Red ImageoutputC(x,y)(0) <= inputA(x,y)(0);
outputC(x,y)(1) <= B"11111111"; outputC(x,y)(2) <= inputA(x,y)(2);
Original Processed
16
![Page 17: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/17.jpg)
Red ImageoutputC(x,y)(0) <= inputA(x,y)(0); outputC(x,y)(1) <= inputA(x,y)(1); outputC(x,y)(2) <= B"11111111";
Original Processed
17
![Page 18: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/18.jpg)
Red = 0
Original Processed
18
![Page 19: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/19.jpg)
Green = 0
Original Processed
19
![Page 20: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/20.jpg)
Blue = 0
Original Processed
20
![Page 21: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/21.jpg)
Add: Image A + Image B
Original Processed
21
![Page 22: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/22.jpg)
Add Immediate: Image A + User Input A
Original Processed
22
![Page 23: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/23.jpg)
Add Immediate Red: Image A(red channel) + User Input A
Original Processed
23
![Page 24: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/24.jpg)
Add Immediate Green: Image A(green channel) + User Input A
Original Processed
24
![Page 25: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/25.jpg)
Add Immediate Blue: Image A(blue channel) + User Input A
Original Processed
25
![Page 26: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/26.jpg)
Subtract: Image A - Image B
Original Processed
26
![Page 27: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/27.jpg)
Subtract Immediate: Image A - User Input A
Original Processed
27
![Page 28: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/28.jpg)
Subtract Immediate Red: Image A(red channel) - User Input A
Original Processed
28
![Page 29: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/29.jpg)
Subtract Immediate Green: Image A(green channel) - User Input A
Original Processed
29
![Page 30: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/30.jpg)
Subtract Immediate Blue: Image A(blue channel) - User Input A
Original Processed
30
![Page 31: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/31.jpg)
Original Processed
Multiply: Image A * Image B 31
![Page 32: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/32.jpg)
Multiply by 2: Image A * 2
Original Processed
32
![Page 33: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/33.jpg)
Multiply Immediate: Image A * User Input A
Original Processed
33
![Page 34: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/34.jpg)
Divide by 2: Image A / 2
Original Processed
34
![Page 35: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/35.jpg)
Divide Immediate: Image A >> User Input A
Original Processed
35
![Page 36: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/36.jpg)
Invert: Image A XOR “11111111”
Original Processed
36
![Page 37: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/37.jpg)
AND: Image A AND Image B
Original Processed
37
![Page 38: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/38.jpg)
OR: Image A OR Image B
Original Processed
38
![Page 39: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/39.jpg)
NOR: Image A NOR Image B
Original Processed
39
![Page 40: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/40.jpg)
XOR: Image A XOR Image B
Original Processed
40
![Page 41: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/41.jpg)
Darken Highlights: Decreases brightness by (User Input B) if average value of the
pixels is above User Input A
Original Processed
41
![Page 42: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/42.jpg)
Brighten Shadows: Increases brightness by (User Input B) if average value of the
pixels is below User Input A
Original Processed
42
![Page 43: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/43.jpg)
Convert to Grayscale using Red Channel
Original Processed
43
![Page 44: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/44.jpg)
Convert to Grayscale using Green Channel
Original Processed
44
![Page 45: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/45.jpg)
Convert to Grayscale using Blue Channel
Original Processed
45
![Page 46: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/46.jpg)
Pixel Flip
Original Processed
46
![Page 47: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/47.jpg)
Convolution Mode
47
![Page 48: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/48.jpg)
Sobel48
![Page 49: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/49.jpg)
Sobel Method 249
![Page 50: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/50.jpg)
Prewitt50
![Page 51: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/51.jpg)
Robinson North51
![Page 52: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/52.jpg)
Robinson Northwest52
![Page 53: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/53.jpg)
Robinson Northeast53
![Page 54: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/54.jpg)
Robinson East54
![Page 55: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/55.jpg)
Robinson West55
![Page 56: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/56.jpg)
Robinson South56
![Page 57: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/57.jpg)
Robinson Southeast57
![Page 58: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/58.jpg)
Robinson Southwest58
![Page 59: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/59.jpg)
3x3 Pixel Average Grayscale 59
![Page 60: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/60.jpg)
3x3 Pixel Average Color 60
![Page 61: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/61.jpg)
User Programmable Filter 61
![Page 62: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/62.jpg)
3x3 Pixel Minimum
Original Processed
62
![Page 63: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/63.jpg)
3x3 Pixel Maximum
Original Processed
63
![Page 64: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/64.jpg)
• Make Image Processor synthesizable
• Move loading files into testbench(HVL) and separate from image processor and Cells(HDL)
• Veloce?
• Enable the processing of larger images by loading and writing only part of the image at a time
• Other convolution ideas(e.g. Noise Reduction)
• Parameterized Kernel Size to enable larger kernels
• Gradient Threshold on “Darken Highlights” and “Brighten Shadows” for a less abrupt transition
• Multiple Operations done on image before writing it
Future Work
64
![Page 65: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/65.jpg)
Horizontal Flip
Original Processed
65
![Page 66: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/66.jpg)
Vertical Flip
Original Processed
66
![Page 67: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/67.jpg)
Rotate 180° Clockwise
Original Processed
67
![Page 68: SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1](https://reader038.vdocument.in/reader038/viewer/2022110102/56649f155503460f94c2a777/html5/thumbnails/68.jpg)
• All Photographs © Eric Liskay
• Lecture 10. Edge detection. Sobel and similar filters. Convolution• http://web.cecs.pdx.edu/~mperkows/CLASS_VHDL_99/2012/2012-lecture010_edge-detection-SobelG.ppt
• Wikipedia - Sobel Operator• http://en.wikipedia.org/wiki/Sobel_operator
References
68