kernel recipes 2013 - overview display in the linux kernel
DESCRIPTION
The Linux kernel has adapted in recent years to cope with the arrival of the embedded platforms using more and more advanced graphics capabilities. This presentation will explore the past challenges, but also the present and future, focusing on the current situation, the problems we face and the solutions that are currently proposed and considered.TRANSCRIPT
![Page 2: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/2.jpg)
![Page 3: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/3.jpg)
Problem Definition
![Page 4: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/4.jpg)
display /
![Page 5: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/5.jpg)
/ graphics /
![Page 6: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/6.jpg)
/ video
![Page 7: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/7.jpg)
Problem - Purpose
display / graphics / video
![Page 8: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/8.jpg)
format
![Page 9: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/9.jpg)
memory / deep pipeline
![Page 10: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/10.jpg)
device / CPU
![Page 11: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/11.jpg)
Problem - Source
formatmemory / deep pipeline
device / CPU
![Page 12: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/12.jpg)
rotation /
![Page 13: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/13.jpg)
/ scaling /
![Page 14: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/14.jpg)
/ composing
![Page 15: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/15.jpg)
Problem - Processing
rotationscaling
composing
![Page 16: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/16.jpg)
X11 /
![Page 17: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/17.jpg)
/ Wayland /
![Page 18: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/18.jpg)
/ MIR /
![Page 19: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/19.jpg)
/ SDL /
![Page 20: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/20.jpg)
/ DirectFB /
![Page 21: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/21.jpg)
/ Raw API
![Page 22: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/22.jpg)
Problem - Stack
X11Wayland
SDLDirectFBRaw API
![Page 23: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/23.jpg)
DRM /
![Page 24: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/24.jpg)
/ FBDEV /
![Page 25: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/25.jpg)
/ V4L2
![Page 26: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/26.jpg)
Problem - API
DRMFBDEVV4L2
![Page 27: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/27.jpg)
Display – Scanout
Frame Buffer
![Page 28: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/28.jpg)
Display – Composition
CRTC
Plane(s)
Composition
![Page 29: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/29.jpg)
![Page 30: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/30.jpg)
![Page 31: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/31.jpg)
![Page 32: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/32.jpg)
![Page 33: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/33.jpg)
![Page 34: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/34.jpg)
Origins
![Page 35: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/35.jpg)
Origins – FBDEV
FBDEV, L
inux 1
.3.9
4
1996 2012
4CC F
orm
ats
2000
Blankin
g
![Page 36: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/36.jpg)
Origins – V4L2
V4L, L
inux 2
.2.0
1999 2002 2006
V4L2,
Linu
x 2.4
.0
V4L d
epre
cate
d
2008
V4L2
subd
ev
2011
Med
ia co
ntro
ller
video
buf2
2012
DMABUF
![Page 37: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/37.jpg)
Origins – DRM/KMS
2000
DRM, L
inux 2
.2.1
8
2009
KMS, T
TM
2008
GEMPlan
es, D
MABUF
2011 2012
Dumb
Buffe
rs
![Page 38: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/38.jpg)
Features
![Page 39: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/39.jpg)
Memory Management
DRM FB V4L2
Dynamic Allocation Yes No Yes
Multiple Buffers Yes panning Yes
Import dmabuf No userptr
Exportdmabufmmap
mmap mmap
![Page 40: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/40.jpg)
Mode Setting
DRM FB V4L2
Formats 4CCRGB4CC
4CC
Enumeration Planes No Yes
Negotiation No No Yes
Atomicity Yes No No
![Page 41: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/41.jpg)
Transformations
DRM FB V4L2
Overlays Yes No Yes
Rotation Yes No Yes
Scaling Yes No Yes
Cropping/Panning Yes Yes Yes
![Page 42: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/42.jpg)
Documentation
api core drivers
drmfbv4l2
![Page 43: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/43.jpg)
DRM API Documentation
The DRM core exports several interfaces to applications, generally intended to be used through corresponding libdrm wrapper functions. In addition, drivers export device-specific interfaces for use by userspace drivers & device-aware applications through ioctls and sysfs files.
External interfaces include: memory mapping, context management, DMA operations, AGP management, vblank control, fence management, memory management, and output management.
Cover generic ioctls and sysfs layout here. We only need high-level info, since man pages should cover the rest.
“
„
![Page 44: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/44.jpg)
Documentation
api core drivers
drmfbv4l2
![Page 45: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/45.jpg)
Code Size
api core drivers
drmfbv4l2
![Page 46: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/46.jpg)
Cumulative Changes - API
0
1000
2000
3000
4000
5000
6000
drm
fb
v4l2
drmfbv4l2
![Page 47: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/47.jpg)
Cumulative Changes - Core
0
5000
10000
15000
20000
25000
30000
35000
40000
45000
50000
drm
fb
v4l2
drmfbv4l2
![Page 48: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/48.jpg)
Cumulative Changes - Drivers
0
200000
400000
600000
800000
1000000
1200000
1400000
1600000
1800000
drm
fb
v4l2
drmfbv4l2
![Page 49: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/49.jpg)
Mailing List Traffic
0
200
400
600
800
1000
1200
1400
1600
1800
drmfbv4l2drm+bug
![Page 50: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/50.jpg)
Device Model – FBDEV
Frame Buffer(memory)
DisplayDevice
![Page 51: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/51.jpg)
Device Model – DRM/KMS
Frame Buffer(memory) CRTC Encoder Connector
Planes(memory)
Encoder
Connector
Connector
![Page 52: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/52.jpg)
Device Model – V4L2
Buffers(memory)
DisplayDevice
Connector
![Page 53: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/53.jpg)
Device Model – V4L2/MC
V4L2subdev
0
1
2V4L2
subdev0 1
V4L2queue
0
![Page 54: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/54.jpg)
Device Model – V4L2/MC
![Page 55: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/55.jpg)
Use Cases
![Page 56: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/56.jpg)
Use Cases - FBDEV
![Page 57: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/57.jpg)
Use Cases - FBDEV
(that's it...)
![Page 58: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/58.jpg)
Use Cases - V4L2
Video
![Page 59: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/59.jpg)
Use Cases – DRM/KMS
Everything else
![Page 60: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/60.jpg)
FB vs. DRM - sloccount
fbdev0
500
1000
1500
2000
2500
2370
![Page 61: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/61.jpg)
FB vs. DRM - sloccount
fbdev drm0
500
1000
1500
2000
2500
2370
1659
![Page 62: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/62.jpg)
DRM/KMS API
drmkmsexynosi810i915mganouveaur128radeonsavagesisvia
![Page 63: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/63.jpg)
KMS - Device Model
Frame Buffer(memory) CRTC Encoder Connector
Plane(memory)
SoCMemory Off-Chip
Encoder Connector
![Page 64: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/64.jpg)
KMS – Frame Buffer
CRTC
Frame Buffer
GEMObject(s)
Memory
● width● height● format● pitches● offsets
Properties
![Page 65: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/65.jpg)
DRM/KMS – GEM Object
CRTC
Frame Buffer
GEM Object
● width● height● bpp● pitch● size
Properties
Memory
![Page 66: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/66.jpg)
DRM – Handles
Process A
LocalHandle
GEMObject
Process B
Send FDSCM_RIGHTS
GlobalFD
1 2Global
FD4
LocalHandle
3
![Page 67: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/67.jpg)
ActiveArea
KMS – Modes (1/2)
sync
back porch
front porch
active area
![Page 68: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/68.jpg)
ActiveArea
KMS – Modes (2/2)
sync
back porch
front porch
active area
hdisplayhsync_start
htotal
hsync_end
![Page 69: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/69.jpg)
KMS – Mode Setting
crtc
fb
ActiveArea
x
y
mode.hdisplay
mo
de
. vdisp
l ay
*connectorsnum_connectors
mode
struct drm_mode_set { struct drm_framebuffer *fb; struct drm_crtc *crtc; struct drm_display_mode *mode; uint32_t x; uint32_t y; struct drm_connector **connectors; size_t num_connectors;};
![Page 70: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/70.jpg)
Source: http://www.flickr.com/photos/buckaroobay/3721809183/
![Page 71: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/71.jpg)
WIP – Display Framework
Common Display Framework
http://lwn.net/Articles/512363/
![Page 72: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/72.jpg)
Entity
Entity
Link Port
CDF - Entity Model
![Page 73: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/73.jpg)
CDF - Device Tree
hdmi_encoder { ports { #address-cells = <1>; #size-cells = <0>;
port@0 { hdmi_input: endpoint@0 { remote = <&display_output>; }; }; port@1 { endpoint@0 { ... }; endpoint@1 { ... }; }; };};
![Page 74: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/74.jpg)
Display Controller
VideoEntity
Pipeline Controller
controlcontrol
Entity Entity
CDF - Configuration Model
![Page 75: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/75.jpg)
Display Controller
VideoEntity
streaming control
streaming control
Entity Entity
CDF - Streaming Control
![Page 76: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/76.jpg)
CDF - Integration
DisplayController
DriverControl Bus API
FBDEV DRM
SPI/I2C Mem Map
Panel Driver
DT
pdata
DSI
![Page 78: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/78.jpg)
? !
![Page 79: Kernel Recipes 2013 - Overview display in the Linux kernel](https://reader033.vdocument.in/reader033/viewer/2022042601/554f85dcb4c9052a518b4ec3/html5/thumbnails/79.jpg)
merci.