tizen v2.3 graphics & ui frameworks
DESCRIPTION
Contents EFL(Enlightenment Foundation Library) Window Systems EFL Features Ecore Evas Edje Elementary Cserve2 Window Systems X Window System Wayland DALi 3D EngineTRANSCRIPT
Embedded Software Lab. @ SKKU
105
1
Tizen v2.3 Graphics & UI Frameworks
Embedded Software Lab. @ SKKU
105
2
• EFL(Enlightenment Foundation Library)– EFL Features– Ecore– Evas– Edje– Elementary– Cserve2
• Window Systems– X Window System– Wayland
• DALi 3D Engine
Contents
Embedded Software Lab. @ SKKU
105
3
• UI Frameworks– Modules to implement interactive GUI– EFL(Elementary/Ecore), OSP, GTK, Qt– Input Service Framework(ISF), Voice Framework
• Graphics Libraries– Modules to render graphic objects– Evas, Cairo, OpenGL ES, EGL
• Window Systems– Modules to manage windows and frame buffers– X Window System, Wayland(Tizen v3.0)
Tizen v2.3 Graphics & UI Frameworks
Embedded Software Lab. @ SKKU
105
4Tizen v2.3 Graphics & UI Frameworks
• Frameworks for UI of core applications & web applications Deprecated
(Bada App FW)
Embedded Software Lab. @ SKKU
105
5
EFL(Enlightenment Foundation
Library)
Embedded Software Lab. @ SKKU
105
6
• Started from a desktop window manager– Default window manager of Bodhi Linux
Enlightenment
Embedded Software Lab. @ SKKU
105
7
• A set of libraries supporting Enlightenment– Enlightenment 0.17(E17)
• Components are divided by functions– Event loop: Ecore– Rendering engine: Evas– UI: Edje, Elementary, …– UI builder tools: Enventor, Eflete, ELM Theme Editor
• EFL in Present– Core components of Tizen Graphics/UI framework– 26% of code is E17, the other is EFL
Enlightenment Foundation Library
Embedded Software Lab. @ SKKU
105
8EFL in Tizen v2.3
LegendApplications EFL & fontsWindow System
H/W
Linux Kernel Frame Buffer Input Devices
H/W
fontconfigfreetype2 OpenGL ES/EGLEet
EvasEcore
Eina
Edje
Elementary
Core Applications
X Window System
InputService
Framework&
Voice Framework
kernel / Frame Buffer/ Input Devices
Graphics
ISF& Voice FW
Cairo
Embedded Software Lab. @ SKKU
105
9
• Immediate Mode Graphics System– Application directly re-paints its portion that is invalidated– Application re-issues all drawing commands to describe the
entire scene each time a new frame is required– GTK+ 3, GDI, GDI+
Immediate vs. Retained Mode (1/2)
Application
Window System
GTK+/GDK-X11
1. Invalidate
2. Expose
Graphics CardFrame Buffer for Screen
3. Drawing
Embedded Software Lab. @ SKKU
105
10
• Retained Mode Graphics System– The graphics system is responsible for rendering the
drawing objects given by application.– Application passes drawing objects to graphics
system.– Evas, GTK+ 4(Clutter), WPF
Immediate vs. Retained Mode (2/2)
ApplicationEvas objects
Evas Rendering Engine(Graphics system)
Expose
Graphics CardFramebuffer for screen
X Window System
Embedded Software Lab. @ SKKU
105
11
• High performance– Retained mode graphics system– 60fps+ on general smartphones
• Fit for Low-end Hardware– Small memory, no swap, no GPUs– OpenGL is not worth for all cases
• Texture keeps speed, but loses memory
• Support for HW rendering– OpenGL ES– SIMD vector(MMX, Neon)
Why is EFL used on Tizen? (1/3)
Desktop Environment
Memory Used (MiB)
Enlightenment 0.18.8
83.8
LXDE 0.5.5 87.0XFCE 4.10.2 110.0LXQt 0.7.0 113.0MATE 1.8.1 123.0Cinnamon 2.2.13
176.3
GNOME3 3.12.2
245.3
KDE 4.13.1 302.6Unity 7.2.0.14 312.5
Embedded Software Lab. @ SKKU
105
12
• Various back-end engine– X11 (~Tizen 2.2.1), Wayland (Tizen 3.0~)– Direct Frame Buffer, DRM, Memory buffers, PS3
Native, Windows (GDI & DirectDraw), …• UI Scalability– Various resolutions (WVGA ~ XQXGA) with the
same layout and resources
Why is EFL used on Tizen? (2/3)
0.6 0.8 1.0Scale factor:
Embedded Software Lab. @ SKKU
105
13
• GUI, Logic separation• Themes– A preset package containing
graphical appearance details• Bindings– Basically C-based– EFL is also binded to
Javascript, C++, Python, Ruby, Perl …
– Eo: Generic object model
Why is EFL used on Tizen? (3/3)
Embedded Software Lab. @ SKKU
105
14
• Entry barrier– Unfamiliar development style– Lack of documentations, developers
• No object-oriented implementation– C-based– Eo project is ongoing
• Incomplete C++ support– Eolian based E++ project ongoing– Eolian is EO object parser and C code generator
• No thread safe
EFL Shortcomings
Embedded Software Lab. @ SKKU
105
15Tizen v2.3 Rendering Path
Textures
OpenGL ES win
X window server
Frame Buffer
Videowin
EFL win
3D App
OpenGL ES/EGL
Video App.
EFL App.
EFLEVAS
CompositeWindowManager
OpenGL ES/EGL
OpenGL ES/EGL
MM FWEFL
Rendering pipeline path; rendering data flow
Legend
3D App. Multimedia App.EFL App. Configurable Enable/disableTexture from
Pixmap
OpenGL ES/EGLProcess Texture
Mem.
Embedded Software Lab. @ SKKU
105
16Tizen v2.3 Rendering Path on SW Backend
ElementaryElementary
EcoreEcore
xlibxlib
Framebuffer
LCDcontroller
60Hz
Update window areasX ServerX Server
X share
d Mem.
EdjeEdje
EvasEvasEvas software backendEvas software backendEcore XEcore X
Embedded Software Lab. @ SKKU
105
17Tizen v2.3 Rendering Path on GLES Backend
EGL for XEGL for X
xlibxlib
Framebuffer
/dev/fb1
LCDcontroller
60Hz
Update window areaX ServerX Server
frontbackDouble buffer
OpenGL ES H/W
OpenGL ESOpenGL ES
contr
olled
by
EGL
EcoreEcore
ElementaryElementary
EdjeEdjeEvasEvas
Evas OpenGL ES backendEvas OpenGL ES backend Ecore XEcore X
Embedded Software Lab. @ SKKU
105
18
• Ecore (E + Core)– Event loop library with convenience modules
• Evas (E + canVAS)– Canvas & rendering library
• Edje– Complex graphical design & layout library
• Elementary– Widget set
EFL Components: Core
Embedded Software Lab. @ SKKU
105
19
• Eo– Generic object model
for binding other languages
• Eina– Library for data types
• Eet– Data codec and storage
• Efreet– Freedesktop.org
standards support
EFL Components: Others
• Edbus– Wrapper of Dbus
• Emotion– Video/Audio Playback
library• Eio
– Asynchronous I/O• Eeze
– Udev hardware detection
• Ethumb– Thumbnailer & cacher
Embedded Software Lab. @ SKKU
105
20
The core loop of EFL applications
EFL: Ecore
Embedded Software Lab. @ SKKU
105
21
• The core loop of EFL application, event & marshaling library– Provides glue to other subsystems– Supports various display system
• X11, FB, Wayland, Win32, WinCE, etc.• Features:– Networking & IPC– Threading & inter-thread communication– Evas input feeding and output driving– More...
Ecore
Embedded Software Lab. @ SKKU
105
22
• elm_main()– Main function of EFL application– After initialization, enter into event loop.
Simple EFL Application
Start Ecore main loop!
Elementary UI configuration(Elementary Window)
On exiting window, it also exits Ecore event loop
Finalization process like freeing resource
Embedded Software Lab. @ SKKU
105
23
• ecore_main_loop_begin()– The wrapper of Ecore main loop– Ecore main loop and Glib main loop can be chosen.
Ecore Main Loop (1/2)
Ecore main loop
Glib main loop
Embedded Software Lab. @ SKKU
105
24
• _ecore_main_loop_iterate_internal()– The implementation of Ecore main loop– It starts all the timer handler, event handler,
rendering in an application.
Ecore Main Loop (2/2)
Embedded Software Lab. @ SKKU
105
25
• Key down event
Ecore: Usage of Ecore Events
Ecore_Event_Handler *handler;handler = ecore_event_handler_add(ECORE_EVENT_KEY_DOWN, func, data);
Eina_Bool func(void *data, int type, void *event_info){ if (…) return ECORE_CALLBACK_DONE; return ECORE_CALLBACK_PASS_ON;}
Ecore_Timer *timer;double time;timer = ecore_timer_add(time, func, data);Eina_Bool func(void *data){ if (…) return ECORE_CALLBACK_RENEW; return ECORE_CALLBACK_CANCEL;}
• Timer event
Embedded Software Lab. @ SKKU
105
26
Canvas & Rendering Library
EFL: Evas
Embedded Software Lab. @ SKKU
105
27
• Canvas & Rendering Library• Features– Retained mode rendering
• Controls rendering behavior in the unit of primitive objects and smart objects.
– Scene graph• Tracks the status of objects → minimizes rendering
behavior.– Multiple output paths
• Pure software/OpenGL ES– Rendering optimization
• 3D H/W Acceleration, SIMD vector operations– Provides GL Glue Layer
Evas
Embedded Software Lab. @ SKKU
105
28
• Evas uses retained mode rendering– Application sends Evas objects to Evas.– Evas does rendering based on Evas objects.
Evas: Retained Mode Rendering
ApplicationEvas objects
Evas Rendering Engine(Graphics system)
Expose
Graphics CardFramebuffer for screen
X Window System
Embedded Software Lab. @ SKKU
105
29
• All Objects Displayed on Canvas– Controlled by Evas_Object handles.
• Loading fonts and images, rendering glyphs and images, scaling, blending, 3D transformation etc.
– Evas manages all Evas objects equally.• Type of Evas Object– Primitive objects: minimum unit of Evas rendering
• Rectangle, Line, Polygon, Text, Textblock, Textgrid, Image
– Smart objects• Box, Grid, Table, Customized smart objects(ELM Widgets)
Evas Objects
Embedded Software Lab. @ SKKU
105
30Evas Primitive Objects
• Rendering units of Evas
Embedded Software Lab. @ SKKU
105
31
• Evas objects composed of primitive objects– Acts as a group.– Evas object generic functions also can be used.
Evas Smart Objects
Embedded Software Lab. @ SKKU
105
32Image Native Surface
Hello World!
Evas_Object_Rect
Evas_Object_Text
Evas_Object_Image
…
GL Rendering Engine
SW Rendering Engine
X11 Rendering Engine
X11 Output
BufferOutput
…
SDLOutput
…
Evas Runtime
OpenGL App(surface_a
ie. Pixmap or texture)
Evas_Object_Image(img_obj_a)
evas_object_image_native_surface_set(img_obj_a, surface_a)
Output
Hello World!
Evas Engine Module
Embedded Software Lab. @ SKKU
105
33
• Scene graph– A structure that arranges the logical
representation of a graphical scene– General data structure in GUI applications
• Scene graph in Evas– Manages Evas objects in graph data structure.– Tracks all the objects’ status and gets changed
area in the next frame → Minimizes rendering area.
Evas: Scene Graph
Embedded Software Lab. @ SKKU
105
34
• Start Here
Evas: Minimizing Rendering Region
Embedded Software Lab. @ SKKU
105
35
• Next frame is…
Evas: Minimizing Rendering Region
Embedded Software Lab. @ SKKU
105
36
• Calculate actual update region deltas
Evas: Minimizing Rendering Region
Embedded Software Lab. @ SKKU
105
37
• Only draw updated regions
Evas: Minimizing Rendering Region
Embedded Software Lab. @ SKKU
105
38
/* Build by $ gcc evas_rectangle.c -o evas_rectangle `pkg-config --cflags --libs ecore ecore-evas` */
#include <Ecore.h>#include <Ecore_Evas.h>
int main(int argc, char **argv) { Ecore_Evas *ee; Evas *evas; Evas_Object *bg; if(ecore_evas_init() <= 0) return 1; ee = ecore_evas_new(NULL, 0, 0, 200, 200, NULL); ecore_evas_show(ee);
evas = ecore_evas_get(ee); // Get the Ecore_Evas's Evas object. bg = evas_object_rectangle_add(evas); // Add a rectangle to the given Evas object. evas_object_color_set(bg, 255, 255, 255, 255); // Set the color of the given Evas object to the given one. evas_object_move(bg, 0, 0); // Move the given Evas object to the given location inside its canvas' viewport. evas_object_resize(bg, 200, 200); // Change the size of the given Evas object. evas_object_show(bg); // Make the given Evas object visible.
ecore_main_loop_begin(); ecore_evas_free(ee); ecore_evas_shutdown();
return 0;}
Evas: Rectangle Example
Embedded Software Lab. @ SKKU
105
39
/* Build by $ gcc evas_image.c -o evas_image `pkg-config --cflags --libs ecore ecore-evas` */
#include <Ecore.h>#include <Ecore_Evas.h>
int main(int argc, char **argv) { Ecore_Evas *ee; Evas *evas; Evas_Object *bg; if(ecore_evas_init() <= 0) return 1; ee = ecore_evas_new(NULL, 0, 0, 200, 200, NULL); ecore_evas_show(ee);
evas = ecore_evas_get(ee); // Get the Ecore_Evas's Evas object. bg = evas_object_image_filled_add(evas); // Add an image to the given Evas object. evas_object_image_file_set(img, “test.png”, NULL); // Set the image file path of the given Evas object. evas_object_move(img, 0, 0); // Move the given Evas object to the given location inside its canvas' viewport. evas_object_resize(img, 200, 200); // Change the size of the given Evas object. evas_object_show(img); // Make the given Evas object visible.
ecore_main_loop_begin(); ecore_evas_free(ee); ecore_evas_shutdown();
return 0;}
Evas: Image Example
Embedded Software Lab. @ SKKU
105
40
Complex graphical design & layout library
EFL: Edje
Embedded Software Lab. @ SKKU
105
41
• Complex graphical design & layout library• EDC script
– GUI layout description separated from application code– Can be changed at or during runtime– Compiled as binary code(edj)
Edje
• Description about:– Images, colors, positioning,
animations, behaviors, …
Embedded Software Lab. @ SKKU
105
42
• Separation of layout and logic– Graphical part: GUI
Layout binary (edj)– Functionality: Executable
logic binary (C)• Utility– edje_cc : compile edc to
edj– edje_decc : de-compile
edj to edc
Edje: Edc & Edj
Embedded Software Lab. @ SKKU
105
43Edje: Relation with Evas, Ecore
Embedded Software Lab. @ SKKU
105
44Edje: EDC Script
Embedded Software Lab. @ SKKU
105
45Edje: EDC Script
Embedded Software Lab. @ SKKU
105
46Edje: Simple Example
part {name: “background”;type: RECT;mouse_events: 0;description {
state: “default” 0.0;color: 255 255 255 255;
}}
part {name: “blue_rect”;type: RECT;mouse_events: 1;description {
state: “default” 0.0;color: 0 0 255 255;rel1 {
relative: 0.1 0.1;}rel2 {
relative: 0.3 0.3;}
}}
Embedded Software Lab. @ SKKU
105
47Edje: Simple Examplepart {
name: “red_rect”;type: RECT;mouse_events: 0;description {
state: “default” 0.0;color: 255 0 0 255;rel1 {
to: “blue_rect”;relative: 0 2;
}rel2 {
to: “blue_rect”;relative: 1 3;
}}
description {state: “down” 0.0;inherit: “default” 0.0;rel1.relative: 3 2;rel2.relative: 4 3;color: 255 0 0 0;
}}
Embedded Software Lab. @ SKKU
105
48Edje: Simple Example
program {name: “blue_rect.clicked”;signal: “mouse,clicked,1”;source: “blue_rect”;action: STATE_SET “down” 0.0;transition: ACCELERATE 0.5;target: “red_rect”;after: “red_rect.restore”;
}program {
name: “red_rect.restore”;action: STATE_SET “default” 0.0;target: “red_rect”;
}
Embedded Software Lab. @ SKKU
105
49
A set of widgets
EFL: Elementary
Embedded Software Lab. @ SKKU
105
50
• A Set of Widgets– 약 80 여 개– Containers: Box, Conformant, Grid, Layout, Panes, Scroller, Table,
…– Non-Containers: Bg, Button, Check, Entry, Label, List, Icon, Win, ...
• Features:– Fast, Finger friendly, Scalability, Themes– Widget hierarchy, Dynamic language change
Elementary
< Scaled and resized relative to the screen> < Same view but different themes>
Embedded Software Lab. @ SKKU
105
51
• Various Widgets
Elementary Widgets
clock
colorselector
entry
frame
flipselector check
diskselector
actionslider
bg
index
map
menuphoto, photocam
panel
toolbar
radio
bubble
slider
ctxpopup
notify
separator
segment control
multibuttonentrypanes glview
fileselector
Labelthumb
Embedded Software Lab. @ SKKU
105
52
• Naviframe– Switching multiple
pages with stack• Toolbar– List of Items
Elementary Widgets
Embedded Software Lab. @ SKKU
105
53
• Entry– Text input field– Copy & paste,
various modes (password, multi-line, ...)
• Genlist– Generic List– Flexible but
complex to use
Elementary Widgets
Embedded Software Lab. @ SKKU
105
54
• Widgets are built in a hierarchical fashion.
• Widget shares it’s functions(APIs) with child widget.
Elementary: Widget Hierarchy
Embedded Software Lab. @ SKKU
105
55
• UI Scalability– Relative Positioning
Elementary: Scalability
– Scale Factor
Embedded Software Lab. @ SKKU
105
56
• Theme can be changed in same layout
Elementary: Theme
Embedded Software Lab. @ SKKU
105
57
• Elementary provides utility APIs for multi language support
Elementary: Dynamic Language Change
Embedded Software Lab. @ SKKU
105
58Elementary: Dynamic Language Change
Embedded Software Lab. @ SKKU
105
59
• Enventor: Dynamic EDC Editor• Eflete: EFL Edje Theme Editor• ELM Theme Viewer: Elementary
Theme Viewer
GUI Tools for Customizing Layout
Embedded Software Lab. @ SKKU
105
60
Optimization of EFL
Cserve2
Embedded Software Lab. @ SKKU
105
61
• Duplicated Resource Problem in EFL– Most images and fonts used in a theme are
used by multiple applications at the same time– EFL contains a variety of caches and
• Images, fonts, ...• Objectives of Cserve2:– Reduce memory consumption by sharing– Optimize load time by pre-loading data
Cserve2: Motivation
Embedded Software Lab. @ SKKU
105
62
• Sharing image and font data between EFL application processes– Started by ProFusion at 2012
• Cache server– Loads images and fonts– Shares resources across applications
• EFL applications– Connect to the server via UNIX socket– Send requests and wait for answers– Don't load anything on their own– Keep local cache pointing to shared data
Cserve2
Embedded Software Lab. @ SKKU
105
63Cserve2
Embedded Software Lab. @ SKKU
105
64
• Problems on Cserve2– IPC overheads:
• messages over socket link• context switches, read, write, ...
– Clients do not know the server's cached data before sending a request
• Solution– Expose indexes of shared objects to all clients
• via /dev/shm– Clients scan the tables and don't wait for the
server's response when the data is already loaded
Cserve2: Shared Indexing
Embedded Software Lab. @ SKKU
105
65
• 3 types of objects:– Data (image data)
• Stored in separate shm files: “img”– Fixed-size arrays of objects
• Indexes• Stored in “array” files
– Variable-size pools of objects• So-called “mempools”• Indexed by offset + length in a separate array
Cserve2: Shared Indexing
Embedded Software Lab. @ SKKU
105
66
• Index tables principles– Arrays of simple structures of a single type– Contain:
• ID• Refcount• Other fixed-size data (int, bool, ...)
Cserve2: Shared Indexing
Embedded Software Lab. @ SKKU
105
67
• Strings– Two shm files are used: index and mempool– Mempool contains the strings actual data– Index table contains:
• String ID• Refcount• Offset (in the mempool)• Length
• So, we can read a string just knowing its string ID
Cserve2: Shared Indexing
Embedded Software Lab. @ SKKU
105
68
• Cserve2 combines image cache and scale cache
• Reuse scale cache logic– Minimizes the number of cached scale images.– Scaling process is done on server side.
Cserve2: Image Scaling
Embedded Software Lab. @ SKKU
105
69
• Cserve2 image loader crash– No crash on client side (just no data)– New slave spawned
• Cserve2 server crash– enlightenment_start restarts cserve2– Clients reconnect to new server
• Discard some data• Keep old image references (no need to load)
• Cserve2 shutdown or restart– cleanup /dev/shm/
Cserve2: Robustness
Embedded Software Lab. @ SKKU
105
70
• Real Phone Usage– Workload & Environment
• 8 threads of “elementary_test -to launcher” on 1280x720 screen– Font size
• ~24x16 (small), ~32x24 (normal), ~40x30 (large)• In case of normal size, 1 font, 50 characters:
1x32x24x50=37.5KB– Icon size
• 50x50 (small), 100x100 (large) in 32-bit color• For one icon, 4Bx50x50=10KB (small), 4Bx100x100=40KB
(large)– Border size
• width 10px, length 720px, 32-bit color, 2x• 4Bx10x720x2=56KB
Cserve2: Benefits
Embedded Software Lab. @ SKKU
105
71
• Overhead– When workload runs with Cserved2 turned on:
72176KB• Size of /dev/shm: 4376KB
– When Cserve2 quits: 66716KB– Total overhead: 72176KB–66716KB–4376KB=1084KB
• Conclusion:– Cserve2 saves memory as soon as 1MB of data can be
shared(fonts & images).• Refer to
“Cserve2: Shared cache subsystem of Evas”.
Cserve2: Overhead
Embedded Software Lab. @ SKKU
105
72
Tizen v2.3 Window Systems:
X Window Systems
Embedded Software Lab. @ SKKU
105
73
• History– X window system is made by MIT in 1984.– X11: the current protocol version, established in 1987.– X.org server: the current reference implementation
developed by the X.Org Foundation• Designed to be used over network
connections• Features
– Event Driven– Network Transparent– Device Independent– Policy Independent– Extensible
X Window System
Embedded Software Lab. @ SKKU
105
74
• Client-server Model– X client is linked with X server
through the X protocol translated by Xlib.
– X client• Sends graphical request• Gets user input.• Ecore and Evas in Tizen
– X server• Accepts request for graphical
output• Sends back user input
X Window System: Architecture (1)
Embedded Software Lab. @ SKKU
105
75X Window System: Architecture (2)• Client-server Architecture of X System
Embedded Software Lab. @ SKKU
105
76X Protocol
• Protocol about communication with X clients and the X server
• Features:– Object-based– Asynchronous
• Types of Messages:– Request (Client → Server)– Reply– Event (Server → Client)– Error
Embedded Software Lab. @ SKKU
105
77X Server
• Handles Output & Input Processes• Manages Screens and Input Devices• Layered Structure
– DIX Layer, DDX Layer, OS Layer, Extension Layer• Resource
– Everything in X Server controlled through “resources”
– Created on demand– Used by clients to request operations– Screens, windows, graphic contexts, pixmaps,
regions, colormaps, fonts, cursors
Embedded Software Lab. @ SKKU
105
78X Server
• Layered Structure:– DIX Layer, DDX Layer, OS Layer, Extension Layer
Embedded Software Lab. @ SKKU
105
79
• DIX(Device Independent X) Layer– Interface between X Clients and
DDX/extensions• DDX(Device Dependent X) Layer
– Deals with graphic and input hardware• OS Layer
– Client connection management, work scheduling, memory allocation routines
• Extension Layer– Enables to get additional functionalities for X
server
X Server
Embedded Software Lab. @ SKKU
105
80DIX Layer
• Interface among DDX/Extensions and X Clients– Delivers requests from clients to
DDX/extensions– Sends input events to clients
• Dispatch loop– DIX Layer manages requests with dispatch loop
• Screens– One for each physical screen– Defines functions used to handle operations on
resources– Main communication channel DIX-DDX
Embedded Software Lab. @ SKKU
105
81DDX Layer
• A layer dealing with graphic and input HW
• Graphics Output• Input Processing– Mouse movement– Key presses– Key mappings
Embedded Software Lab. @ SKKU
105
82DDX Layer: Graphics Output
• Pixel-based• Screen basic data structure– Manage all resources
• Output done on drawables– Pixel data format defined by DDX– Pixmap format constant across screens
• XYPixmap -> one bitmap for each plane• Zpixmap -> series of pixel values
– Pixel values interpreted at display time– Windows, Pixmaps, …
82
Embedded Software Lab. @ SKKU
105
83DDX Drawables
• Windows– Visible rectangle on screen– Border drawn by DDX
• Create, move, resize– Contents drawn by client
• Server may provide backing store• Bit gravity (resizing)
– Clipped by parent and sibbling windows• Pixmaps– Rectangular array of pixel values– Reference counted
Embedded Software Lab. @ SKKU
105
84OS Layer
• Client Monnection Management• Work Scheduling– It schedules work to be done for clients– Ignore/AttendClient()
• Memory Allocation Routines– Xalloc()/Xrealloc()/Xfree()
Embedded Software Lab. @ SKKU
105
85Porting X Server
• Code layers– DIX: Dev.Independent, don’t touch– OS: OS specific– DDX: Graphics Device specific– Extensions: Add features
• Porting Layer: OS + DDX• Porting process:– Define functions required by DIX to handle
resources
Embedded Software Lab. @ SKKU
105
86
Tizen v3.0 Window Systems:
Wayland
Embedded Software Lab. @ SKKU
105
87
• Display management architecture & protocol– Window management and composition– Weston: main open source implementation– Maintained by Freedesktop.org– Adopted by Tizen 3.0
• Replacement for X– No rendering API in Wayland
• Rendering has already been supported by Evas, Cairo, OpenGL ES
– SHM(Shared memory) protocol: support for SW rendering– Less IPC: single process architecture
Wayland
Embedded Software Lab. @ SKKU
105
88
• Wayland Protocol– Weston compositor interacts with Wayland
clients(applications) via Wayland protocol.
Wayland Protocol
Embedded Software Lab. @ SKKU
105
89
• EFL and E17(X compositor) are linked to X.org(X server) via X protocol
Tizen v2.3 Graphic Stack with X.org
Embedded Software Lab. @ SKKU
105
90
• EFL is linked to Weston/E17 compositor via Wayland protocols
Tizen v3.0 Graphic Stack with Wayland
Embedded Software Lab. @ SKKU
105
91
• Weston provides surface to be rendered to Evas
• Weston sends input events to Ecore
Wayland with EFL
Embedded Software Lab. @ SKKU
105
92
• EFL controls display and input devices through several backends.– Ecore_Evas: Glue to the backends
Backends of EFL
• X Server Backend– Software_X11, OpenGL_X11
• Wayland Backend– Wayland_SHM: SW rendering– Wayland_EGL: HW rendering– Ecore_Wayland
• Surface Handling and Input
Embedded Software Lab. @ SKKU
105
93
• Performance of Wayland– 60 FPS(Frames Per Second)– 16ms for one frame from client to compositor
• Visualization Tool: E-Graph– A tool to visualize log information and draw
FPS curve– https://gitorious.org/e-graph/e-graph
Wayland Optimization
Embedded Software Lab. @ SKKU
105
94
• Problem– Rendering when resource is blocked by waiting
VSync• Triple Buffering– Add one additional buffer for client and compositor– Before: ~40fps → After: ~48fps– Time spend on composition: 13ms
Wayland Optimization
Embedded Software Lab. @ SKKU
105
95
• Problem– Redraw overlapped surface → heavy load
during composition• Opaque Region– Overlapped surface, set by Ecore and sent to
Weston– Compositing time: ~13ms → ~5ms– FPS: 40fps → 60fps
Wayland Optimization
Embedded Software Lab. @ SKKU
105
96
DALi 3D Engine
Embedded Software Lab. @ SKKU
105
97
• Tizen 3D UI– DALi 3D Engine & UI Toolkit
• DALi is a 3D Engine– UI is represented as a 3D Scene Graph– Animations and Transitions are done using 3D Math
(Vector, Quaternion & Matrix)– Rendering and Visual Effects are done using Open
GL ES Shaders, Vertices and Textures– OpenGL ES 2 and 3 support
• 2D world is the Z plane 0 in the 3D world– When using default camera
DALi 3D Engine
Embedded Software Lab. @ SKKU
105
98
• DALi is part of the Tizen Native Subsystem– Graphics & UI native module– Mobile and TV profiles
• Implemented in C++• DALi (Dynamic
Animation Library)– 2D and 3D Application UIs
with Realistic Effects & Animations
– Home Screen, Lock Screen, Gallery, Music Player …
DALi in Tizen System Architecture
Embedded Software Lab. @ SKKU
105
99
• Evas– EFL drawing canvas– SW/GL rendering
backends• DALi
– OpenGL based Toolkit– Differential UX/
heavy contents• CoreGL
– OpenGL wrapper– Performance
optimization
DALi and Graphics Framework
Embedded Software Lab. @ SKKU
105
100
• Core Library– Event handling, Scene
Graph, Rendering, Resource management
• Adaptor– Threading model– Integration with the
main loop• Platform abstraction
– Resource loading and decoding with multiple threads
• Toolkit– Reusable UI controls,– Effects and Scripting
support
DALi Architecture
Embedded Software Lab. @ SKKU
105
101
• Scene graph based UI is a tree of Nodes– Each Node can have 0-N Children– Each Node inherits its parent
Transformation– $ Position, Rotation, Scale
• Allows easy layout and animation management
– Each Node’s Transformation is relative to a reference point in the parent’s space• Anchor point in the Nodes own coordinate
space• Parent origin in the Parents coordinate
space– Child does not have to be inside its
parent area
DALi: 3D Scene Graph
Embedded Software Lab. @ SKKU
105
102
• DALi uses multithreaded architecture – Best performance and scalability
• Event Thread – The main thread in which
application code and event handling runs
• Update Thread – Updates the nodes on scene – Runs animations, constraints and
physics • Render Thread
– OpenGL drawing, texture and geometry uploading etc
• Resource Threads – Loads font, image and model
resources and decodes into bitmaps etc
Multi-threaded Engine
Embedded Software Lab. @ SKKU
105
103
• Actors & UI Controls– Stage: root of the world– Actors: image, text, mesh, …– UI controls: provide additional
layout and scroll• Animations• Shader Effects
– modify the appearance of objects during rendering
• Image Effects– Cube transition effect, shadow
view, bubble effect, motion blur effect
DALi Features
Embedded Software Lab. @ SKKU
105
104
• ItemView– Scrolling container
based on data source provided by application
• ScrollView– Scrolling container with
scroll effect support• 3D Models & Bone
Animation– Industry standard model
supports(Maya, 3DS, …)• Physical Integration
– Rigid & soft body physics effects
DALi Features
Embedded Software Lab. @ SKKU
105
105
• Tizen Developer Conference 2012– Creating Fancy UIs
with EDJE– Overview of Graphics
and Input in Tizen– Tizen Graphics Core
Evas and Scene Graph• Tizen Developer
Conference 2014– The Art of Tizen UI
Theme Technology in Various Profiles
– Tizen 3D UI DALi 3D Engine
References• LinuxCon #
– 2012: E17 and EFL– 2013: The EFL Toolkit
• EFL Workshop 2012 #– Tizen Native Display
Layer: Architecture and Usage
• Tizen Technical Workshop #
• EFL Dev Day 2013 #– Cserve2: Shared Cache
Subsystem for Evas– EFL on Wayland