tizen v2.3 graphics & ui frameworks

105
Embedded Software Lab. @ SKKU 105 1 Tizen v2.3 Graphics & UI Frameworks

Upload: justin-cobb

Post on 18-Jan-2018

271 views

Category:

Documents


0 download

DESCRIPTION

Contents EFL(Enlightenment Foundation Library) Window Systems EFL Features Ecore Evas Edje Elementary Cserve2 Window Systems X Window System Wayland DALi 3D Engine

TRANSCRIPT

Page 1: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

1

Tizen v2.3 Graphics & UI Frameworks

Page 2: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

2

• EFL(Enlightenment Foundation Library)– EFL Features– Ecore– Evas– Edje– Elementary– Cserve2

• Window Systems– X Window System– Wayland

• DALi 3D Engine

Contents

Page 3: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

3

• UI Frameworks– Modules to implement interactive GUI– EFL(Elementary/Ecore), OSP, GTK, Qt– Input Service Framework(ISF), Voice Framework

• Graphics Libraries– Modules to render graphic objects– Evas, Cairo, OpenGL ES, EGL

• Window Systems– Modules to manage windows and frame buffers– X Window System, Wayland(Tizen v3.0)

Tizen v2.3 Graphics & UI Frameworks

Page 4: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

4Tizen v2.3 Graphics & UI Frameworks

• Frameworks for UI of core applications & web applications Deprecated

(Bada App FW)

Page 5: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

5

EFL(Enlightenment Foundation

Library)

Page 6: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

6

• Started from a desktop window manager– Default window manager of Bodhi Linux

Enlightenment

Page 7: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

7

• A set of libraries supporting Enlightenment– Enlightenment 0.17(E17)

• Components are divided by functions– Event loop: Ecore– Rendering engine: Evas– UI: Edje, Elementary, …– UI builder tools: Enventor, Eflete, ELM Theme Editor

• EFL in Present– Core components of Tizen Graphics/UI framework– 26% of code is E17, the other is EFL

Enlightenment Foundation Library

Page 8: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

8EFL in Tizen v2.3

LegendApplications EFL & fontsWindow System

H/W

Linux Kernel Frame Buffer Input Devices

H/W

fontconfigfreetype2 OpenGL ES/EGLEet

EvasEcore

Eina

Edje

Elementary

Core Applications

X Window System

InputService

Framework&

Voice Framework

kernel / Frame Buffer/ Input Devices

Graphics

ISF& Voice FW

Cairo

Page 9: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

9

• Immediate Mode Graphics System– Application directly re-paints its portion that is invalidated– Application re-issues all drawing commands to describe the

entire scene each time a new frame is required– GTK+ 3, GDI, GDI+

Immediate vs. Retained Mode (1/2)

Application

Window System

GTK+/GDK-X11

1. Invalidate

2. Expose

Graphics CardFrame Buffer for Screen

3. Drawing

Page 10: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

10

• Retained Mode Graphics System– The graphics system is responsible for rendering the

drawing objects given by application.– Application passes drawing objects to graphics

system.– Evas, GTK+ 4(Clutter), WPF

Immediate vs. Retained Mode (2/2)

ApplicationEvas objects

Evas Rendering Engine(Graphics system)

Expose

Graphics CardFramebuffer for screen

X Window System

Page 11: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

11

• High performance– Retained mode graphics system– 60fps+ on general smartphones

• Fit for Low-end Hardware– Small memory, no swap, no GPUs– OpenGL is not worth for all cases

• Texture keeps speed, but loses memory

• Support for HW rendering– OpenGL ES– SIMD vector(MMX, Neon)

Why is EFL used on Tizen? (1/3)

Desktop Environment

Memory Used (MiB)

Enlightenment 0.18.8

83.8

LXDE 0.5.5 87.0XFCE 4.10.2 110.0LXQt 0.7.0 113.0MATE 1.8.1 123.0Cinnamon 2.2.13

176.3

GNOME3 3.12.2

245.3

KDE 4.13.1 302.6Unity 7.2.0.14 312.5

Page 12: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

12

• Various back-end engine– X11 (~Tizen 2.2.1), Wayland (Tizen 3.0~)– Direct Frame Buffer, DRM, Memory buffers, PS3

Native, Windows (GDI & DirectDraw), …• UI Scalability– Various resolutions (WVGA ~ XQXGA) with the

same layout and resources

Why is EFL used on Tizen? (2/3)

0.6 0.8 1.0Scale factor:

Page 13: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

13

• GUI, Logic separation• Themes– A preset package containing

graphical appearance details• Bindings– Basically C-based– EFL is also binded to

Javascript, C++, Python, Ruby, Perl …

– Eo: Generic object model

Why is EFL used on Tizen? (3/3)

Page 14: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

14

• Entry barrier– Unfamiliar development style– Lack of documentations, developers

• No object-oriented implementation– C-based– Eo project is ongoing

• Incomplete C++ support– Eolian based E++ project ongoing– Eolian is EO object parser and C code generator

• No thread safe

EFL Shortcomings

Page 15: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

15Tizen v2.3 Rendering Path

Textures

OpenGL ES win

X window server

Frame Buffer

Videowin

EFL win

3D App

OpenGL ES/EGL

Video App.

EFL App.

EFLEVAS

CompositeWindowManager

OpenGL ES/EGL

OpenGL ES/EGL

MM FWEFL

Rendering pipeline path; rendering data flow

Legend

3D App. Multimedia App.EFL App. Configurable Enable/disableTexture from

Pixmap

OpenGL ES/EGLProcess Texture

Mem.

Page 16: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

16Tizen v2.3 Rendering Path on SW Backend

ElementaryElementary

EcoreEcore

xlibxlib

Framebuffer

LCDcontroller

60Hz

Update window areasX ServerX Server

X share

d Mem.

EdjeEdje

EvasEvasEvas software backendEvas software backendEcore XEcore X

Page 17: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

17Tizen v2.3 Rendering Path on GLES Backend

EGL for XEGL for X

xlibxlib

Framebuffer

/dev/fb1

LCDcontroller

60Hz

Update window areaX ServerX Server

frontbackDouble buffer

OpenGL ES H/W

OpenGL ESOpenGL ES

contr

olled

by

EGL

EcoreEcore

ElementaryElementary

EdjeEdjeEvasEvas

Evas OpenGL ES backendEvas OpenGL ES backend Ecore XEcore X

Page 18: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

18

• Ecore (E + Core)– Event loop library with convenience modules

• Evas (E + canVAS)– Canvas & rendering library

• Edje– Complex graphical design & layout library

• Elementary– Widget set

EFL Components: Core

Page 19: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

19

• Eo– Generic object model

for binding other languages

• Eina– Library for data types

• Eet– Data codec and storage

• Efreet– Freedesktop.org

standards support

EFL Components: Others

• Edbus– Wrapper of Dbus

• Emotion– Video/Audio Playback

library• Eio

– Asynchronous I/O• Eeze

– Udev hardware detection

• Ethumb– Thumbnailer & cacher

Page 20: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

20

The core loop of EFL applications

EFL: Ecore

Page 21: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

21

• The core loop of EFL application, event & marshaling library– Provides glue to other subsystems– Supports various display system

• X11, FB, Wayland, Win32, WinCE, etc.• Features:– Networking & IPC– Threading & inter-thread communication– Evas input feeding and output driving– More...

Ecore

Page 22: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

22

• elm_main()– Main function of EFL application– After initialization, enter into event loop.

Simple EFL Application

Start Ecore main loop!

Elementary UI configuration(Elementary Window)

On exiting window, it also exits Ecore event loop

Finalization process like freeing resource

Page 23: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

23

• ecore_main_loop_begin()– The wrapper of Ecore main loop– Ecore main loop and Glib main loop can be chosen.

Ecore Main Loop (1/2)

Ecore main loop

Glib main loop

Page 24: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

24

• _ecore_main_loop_iterate_internal()– The implementation of Ecore main loop– It starts all the timer handler, event handler,

rendering in an application.

Ecore Main Loop (2/2)

Page 25: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

25

• Key down event

Ecore: Usage of Ecore Events

Ecore_Event_Handler *handler;handler = ecore_event_handler_add(ECORE_EVENT_KEY_DOWN, func, data);

Eina_Bool func(void *data, int type, void *event_info){ if (…) return ECORE_CALLBACK_DONE; return ECORE_CALLBACK_PASS_ON;}

Ecore_Timer *timer;double time;timer = ecore_timer_add(time, func, data);Eina_Bool func(void *data){ if (…) return ECORE_CALLBACK_RENEW; return ECORE_CALLBACK_CANCEL;}

• Timer event

Page 26: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

26

Canvas & Rendering Library

EFL: Evas

Page 27: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

27

• Canvas & Rendering Library• Features– Retained mode rendering

• Controls rendering behavior in the unit of primitive objects and smart objects.

– Scene graph• Tracks the status of objects → minimizes rendering

behavior.– Multiple output paths

• Pure software/OpenGL ES– Rendering optimization

• 3D H/W Acceleration, SIMD vector operations– Provides GL Glue Layer

Evas

Page 28: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

28

• Evas uses retained mode rendering– Application sends Evas objects to Evas.– Evas does rendering based on Evas objects.

Evas: Retained Mode Rendering

ApplicationEvas objects

Evas Rendering Engine(Graphics system)

Expose

Graphics CardFramebuffer for screen

X Window System

Page 29: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

29

• All Objects Displayed on Canvas– Controlled by Evas_Object handles.

• Loading fonts and images, rendering glyphs and images, scaling, blending, 3D transformation etc.

– Evas manages all Evas objects equally.• Type of Evas Object– Primitive objects: minimum unit of Evas rendering

• Rectangle, Line, Polygon, Text, Textblock, Textgrid, Image

– Smart objects• Box, Grid, Table, Customized smart objects(ELM Widgets)

Evas Objects

Page 30: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

30Evas Primitive Objects

• Rendering units of Evas

Page 31: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

31

• Evas objects composed of primitive objects– Acts as a group.– Evas object generic functions also can be used.

Evas Smart Objects

Page 32: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

32Image Native Surface

Hello World!

Evas_Object_Rect

Evas_Object_Text

Evas_Object_Image

GL Rendering Engine

SW Rendering Engine

X11 Rendering Engine

X11 Output

BufferOutput

SDLOutput

Evas Runtime

OpenGL App(surface_a

ie. Pixmap or texture)

Evas_Object_Image(img_obj_a)

evas_object_image_native_surface_set(img_obj_a, surface_a)

Output

Hello World!

Evas Engine Module

Page 33: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

33

• Scene graph– A structure that arranges the logical

representation of a graphical scene– General data structure in GUI applications

• Scene graph in Evas– Manages Evas objects in graph data structure.– Tracks all the objects’ status and gets changed

area in the next frame → Minimizes rendering area.

Evas: Scene Graph

Page 34: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

34

• Start Here

Evas: Minimizing Rendering Region

Page 35: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

35

• Next frame is…

Evas: Minimizing Rendering Region

Page 36: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

36

• Calculate actual update region deltas

Evas: Minimizing Rendering Region

Page 37: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

37

• Only draw updated regions

Evas: Minimizing Rendering Region

Page 38: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

38

/* Build by $ gcc evas_rectangle.c -o evas_rectangle `pkg-config --cflags --libs ecore ecore-evas` */

#include <Ecore.h>#include <Ecore_Evas.h>

int main(int argc, char **argv) { Ecore_Evas *ee; Evas *evas; Evas_Object *bg; if(ecore_evas_init() <= 0) return 1; ee = ecore_evas_new(NULL, 0, 0, 200, 200, NULL); ecore_evas_show(ee);

evas = ecore_evas_get(ee); // Get the Ecore_Evas's Evas object. bg = evas_object_rectangle_add(evas); // Add a rectangle to the given Evas object. evas_object_color_set(bg, 255, 255, 255, 255); // Set the color of the given Evas object to the given one. evas_object_move(bg, 0, 0); // Move the given Evas object to the given location inside its canvas' viewport. evas_object_resize(bg, 200, 200); // Change the size of the given Evas object. evas_object_show(bg); // Make the given Evas object visible.

ecore_main_loop_begin(); ecore_evas_free(ee); ecore_evas_shutdown();

return 0;}

Evas: Rectangle Example

Page 39: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

39

/* Build by $ gcc evas_image.c -o evas_image `pkg-config --cflags --libs ecore ecore-evas` */

#include <Ecore.h>#include <Ecore_Evas.h>

int main(int argc, char **argv) { Ecore_Evas *ee; Evas *evas; Evas_Object *bg; if(ecore_evas_init() <= 0) return 1; ee = ecore_evas_new(NULL, 0, 0, 200, 200, NULL); ecore_evas_show(ee);

evas = ecore_evas_get(ee); // Get the Ecore_Evas's Evas object. bg = evas_object_image_filled_add(evas); // Add an image to the given Evas object. evas_object_image_file_set(img, “test.png”, NULL); // Set the image file path of the given Evas object. evas_object_move(img, 0, 0); // Move the given Evas object to the given location inside its canvas' viewport. evas_object_resize(img, 200, 200); // Change the size of the given Evas object. evas_object_show(img); // Make the given Evas object visible.

ecore_main_loop_begin(); ecore_evas_free(ee); ecore_evas_shutdown();

return 0;}

Evas: Image Example

Page 40: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

40

Complex graphical design & layout library

EFL: Edje

Page 41: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

41

• Complex graphical design & layout library• EDC script

– GUI layout description separated from application code– Can be changed at or during runtime– Compiled as binary code(edj)

Edje

• Description about:– Images, colors, positioning,

animations, behaviors, …

Page 42: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

42

• Separation of layout and logic– Graphical part: GUI

Layout binary (edj)– Functionality: Executable

logic binary (C)• Utility– edje_cc : compile edc to

edj– edje_decc : de-compile

edj to edc

Edje: Edc & Edj

Page 43: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

43Edje: Relation with Evas, Ecore

Page 44: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

44Edje: EDC Script

Page 45: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

45Edje: EDC Script

Page 46: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

46Edje: Simple Example

part {name: “background”;type: RECT;mouse_events: 0;description {

state: “default” 0.0;color: 255 255 255 255;

}}

part {name: “blue_rect”;type: RECT;mouse_events: 1;description {

state: “default” 0.0;color: 0 0 255 255;rel1 {

relative: 0.1 0.1;}rel2 {

relative: 0.3 0.3;}

}}

Page 47: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

47Edje: Simple Examplepart {

name: “red_rect”;type: RECT;mouse_events: 0;description {

state: “default” 0.0;color: 255 0 0 255;rel1 {

to: “blue_rect”;relative: 0 2;

}rel2 {

to: “blue_rect”;relative: 1 3;

}}

description {state: “down” 0.0;inherit: “default” 0.0;rel1.relative: 3 2;rel2.relative: 4 3;color: 255 0 0 0;

}}

Page 48: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

48Edje: Simple Example

program {name: “blue_rect.clicked”;signal: “mouse,clicked,1”;source: “blue_rect”;action: STATE_SET “down” 0.0;transition: ACCELERATE 0.5;target: “red_rect”;after: “red_rect.restore”;

}program {

name: “red_rect.restore”;action: STATE_SET “default” 0.0;target: “red_rect”;

}

Page 49: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

49

A set of widgets

EFL: Elementary

Page 50: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

50

• A Set of Widgets– 약 80 여 개– Containers: Box, Conformant, Grid, Layout, Panes, Scroller, Table,

…– Non-Containers: Bg, Button, Check, Entry, Label, List, Icon, Win, ...

• Features:– Fast, Finger friendly, Scalability, Themes– Widget hierarchy, Dynamic language change

Elementary

< Scaled and resized relative to the screen> < Same view but different themes>

Page 51: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

51

• Various Widgets

Elementary Widgets

clock

colorselector

entry

frame

flipselector check

diskselector

actionslider

bg

index

map

menuphoto, photocam

panel

toolbar

radio

bubble

slider

ctxpopup

notify

separator

segment control

multibuttonentrypanes glview

fileselector

Labelthumb

Page 52: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

52

• Naviframe– Switching multiple

pages with stack• Toolbar– List of Items

Elementary Widgets

Page 53: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

53

• Entry– Text input field– Copy & paste,

various modes (password, multi-line, ...)

• Genlist– Generic List– Flexible but

complex to use

Elementary Widgets

Page 54: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

54

• Widgets are built in a hierarchical fashion.

• Widget shares it’s functions(APIs) with child widget.

Elementary: Widget Hierarchy

Page 55: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

55

• UI Scalability– Relative Positioning

Elementary: Scalability

– Scale Factor

Page 56: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

56

• Theme can be changed in same layout

Elementary: Theme

Page 57: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

57

• Elementary provides utility APIs for multi language support

Elementary: Dynamic Language Change

Page 58: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

58Elementary: Dynamic Language Change

Page 59: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

59

• Enventor: Dynamic EDC Editor• Eflete: EFL Edje Theme Editor• ELM Theme Viewer: Elementary

Theme Viewer

GUI Tools for Customizing Layout

Page 60: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

60

Optimization of EFL

Cserve2

Page 61: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

61

• Duplicated Resource Problem in EFL– Most images and fonts used in a theme are

used by multiple applications at the same time– EFL contains a variety of caches and

• Images, fonts, ...• Objectives of Cserve2:– Reduce memory consumption by sharing– Optimize load time by pre-loading data

Cserve2: Motivation

Page 62: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

62

• Sharing image and font data between EFL application processes– Started by ProFusion at 2012

• Cache server– Loads images and fonts– Shares resources across applications

• EFL applications– Connect to the server via UNIX socket– Send requests and wait for answers– Don't load anything on their own– Keep local cache pointing to shared data

Cserve2

Page 63: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

63Cserve2

Page 64: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

64

• Problems on Cserve2– IPC overheads:

• messages over socket link• context switches, read, write, ...

– Clients do not know the server's cached data before sending a request

• Solution– Expose indexes of shared objects to all clients

• via /dev/shm– Clients scan the tables and don't wait for the

server's response when the data is already loaded

Cserve2: Shared Indexing

Page 65: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

65

• 3 types of objects:– Data (image data)

• Stored in separate shm files: “img”– Fixed-size arrays of objects

• Indexes• Stored in “array” files

– Variable-size pools of objects• So-called “mempools”• Indexed by offset + length in a separate array

Cserve2: Shared Indexing

Page 66: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

66

• Index tables principles– Arrays of simple structures of a single type– Contain:

• ID• Refcount• Other fixed-size data (int, bool, ...)

Cserve2: Shared Indexing

Page 67: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

67

• Strings– Two shm files are used: index and mempool– Mempool contains the strings actual data– Index table contains:

• String ID• Refcount• Offset (in the mempool)• Length

• So, we can read a string just knowing its string ID

Cserve2: Shared Indexing

Page 68: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

68

• Cserve2 combines image cache and scale cache

• Reuse scale cache logic– Minimizes the number of cached scale images.– Scaling process is done on server side.

Cserve2: Image Scaling

Page 69: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

69

• Cserve2 image loader crash– No crash on client side (just no data)– New slave spawned

• Cserve2 server crash– enlightenment_start restarts cserve2– Clients reconnect to new server

• Discard some data• Keep old image references (no need to load)

• Cserve2 shutdown or restart– cleanup /dev/shm/

Cserve2: Robustness

Page 70: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

70

• Real Phone Usage– Workload & Environment

• 8 threads of “elementary_test -to launcher” on 1280x720 screen– Font size

• ~24x16 (small), ~32x24 (normal), ~40x30 (large)• In case of normal size, 1 font, 50 characters:

1x32x24x50=37.5KB– Icon size

• 50x50 (small), 100x100 (large) in 32-bit color• For one icon, 4Bx50x50=10KB (small), 4Bx100x100=40KB

(large)– Border size

• width 10px, length 720px, 32-bit color, 2x• 4Bx10x720x2=56KB

Cserve2: Benefits

Page 71: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

71

• Overhead– When workload runs with Cserved2 turned on:

72176KB• Size of /dev/shm: 4376KB

– When Cserve2 quits: 66716KB– Total overhead: 72176KB–66716KB–4376KB=1084KB

• Conclusion:– Cserve2 saves memory as soon as 1MB of data can be

shared(fonts & images).• Refer to

“Cserve2: Shared cache subsystem of Evas”.

Cserve2: Overhead

Page 72: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

72

Tizen v2.3 Window Systems:

X Window Systems

Page 73: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

73

• History– X window system is made by MIT in 1984.– X11: the current protocol version, established in 1987.– X.org server: the current reference implementation

developed by the X.Org Foundation• Designed to be used over network

connections• Features

– Event Driven– Network Transparent– Device Independent– Policy Independent– Extensible

X Window System

Page 74: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

74

• Client-server Model– X client is linked with X server

through the X protocol translated by Xlib.

– X client• Sends graphical request• Gets user input.• Ecore and Evas in Tizen

– X server• Accepts request for graphical

output• Sends back user input

X Window System: Architecture (1)

Page 75: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

75X Window System: Architecture (2)• Client-server Architecture of X System

Page 76: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

76X Protocol

• Protocol about communication with X clients and the X server

• Features:– Object-based– Asynchronous

• Types of Messages:– Request (Client → Server)– Reply– Event (Server → Client)– Error

Page 77: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

77X Server

• Handles Output & Input Processes• Manages Screens and Input Devices• Layered Structure

– DIX Layer, DDX Layer, OS Layer, Extension Layer• Resource

– Everything in X Server controlled through “resources”

– Created on demand– Used by clients to request operations– Screens, windows, graphic contexts, pixmaps,

regions, colormaps, fonts, cursors

Page 78: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

78X Server

• Layered Structure:– DIX Layer, DDX Layer, OS Layer, Extension Layer

Page 79: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

79

• DIX(Device Independent X) Layer– Interface between X Clients and

DDX/extensions• DDX(Device Dependent X) Layer

– Deals with graphic and input hardware• OS Layer

– Client connection management, work scheduling, memory allocation routines

• Extension Layer– Enables to get additional functionalities for X

server

X Server

Page 80: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

80DIX Layer

• Interface among DDX/Extensions and X Clients– Delivers requests from clients to

DDX/extensions– Sends input events to clients

• Dispatch loop– DIX Layer manages requests with dispatch loop

• Screens– One for each physical screen– Defines functions used to handle operations on

resources– Main communication channel DIX-DDX

Page 81: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

81DDX Layer

• A layer dealing with graphic and input HW

• Graphics Output• Input Processing– Mouse movement– Key presses– Key mappings

Page 82: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

82DDX Layer: Graphics Output

• Pixel-based• Screen basic data structure– Manage all resources

• Output done on drawables– Pixel data format defined by DDX– Pixmap format constant across screens

• XYPixmap -> one bitmap for each plane• Zpixmap -> series of pixel values

– Pixel values interpreted at display time– Windows, Pixmaps, …

82

Page 83: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

83DDX Drawables

• Windows– Visible rectangle on screen– Border drawn by DDX

• Create, move, resize– Contents drawn by client

• Server may provide backing store• Bit gravity (resizing)

– Clipped by parent and sibbling windows• Pixmaps– Rectangular array of pixel values– Reference counted

Page 84: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

84OS Layer

• Client Monnection Management• Work Scheduling– It schedules work to be done for clients– Ignore/AttendClient()

• Memory Allocation Routines– Xalloc()/Xrealloc()/Xfree()

Page 85: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

85Porting X Server

• Code layers– DIX: Dev.Independent, don’t touch– OS: OS specific– DDX: Graphics Device specific– Extensions: Add features

• Porting Layer: OS + DDX• Porting process:– Define functions required by DIX to handle

resources

Page 86: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

86

Tizen v3.0 Window Systems:

Wayland

Page 87: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

87

• Display management architecture & protocol– Window management and composition– Weston: main open source implementation– Maintained by Freedesktop.org– Adopted by Tizen 3.0

• Replacement for X– No rendering API in Wayland

• Rendering has already been supported by Evas, Cairo, OpenGL ES

– SHM(Shared memory) protocol: support for SW rendering– Less IPC: single process architecture

Wayland

Page 88: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

88

• Wayland Protocol– Weston compositor interacts with Wayland

clients(applications) via Wayland protocol.

Wayland Protocol

Page 89: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

89

• EFL and E17(X compositor) are linked to X.org(X server) via X protocol

Tizen v2.3 Graphic Stack with X.org

Page 90: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

90

• EFL is linked to Weston/E17 compositor via Wayland protocols

Tizen v3.0 Graphic Stack with Wayland

Page 91: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

91

• Weston provides surface to be rendered to Evas

• Weston sends input events to Ecore

Wayland with EFL

Page 92: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

92

• EFL controls display and input devices through several backends.– Ecore_Evas: Glue to the backends

Backends of EFL

• X Server Backend– Software_X11, OpenGL_X11

• Wayland Backend– Wayland_SHM: SW rendering– Wayland_EGL: HW rendering– Ecore_Wayland

• Surface Handling and Input

Page 93: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

93

• Performance of Wayland– 60 FPS(Frames Per Second)– 16ms for one frame from client to compositor

• Visualization Tool: E-Graph– A tool to visualize log information and draw

FPS curve– https://gitorious.org/e-graph/e-graph

Wayland Optimization

Page 94: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

94

• Problem– Rendering when resource is blocked by waiting

VSync• Triple Buffering– Add one additional buffer for client and compositor– Before: ~40fps → After: ~48fps– Time spend on composition: 13ms

Wayland Optimization

Page 95: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

95

• Problem– Redraw overlapped surface → heavy load

during composition• Opaque Region– Overlapped surface, set by Ecore and sent to

Weston– Compositing time: ~13ms → ~5ms– FPS: 40fps → 60fps

Wayland Optimization

Page 96: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

96

DALi 3D Engine

Page 97: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

97

• Tizen 3D UI– DALi 3D Engine & UI Toolkit

• DALi is a 3D Engine– UI is represented as a 3D Scene Graph– Animations and Transitions are done using 3D Math

(Vector, Quaternion & Matrix)– Rendering and Visual Effects are done using Open

GL ES Shaders, Vertices and Textures– OpenGL ES 2 and 3 support

• 2D world is the Z plane 0 in the 3D world– When using default camera

DALi 3D Engine

Page 98: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

98

• DALi is part of the Tizen Native Subsystem– Graphics & UI native module– Mobile and TV profiles

• Implemented in C++• DALi (Dynamic

Animation Library)– 2D and 3D Application UIs

with Realistic Effects & Animations

– Home Screen, Lock Screen, Gallery, Music Player …

DALi in Tizen System Architecture

Page 99: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

99

• Evas– EFL drawing canvas– SW/GL rendering

backends• DALi

– OpenGL based Toolkit– Differential UX/

heavy contents• CoreGL

– OpenGL wrapper– Performance

optimization

DALi and Graphics Framework

Page 100: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

100

• Core Library– Event handling, Scene

Graph, Rendering, Resource management

• Adaptor– Threading model– Integration with the

main loop• Platform abstraction

– Resource loading and decoding with multiple threads

• Toolkit– Reusable UI controls,– Effects and Scripting

support

DALi Architecture

Page 101: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

101

• Scene graph based UI is a tree of Nodes– Each Node can have 0-N Children– Each Node inherits its parent

Transformation– $ Position, Rotation, Scale

• Allows easy layout and animation management

– Each Node’s Transformation is relative to a reference point in the parent’s space• Anchor point in the Nodes own coordinate

space• Parent origin in the Parents coordinate

space– Child does not have to be inside its

parent area

DALi: 3D Scene Graph

Page 102: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

102

• DALi uses multithreaded architecture – Best performance and scalability

• Event Thread – The main thread in which

application code and event handling runs

• Update Thread – Updates the nodes on scene – Runs animations, constraints and

physics • Render Thread

– OpenGL drawing, texture and geometry uploading etc

• Resource Threads – Loads font, image and model

resources and decodes into bitmaps etc

Multi-threaded Engine

Page 103: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

103

• Actors & UI Controls– Stage: root of the world– Actors: image, text, mesh, …– UI controls: provide additional

layout and scroll• Animations• Shader Effects

– modify the appearance of objects during rendering

• Image Effects– Cube transition effect, shadow

view, bubble effect, motion blur effect

DALi Features

Page 104: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

104

• ItemView– Scrolling container

based on data source provided by application

• ScrollView– Scrolling container with

scroll effect support• 3D Models & Bone

Animation– Industry standard model

supports(Maya, 3DS, …)• Physical Integration

– Rigid & soft body physics effects

DALi Features

Page 105: Tizen v2.3 Graphics & UI Frameworks

Embedded Software Lab. @ SKKU

105

105

• Tizen Developer Conference 2012– Creating Fancy UIs

with EDJE– Overview of Graphics

and Input in Tizen– Tizen Graphics Core

Evas and Scene Graph• Tizen Developer

Conference 2014– The Art of Tizen UI

Theme Technology in Various Profiles

– Tizen 3D UI DALi 3D Engine

References• LinuxCon #

– 2012: E17 and EFL– 2013: The EFL Toolkit

• EFL Workshop 2012 #– Tizen Native Display

Layer: Architecture and Usage

• Tizen Technical Workshop #

• EFL Dev Day 2013 #– Cserve2: Shared Cache

Subsystem for Evas– EFL on Wayland