high performance graphics and computation - opengl es and renderscript

High Performance Graphics and Compute

OpenGLES and RenderScript

Arvind Devaraj

Agenda

● Introduction : Graphics terms

● Graphics on GPU

● OpenGL Android Graphics

● Graphics Pipeline

● Shaders

● High Performance Compute on GPU

● RenderScript

Graphics Terms

● OpenGLES : Graphics API for doing 3D operations on GPU / CPU

● Primitives : lines, point, triangles

● Texture : make the image realistic by adding bitmap

CPU versus GPU

● CPU

– good at executing sequential code

– Handles branches well

● GPU

– Same code, multiple data

– Parallelism (ideal for image rendering)

Graphics on GPU

Graphics Pipeline

OpenGLES Android Graphics

● Graphics Library for 3D

Android Graphics Classes

GLSurfaceViewGLSurfaceView.Renderer

➢View - connects SurfaceView to OpenGLES library

➢Renderer - responsible for making OpenGL calls to render a frame

GLSurfaceView    

    GLSurfaceView view = new GLSurfaceView(this);

    view.setRenderer(new SquareRenderer());

GLSurfaceView.Renderer

● The renderer is responsible for making OpenGL calls to render a frame.

– onDrawFrame() responsible for drawing the current frame

– OnSurfaceChanged() called when surface size changes

– OnSurfaceCreated() called when surface is created

GLSurfaceView.Renderer

public class SquareRenderer implements GLSurfaceView.Renderer {

    public void onSurfaceCreated(GL10 unused, EGLConfig config) {              }    public void onDrawFrame(GL10 unused) {

    }    public void onSurfaceChanged(GL10 unused, int width, int height)   {

           }}

When surface is changed (rotated etc)

Code for drawing the frame ( Square )

Code when surface is created

draw() {

  float coords[] = { ....     };

vertex 'buffer' created with the coords

glVertexPointer(buffer) :

glDrawArrays(TRIANGLE_STRIP, ...);}

Drawing a Square

draw() {

  float squareCoords[] = { ....     };

ByteBuffer vbb = ....

  squareVB = vbb.asFloatBuffer();    squareVB.put(squareCoords); 

gl.glVertexPointer(3, GL10.GL_FLOAT, 0, vertexBuffer);

gl.glDrawArrays(GL10.GL_TRIANGLE_STRIP, 0, vertices.length / 3);}

Full code:https://github.com/arvind-devaraj/android-opengles/blob/master/3_SquareRenderer/src/com/example/graphics1/Square.java

Drawing a Square

OpenGL Rendering Pipeline

Shaders

● Shaders are programs that execute on the GPU

● Shader programs operate on

– Each vertex

– Each pixel

Shader Programs

• Vertex Shader – operates on each vertex

• Fragment Shader – operates on each pixel

• Shaders are compiled and linked ( like any program )

• Executable is sent to the GPU

Vertex Shader

Fragment Shader

attribute vec4 vertexPosition;void main(){  gl_Position = vertexPosition;

}

precision mediump float;void main(){ gl_FragColor = vec4(0.5, 0.5, 0.5, 1.0);}

CPU – GPU communication

GPU for General Programs

● Graphics is accelerated by the GPU

● Can GPU accelerate other programs ?

– e.g. Matrix multiply, encryption

● OpenCL, CUDA, Renderscript are API

– used for general purpose GPU

RenderScript

● A high level API to access GPU● Provides high performance Compute● Provides uniform API across multiple SoC● Alternative is NDK (but NDK is platform

specific)

RenderScript – Use cases

● What functions can be accelerated

– Graphics

– Image Processing

– Encryption

– Signal processing

– Mathematical functions

RenderScript - Flow

RenderScript – CPU side

Class Hello extends Activity {

Allocation input; Allocation output;

RenderScript rs = new RenderScript() ScriptC_func script = new ScriptC_func(...)

script.forEach_root(input, output) }

RenderScript – GPU side

Hello.rs

void root (char *in , char *out) { *out = *in * 2

}

RenderScript

● Advantages : Compared to NDK, provides an easy device agnostic way to accelerate performance on GPU

● Disadvantages : C99 standard, debugging is restricted

Renderscipt - Summary

● Renderscript is an API to access GPU

● Used for High Performance

● High Performance Compute / Graphics

● Compute, Math , FFT, convolution, Signal processing

● Support Graphics – but not a replacement for OpenGL

● Works on all GPUs ( if supported by SoC ) otherwise on CPU

●

●