Human Emotion Recognition ModuleProject: Automatic Surveillance System for Video Streams (ASSVS)

November 24, 2016

Contents

1

Chapter 1

Introduction

1.1 Purpose

The purpose of this document is to provide detailed requirements of emotion recognition module.It will explain scope of the module, features, constraints and limitations, assumptions and otherrequirements.

1.2 Module Scope

The emotions that will be covered in this module include:

• Joy

• Surprise

• Anger

• Sadness

• Fear

• Neutral

2

Chapter 2

Overall Description

2.1 Module Perspective

Human emotion recognition shall play a vital role in ASSVS project - it helps determine moodof a human in video which facilitates in automatic survellience. The output of the module shallbe used for event recognition. The diagram in figure ?? depicts the role of emotion recognitionmodule in the context of complete framework.

Figure 2.1: Emotion recognition module in the context of ASSVS

3

CHAPTER 2. OVERALL DESCRIPTION 4

2.2 Module Functions

The module will be capable of identifying seven categories of emotions based on changes in facialfeatures.Functions Description

Basic Emotion Detection Joy, sadness, surprise, anger, fear, neutral

Generate List Maintain list of emotions captured and provide selective filtration

Generate Statistics Generate statistical graphs of all emotions in the video or frame

Enlarged View Provide enlarged view of each captured emotion

2.3 User Classes and Characteristics

The different types of user classes for the module will be:

• It can be used by CCTV staff operating in government institutions like police stations.

• Intelligence officers to prevent criminal activity.

• Employers in corporate sectors to find out what their employees are doing.

• Staff in intelligence agencies to find and track suspicious person based on his facial expres-sions.

2.4 Design and Implementation Constraints

In this section, we present some limitations and constraints faced in recognizing emotions.

• There exists overlap between different emotions such as it is hard to differentiate betweenjoy and surprise.

• To detect emotion we should input frontal face.

• Face features should be clear and image should not be dark.

• Illumination condition should be same for complete face.

• Images captured from near/long distance may decrease accuracy.

• Any occlusion may create hinder for detection of facial parts.

2.5 Assumptions and Dependencies

There are multiple assumptions for the module including:

• Segmented face will be used as input to emotion detection system.

• Lips and eyes of subject should be clearly visible.

• In case of hand detection, hand should not cover essential face features.

• Image will contain face and face features.

Chapter 3

External Interface Requirements

3.1 User Interfaces

The module consists of two major user interfaces screens. The interface will provide option forcamera or video input. User can start or stop using player button. The UI in figure ?? consistsof three views: first view is live view of user or video selected by user, a list of emotions capturedand real time graph and statistics.

Figure 3.1: Main UI of the module

The other UI shown in figure ?? will be a pop up which will be shown when user clicks ondetailed view button showing statistics in a tabular form.

5

CHAPTER 3. EXTERNAL INTERFACE REQUIREMENTS 6

Figure 3.2: Pop-up summarized view

3.2 User Documentation

Installation and usage guide shall be provided with the module.

3.3 Software Interfaces

Operating System Windows

.NET Framework v4.0

IDE Visual Studio 2013

Database MS SQL SERVER 2008

Image Processing Libraries OpenCV 3.0, EmguCV 3.0

Machine Learning API Accord.NET

Database IDE SQL Server Management Studio

Web Server IIS 8

3.4 Hardware Interface

Processor Quad-Core 3.4 GHz

RAM DDR3 16GB RAM

Camera FHD

DVR Any

Storage Device 1TB

Network Bandwidth 1MBps

Graphics Card Nvidia

CHAPTER 3. EXTERNAL INTERFACE REQUIREMENTS 7

3.5 Operating Environment

As this module is a part of surveillance system, monitoring the behavior of humans. This caninclude observing a subject from a distance using electronic devices like CCTV cameras. Systemcan work in indoor environment as well as outdoor environment with the concerns that minimumlighting conditions are satisfied. This module is an independent system that can act as an inputof different applications and can also act as a subsystem for a surveillance system.

Chapter 4

System Features

4.1 Product Perspective

Figure 4.1: Use case diagram

8

CHAPTER 4. SYSTEM FEATURES 9

4.2 Emotion Recognition

4.2.1 Description and Priority

The emotional state of the person shall be recognized by this module.

4.2.2 Stimulus/Response Sequences

Stimulus: User provide live or recorded video and requests for emotion recognition.Response: The system identifies the emotion and returns the name of the emotion.

4.2.3 Functional Requirements

FR-1: All the emotions described in the scope shall be covered.FR-2: The system shall provide statistics and some graphical representation of identified emo-tions.