A Block Matching Based Method for Moving Object Detection in Active Camera

Mohsen Kheirandish Fard, Mehran Yazdi, Mohammadali MasnadiShirazi Department of Electrical Engineering and Computer Science, Shiraz University

, Shiraz, Iran { Mohsen_Kheirandish , Yazdi , Masnadi }@shirazu.ac.ir

Abstract—One of the most important challenging issues in visual surveillance systems is about detecting moving objects from video sequences captured by an active camera. In contrast to the other proposed methods which are focused on fixed cameras, approaches based on moving cameras are more complex, because making a distinction between moving object and background is difficult. Thus, detecting moving objects independently of the background from video frames is desirable. This paper introduces a new method consisting two phases. Firstly, motion vector field is extracted from output of TSS block matching algorithm. Then, it is classified into many clusters. The biggest one represents the motion direction of blocks which belong to the background and the other blocks can be regarded as moving objects. Secondly, k-means clustering algorithm segments the image to achieve the sharp boundary of moving objects. Simulation results showed that this new method can extract moving object with high accuracy and easy implementation.

Keywords-component; visual surveillance, block matching, image segmentation and motion vector.

I. INTRODUCTION Due to great usage of network camera as a sensing device

for airport, highway surveillance, homeland security, law enforcement, and military agencies, visual surveillance system has widely attracted massive interest to itself in recent years. Beside the wide usage of network camera due to reduction of cost and size, a number of critical issues emerged because of high variability of environment such as changing scene, lighting, and weather conditions. Thus sophisticated algorithm was proposed for different areas such as video acquisition, video compression, camera calibration, noise filtering and motion detection. One of the key challenges focused on this paper is about camera movements.

There are many problems in moving object detection under active camera. The first and the most important problem is that the camera motion and the object motion are mixed. As a consequence, it is very difficult to detect the moving object.

Moving objects can only be found in an image sequence. Conventional vision based methods can simply detect moving object under fixed camera. For example, background subtraction is one of the most common methods for moving object detection when the camera is fixed. Pixels which have the constant intensity throughout the consecutive frames can be considered as background and the pixels having sudden changes can be regarded as candidate for moving objects.

These objects can be simply extracted by a subtraction between the current image and the background. However, in applications which have moving platform such as automatic vehicles, mobile robots, it has poor performance. The reason is that the value of each pixel in the successive frames changes and the background is not detectable. Thus the methods based on motion estimation are proposed.

The main goal of motion estimation is to estimate the motion direction of pixels which can contribute to making a distinction between background and moving objects. TSS block matching algorithm finds the best block in current frame for corresponding block in the previous frame and then defines

as the displacement of block k in which x, y define the value of displacement along x-axis and y-axis respectively. In this step all the pixels of each block have the same motion vector. The motion vector field is a matrix in which each element represents the relative displacement of the block in row and column of previous frame to next frame in a time-varying image sequences. It is a useful analysis to find out the motion direction of camera. So, the regions which have different motion vectors can be perceived as moving objects.

One of the first steps in direction of finding out different objects in images is image segmentation. Image segmentation is done by clustering the pixels according to the similarity of their intensity value. In this process, image is divided into different homogeneous regions, which can contribute to decreasing the complexity of methods.

II. RELATED WORK Number of proposed methods for moving objects detection

assume that the camera is fixed [1, 2]. However, many tried to address the problem of moving object detection in moving camera. For the moving object detection, many approaches have been studied; e.g. background subtraction method [3, 4] and the technique using the optical flow [5, 6, 7]. In addition, some other methods are suggested which do not fall into mentioned categories [8, 9]. In the following, some of these methods are described.

An algorithm in [3] is introduced which is mainly based on motion recovery and segmentation. Background subtraction is used to develop estimation model which in turn leads to performing the motion recovery. Then, the corresponded feature points between two consecutive frames are matched in order to gain feature motion vectors.

443

2013 5th Conference on Information and Knowledge Technology (IKT)

978-1-4673-6490-4/13/$31.00 ©2013 IEEE

A precise method is employed in [5], which is useful for images recorded by an active camera. This method is described in two phases. Firstly, motion is clustered based on optical flow. This part uses the fact that the feature points of the background are more spread than feature points of the foreground. Secondly, by compensating the background motion, the foreground can be detected. A new approach is explained by Shibata in [6]. It is used for moving object detection in a moving camera by optical flow. The camera motion is managed by a controller to obtain a precise optical flow generation. This method uses this fact that the object motion significantly affects the optical flow. As a result, moving objects can be found.

A method is extended in [8] for detecting and tracking moving objects in active camera which is mainly based on Recursive Density Estimation (RDE) method and Scale Invariant Feature Transformation (SIFT). RDE is used for finding out the object moving. The output of RDE is a group of pixels which belong to foreground and can be regarded as target. So the regions which consist of moving object appears and then SIFT is used for tracking.

III. PROPOSED METHOD In this paper, a new method for detecting moving object in

image sequences captured by a moving camera is introduced. It tries to make a distinction between camera motion and object motion in order to find the pixels which belong to moving object. In the following, it will be introduced in two phases. The goal of the first phase is to determine the regions which can be considered as moving objects and the second phase tries to give a precise boundary for moving object and remove the clutters. Consequently, the output of the first step and the second step are processed alongside with each other, which ends in a more acceptable result. In addition, a flowchart is prepared to improve the understanding of this method details.

Phase I . TSS Block Matching Algorithm In this section, the regions which are more likely to be

moving objects are determined. To serve this purpose, TSS block matching algorithm is applied to image sequences in order to extract the motion vector field [10].

The TSS algorithm is an improvement of the traditional block matching algorithm. It greatly decreases the number of checking points in search area. The Full Search (FS) as a traditional search algorithm tries to find the best candidate of the current frame to corresponding blocks in previous frame by searching all area. However, TSS assumes that the error increases monotonically as the searched area moves away from the direction of the minimum error. TSS starts search to find out best point with minimum MSE. This search area include nine points consist of one central point and eight neighbor points on the borders of current frame. The best point with the minimum MSE chosen as the best point for next step. The second and third step will be done similarly but in smaller scale. After third step, suboptimum point is obtained. (Figure 1)

Figure 1. The TSS procedure for p = 7. The motion vector is (-3,-7)

Besides, we should select a good criterion in order to make a comparison between corresponding blocks in current and previous frame. We use the Mean Square Error (MSE):

In the above equation, is represented as previous

frame and as current frame. Size of block is . Although TSS might not attain the best point with the minimum MSE, it can reach to an acceptable point with great reduction in computation cost. This makes it suitable for implementation in real-time applications.

As it is shown in Figure 2, dotted rectangle depicts the three mentioned steps of TSS. At the end of dotted rectangle, a matrix in which each element represents the displacement of blocks in the previous frame T is gained. It is called motion vector field. By clustering matrix data, the cluster with the most members is most likely to be related to the background based on this assumption; the size of moving object is small part of background. Motion camera direction can be computed as following.

In the above equation, C represents the number of the biggest cluster’s members.

By recognizing camera motion direction, the number of checking points in TSS algorithm will be decreased, because many of block comparisons in TSS can be ignored which can affect the speed of algorithm.

444

Figure 2. Phase I procedure

Phase II . K-Means Algorithm The regions which include the moving objects are detected

in the previous step. In this section, sharp boundary of moving object should be defined and clutters are removed. As a consequence, region of moving object with a sharp boundary is obtained after implementing this phase.

In this section, clustering algorithm which should cluster our data based on their features is introduced. From the previous part, relative displacement for each block which represents the movement of that block in x-axis and y-axis throughout two consecutive frames.

In the segmentation process, pixels of the input image should be clustered into many non-over lapped classes based on their colors. Thus, the segmented image consists of disjoint regions with similar color. The K-means clustering algorithm which is an unsupervised algorithm used to clusters data into K disjoint clusters. Clusters which are fallen into moving regions are more likely belong to a moving object. Since the result of k-means is very sensitive to the initialization process that select the initial cluster centers, histogram information is used to achieve better initial guess. As Figure 3 illustrates, K-means algorithm execute as follows: K-Means Algorithm {

Initialization // color histogram of image used to determine the initial cluster centers; pixels are randomly assigned to clusters

Do { Flag=0 Calculate new center of each cluster Partitioning // dotted rectangle

} while Flag is True; Return segmented image }

Figure 3. Phase II procedure

The output of Phase I and Phase II should be processed to obtain clear result. Since the sets of square blocks cannot clearly depict the shape of moving object, some pixels should be added or removed based on the output of phase II. Figure 4 illustrates the final result by combining result of Phase I and Phase II.

Figure 4. Combination of two outputs

IV. SIMULATION AND RESULT The proposed method is applied to many various image

sequences recorded by hand held cameras. In our simulations, p considers as 7 and the size of blocks is 6. The performance of algorithm is high when the difference between the speed of camera and moving object in two consecutive frames is lower than p pixel. That is why we can precisely match blocks in previous frame and current frame. Also, we use many morphological operators in order to remove the clutter which is detected in result image. The performance of the proposed algorithm is shown in Table 1. The object is detected with a high level of precision.

Make biggest cluster block as camera

direction

Clustering

Ti=Vector

point with min ( MSE( ) and

MSE( neighbors ) )

For all blocks of T:

counter=1

Counter 3

Yes

First frame T Second frame T`

No

Counter+1

Candidate for moving object

Yes

No

Initialize

For all pixels: d=distance to its cluster center

Segmented Image

Pixel cluster=new cluster; Flag=1

d> min (distance to other clusters

center)

Flag=1

Input Image

Yes

No

Flag=0 Calculate new center of clusters

Adding and removing pixels to catch clear result

Candidate for moving object

Segmented Image

445

.

Figure 5. Our method precisely separates the moving object from the background. Proposed method is applied on various image sequences. Many result frames of two tested sequences are showed in the above pictures. Many morphological operators are used in order to clear the results.

Moving object detection in frames which has been recorded by a camera has a number of applications in visual tracking, image compression and etc.

TABLE 1. OBJECT DETECTION PERFORMANCE

Image Sequences

Number of Objects

Detected Objects

False-Positive

Detection 1 153 136 (88%) 6 2 179 161 (90%) 9

V. CONCLUSION The main challenge of detecting moving objects from video

sequences captured by an active camera is distinguishing between moving objects from moving background. After applying TSS algorithm, the biggest cluster from division of motion vector field is considered as camera direction and other clusters can be candidate for motion of moving object. Moreover, result of parallel procedure of phase II is image segmented into many regions. Since the sets of square blocks (phase I output) cannot clearly depict the shape of moving object, some pixels should be added or removed based on the output of phase II. Simulation results shows that new approach can effectively detect moving object from image sequences captured by active camera.

REFERENCES [1] Lianqiang Niu , Nan Jiang , “A Moving Objects Detection Algorithm

Based on Improved Background Subtraction, ” Eighth International Conference on Intelligent Systems Design and Applications, 2008. ISDA'08, vol.3, pp.604-607, 26-28 Nov.2008.

[2] J.C. Nascimento, J.S. Marques, “Performance evaluation of object detection algorithms for video surveillance,” IEEE Trans. Multimedia, vol. 8,pp. 761-774, Aug. 2006.

[3] Y. Sheikh , O. Javad , T. Kanade, “Background Subtraction for Freely Moving Cameras , ” IEEE 12th International Conference on Computer Vision 2009 ,pp. 1219-1225, Sept. 29 2009 - Oct. 2 2009.

[4] Chuung-Ching Lin , M. Wolf , “Detecting Moving Objects Using a Camera on a Moving Platform, ”20th International Conference on Pattern Recognition(ICPR) 2010 ,pp. 460-463 , 23-26 Aug. 2010.

[5] Jiman Kim, Guensu Ye, Daijin Kim, “ Moving object detection under free-moving camera, ” 17th IEEE International Conference on Image Processing (ICIP) 2010 ,pp. 4669-4672 , 26-29 Sept. 2010.

[6] Shibata Masaaki, Yasuda Yuichiro ,Ito Masahide “ Moving Object Detection for Active Camera Based on Optical Flow Distortion, ” World Congress, COEX, Korea, South ,vol. 17, pp. 14720 – 14725 ,Jul. 2008

[7] D. Murray, A. Basu, “Motion tracking with an active camera, ” IEEE Trans. Pattern Analysis and Machine Intelligence ,vol. 16 ,pp. 449-459 , May 1994

[8] Gude, C. ; Sadeghi-Tehran, P. ; Ivanov, T. “ARTOT: Autonomous Real-Time Object Detection and Tracking by a Moving Camera, ” 6t h IEEE International Conference, Intelligent System (IS), pp. 446-452 , 6-8 Sep. 2012

[9] Hui Kong , J.-Y. Audibert , J. Ponce , “Detecting Abandoned Objects With a Moving Camera, ” IEEE Trans. Image Processing, vol. 19,pp. 2201 – 2210 , Aug. 2010

[10] Jianhua Lu , Liou, M.L. , “A simple and efficient search algorithm for block-matching motion estimation, ” IEEE Trans. Circuits and Systems for Video Technology, vol. 7 , pp.429-433, Apr 1997

446