Review ArticleReal-Time Height Measurement for Moving Pedestrians

Wenju Zhou1 Fulong Yao1 Wei Feng 2 and Haikuan Wang1

1School of Mechanical Engineering and Automation Shanghai University Shanghai 200444 China2Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences Shenzhen 518055 China

Correspondence should be addressed to Wei Feng weifengsiataccn

Received 11 June 2020 Revised 27 July 2020 Accepted 18 August 2020 Published 27 August 2020

Academic Editor Kailong Liu

Copyright copy 2020Wenju Zhou et al +is is an open access article distributed under the Creative Commons Attribution Licensewhich permits unrestricted use distribution and reproduction in any medium provided the original work is properly cited

Height measurement for moving pedestrians is quite significant in many scenarios such as pedestrian positioning criminalsuspect tracking and virtual reality Although some existing height measurement methods can detect the height of the staticpeople it is hard to measure height accurately for moving pedestrians Considering the height fluctuations in dynamic situationthis paper proposes a real-time height measurement based on the Time-of-Flight (TOF) camera Depth images in a continuoussequence are addressed to obtain the real-time height of the pedestrian with moving Firstly a normalization equation is presentedto convert the depth image into the grey image for a lower time cost and better performance Secondly a difference-particle swarmoptimization (D-PSO) algorithm is proposed to remove the complex background and reduce the noises +irdly a segmentationalgorithm based on the maximally stable extremal regions (MSERs) is introduced to extract the pedestrian head region +en anovel multilayer iterative average algorithm (MLIA) is developed for obtaining the height of dynamic pedestrians Finally Kalmanfiltering is used to improve the measurement accuracy by combining the current measurement and the height at the last momentIn addition the VICON system is adopted as the ground truth to verify the proposed method and the result shows that ourmethod can accurately measure the real-time height of moving pedestrians

1 Introduction

Whether in reality or in virtual scene it is crucial to evaluatethe height of moving pedestrians Although there are manyworks related to dynamic pedestrians such as detection andrecognition [1ndash3] positioning [4ndash6] and tracking [7ndash9] it isstill a serious challenge to measure the human height ac-curately in the dynamic case As a vital state attribute ofpedestrians the height can not only help locate dynamicpedestrians or track criminal suspects in reality but also helppeople get rid of the 3D glasses or helmets in virtual scene[10] For example Nilsson et al adopted the pedestrianheight and positions of feet as constraint factors to design aKalman filter model [11 12] which can be used for pe-destrian positioning and navigation

In the past years some height detection methods aredeveloped Chen et al developed a novel action-based pe-destrian recognition method [13] which could get the roughheight Also the pneumatic sensor-based height measure-ment methods are developed to detect the pedestrian height

in literatures [14 15] However these methods did not takeheight measurement as the main research content whichleads to an overall low accuracy Besides a significant issue isoften ignored that the pedestrian height is changing whenthe pedestrian is walking which may reduce the accuracy+ere are few state-of-the-art motion tracking systems(MTS) such as VICON which can precisely detect the real-time height of the dynamic pedestrian [16] Sheng et al usedthe VICON motion capture system one of the popularspatial positioning systems in the world to evaluate thehuman behaviours by analysing pedestrian attributes in-cluding height [17] However the MTSrsquos costs for instal-lation and maintenance are extremely expensive +ereforeit is necessary to develop a cheap system to accuratelymeasure the real-time height for moving pedestrians

With the rapid development of the visual sensingtechnology TOF camera is widely used in many fields suchas robot research [18ndash20] object detection [21ndash23] 3Dreconstruction and gesture recognition [24 25] due to itscompact structure and stable characteristics In this paper

HindawiComplexityVolume 2020 Article ID 5708593 15 pageshttpsdoiorg10115520205708593

the TOF camera is adopted as the input and a real-timeheight measurement is developed for moving pedestriansthe flowchart of the measurement is shown in Figure 1 Eachframe of depth image in a continuous sequence is addressedby the proposed algorithms to obtain the real-time height ofthe pedestrian with moving +e algorithm can be roughlydivided into two steps image processing and dataprocessing

Image processing is dedicated to extract the regions ofinterest (ROI)mdashhead region When the TOF camera isadopted the depth value in depth images may be large due toa huge conversion ratio occurring between the actual dis-tance in the world coordinate and the depth data in theimage coordinate Also the gap between the maximumdepth value and minimum depth value is massive which isnot conducive to the subsequent extraction of ROI To re-duce computation and improve the efficiency a normali-zation equation is developed in the paper +en a D-PSOevolutionary algorithm is developed to reduce the effects ofthe complicated background In the D-PSO the differencepart is dedicated to removing the complex background insurroundings while the PSO part is responsible for thenoises that appear after applying the difference part Afterthat an MSER-based segmentation algorithm is adopted toextract the head region Image processing is devoted toheight calculation and correction In this step a novelmultilayer iterative average algorithm based on the actualsituation is proposed to remove the outliers and possiblenoises among the head data +en the pinhole modelproposed in our previous work [26] is adopted to allow ourmethod to work for pedestrians who are not vertically belowthe TOF camera After that considering the continuity of theheight change of the moving pedestrian Kalman filtering isadopted to combine the current measurement and previousheight to improve the accuracy In addition the VICONsystem whose measurement accuracy reached 001mm [27]is used as the ground truth to verify the proposed method

To this end our main contributions are listed as follows

(1) A real-time height detection method is developed fordynamic pedestrians It considers the fluctuation ofheight while the pedestrian is walking which isscarcely mentioned in the existing paper

(2) A new D-PSO algorithm is proposed to reduce theeffects of the complicated background

(3) A novel multilayer iterative average algorithm isdeveloped to remove the outliers and possible noisesfor a better performance

(4) Kalman filtering is used to improve the accuracy ofthe real-time height by combining current mea-surements and the data at last moment

+e rest of this paper is organised as follows Section 2presents a real-time extraction for head region Section 3shows a real-time correction and estimation for pedestrianheight Some experiments are introduced in Section 4 toshow the feasibility and good performance of the proposedmethod while the conclusion and further work are shown inSection 5

2 Real-Time Extraction for the PedestrianHead Region

+e pedestrian head data are used in this paper to calculatethe real-time height of dynamic pedestrians Each frame ofdepth image in a continuous sequence is addressed by thefollowing algorithms to extract the pedestrian head region

21 Normalization According to the imaging principle ofthe TOF camera the depth value in depth image is large andthe gap among depth data is massive Figures 2(a) and 2(b)show the depth images captured by TOF camera the depthimages are shown in Hue Saturation and Value (HSV)format for clarity different colours represent differentdistances Figure 2(a) is the background depth image cap-tured in advance while Figure 2(b) shows the depth imagewith the pedestrian

To reduce computation and improve the efficiency ofsubsequent algorithms the depth image is converted into thegrey image by (1) Figures 2(c) and 2(d) are the grey image(image with pixel values between 0 and 255) correspondingto Figures 2(a) and 2(b) respectively

pi 255lowastdi minus dmin

dmax minus dmin (1)

where pi is the pixel value of a point in the grey imagecorresponding to the depth value di in the depth image anddi is only related to the characteristics of the camera and thedistance from the head to the camera dmax and dmin are themaximum and minimum depth values in the depth imageDifferent depth values correspond to different pixel valuesbetween 0 and 255 +e larger the depth value the bigger thepixel value

22 Difference-Particle Swarm Optimization (D-PSO)Denoising It is obviously hard to obtain the accurate headdata of the pedestrian due to disturbance from the com-plicated background +e difference algorithm as shown inthe following equation is used in this paper to mitigate theeffects of the complex background

Gdif Gback minus Gped (2)

where Gback shows the background grey image (such as inFigure 2(c)) Gped represents the grey pedestrian image (suchas in Figure 2(d)) and Gdif denotes the result of difference(such as in Figure 2(e))

+e difference algorithm produces a large amount ofnoises while extracting the human body region successfullyFor clarity the 3D perspective view of Figure 2(e) is shownin Figure 2(f) To eliminate the effect of noises severalcommon denoising algorithms have been applied with ap-propriate parameters Figures 2(g)ndash2(j) show the resultsobtained by the common denoising algorithms along withthe grey image (Figure 2(e)) To compare these algorithmsclearly the results of these algorithms are also shown here in3D perspective view In these figures we can easily see thestrength of the noise from the value of Pixel-axis +erefore

2 Complexity

TOF Camera Continuous depth imageImage processing (head extraction)

Normalization

Difference-particleswarm optimization(D-PSO) denoising

Head segmentation basedon maximally stable

extremal regions(MSER)

Everyframe

Headregion

Multilayer iterativealgorithm for average

Height calculation

Centroidcalculation

Pinhole model

Kalman filter

Real-time heights

1680

1700

1720

1740

1760

1780

Hei

ght (

mm

)

2523211911 13 15 17 27 29753 91Frames

Data processing(height calculation and correction)

Figure 1 Overall framework of the proposed method

(a) (b) (c)

(d) (e)

Origin

200

150

100

50

0

Pixe

l

Width Height0 50 100 150 200 250 300

200100

0

(f )

Figure 2 Continued

Complexity 3

the value of the Pixel-axis can be used as a criterion forevaluating the denoising effect Although these algorithmscan reduce the influence of noises to some extent they mayalso blur the target contour and damage the pixel in headregion which is not conducive to the extraction of the headregion Figure 2(k) shows the result that is got by adoptingthe two-stage PCA filtering algorithm proposed in [28] Itcan be seen from Figures 2(f)ndash2(k) that compared tocommon filtering algorithms PCA can reduce noise betterand has little influence on target contour and head regionHowever the average time consumed by the PCA algorithmis greater than 15 seconds which is beyond our tolerance

Particle swarm optimization (PSO) developed by DrKenney and Dr Eberhart [29] is an evolutionary algorithm

based on the study of bird or fish predation behaviour andmainly seeks an optimal global solution by following thesearched optimal values of current particles [30] Because of itsfast speed no need to manually set the threshold etc it hasbeen widely used in the field of image processing [31ndash33] andhas achieved excellent results +us PSO is adopted here toremove the background noises In the PSO algorithm eachparticle travels in a multidimensional search space and adjustsits position in search space based on the experience of itself andneighbouring particles [34]+e performance of each particle isevaluated by a predefined fitness function that encapsulates thecore characteristics of the optimization problem

In each iteration every particle in the particle swarm getsits velocity and position by (3) and (4) respectively

Average

200

150

100

50

0

Pixe

l

Width Height0 50 100 150 200 250 300

200100

0

(g)

Median

200

150

100

50

0

Pixe

l

Width Height0 50 100 150 200 250 300

200100

0

(h)

Gaussian

200

150

100

50

0

Piex

l

Width Height0 50 100 150 200 250 300

200100

0

(i)Bilateral

200

150

100

50

0

Piex

l

Width Height0 50 100 150 200 250 300

200100

0

(j)

PCA

200

150

100

50

0

Piex

l

Width Height0 50 100 150 200 250 300

200100

0

(k)

PSO

200

150

100

50

0

Piex

l

Width Height0 50 100 150 200 250 300

200100

0

(l)

(m) (n) (o)

Figure 2 Representative images at each stage (a b) +e depth images represented by HSV format (c d) +e grey image corresponding tothe depth images in (a b) respectively (e) +e result of difference algorithm (f ) +e 3D perspective view of the result in (e) +erepresentative results (represented in 3D perspective view) using the different denoising algorithms (g) average filtering (h) medianfiltering (i) Gaussian filtering (j) bilateral filtering and (k) two-stage PCA filtering (l m) +e PSO denoising results in 3D and 2Drespectively (n o) +e results obtained by the segmentation algorithm based on MSER

4 Complexity

vk+1i w

kv

ki + c1r1 pbesti minus x

ki1113872 1113873 + c2r2 gbest minus x

ki1113872 1113873 (3)

xk+1i x

ki + v

ki (4)

where k is the current number of iterations xki and vk

i arerespectively the position and velocity of the ith particle inthe particle swarm during the kth iteration r1 and r2 are tworandom numbers in [0 1] respectively wk is the inertiaweight in the kth iteration pbesti is the optimal solutionavailable for the ith particle gbest is the optimal solutioncurrently available for all particles and c1 and c2 are indi-vidual learning factors and social learning factors respec-tively which are generally constant As recommended by DrKenney and Dr Eberhart [29] we define learning factorsc1 c2 2 In this case r1 or r2 multiplied by 2 to give it amean of 1 PSO can well take into account both sociallearning and individual learning [35] +e scale of theparticle swarm called M is directly related to the optimi-zation result and time consumption A small scale may causethe PSO to fail to find the optimal solution and a large scalewill cause unnecessary time costs [36] Consider the twopoints the particle swarm scale is defined as M 20

+e larger the inertia weight w is the stronger the globaloptimization ability is and the weaker the local optimizationability is [37] Otherwise the local optimization ability isstronger In order to strike a balance between search speedand search accuracy w should not be a fixed constant Anonlinear decreasing function for w is adopted in the paperas shown in the following equation

wk

wmax minus wmax minus wmin( 1113857lowast1

1 + alowast bmlowast kkmax( )

(5)

where wmax and wmin are the predefined maximum andminimum inertia weights respectively k and kmax are thecurrent and maximum number of iterations and m isin Nlowastagt 0 and 0lt blt 1 are adjustment factors of the polynomialAfter trial and error we define wmax 09 wmin 03kmax 100 a 2 b 06 and m 10 +e inertia weightcurve corresponding to the above parameters is shown inFigure 3 It guarantees that PSO has a high global search-ability in the early stage to get the appropriate seed and hashigher local searchability in the later stage to improve theconvergence accuracy

Besides we adopted the maximum interclass varianceequation (6) as the fitness function in this paper +e largerthe value of the fitness function is the closer to the optimalsolution it will be

f v0 lowast v1 lowast u0 minus u1( 11138572 (6)

where v0 and v1 are respectively the proportion of theforeground and background images to the image u0 and u1represent respectively the average grayscale of the fore-ground and background images

Figures 2(l) and 2(m) show the denoising results of PSOalgorithm in 3D and 2D perspective view respectivelyCompared with other denoising algorithms this algorithmcan achieve better denoising effect without blurring the

target contour In this section a D-PSO is introduced toremove the complicated background Compared with usingthe difference algorithm alone D-PSO can not only removethe complex background in surroundings but it can alsoreduce the noises that appear after applying the differencealgorithm

23 Head Segmentation Based on Maximally Stable ExtremalRegions (MSER) When the TOF camera is used the depthvalue for different parts of the pedestrian body varies greatlyIn order to extract the head region the maximally stableextremal regions (MSER) algorithm is used in the paper +eMSER algorithm refers to performing successive binariza-tion operations on a picture the binarization threshold iscontinuously increased from 0 to 255 [38] If a connectedregion in the image is changed a little or even is not changedwithin a wide range of the binarization threshold this regionis called the maximum stable extreme region Figure 2(n)shows the result obtained by the MSER along withFigure 2(m) In the figure different connected regions aremarked with different colours for clarity It is obvious thatMSER can separate different levels of pedestrian body parts

Fortunately regardless of the height and position ofpedestrians the head shapes of pedestrians are relativelystable ellipse even for pedestrians without hair +us thecircularity is used as a constraint to get the head region +ecircularity of each region is calculated by the followingequation

C 4π lowastA

l2 (7)

where C represents the circularity of the connected region lrepresents the number of pixels in the boundary of theconnected region and A represents the number of pixelswithin the connected region

+e standard circularity is 1 and the circularity of othernoncircular objects is less than 1 According to the exper-imental equipment and environment we had an empiricalconclusion that the circularity of head region is better be-tween 06 and 10 If a connected regionrsquos circularity isbeyond this range it would be remarked as the nonndashheadregion and deleted Due to the size of the pedestrian head inpractice the number of pixelsA is used as another constraintcondition After repeated tests we conclude that the A ofhead region should be during (300 900) In other words it ispossible to be a head region only if the A of the connectedregion is within the range As stated above the constraintscan be summarized in the following equation

300leAle 900

C 4π lowastA

l2

06leCle 10

⎧⎪⎪⎪⎪⎪⎪⎪⎪⎨

⎪⎪⎪⎪⎪⎪⎪⎪⎩

(8)

By calculating and comparing the above two parametersof each connected region in Figure 2(n) the head region is

Complexity 5

extracted as shown in the yellow part of Figure 2(o) Figure 4is the pixel distribution map of the extracted head regionwhere the black dots represent pixel points and the coor-dinates represent the positions of the pixels in the imageFrom this figure we can discover another advantage of theproposed MSER-based segmentation algorithm which canremove the notable noises in the head region such as salt-and-pepper noise Since the notable noise is very differentfrom its neighbour pixels it will not be incorporated into thehead region when the MSER algorithm is used to obtain thestable region +erefore the MSER-based segmentation caneffectively filter out notable noises in the head region asshown in the red rectangles in Figure 4 Note that the redrectangles are the manual markers for easy viewing

3 Real-Time Calculation for Pedestrian Height

31 Multilayer Iterative Average Algorithm for Pixel ValueAlthough the MSER algorithm can filter out the notablenoises there will still be some noises in the head region asshown in the 3D representation of the head region inFigure 2(f ) +e typical height measurement of only usingthe head top is not accurate +us a novel multilayer it-erative average algorithm (MLIA) is proposed to get thepixel average for getting the pedestrian height +e MLIAalgorithm not only can improve accuracy but also can ef-fectively remove some outliers that MSER cannot filter out+e MLIA can be broken down into the following steps

(1) Calculating the average of pixel value adopting thefollowing equation to get the average of pixel value inthe head region as

pave 1n

1113944

n

i1pi (9)

where pave is the pixel value average n is the numberof pixels in current head region and pi representsi minus th pixel value in current head region

(2) Updating the head region traverse all the pixels inthe head region and delete the pixels that do notmeet the following equation +e remaining pixelsare combined to update the head region

pi minus pave1113868111386811138681113868

1113868111386811138681113868leT pave( 1113857 (10)

where T(pave) is a threshold function related to thecurrent average pave and it is defined as follows

T pave( 1113857 Min pmax minus pave pave minus pmin( 1113857 (11)

where pmax and pmin are the maximum pixel valueand the minimum pixel value in the head regionrespectively

(3) Repeat step (1) and step (2) above until pave satisfythe following equation

Pave minuspmin + pmax

2le δ (12)

where δ is the empirical constant In this paper δ isselected as 20 according to the actual situation

+e above steps can be summarized as the followingpseudocode (Algorithm 1)

By the way the MLIA algorithm can also be applied tothe multipedestrian situation When the image containsmore than one pedestrian the MSER-based segmentationcan get more than one head region Meanwhile the pixelvalue average of each head region needs to be calculated bythe MLIA algorithm

32HeightCalculation Once pave is obtained the average ofthe head region in original pedestrian grey image (such as inFigure 2(d)) defined as pavg can be obtained through thedeformation of (2)

+en substituting pavg into (1) to replace pi we canobtain the following equation

10 807040 60 90 1003020 500Number of iterations

04

045

05

055

06

065

07

075

Iner

tia w

eigh

ts

Figure 3 +e curve of inertia weight in PSO

175 180 185 190 195170Width

120

125

130

135

140

145

150

Hei

ght

Figure 4 +e pixel distribution map of the extracted head region

6 Complexity

davg pavg dmax minus dmin( 1113857

255+ dmin (13)

where davg is the depth value corresponding to pavg and dmaxand dmin are the maximum and minimum depth values inthe pedestrian depth image

According to the physical properties of the TOF camerathe following conversion equation can be used to recover thephysical distance from the depth data davg[39]

Ddis Ktofdavg + E Ktofpavg dmax minus dmin( 1113857

255+ dmin1113888 1113889 + E

(14)

where Ddis represents the physical distance between the TOFcamera and the pedestrian head (unit mm) E is the de-viation constant associated with the physical structure andplacement height of the TOF camera while Ktof (512) isthe conversion coefficient only associated with the physicalstructure of the TOF camera

To allow our method to work for pedestrians who are notvertically below the TOF camera the pinhole model pro-posed in our previous work [26] is adopted to correct Ddis

Dco Ddis times cos arctanOMf

1113888 11138891113888 1113889 (15)

where Dco is the corrected physical distance f is the focallength and OM is the distance between the centroid of thehead region in the grey image M and the centre of the greyimage O the coordinates of the centroidM can be got by thefollowing equation More detailed information about thepinhole model can be found in the literature [26]

hp 1n

1113944

n

i1mihi

wp 1n

1113944

n

i1miwi

⎧⎪⎪⎪⎪⎪⎪⎪⎨

⎪⎪⎪⎪⎪⎪⎪⎩

(16)

where n is the number of pixels in current head region wp

and hp are the horizontal and vertical coordinates of thecentroid M and wi and hi are the horizontal and verticalcoordinates of the ith pixel respectively mi is the mass of theith pixel which is defined as mi 1 in this paper

Finally the pedestrian height H is calculated by thefollowing equation

H Htof minus Dco (17)

where Htof is the distance between the TOF camera and theground

33 Kalman Estimation of Real-Time Height In the exper-iments we found that the fluctuations of the pedestrianheights all approximately conform to the Gaussian distri-bution with variance 256 (unit mm2) and the variance didnot change with the state of the system +erefore Kalmanfiltering is further introduced to estimate the pedestrianheights got by (17) to achieve the more accurate real-timeheights Kalman filtering is a highly efficient recursive filterthat can estimate the state of a dynamic system from a seriesof measurements containing redundant noise [40] It cangenerate estimates of unknown variables which have provento be more accurate than those only based on a singlemeasurement [4 41] +e Kalman filter can be implementedin two stages time update stage and measurement updatestage [42]

+e time update stage is dedicated to predicting thecurrently a priori estimates through past state and the errorcovariance Equations (18) and (19) are responsible forpredicting the a priori state estimate 1113954xk and the a priori errorcovariance estimate 1113954Pk in current (kth) frame respectively

1113954xk Akminus 1xkminus 1 + Bukminus 1 (18)

1113954Pk Akminus 1Pkminus 1ATkminus 1 + Q (19)

where xkminus 1 and Pkminus 1 are respectively the state and the errorcovariance of the previous step Akminus 1 is the transfer matrixthat relates the state of the previous step to the state of the

Input S-initial head region extracted by the MSER-based segmentationProcedure

(1) n count (S)(2) pave (1n) 1113936

ni1 pi pi isin S

(3) pmax Max(pi) i 1 2 n(4) pmin Min(pi) i 1 2 n(5) while Pave minus ((pmin + pmax)2)gt δ do(7) T(pave) min (pmax minus pave pave minus pmin)(8) S pi | |pi minus pave|leT(pave) i 1 2 n1113864 1113865(9) n count (S)(10) pave (1n) 1113936

ni1 pi pi isin S

(11) pmax max(pi) i 1 2 n(12) pmin min(pi) i 1 2 n(13) end while

Output pave-the average of the pixels in the head region

ALGORITHM 1 Multilayer iterative average algorithm (MLIA)

Complexity 7

current step B is the control matrix that relates the previousinput ukminus 1 and Q is the variance of the Gaussian processnoise Based on the actual situation of pedestrians during themovement (no external input Gaussian distribution of theheight fluctuation and continuity of the height change) theparameters in time update stage are defined as followsukminus 1 equiv 0 Q equiv 256 Akminus 1 equiv 1 1113954xk is the a priori height estimatefrom the current depth image

+e measurement update stage is devoted to combiningactual measurements with a priori estimates to get theimproved posteriori estimates [42] It can be achieved by thefollowing equations

Kk 1113954PkHTk Hk

1113954PkHTk + R1113872 1113873

minus 1 (20)

xk 1113954xk + K Zk minus Hk1113954xk( 1113857 (21)

Pk I minus KkHk( 11138571113954Pk (22)

where xk and Pk are the posteriori state estimate and theposteriori error covariance estimate in current (kth) step Kk

is the Kalman gain in current step Hk is the matrix thatrelates the state to the measurement Zk I is a unit matrixand R is the variance of the Gaussian measurement noiseBased on the actual situation of measurements (cameraaccuracy and measurement process) the parameters inmeasurement update stage are defined as follows R equiv 144Hk equiv 1 xk is the posteriori height estimate from the currentdepth image and Zk is the pedestrian heights got by (17) Inaddition the initialization is defined as x1 Z1 and P1 10

4 Experiments and Analysis

41 Experimental Setup In this paper an EPC660 is used asthe TOF chip to offer a fully digital interface for the controlcircuitry and the communication between computer andcamera is realized through Gigabit network In addition theexperiment is completed with the support of the computerwith Windows 10 OS Intelreg Coretrade i3-8100 360GHz CPUand 8GB RAM +e campus corridor is selected as the firsttest site and the experimental scene is shown in Figure 5(a)+en considering the fluctuation of pedestrian height indynamic situations the research room is chosen as thesecond test site and the VICON system fixed in this site isadopted as the ground truth to confirm the feasibility of theproposed method +e experimental scene in research roomis shown in Figure 5(b) where a portion of the VICONsystem two of the 12 infrared cameras is shown While theVICON is running four lightweight reflective balls are stuckto the pedestrianrsquos head the placement layout of the balls isshown in Figure 5(c) And the average height of the four ballsis adopted as the real-time height of the pedestrian

42 Comparison with Other Popular Algorithms Before thePSO algorithm is adopted to process the images with un-wanted noise other popular algorithms are deployed toprocess the same images for a comparison More specificallythree algorithms are implemented for comparison here

(1) Maximum Connected Region (MCR) As the nameimplies MCR refers to the method of extracting thelargest connected region in an image When only asingle person appears in the field of view such as inFigure 2(e) MCR is more likely to get desirableresults than PSO In the actual situation however wedo not know in advance how many people will gothrough the test site Take Figure 6(a) as an examplewhen two people go through the test site at the sametime MCR may get a wrong result as shown inFigures 6(b) and 6(c)

(2) Edge 9reshold Method (ETM) In ETM the edgeoperators such as Canny is firstly used to obtain thepossible target contours and the number of pixels inthese contours is then calculated respectively Oncethe number is bigger than a specific threshold theregion enclosed by the corresponding contour isconsidered as the useful region and is retainedotherwise this region is considered as the uselessregion and is removed In the paper the boundarybetween the target person and the redundant noise isusually solid which makes it possible to split thetarget from the background with the ETM Moreimportantly the ETM can also get good results inmultipedestrian images with appropriate parame-ters However it is a very difficult task for the ETM toadaptively select parameters Once the test envi-ronment changes the parameters of ETM need to bereselected which limits the application of the ETM

(3) Reaction Diffusion-Level Set Evolution (RD-LSE) +eRD-LES proposed by Zhang et al [43] is an im-proved level set algorithm which is widely used inthe field of image segmentation Figure 6(d) showsthe search process using the RD-LSE algorithm forthe Figure 6(a) in which the yellow curves show theevolution processes the green curve represents theinitial contour and the red curve represents the finalcontour +is algorithm can achieve a better resultthan PSO algorithm even in the case of multiplepedestrians as shown in Figures 6(e) and 6(f ) In thepaper we take 4 different types of pictures as ex-amples to compare the performance of RD-FLS andPSO in terms of converged iterations and CPU time+e experimental results are shown in Table 1 whereimages 1ndash4 represent Figures 2(e) 6(a) 7(a) and7(g) respectively +e values in table are the averageof 100 experiments Table 1 shows that the com-putational efficiency of the PSO algorithm far ex-ceeds the RD-FLS which is the main reason why wechoose PSO

43 Experimental Results Apart from the multipedestriancases such as in Figure 6 many other cases with the pedestrianin different states are studied to verify the effectiveness androbustness of the proposed method In Figure 7(a) the pe-destrian raised his left hand above his head Figures 7(c) 7(e)and 7(f) show the experimental process and result of adoptingthe proposed method for Figure 7(a) For clarity the 3D

8 Complexity

(a) (b)

Front

Back

Mark 3

Mark 4

Mark 1

Mark 2

(c)

Figure 5 Experimental setup (a) Site campus corridor (b) Site research room (c) Placement layout of the lightweight reflective balls

(a) (b) (c)

(d) (e) (f )

Figure 6 Experiments with two-pedestrian image (a) Original image with two pedestrians (b) Image obtained by theMCR algorithm alongwith the original image (c) Image obtained by theMSER-based segmentation along with (b) (d e)+e processes and result images obtainedby the RD-LSE along with the original image (f ) Image obtained by the MSER-based segmentation along with that in (e)

Table 1 Iterations (Iter) and CPU time (Time) by FRFLS and PSO methods

MethodsImage 1 Image 2 Image 3 Image 4

Time (s) Iter Time (s) Iter Time (s) Iter Time (s) IterFRFLS 521 643 601 800 532 665 487 611PSO 0049 129 0057 86 0053 185 0050 147Image size 320 lowast 240 pixels

Complexity 9

representations of Figures 7(a) and 7(c) are shown inFigures 7(b) and 7(d) respectively Although the height of thehead is lower than that of the left hand the proposed methodcan still get the correct result Figures 7(i) 7(k) and 7(l) showthe experimental process and result of adopting the proposed

method for Figure 7(g) in which a pedestrian is kneelingAlthough the proposed D-PSO algorithm does not eliminateall redundant noises as shown in Figure 7(j) it also yieldsideal experimental results due to MSERrsquos insensitivity to asmall amount of the sporadic noise All the above experiments

(a)

200

150

100

50

0

Pixe

l

050

100150

200HeightWidth

050100150200250300

(b) (c)

200

150

100

50

0

Pixe

l

050

100150

200HeightWidth

050100150200250300

(d) (e) (f )

(g)

10080604020

0

Pixe

l

250200

150100

500Height

Width

300

200

0100

(h) (i)

150

100

50

0

Pixe

l

200150

10050

0Height

Width

200100

0

300

(j) (k) (l)

Figure 7 Experiments with the pedestrian in different states (a) Original image with the pedestrian raising his left hand (c) Image obtainedby the PSO algorithm along with that in (a) (e f ) Images obtained by the MSER-based segmentation along with that in (c) (b d) +e 3Drepresentation of images in (a c) respectively (g) Original image with the pedestrian who is kneeling (i) Image obtained by the PSOalgorithm along with that in (g) (k l) Images obtained by theMSER-based segmentation along with those in (i) (h j)+e 3D representationof images in (g i) respectively

10 Complexity

show that the performance of our method is very stable andreliable

To further verify the accuracy of the proposed method alot of experiments are conducted based on 6 subjects fourmen and two women who are asked to walk through the testsites at the usual speed Here we take a set of data obtainedfrom the research room as an example to analyse the resultsFigure 8 shows the height results obtained from the sixsubjects using the VICON alone in several continuousseconds the sex and static height of the six subjects arepresented in the legend It explains that it is unrealistic to

keep the height on the static level when the pedestrian iswalking +us it is essential to study the pedestrian height inthe dynamic situation

Due to the high speed of pictures taken by VICON andTOF cameras and the slowness of pedestrian movement(07ndash12 meters per second) we only select 5 height data persecond to show a real-time height comparison between theVICON and the proposedmethod Every fifth of one secondan image is collected with the TOF camera +e pedestrianheight in the image is obtained by the proposed method andcompared with the height collected with VICON at the same

0 100 200 300 400 500 600 700 800 900 1000 1100 1200Number

160016101620163016401650166016701680169017001710172017301740175017601770178017901800

Hei

ght (

mm

)

Men1760167617611728

Women16481629

Figure 8 +e height results got from the six subjects using the VICON alone in several continuous seconds

1800179017801770176017501740173017201710170016901680167016601650

Mea

sure

d he

ight

(mm

)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30Number

Our algorithm without Kalman filtering1760167617611728

Our algorithm with Kalman filtering1760167617611728

VICON (ground truth)1760167617611728

Figure 9 Experimental results of men with different heights in the six consecutive seconds

Complexity 11

1700

1690

1680

1670

1660

1650

1640

1630

1620

1610

1600

1590

1580

1570

Mea

sure

d he

ight

(mm

)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30Number

Our algorithm without Kalman filtering16481629

Our algorithm with Kalman filtering16481629

VICON (ground truth)16481629

Figure 10 Experimental results of women with different heights in the six consecutive seconds

28272625242322211011121314151617181920 29308765432 91

Number

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

(a)

2 3 4 5 6 7 8 91 111213141516171819202122232425262728293010

Number

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

(b)

43 61 2 7 8 95 111213141516171819202122232425262728293010

Number

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

(c)

43 61 2 7 8 95 111213141516171819202122232425262728293010

Number

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

(d)

Figure 11 +e error plot of men in the six consecutive seconds (andashd) +e men with static heights of 1760 1676 1761 and 1728

12 Complexity

time Figures 9 and 10 show the experimental results of fourmen and two women in six consecutive seconds In thefigures the dotted line represents our algorithm withoutKalman filtering the solid line represents our algorithmwithout Kalman filtering and the dotted line with the markldquo+rdquo indicates the VICON+e waveforms show the real-timeheight value in 6 consecutive seconds the static heights ofmen are 1760mm 1676mm 1761mm and 1728mm asshown in the legend of Figure 9 while the static heights ofwomen are 1648mm and 1629mm as shown in Figure 10

It can be seen from the curves that the height datameasured by our algorithm is almost consistent with the dataobtained by VICON In order to analyse the error of ouralgorithm we sort out the errors of all the data in the sixconsecutive seconds the results are shown in Figures 11 and12 +e figures show that Kalman filtering can effectivelyimprove the accuracy of height measurement which indi-cates the pedestrian height at the preceding moment facil-itates the estimate of the pedestrian height in the lattermoment

Also the sums of errors per second of the algorithmswith and without Kalman filtering are given in Table 2where the subscript ldquolowastrdquo represents male and ldquordquo representsfemale Table 2 shows that our algorithm with Kalmanfiltering has a smaller cumulative error and can moreaccurately measure the real-time height of the movingpedestrians which proves the feasibility and validity of theproposed method

5 Conclusion and Future Work

In this paper a real-time height measurement based onthe TOF camera is proposed for moving pedestrians Toget the target region a new D-PSO denoising algorithmand a segmentation algorithm based on MSER are de-veloped in the paper In addition a novel multilayer it-erative average algorithm is designed for calculating thepedestrian height Also the Kalman filtering is used toimprove the measurement accuracy +e experimentalresults demonstrate the effectiveness and practicability of

2 3 4 5 6 7 8 9 1011121314151617181920212223242526272829301Number

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2Er

ror (

)

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

(a)

2 3 4 5 6 7 8 9 1011121314151617181920212223242526272829301Number

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

(b)

Figure 12 +e error plot of women in the six consecutive seconds (a) +e woman with static height of 1648 (b) +e woman with staticheight of 1629

Table 2 +e sum of errors per second of the algorithms with and without Kalman filtering

Heights (mm) Kalman filteringSum of errors per second ()

Sum1st second 2nd second 3rd second 4th second 5th second 6th second

1760lowast Yes 1202 0956 1836 1242 1611 1525 8372No 1868 1003 2013 1362 1898 1758 9902

1676lowast Yes 2002 1799 1977 0863 1648 2137 10426No 2249 1968 2087 1602 1827 3261 12994

1761lowast Yes 1282 1483 0963 1132 0632 1487 6979No 1562 1702 1333 1617 1234 1714 9162

1728lowast Yes 1629 1652 1354 1453 1224 0902 8214No 2201 2159 1912 1592 1984 1336 11184

1648 Yes 2006 1194 1818 1014 1585 1693 9310No 2488 1245 2152 1906 2078 2087 11956

1629 Yes 1509 1838 0652 2344 1398 1109 8850No 1632 2536 1328 2508 1497 1340 10841

lowastMale female

Complexity 13

the proposed method Our future work is going to furtherimprove the measurement accuracy and focus on trackingpedestrians in real time by using the real-time height ofmoving pedestrians

Data Availability

+e data used to support the findings of this study areavailable from the corresponding author upon request

Conflicts of Interest

+e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

+e authors are grateful to the financial support from theNatural Science Foundation of China (61877065) the NationalKey Research and Development Program of China(2019YFB1405500) the National Natural Science Foundationof Guangdong (2016A030313177) Guangdong Frontier andKey Technological Innovation (2017B090910013) and theScience and Technology Innovation Commission of Shenzhen(JCYJ20170818153048647 and JCYJ20180507182239617)

References

[1] J Li X Liang S Shen et al ldquoScale-aware fast R-CNN forpedestrian detectionrdquo IEEE Transactions on Multimediavol 20 no 4 pp 985ndash996 2017

[2] F P An ldquoPedestrian re-recognition algorithm based onoptimization deep learning-sequence memory modelrdquoComplexity vol 2019 Article ID 5069026 16 pages 2019

[3] J Cao Y Pang and X Li ldquoLearning multilayer channelfeatures for pedestrian detectionrdquo IEEE Transactions on ImageProcessing vol 26 no 7 pp 3210ndash3220 2017

[4] M Ji J Liu X Xu Y Guo and Z Lu ldquoImproved pedestrianpositioning with inertial sensor based on adaptive gradientdescent and double-constrained extended kalman filterrdquoComplexity vol 2020 Article ID 4361812 11 pages 2020

[5] C Li Z Su Q Li and H Zhao ldquoAn indoor positioning errorcorrection method of pedestrian multi-motions recognized byhybrid-orders fraction domain transformationrdquo IEEE Accessvol 7 pp 11360ndash11377 2019

[6] H Zhao W Cheng N Yang et al ldquoSmartphone-based 3Dindoor pedestrian positioning through multi-modal datafusionrdquo Sensors vol 19 no 20 Article ID s19204554 2019

[7] B Wang T Su X Jin J Kong and Y Bai ldquo3D reconstructionof pedestrian trajectory with moving direction learning andoptimal gait recognitionrdquo Complexity vol 2018 Article ID8735846 10 pages 2018

[8] Y Jiang Z Li and J B Wang ldquoPtrack enhancing the ap-plicability of pedestrian tracking with wearablesrdquo IEEETransactions on Mobile Computing vol 18 no 2 pp 431ndash4432018

[9] W Xu L Liu S Zlatanova W Penard and Q Xiong ldquoApedestrian tracking algorithm using grid-based indoormodelrdquo Automation in Construction vol 92 pp 173ndash1872018

[10] L Bozgeyikli A Raij S Katkoori and R Alqasemi ldquoA surveyon virtual reality for individuals with autism spectrum

disorder design considerationsrdquo IEEE Transactions onLearning Technologies vol 11 no 2 pp 133ndash151 2017

[11] J O Nilsson D Zachariah I Skog and P Handel ldquoCoop-erative localization by dual foot-mounted inertial sensors andinter-agent rangingrdquo EURASIP Journal on Advances in SignalProcessing vol 2013 no 1 p 164 2013

[12] I Skog J-O Nilsson D Zachariah and P Handel ldquoFusingthe information from two navigation systems using an upperbound on their maximum spatial separationrdquo in Proceedingsof the 2012 International Conference on Indoor Positioning andIndoor Navigation Article ID 6418862 Sydney AustraliaNovember 2012

[13] S-B Chen Y Xin and B Luo ldquoAction-based pedestrianidentification via hierarchical matching pursuit and orderpreserving sparse codingrdquo Cognitive Computation vol 8no 5 pp 797ndash805 2016

[14] B Shin C Kim J Kim et al ldquoMotion recognition based 3Dpedestrian navigation system using smartphonerdquo IEEE Sen-sors Journal vol 16 no 18 pp 6977ndash6989 2016

[15] M Romanovas V Goridko A Al-Jawad et al ldquoA study onindoor pedestrian localization algorithms with foot-mountedsensorsrdquo in Proceedings of the International Conference onIndoor Positioning and Indoor Navigation pp 1ndash10 SydneyAustralia November 2012

[16] A Azaman ldquoComparative study on gait kinematics betweenmicrosoft kinect and vicon across different anthropometricmeasurementsrdquo Journal of Tomography System and SensorApplication vol 2 no 2 pp 12ndash17 2019

[17] W Sheng A +obbi and Y Gu ldquoAn integrated frameworkfor human-robot collaborative manipulationrdquo IEEE Trans-actions on Cybernetics vol 45 no 10 pp 2030ndash2041 2014

[18] S Tsuji and T Kohama ldquoProximity skin sensor using time-of-flight sensor for human collaborative robotrdquo IEEE SensorsJournal vol 19 no 14 pp 5859ndash5864 2019

[19] C Oprea I Pirnog I Marcu and M Udrea ldquoRobust poseestimation using Time-of-Flight imagingrdquo in Proceedings ofthe IEEE International Semiconductor Conference pp 301ndash304 Sinaia Romania January 2019

[20] A Vysocky R Pastor and P Novak ldquoInteraction with col-laborative robot using 2D and TOF camerardquo in InternationalConference on Modelling and Simulation for AutonomousSystems pp 477ndash489 Springer Cham Switzerland 2018

[21] M Gao Y Du Y Yang and J Zhang ldquoAdaptive anchor boxmechanism to improve the accuracy in the object detectionsystemrdquo Multimedia Tools and Applications vol 78 no 19pp 27383ndash27402 2019

[22] A Anwer S S Azhar Ali A Khan and F MeriaudeauldquoUnderwater 3-d scene reconstruction using kinect v2 basedon physical models for refraction and time of flight correc-tionrdquo IEEE Access vol 5 pp 15960ndash15970 2017

[23] A R Garcıa L R Miller C F Andres and P J N LorenteldquoObstacle detection using a time of flight range camerardquo inProceedings of the 2018 IEEE International Conference onVehicular Electronics and Safety (ICVES) pp 1ndash6 MadridSpain September 2018

[24] N Zengeler T Kopinski and U Handmann ldquoHand gesturerecognition in automotive humanndashmachine interaction usingdepth camerasrdquo Sensors vol 19 no 1 Article ID s190100592019

[25] M A Garduntildeo-Ramon I R Terol-Villalobos R A Osornio-Rios and L A Morales-Hernandez ldquoA new method forinpainting of depthmaps from time-of-flight sensors based ona modified closing by reconstruction algorithmrdquo Journal of

14 Complexity

Visual Communication and Image Representation vol 47pp 36ndash47 2019

[26] L Wang Y Luo H Wang and M Fei ldquoMeasurement errorcorrection model of TOF depth camerardquo Chinese Journal ofSystem Simulation vol 29 no 10 pp 2323ndash2329 2017

[27] VICON ldquoOfficial website of oxford metrics companyrdquo 2020httpswwwviconcom

[28] L Zhang W Dong D Zhang and G Shi ldquoTwo-stage imagedenoising by principal component analysis with local pixelgroupingrdquo Pattern Recognition vol 43 no 4 pp 1531ndash15492010

[29] J Kennedy and R Eberhart ldquoParticle swarm optimizationrdquo inProceedings of ICNNrsquo95-International Conference on NeuralNetworks (ICW) vol 4 pp 1942ndash1948 Perth AustraliaNovember 1995

[30] M A M De Oca T Stutzle M Birattari and M DorigoldquoFrankensteinrsquos PSO a composite particle swarm optimiza-tion algorithmrdquo IEEE Transactions on Evolutionary Compu-tation vol 13 no 5 pp 1120ndash1132 2009

[31] Z Zhen S Pang F Wang et al ldquoPattern classification andPSO optimal weights based sky images cloud motion speedcalculation method for solar PV power forecastingrdquo IEEETransactions on Industry Applications vol 55 no 4pp 3331ndash3342 2019

[32] X Wang J-S Pan and S-C Chu ldquoA parallel multi-verseoptimizer for application in multilevel image segmentationrdquoIEEE Access vol 8 pp 32018ndash32030 2020

[33] Z A Bashir and M E El-Hawary ldquoApplying wavelets toshort-term load forecasting using PSO-based neural net-worksrdquo IEEE Transactions on Power Systems vol 24 no 1pp 20ndash27 2009

[34] L Liu Y Wang F Xie and J Gao ldquoLegendre cooperativePSO strategies for trajectory optimizationrdquo Complexityvol 2018 Article ID 5036791 13 pages 2018

[35] Y Shi and R C Eberhart ldquoA modified particle swarm op-timizerrdquo in Proceedings of the 1998 IEEE InternationalConference on Evolutionary Computation Proceedingspp 69ndash73 Anchorage AK USA May 1998

[36] Y Shi and R C Eberhart ldquoParameter selection in particleswarm optimizationrdquo in International Conference on Evolu-tionary Programming pp 591ndash600 Springer Berlin Ger-many 1998

[37] X Lv D Zhou Y Tang and L Ma ldquoAn improved test se-lection optimization model based on fault ambiguity groupisolation and chaotic discrete PSOrdquo Complexity vol 2018Article ID 3942723 10 pages 2018

[38] J Matas O Chum M Urban and T Pajdla ldquoRobust wide-baseline stereo from maximally stable extremal regionsrdquoImage and Vision Computing vol 22 no 10 pp 761ndash7672004

[39] H Shim and S Lee ldquoRecovering translucent objects using asingle time-of-flight depth camerardquo IEEE Transactions onCircuits and Systems for Video Technology vol 26 no 5pp 841ndash854 2015

[40] C K Chui and G ChenKalman Filtering pp 19ndash26 SpringerInternational Publishing Berlin Germany 2017

[41] L Cui X Wang Y Xu H Jiang and J Zhou ldquoA novelswitching unscented Kalman filter method for remaininguseful life prediction of rolling bearingrdquo Measurementvol 135 pp 678ndash684 2019

[42] GWelch and G BishopAn Introduction to the Kalman FilterMacmillan New York NY USA 1995

[43] K Zhang L Zhang H Song and D Zhang ldquoReinitialization-free level set evolution via reaction diffusionrdquo IEEE Trans-actions on Image Processing vol 22 no 1 pp 258ndash271 2012

Complexity 15

TOF Camera Continuous depth imageImage processing (head extraction)

Normalization

Difference-particleswarm optimization(D-PSO) denoising

Head segmentation basedon maximally stable

extremal regions(MSER)

Everyframe

Headregion

Multilayer iterativealgorithm for average

Height calculation

Centroidcalculation

Pinhole model

Kalman filter

Real-time heights

1680

1700

1720

1740

1760

1780

Hei

ght (

mm

)

2523211911 13 15 17 27 29753 91Frames

Data processing(height calculation and correction)

Figure 1 Overall framework of the proposed method

(a) (b) (c)

(d) (e)

Origin

200

150

100

50

0

Pixe

l

Width Height0 50 100 150 200 250 300

200100

0

(f )

Figure 2 Continued

Complexity 3

the value of the Pixel-axis can be used as a criterion forevaluating the denoising effect Although these algorithmscan reduce the influence of noises to some extent they mayalso blur the target contour and damage the pixel in headregion which is not conducive to the extraction of the headregion Figure 2(k) shows the result that is got by adoptingthe two-stage PCA filtering algorithm proposed in [28] Itcan be seen from Figures 2(f)ndash2(k) that compared tocommon filtering algorithms PCA can reduce noise betterand has little influence on target contour and head regionHowever the average time consumed by the PCA algorithmis greater than 15 seconds which is beyond our tolerance

Particle swarm optimization (PSO) developed by DrKenney and Dr Eberhart [29] is an evolutionary algorithm

based on the study of bird or fish predation behaviour andmainly seeks an optimal global solution by following thesearched optimal values of current particles [30] Because of itsfast speed no need to manually set the threshold etc it hasbeen widely used in the field of image processing [31ndash33] andhas achieved excellent results +us PSO is adopted here toremove the background noises In the PSO algorithm eachparticle travels in a multidimensional search space and adjustsits position in search space based on the experience of itself andneighbouring particles [34]+e performance of each particle isevaluated by a predefined fitness function that encapsulates thecore characteristics of the optimization problem

In each iteration every particle in the particle swarm getsits velocity and position by (3) and (4) respectively

Average

200

150

100

50

0

Pixe

l

Width Height0 50 100 150 200 250 300

200100

0

(g)

Median

200

150

100

50

0

Pixe

l

Width Height0 50 100 150 200 250 300

200100

0

(h)

Gaussian

200

150

100

50

0

Piex

l

Width Height0 50 100 150 200 250 300

200100

0

(i)Bilateral

200

150

100

50

0

Piex

l

Width Height0 50 100 150 200 250 300

200100

0

(j)

PCA

200

150

100

50

0

Piex

l

Width Height0 50 100 150 200 250 300

200100

0

(k)

PSO

200

150

100

50

0

Piex

l

Width Height0 50 100 150 200 250 300

200100

0

(l)

(m) (n) (o)

Figure 2 Representative images at each stage (a b) +e depth images represented by HSV format (c d) +e grey image corresponding tothe depth images in (a b) respectively (e) +e result of difference algorithm (f ) +e 3D perspective view of the result in (e) +erepresentative results (represented in 3D perspective view) using the different denoising algorithms (g) average filtering (h) medianfiltering (i) Gaussian filtering (j) bilateral filtering and (k) two-stage PCA filtering (l m) +e PSO denoising results in 3D and 2Drespectively (n o) +e results obtained by the segmentation algorithm based on MSER

4 Complexity

vk+1i w

kv

ki + c1r1 pbesti minus x

ki1113872 1113873 + c2r2 gbest minus x

ki1113872 1113873 (3)

xk+1i x

ki + v

ki (4)

where k is the current number of iterations xki and vk

i arerespectively the position and velocity of the ith particle inthe particle swarm during the kth iteration r1 and r2 are tworandom numbers in [0 1] respectively wk is the inertiaweight in the kth iteration pbesti is the optimal solutionavailable for the ith particle gbest is the optimal solutioncurrently available for all particles and c1 and c2 are indi-vidual learning factors and social learning factors respec-tively which are generally constant As recommended by DrKenney and Dr Eberhart [29] we define learning factorsc1 c2 2 In this case r1 or r2 multiplied by 2 to give it amean of 1 PSO can well take into account both sociallearning and individual learning [35] +e scale of theparticle swarm called M is directly related to the optimi-zation result and time consumption A small scale may causethe PSO to fail to find the optimal solution and a large scalewill cause unnecessary time costs [36] Consider the twopoints the particle swarm scale is defined as M 20

+e larger the inertia weight w is the stronger the globaloptimization ability is and the weaker the local optimizationability is [37] Otherwise the local optimization ability isstronger In order to strike a balance between search speedand search accuracy w should not be a fixed constant Anonlinear decreasing function for w is adopted in the paperas shown in the following equation

wk

wmax minus wmax minus wmin( 1113857lowast1

1 + alowast bmlowast kkmax( )

(5)

where wmax and wmin are the predefined maximum andminimum inertia weights respectively k and kmax are thecurrent and maximum number of iterations and m isin Nlowastagt 0 and 0lt blt 1 are adjustment factors of the polynomialAfter trial and error we define wmax 09 wmin 03kmax 100 a 2 b 06 and m 10 +e inertia weightcurve corresponding to the above parameters is shown inFigure 3 It guarantees that PSO has a high global search-ability in the early stage to get the appropriate seed and hashigher local searchability in the later stage to improve theconvergence accuracy

Besides we adopted the maximum interclass varianceequation (6) as the fitness function in this paper +e largerthe value of the fitness function is the closer to the optimalsolution it will be

f v0 lowast v1 lowast u0 minus u1( 11138572 (6)

where v0 and v1 are respectively the proportion of theforeground and background images to the image u0 and u1represent respectively the average grayscale of the fore-ground and background images

Figures 2(l) and 2(m) show the denoising results of PSOalgorithm in 3D and 2D perspective view respectivelyCompared with other denoising algorithms this algorithmcan achieve better denoising effect without blurring the

target contour In this section a D-PSO is introduced toremove the complicated background Compared with usingthe difference algorithm alone D-PSO can not only removethe complex background in surroundings but it can alsoreduce the noises that appear after applying the differencealgorithm

23 Head Segmentation Based on Maximally Stable ExtremalRegions (MSER) When the TOF camera is used the depthvalue for different parts of the pedestrian body varies greatlyIn order to extract the head region the maximally stableextremal regions (MSER) algorithm is used in the paper +eMSER algorithm refers to performing successive binariza-tion operations on a picture the binarization threshold iscontinuously increased from 0 to 255 [38] If a connectedregion in the image is changed a little or even is not changedwithin a wide range of the binarization threshold this regionis called the maximum stable extreme region Figure 2(n)shows the result obtained by the MSER along withFigure 2(m) In the figure different connected regions aremarked with different colours for clarity It is obvious thatMSER can separate different levels of pedestrian body parts

Fortunately regardless of the height and position ofpedestrians the head shapes of pedestrians are relativelystable ellipse even for pedestrians without hair +us thecircularity is used as a constraint to get the head region +ecircularity of each region is calculated by the followingequation

C 4π lowastA

l2 (7)

where C represents the circularity of the connected region lrepresents the number of pixels in the boundary of theconnected region and A represents the number of pixelswithin the connected region

+e standard circularity is 1 and the circularity of othernoncircular objects is less than 1 According to the exper-imental equipment and environment we had an empiricalconclusion that the circularity of head region is better be-tween 06 and 10 If a connected regionrsquos circularity isbeyond this range it would be remarked as the nonndashheadregion and deleted Due to the size of the pedestrian head inpractice the number of pixelsA is used as another constraintcondition After repeated tests we conclude that the A ofhead region should be during (300 900) In other words it ispossible to be a head region only if the A of the connectedregion is within the range As stated above the constraintscan be summarized in the following equation

300leAle 900

C 4π lowastA

l2

06leCle 10

⎧⎪⎪⎪⎪⎪⎪⎪⎪⎨

⎪⎪⎪⎪⎪⎪⎪⎪⎩

(8)

By calculating and comparing the above two parametersof each connected region in Figure 2(n) the head region is

Complexity 5

extracted as shown in the yellow part of Figure 2(o) Figure 4is the pixel distribution map of the extracted head regionwhere the black dots represent pixel points and the coor-dinates represent the positions of the pixels in the imageFrom this figure we can discover another advantage of theproposed MSER-based segmentation algorithm which canremove the notable noises in the head region such as salt-and-pepper noise Since the notable noise is very differentfrom its neighbour pixels it will not be incorporated into thehead region when the MSER algorithm is used to obtain thestable region +erefore the MSER-based segmentation caneffectively filter out notable noises in the head region asshown in the red rectangles in Figure 4 Note that the redrectangles are the manual markers for easy viewing

3 Real-Time Calculation for Pedestrian Height

31 Multilayer Iterative Average Algorithm for Pixel ValueAlthough the MSER algorithm can filter out the notablenoises there will still be some noises in the head region asshown in the 3D representation of the head region inFigure 2(f ) +e typical height measurement of only usingthe head top is not accurate +us a novel multilayer it-erative average algorithm (MLIA) is proposed to get thepixel average for getting the pedestrian height +e MLIAalgorithm not only can improve accuracy but also can ef-fectively remove some outliers that MSER cannot filter out+e MLIA can be broken down into the following steps

(1) Calculating the average of pixel value adopting thefollowing equation to get the average of pixel value inthe head region as

pave 1n

1113944

n

i1pi (9)

where pave is the pixel value average n is the numberof pixels in current head region and pi representsi minus th pixel value in current head region

(2) Updating the head region traverse all the pixels inthe head region and delete the pixels that do notmeet the following equation +e remaining pixelsare combined to update the head region

pi minus pave1113868111386811138681113868

1113868111386811138681113868leT pave( 1113857 (10)

where T(pave) is a threshold function related to thecurrent average pave and it is defined as follows

T pave( 1113857 Min pmax minus pave pave minus pmin( 1113857 (11)

where pmax and pmin are the maximum pixel valueand the minimum pixel value in the head regionrespectively

(3) Repeat step (1) and step (2) above until pave satisfythe following equation

Pave minuspmin + pmax

2le δ (12)

where δ is the empirical constant In this paper δ isselected as 20 according to the actual situation

+e above steps can be summarized as the followingpseudocode (Algorithm 1)

By the way the MLIA algorithm can also be applied tothe multipedestrian situation When the image containsmore than one pedestrian the MSER-based segmentationcan get more than one head region Meanwhile the pixelvalue average of each head region needs to be calculated bythe MLIA algorithm

32HeightCalculation Once pave is obtained the average ofthe head region in original pedestrian grey image (such as inFigure 2(d)) defined as pavg can be obtained through thedeformation of (2)

+en substituting pavg into (1) to replace pi we canobtain the following equation

10 807040 60 90 1003020 500Number of iterations

04

045

05

055

06

065

07

075

Iner

tia w

eigh

ts

Figure 3 +e curve of inertia weight in PSO

175 180 185 190 195170Width

120

125

130

135

140

145

150

Hei

ght

Figure 4 +e pixel distribution map of the extracted head region

6 Complexity

davg pavg dmax minus dmin( 1113857

255+ dmin (13)

where davg is the depth value corresponding to pavg and dmaxand dmin are the maximum and minimum depth values inthe pedestrian depth image

According to the physical properties of the TOF camerathe following conversion equation can be used to recover thephysical distance from the depth data davg[39]

Ddis Ktofdavg + E Ktofpavg dmax minus dmin( 1113857

255+ dmin1113888 1113889 + E

(14)

where Ddis represents the physical distance between the TOFcamera and the pedestrian head (unit mm) E is the de-viation constant associated with the physical structure andplacement height of the TOF camera while Ktof (512) isthe conversion coefficient only associated with the physicalstructure of the TOF camera

To allow our method to work for pedestrians who are notvertically below the TOF camera the pinhole model pro-posed in our previous work [26] is adopted to correct Ddis

Dco Ddis times cos arctanOMf

1113888 11138891113888 1113889 (15)

where Dco is the corrected physical distance f is the focallength and OM is the distance between the centroid of thehead region in the grey image M and the centre of the greyimage O the coordinates of the centroidM can be got by thefollowing equation More detailed information about thepinhole model can be found in the literature [26]

hp 1n

1113944

n

i1mihi

wp 1n

1113944

n

i1miwi

⎧⎪⎪⎪⎪⎪⎪⎪⎨

⎪⎪⎪⎪⎪⎪⎪⎩

(16)

where n is the number of pixels in current head region wp

and hp are the horizontal and vertical coordinates of thecentroid M and wi and hi are the horizontal and verticalcoordinates of the ith pixel respectively mi is the mass of theith pixel which is defined as mi 1 in this paper

Finally the pedestrian height H is calculated by thefollowing equation

H Htof minus Dco (17)

where Htof is the distance between the TOF camera and theground

33 Kalman Estimation of Real-Time Height In the exper-iments we found that the fluctuations of the pedestrianheights all approximately conform to the Gaussian distri-bution with variance 256 (unit mm2) and the variance didnot change with the state of the system +erefore Kalmanfiltering is further introduced to estimate the pedestrianheights got by (17) to achieve the more accurate real-timeheights Kalman filtering is a highly efficient recursive filterthat can estimate the state of a dynamic system from a seriesof measurements containing redundant noise [40] It cangenerate estimates of unknown variables which have provento be more accurate than those only based on a singlemeasurement [4 41] +e Kalman filter can be implementedin two stages time update stage and measurement updatestage [42]

+e time update stage is dedicated to predicting thecurrently a priori estimates through past state and the errorcovariance Equations (18) and (19) are responsible forpredicting the a priori state estimate 1113954xk and the a priori errorcovariance estimate 1113954Pk in current (kth) frame respectively

1113954xk Akminus 1xkminus 1 + Bukminus 1 (18)

1113954Pk Akminus 1Pkminus 1ATkminus 1 + Q (19)

where xkminus 1 and Pkminus 1 are respectively the state and the errorcovariance of the previous step Akminus 1 is the transfer matrixthat relates the state of the previous step to the state of the

Input S-initial head region extracted by the MSER-based segmentationProcedure

(1) n count (S)(2) pave (1n) 1113936

ni1 pi pi isin S

(3) pmax Max(pi) i 1 2 n(4) pmin Min(pi) i 1 2 n(5) while Pave minus ((pmin + pmax)2)gt δ do(7) T(pave) min (pmax minus pave pave minus pmin)(8) S pi | |pi minus pave|leT(pave) i 1 2 n1113864 1113865(9) n count (S)(10) pave (1n) 1113936

ni1 pi pi isin S

(11) pmax max(pi) i 1 2 n(12) pmin min(pi) i 1 2 n(13) end while

Output pave-the average of the pixels in the head region

ALGORITHM 1 Multilayer iterative average algorithm (MLIA)

Complexity 7

current step B is the control matrix that relates the previousinput ukminus 1 and Q is the variance of the Gaussian processnoise Based on the actual situation of pedestrians during themovement (no external input Gaussian distribution of theheight fluctuation and continuity of the height change) theparameters in time update stage are defined as followsukminus 1 equiv 0 Q equiv 256 Akminus 1 equiv 1 1113954xk is the a priori height estimatefrom the current depth image

+e measurement update stage is devoted to combiningactual measurements with a priori estimates to get theimproved posteriori estimates [42] It can be achieved by thefollowing equations

Kk 1113954PkHTk Hk

1113954PkHTk + R1113872 1113873

minus 1 (20)

xk 1113954xk + K Zk minus Hk1113954xk( 1113857 (21)

Pk I minus KkHk( 11138571113954Pk (22)

where xk and Pk are the posteriori state estimate and theposteriori error covariance estimate in current (kth) step Kk

is the Kalman gain in current step Hk is the matrix thatrelates the state to the measurement Zk I is a unit matrixand R is the variance of the Gaussian measurement noiseBased on the actual situation of measurements (cameraaccuracy and measurement process) the parameters inmeasurement update stage are defined as follows R equiv 144Hk equiv 1 xk is the posteriori height estimate from the currentdepth image and Zk is the pedestrian heights got by (17) Inaddition the initialization is defined as x1 Z1 and P1 10

4 Experiments and Analysis

41 Experimental Setup In this paper an EPC660 is used asthe TOF chip to offer a fully digital interface for the controlcircuitry and the communication between computer andcamera is realized through Gigabit network In addition theexperiment is completed with the support of the computerwith Windows 10 OS Intelreg Coretrade i3-8100 360GHz CPUand 8GB RAM +e campus corridor is selected as the firsttest site and the experimental scene is shown in Figure 5(a)+en considering the fluctuation of pedestrian height indynamic situations the research room is chosen as thesecond test site and the VICON system fixed in this site isadopted as the ground truth to confirm the feasibility of theproposed method +e experimental scene in research roomis shown in Figure 5(b) where a portion of the VICONsystem two of the 12 infrared cameras is shown While theVICON is running four lightweight reflective balls are stuckto the pedestrianrsquos head the placement layout of the balls isshown in Figure 5(c) And the average height of the four ballsis adopted as the real-time height of the pedestrian

42 Comparison with Other Popular Algorithms Before thePSO algorithm is adopted to process the images with un-wanted noise other popular algorithms are deployed toprocess the same images for a comparison More specificallythree algorithms are implemented for comparison here

(1) Maximum Connected Region (MCR) As the nameimplies MCR refers to the method of extracting thelargest connected region in an image When only asingle person appears in the field of view such as inFigure 2(e) MCR is more likely to get desirableresults than PSO In the actual situation however wedo not know in advance how many people will gothrough the test site Take Figure 6(a) as an examplewhen two people go through the test site at the sametime MCR may get a wrong result as shown inFigures 6(b) and 6(c)

(2) Edge 9reshold Method (ETM) In ETM the edgeoperators such as Canny is firstly used to obtain thepossible target contours and the number of pixels inthese contours is then calculated respectively Oncethe number is bigger than a specific threshold theregion enclosed by the corresponding contour isconsidered as the useful region and is retainedotherwise this region is considered as the uselessregion and is removed In the paper the boundarybetween the target person and the redundant noise isusually solid which makes it possible to split thetarget from the background with the ETM Moreimportantly the ETM can also get good results inmultipedestrian images with appropriate parame-ters However it is a very difficult task for the ETM toadaptively select parameters Once the test envi-ronment changes the parameters of ETM need to bereselected which limits the application of the ETM

(3) Reaction Diffusion-Level Set Evolution (RD-LSE) +eRD-LES proposed by Zhang et al [43] is an im-proved level set algorithm which is widely used inthe field of image segmentation Figure 6(d) showsthe search process using the RD-LSE algorithm forthe Figure 6(a) in which the yellow curves show theevolution processes the green curve represents theinitial contour and the red curve represents the finalcontour +is algorithm can achieve a better resultthan PSO algorithm even in the case of multiplepedestrians as shown in Figures 6(e) and 6(f ) In thepaper we take 4 different types of pictures as ex-amples to compare the performance of RD-FLS andPSO in terms of converged iterations and CPU time+e experimental results are shown in Table 1 whereimages 1ndash4 represent Figures 2(e) 6(a) 7(a) and7(g) respectively +e values in table are the averageof 100 experiments Table 1 shows that the com-putational efficiency of the PSO algorithm far ex-ceeds the RD-FLS which is the main reason why wechoose PSO

43 Experimental Results Apart from the multipedestriancases such as in Figure 6 many other cases with the pedestrianin different states are studied to verify the effectiveness androbustness of the proposed method In Figure 7(a) the pe-destrian raised his left hand above his head Figures 7(c) 7(e)and 7(f) show the experimental process and result of adoptingthe proposed method for Figure 7(a) For clarity the 3D

8 Complexity

(a) (b)

Front

Back

Mark 3

Mark 4

Mark 1

Mark 2

(c)

Figure 5 Experimental setup (a) Site campus corridor (b) Site research room (c) Placement layout of the lightweight reflective balls

(a) (b) (c)

(d) (e) (f )

Figure 6 Experiments with two-pedestrian image (a) Original image with two pedestrians (b) Image obtained by theMCR algorithm alongwith the original image (c) Image obtained by theMSER-based segmentation along with (b) (d e)+e processes and result images obtainedby the RD-LSE along with the original image (f ) Image obtained by the MSER-based segmentation along with that in (e)

Table 1 Iterations (Iter) and CPU time (Time) by FRFLS and PSO methods

MethodsImage 1 Image 2 Image 3 Image 4

Time (s) Iter Time (s) Iter Time (s) Iter Time (s) IterFRFLS 521 643 601 800 532 665 487 611PSO 0049 129 0057 86 0053 185 0050 147Image size 320 lowast 240 pixels

Complexity 9

representations of Figures 7(a) and 7(c) are shown inFigures 7(b) and 7(d) respectively Although the height of thehead is lower than that of the left hand the proposed methodcan still get the correct result Figures 7(i) 7(k) and 7(l) showthe experimental process and result of adopting the proposed

method for Figure 7(g) in which a pedestrian is kneelingAlthough the proposed D-PSO algorithm does not eliminateall redundant noises as shown in Figure 7(j) it also yieldsideal experimental results due to MSERrsquos insensitivity to asmall amount of the sporadic noise All the above experiments

(a)

200

150

100

50

0

Pixe

l

050

100150

200HeightWidth

050100150200250300

(b) (c)

200

150

100

50

0

Pixe

l

050

100150

200HeightWidth

050100150200250300

(d) (e) (f )

(g)

10080604020

0

Pixe

l

250200

150100

500Height

Width

300

200

0100

(h) (i)

150

100

50

0

Pixe

l

200150

10050

0Height

Width

200100

0

300

(j) (k) (l)

Figure 7 Experiments with the pedestrian in different states (a) Original image with the pedestrian raising his left hand (c) Image obtainedby the PSO algorithm along with that in (a) (e f ) Images obtained by the MSER-based segmentation along with that in (c) (b d) +e 3Drepresentation of images in (a c) respectively (g) Original image with the pedestrian who is kneeling (i) Image obtained by the PSOalgorithm along with that in (g) (k l) Images obtained by theMSER-based segmentation along with those in (i) (h j)+e 3D representationof images in (g i) respectively

10 Complexity

show that the performance of our method is very stable andreliable

To further verify the accuracy of the proposed method alot of experiments are conducted based on 6 subjects fourmen and two women who are asked to walk through the testsites at the usual speed Here we take a set of data obtainedfrom the research room as an example to analyse the resultsFigure 8 shows the height results obtained from the sixsubjects using the VICON alone in several continuousseconds the sex and static height of the six subjects arepresented in the legend It explains that it is unrealistic to

keep the height on the static level when the pedestrian iswalking +us it is essential to study the pedestrian height inthe dynamic situation

Due to the high speed of pictures taken by VICON andTOF cameras and the slowness of pedestrian movement(07ndash12 meters per second) we only select 5 height data persecond to show a real-time height comparison between theVICON and the proposedmethod Every fifth of one secondan image is collected with the TOF camera +e pedestrianheight in the image is obtained by the proposed method andcompared with the height collected with VICON at the same

0 100 200 300 400 500 600 700 800 900 1000 1100 1200Number

160016101620163016401650166016701680169017001710172017301740175017601770178017901800

Hei

ght (

mm

)

Men1760167617611728

Women16481629

Figure 8 +e height results got from the six subjects using the VICON alone in several continuous seconds

1800179017801770176017501740173017201710170016901680167016601650

Mea

sure

d he

ight

(mm

)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30Number

Our algorithm without Kalman filtering1760167617611728

Our algorithm with Kalman filtering1760167617611728

VICON (ground truth)1760167617611728

Figure 9 Experimental results of men with different heights in the six consecutive seconds

Complexity 11

1700

1690

1680

1670

1660

1650

1640

1630

1620

1610

1600

1590

1580

1570

Mea

sure

d he

ight

(mm

)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30Number

Our algorithm without Kalman filtering16481629

Our algorithm with Kalman filtering16481629

VICON (ground truth)16481629

Figure 10 Experimental results of women with different heights in the six consecutive seconds

28272625242322211011121314151617181920 29308765432 91

Number

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

(a)

2 3 4 5 6 7 8 91 111213141516171819202122232425262728293010

Number

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

(b)

43 61 2 7 8 95 111213141516171819202122232425262728293010

Number

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

(c)

43 61 2 7 8 95 111213141516171819202122232425262728293010

Number

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

(d)

Figure 11 +e error plot of men in the six consecutive seconds (andashd) +e men with static heights of 1760 1676 1761 and 1728

12 Complexity

time Figures 9 and 10 show the experimental results of fourmen and two women in six consecutive seconds In thefigures the dotted line represents our algorithm withoutKalman filtering the solid line represents our algorithmwithout Kalman filtering and the dotted line with the markldquo+rdquo indicates the VICON+e waveforms show the real-timeheight value in 6 consecutive seconds the static heights ofmen are 1760mm 1676mm 1761mm and 1728mm asshown in the legend of Figure 9 while the static heights ofwomen are 1648mm and 1629mm as shown in Figure 10

It can be seen from the curves that the height datameasured by our algorithm is almost consistent with the dataobtained by VICON In order to analyse the error of ouralgorithm we sort out the errors of all the data in the sixconsecutive seconds the results are shown in Figures 11 and12 +e figures show that Kalman filtering can effectivelyimprove the accuracy of height measurement which indi-cates the pedestrian height at the preceding moment facil-itates the estimate of the pedestrian height in the lattermoment

Also the sums of errors per second of the algorithmswith and without Kalman filtering are given in Table 2where the subscript ldquolowastrdquo represents male and ldquordquo representsfemale Table 2 shows that our algorithm with Kalmanfiltering has a smaller cumulative error and can moreaccurately measure the real-time height of the movingpedestrians which proves the feasibility and validity of theproposed method

5 Conclusion and Future Work

In this paper a real-time height measurement based onthe TOF camera is proposed for moving pedestrians Toget the target region a new D-PSO denoising algorithmand a segmentation algorithm based on MSER are de-veloped in the paper In addition a novel multilayer it-erative average algorithm is designed for calculating thepedestrian height Also the Kalman filtering is used toimprove the measurement accuracy +e experimentalresults demonstrate the effectiveness and practicability of

2 3 4 5 6 7 8 9 1011121314151617181920212223242526272829301Number

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2Er

ror (

)

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

(a)

2 3 4 5 6 7 8 9 1011121314151617181920212223242526272829301Number

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

(b)

Figure 12 +e error plot of women in the six consecutive seconds (a) +e woman with static height of 1648 (b) +e woman with staticheight of 1629

Table 2 +e sum of errors per second of the algorithms with and without Kalman filtering

Heights (mm) Kalman filteringSum of errors per second ()

Sum1st second 2nd second 3rd second 4th second 5th second 6th second

1760lowast Yes 1202 0956 1836 1242 1611 1525 8372No 1868 1003 2013 1362 1898 1758 9902

1676lowast Yes 2002 1799 1977 0863 1648 2137 10426No 2249 1968 2087 1602 1827 3261 12994

1761lowast Yes 1282 1483 0963 1132 0632 1487 6979No 1562 1702 1333 1617 1234 1714 9162

1728lowast Yes 1629 1652 1354 1453 1224 0902 8214No 2201 2159 1912 1592 1984 1336 11184

1648 Yes 2006 1194 1818 1014 1585 1693 9310No 2488 1245 2152 1906 2078 2087 11956

1629 Yes 1509 1838 0652 2344 1398 1109 8850No 1632 2536 1328 2508 1497 1340 10841

lowastMale female

Complexity 13

the proposed method Our future work is going to furtherimprove the measurement accuracy and focus on trackingpedestrians in real time by using the real-time height ofmoving pedestrians

Data Availability

+e data used to support the findings of this study areavailable from the corresponding author upon request

Conflicts of Interest

+e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

+e authors are grateful to the financial support from theNatural Science Foundation of China (61877065) the NationalKey Research and Development Program of China(2019YFB1405500) the National Natural Science Foundationof Guangdong (2016A030313177) Guangdong Frontier andKey Technological Innovation (2017B090910013) and theScience and Technology Innovation Commission of Shenzhen(JCYJ20170818153048647 and JCYJ20180507182239617)

References

[1] J Li X Liang S Shen et al ldquoScale-aware fast R-CNN forpedestrian detectionrdquo IEEE Transactions on Multimediavol 20 no 4 pp 985ndash996 2017

[2] F P An ldquoPedestrian re-recognition algorithm based onoptimization deep learning-sequence memory modelrdquoComplexity vol 2019 Article ID 5069026 16 pages 2019

[3] J Cao Y Pang and X Li ldquoLearning multilayer channelfeatures for pedestrian detectionrdquo IEEE Transactions on ImageProcessing vol 26 no 7 pp 3210ndash3220 2017

[4] M Ji J Liu X Xu Y Guo and Z Lu ldquoImproved pedestrianpositioning with inertial sensor based on adaptive gradientdescent and double-constrained extended kalman filterrdquoComplexity vol 2020 Article ID 4361812 11 pages 2020

[5] C Li Z Su Q Li and H Zhao ldquoAn indoor positioning errorcorrection method of pedestrian multi-motions recognized byhybrid-orders fraction domain transformationrdquo IEEE Accessvol 7 pp 11360ndash11377 2019

[6] H Zhao W Cheng N Yang et al ldquoSmartphone-based 3Dindoor pedestrian positioning through multi-modal datafusionrdquo Sensors vol 19 no 20 Article ID s19204554 2019

[7] B Wang T Su X Jin J Kong and Y Bai ldquo3D reconstructionof pedestrian trajectory with moving direction learning andoptimal gait recognitionrdquo Complexity vol 2018 Article ID8735846 10 pages 2018

[8] Y Jiang Z Li and J B Wang ldquoPtrack enhancing the ap-plicability of pedestrian tracking with wearablesrdquo IEEETransactions on Mobile Computing vol 18 no 2 pp 431ndash4432018

[9] W Xu L Liu S Zlatanova W Penard and Q Xiong ldquoApedestrian tracking algorithm using grid-based indoormodelrdquo Automation in Construction vol 92 pp 173ndash1872018

[10] L Bozgeyikli A Raij S Katkoori and R Alqasemi ldquoA surveyon virtual reality for individuals with autism spectrum

disorder design considerationsrdquo IEEE Transactions onLearning Technologies vol 11 no 2 pp 133ndash151 2017

[11] J O Nilsson D Zachariah I Skog and P Handel ldquoCoop-erative localization by dual foot-mounted inertial sensors andinter-agent rangingrdquo EURASIP Journal on Advances in SignalProcessing vol 2013 no 1 p 164 2013

[12] I Skog J-O Nilsson D Zachariah and P Handel ldquoFusingthe information from two navigation systems using an upperbound on their maximum spatial separationrdquo in Proceedingsof the 2012 International Conference on Indoor Positioning andIndoor Navigation Article ID 6418862 Sydney AustraliaNovember 2012

[13] S-B Chen Y Xin and B Luo ldquoAction-based pedestrianidentification via hierarchical matching pursuit and orderpreserving sparse codingrdquo Cognitive Computation vol 8no 5 pp 797ndash805 2016

[14] B Shin C Kim J Kim et al ldquoMotion recognition based 3Dpedestrian navigation system using smartphonerdquo IEEE Sen-sors Journal vol 16 no 18 pp 6977ndash6989 2016

[15] M Romanovas V Goridko A Al-Jawad et al ldquoA study onindoor pedestrian localization algorithms with foot-mountedsensorsrdquo in Proceedings of the International Conference onIndoor Positioning and Indoor Navigation pp 1ndash10 SydneyAustralia November 2012

[16] A Azaman ldquoComparative study on gait kinematics betweenmicrosoft kinect and vicon across different anthropometricmeasurementsrdquo Journal of Tomography System and SensorApplication vol 2 no 2 pp 12ndash17 2019

[17] W Sheng A +obbi and Y Gu ldquoAn integrated frameworkfor human-robot collaborative manipulationrdquo IEEE Trans-actions on Cybernetics vol 45 no 10 pp 2030ndash2041 2014

[18] S Tsuji and T Kohama ldquoProximity skin sensor using time-of-flight sensor for human collaborative robotrdquo IEEE SensorsJournal vol 19 no 14 pp 5859ndash5864 2019

[19] C Oprea I Pirnog I Marcu and M Udrea ldquoRobust poseestimation using Time-of-Flight imagingrdquo in Proceedings ofthe IEEE International Semiconductor Conference pp 301ndash304 Sinaia Romania January 2019

[20] A Vysocky R Pastor and P Novak ldquoInteraction with col-laborative robot using 2D and TOF camerardquo in InternationalConference on Modelling and Simulation for AutonomousSystems pp 477ndash489 Springer Cham Switzerland 2018

[21] M Gao Y Du Y Yang and J Zhang ldquoAdaptive anchor boxmechanism to improve the accuracy in the object detectionsystemrdquo Multimedia Tools and Applications vol 78 no 19pp 27383ndash27402 2019

[22] A Anwer S S Azhar Ali A Khan and F MeriaudeauldquoUnderwater 3-d scene reconstruction using kinect v2 basedon physical models for refraction and time of flight correc-tionrdquo IEEE Access vol 5 pp 15960ndash15970 2017

[23] A R Garcıa L R Miller C F Andres and P J N LorenteldquoObstacle detection using a time of flight range camerardquo inProceedings of the 2018 IEEE International Conference onVehicular Electronics and Safety (ICVES) pp 1ndash6 MadridSpain September 2018

[24] N Zengeler T Kopinski and U Handmann ldquoHand gesturerecognition in automotive humanndashmachine interaction usingdepth camerasrdquo Sensors vol 19 no 1 Article ID s190100592019

[25] M A Garduntildeo-Ramon I R Terol-Villalobos R A Osornio-Rios and L A Morales-Hernandez ldquoA new method forinpainting of depthmaps from time-of-flight sensors based ona modified closing by reconstruction algorithmrdquo Journal of

14 Complexity

Visual Communication and Image Representation vol 47pp 36ndash47 2019

[26] L Wang Y Luo H Wang and M Fei ldquoMeasurement errorcorrection model of TOF depth camerardquo Chinese Journal ofSystem Simulation vol 29 no 10 pp 2323ndash2329 2017

[27] VICON ldquoOfficial website of oxford metrics companyrdquo 2020httpswwwviconcom

[28] L Zhang W Dong D Zhang and G Shi ldquoTwo-stage imagedenoising by principal component analysis with local pixelgroupingrdquo Pattern Recognition vol 43 no 4 pp 1531ndash15492010

[29] J Kennedy and R Eberhart ldquoParticle swarm optimizationrdquo inProceedings of ICNNrsquo95-International Conference on NeuralNetworks (ICW) vol 4 pp 1942ndash1948 Perth AustraliaNovember 1995

[30] M A M De Oca T Stutzle M Birattari and M DorigoldquoFrankensteinrsquos PSO a composite particle swarm optimiza-tion algorithmrdquo IEEE Transactions on Evolutionary Compu-tation vol 13 no 5 pp 1120ndash1132 2009

[31] Z Zhen S Pang F Wang et al ldquoPattern classification andPSO optimal weights based sky images cloud motion speedcalculation method for solar PV power forecastingrdquo IEEETransactions on Industry Applications vol 55 no 4pp 3331ndash3342 2019

[32] X Wang J-S Pan and S-C Chu ldquoA parallel multi-verseoptimizer for application in multilevel image segmentationrdquoIEEE Access vol 8 pp 32018ndash32030 2020

[33] Z A Bashir and M E El-Hawary ldquoApplying wavelets toshort-term load forecasting using PSO-based neural net-worksrdquo IEEE Transactions on Power Systems vol 24 no 1pp 20ndash27 2009

[34] L Liu Y Wang F Xie and J Gao ldquoLegendre cooperativePSO strategies for trajectory optimizationrdquo Complexityvol 2018 Article ID 5036791 13 pages 2018

[35] Y Shi and R C Eberhart ldquoA modified particle swarm op-timizerrdquo in Proceedings of the 1998 IEEE InternationalConference on Evolutionary Computation Proceedingspp 69ndash73 Anchorage AK USA May 1998

[36] Y Shi and R C Eberhart ldquoParameter selection in particleswarm optimizationrdquo in International Conference on Evolu-tionary Programming pp 591ndash600 Springer Berlin Ger-many 1998

[37] X Lv D Zhou Y Tang and L Ma ldquoAn improved test se-lection optimization model based on fault ambiguity groupisolation and chaotic discrete PSOrdquo Complexity vol 2018Article ID 3942723 10 pages 2018

[38] J Matas O Chum M Urban and T Pajdla ldquoRobust wide-baseline stereo from maximally stable extremal regionsrdquoImage and Vision Computing vol 22 no 10 pp 761ndash7672004

[39] H Shim and S Lee ldquoRecovering translucent objects using asingle time-of-flight depth camerardquo IEEE Transactions onCircuits and Systems for Video Technology vol 26 no 5pp 841ndash854 2015

[40] C K Chui and G ChenKalman Filtering pp 19ndash26 SpringerInternational Publishing Berlin Germany 2017

[41] L Cui X Wang Y Xu H Jiang and J Zhou ldquoA novelswitching unscented Kalman filter method for remaininguseful life prediction of rolling bearingrdquo Measurementvol 135 pp 678ndash684 2019

[42] GWelch and G BishopAn Introduction to the Kalman FilterMacmillan New York NY USA 1995

[43] K Zhang L Zhang H Song and D Zhang ldquoReinitialization-free level set evolution via reaction diffusionrdquo IEEE Trans-actions on Image Processing vol 22 no 1 pp 258ndash271 2012

Complexity 15

the value of the Pixel-axis can be used as a criterion forevaluating the denoising effect Although these algorithmscan reduce the influence of noises to some extent they mayalso blur the target contour and damage the pixel in headregion which is not conducive to the extraction of the headregion Figure 2(k) shows the result that is got by adoptingthe two-stage PCA filtering algorithm proposed in [28] Itcan be seen from Figures 2(f)ndash2(k) that compared tocommon filtering algorithms PCA can reduce noise betterand has little influence on target contour and head regionHowever the average time consumed by the PCA algorithmis greater than 15 seconds which is beyond our tolerance

Particle swarm optimization (PSO) developed by DrKenney and Dr Eberhart [29] is an evolutionary algorithm

based on the study of bird or fish predation behaviour andmainly seeks an optimal global solution by following thesearched optimal values of current particles [30] Because of itsfast speed no need to manually set the threshold etc it hasbeen widely used in the field of image processing [31ndash33] andhas achieved excellent results +us PSO is adopted here toremove the background noises In the PSO algorithm eachparticle travels in a multidimensional search space and adjustsits position in search space based on the experience of itself andneighbouring particles [34]+e performance of each particle isevaluated by a predefined fitness function that encapsulates thecore characteristics of the optimization problem

In each iteration every particle in the particle swarm getsits velocity and position by (3) and (4) respectively

Average

200

150

100

50

0

Pixe

l

Width Height0 50 100 150 200 250 300

200100

0

(g)

Median

200

150

100

50

0

Pixe

l

Width Height0 50 100 150 200 250 300

200100

0

(h)

Gaussian

200

150

100

50

0

Piex

l

Width Height0 50 100 150 200 250 300

200100

0

(i)Bilateral

200

150

100

50

0

Piex

l

Width Height0 50 100 150 200 250 300

200100

0

(j)

PCA

200

150

100

50

0

Piex

l

Width Height0 50 100 150 200 250 300

200100

0

(k)

PSO

200

150

100

50

0

Piex

l

Width Height0 50 100 150 200 250 300

200100

0

(l)

(m) (n) (o)

Figure 2 Representative images at each stage (a b) +e depth images represented by HSV format (c d) +e grey image corresponding tothe depth images in (a b) respectively (e) +e result of difference algorithm (f ) +e 3D perspective view of the result in (e) +erepresentative results (represented in 3D perspective view) using the different denoising algorithms (g) average filtering (h) medianfiltering (i) Gaussian filtering (j) bilateral filtering and (k) two-stage PCA filtering (l m) +e PSO denoising results in 3D and 2Drespectively (n o) +e results obtained by the segmentation algorithm based on MSER

4 Complexity

vk+1i w

kv

ki + c1r1 pbesti minus x

ki1113872 1113873 + c2r2 gbest minus x

ki1113872 1113873 (3)

xk+1i x

ki + v

ki (4)

where k is the current number of iterations xki and vk

i arerespectively the position and velocity of the ith particle inthe particle swarm during the kth iteration r1 and r2 are tworandom numbers in [0 1] respectively wk is the inertiaweight in the kth iteration pbesti is the optimal solutionavailable for the ith particle gbest is the optimal solutioncurrently available for all particles and c1 and c2 are indi-vidual learning factors and social learning factors respec-tively which are generally constant As recommended by DrKenney and Dr Eberhart [29] we define learning factorsc1 c2 2 In this case r1 or r2 multiplied by 2 to give it amean of 1 PSO can well take into account both sociallearning and individual learning [35] +e scale of theparticle swarm called M is directly related to the optimi-zation result and time consumption A small scale may causethe PSO to fail to find the optimal solution and a large scalewill cause unnecessary time costs [36] Consider the twopoints the particle swarm scale is defined as M 20

+e larger the inertia weight w is the stronger the globaloptimization ability is and the weaker the local optimizationability is [37] Otherwise the local optimization ability isstronger In order to strike a balance between search speedand search accuracy w should not be a fixed constant Anonlinear decreasing function for w is adopted in the paperas shown in the following equation

wk

wmax minus wmax minus wmin( 1113857lowast1

1 + alowast bmlowast kkmax( )

(5)

where wmax and wmin are the predefined maximum andminimum inertia weights respectively k and kmax are thecurrent and maximum number of iterations and m isin Nlowastagt 0 and 0lt blt 1 are adjustment factors of the polynomialAfter trial and error we define wmax 09 wmin 03kmax 100 a 2 b 06 and m 10 +e inertia weightcurve corresponding to the above parameters is shown inFigure 3 It guarantees that PSO has a high global search-ability in the early stage to get the appropriate seed and hashigher local searchability in the later stage to improve theconvergence accuracy

Besides we adopted the maximum interclass varianceequation (6) as the fitness function in this paper +e largerthe value of the fitness function is the closer to the optimalsolution it will be

f v0 lowast v1 lowast u0 minus u1( 11138572 (6)

where v0 and v1 are respectively the proportion of theforeground and background images to the image u0 and u1represent respectively the average grayscale of the fore-ground and background images

Figures 2(l) and 2(m) show the denoising results of PSOalgorithm in 3D and 2D perspective view respectivelyCompared with other denoising algorithms this algorithmcan achieve better denoising effect without blurring the

target contour In this section a D-PSO is introduced toremove the complicated background Compared with usingthe difference algorithm alone D-PSO can not only removethe complex background in surroundings but it can alsoreduce the noises that appear after applying the differencealgorithm

23 Head Segmentation Based on Maximally Stable ExtremalRegions (MSER) When the TOF camera is used the depthvalue for different parts of the pedestrian body varies greatlyIn order to extract the head region the maximally stableextremal regions (MSER) algorithm is used in the paper +eMSER algorithm refers to performing successive binariza-tion operations on a picture the binarization threshold iscontinuously increased from 0 to 255 [38] If a connectedregion in the image is changed a little or even is not changedwithin a wide range of the binarization threshold this regionis called the maximum stable extreme region Figure 2(n)shows the result obtained by the MSER along withFigure 2(m) In the figure different connected regions aremarked with different colours for clarity It is obvious thatMSER can separate different levels of pedestrian body parts

Fortunately regardless of the height and position ofpedestrians the head shapes of pedestrians are relativelystable ellipse even for pedestrians without hair +us thecircularity is used as a constraint to get the head region +ecircularity of each region is calculated by the followingequation

C 4π lowastA

l2 (7)

where C represents the circularity of the connected region lrepresents the number of pixels in the boundary of theconnected region and A represents the number of pixelswithin the connected region

+e standard circularity is 1 and the circularity of othernoncircular objects is less than 1 According to the exper-imental equipment and environment we had an empiricalconclusion that the circularity of head region is better be-tween 06 and 10 If a connected regionrsquos circularity isbeyond this range it would be remarked as the nonndashheadregion and deleted Due to the size of the pedestrian head inpractice the number of pixelsA is used as another constraintcondition After repeated tests we conclude that the A ofhead region should be during (300 900) In other words it ispossible to be a head region only if the A of the connectedregion is within the range As stated above the constraintscan be summarized in the following equation

300leAle 900

C 4π lowastA

l2

06leCle 10

⎧⎪⎪⎪⎪⎪⎪⎪⎪⎨

⎪⎪⎪⎪⎪⎪⎪⎪⎩

(8)

By calculating and comparing the above two parametersof each connected region in Figure 2(n) the head region is

Complexity 5

extracted as shown in the yellow part of Figure 2(o) Figure 4is the pixel distribution map of the extracted head regionwhere the black dots represent pixel points and the coor-dinates represent the positions of the pixels in the imageFrom this figure we can discover another advantage of theproposed MSER-based segmentation algorithm which canremove the notable noises in the head region such as salt-and-pepper noise Since the notable noise is very differentfrom its neighbour pixels it will not be incorporated into thehead region when the MSER algorithm is used to obtain thestable region +erefore the MSER-based segmentation caneffectively filter out notable noises in the head region asshown in the red rectangles in Figure 4 Note that the redrectangles are the manual markers for easy viewing

3 Real-Time Calculation for Pedestrian Height

31 Multilayer Iterative Average Algorithm for Pixel ValueAlthough the MSER algorithm can filter out the notablenoises there will still be some noises in the head region asshown in the 3D representation of the head region inFigure 2(f ) +e typical height measurement of only usingthe head top is not accurate +us a novel multilayer it-erative average algorithm (MLIA) is proposed to get thepixel average for getting the pedestrian height +e MLIAalgorithm not only can improve accuracy but also can ef-fectively remove some outliers that MSER cannot filter out+e MLIA can be broken down into the following steps

(1) Calculating the average of pixel value adopting thefollowing equation to get the average of pixel value inthe head region as

pave 1n

1113944

n

i1pi (9)

where pave is the pixel value average n is the numberof pixels in current head region and pi representsi minus th pixel value in current head region

(2) Updating the head region traverse all the pixels inthe head region and delete the pixels that do notmeet the following equation +e remaining pixelsare combined to update the head region

pi minus pave1113868111386811138681113868

1113868111386811138681113868leT pave( 1113857 (10)

where T(pave) is a threshold function related to thecurrent average pave and it is defined as follows

T pave( 1113857 Min pmax minus pave pave minus pmin( 1113857 (11)

where pmax and pmin are the maximum pixel valueand the minimum pixel value in the head regionrespectively

(3) Repeat step (1) and step (2) above until pave satisfythe following equation

Pave minuspmin + pmax

2le δ (12)

where δ is the empirical constant In this paper δ isselected as 20 according to the actual situation

+e above steps can be summarized as the followingpseudocode (Algorithm 1)

By the way the MLIA algorithm can also be applied tothe multipedestrian situation When the image containsmore than one pedestrian the MSER-based segmentationcan get more than one head region Meanwhile the pixelvalue average of each head region needs to be calculated bythe MLIA algorithm

32HeightCalculation Once pave is obtained the average ofthe head region in original pedestrian grey image (such as inFigure 2(d)) defined as pavg can be obtained through thedeformation of (2)

+en substituting pavg into (1) to replace pi we canobtain the following equation

10 807040 60 90 1003020 500Number of iterations

04

045

05

055

06

065

07

075

Iner

tia w

eigh

ts

Figure 3 +e curve of inertia weight in PSO

175 180 185 190 195170Width

120

125

130

135

140

145

150

Hei

ght

Figure 4 +e pixel distribution map of the extracted head region

6 Complexity

davg pavg dmax minus dmin( 1113857

255+ dmin (13)

where davg is the depth value corresponding to pavg and dmaxand dmin are the maximum and minimum depth values inthe pedestrian depth image

According to the physical properties of the TOF camerathe following conversion equation can be used to recover thephysical distance from the depth data davg[39]

Ddis Ktofdavg + E Ktofpavg dmax minus dmin( 1113857

255+ dmin1113888 1113889 + E

(14)

where Ddis represents the physical distance between the TOFcamera and the pedestrian head (unit mm) E is the de-viation constant associated with the physical structure andplacement height of the TOF camera while Ktof (512) isthe conversion coefficient only associated with the physicalstructure of the TOF camera

To allow our method to work for pedestrians who are notvertically below the TOF camera the pinhole model pro-posed in our previous work [26] is adopted to correct Ddis

Dco Ddis times cos arctanOMf

1113888 11138891113888 1113889 (15)

where Dco is the corrected physical distance f is the focallength and OM is the distance between the centroid of thehead region in the grey image M and the centre of the greyimage O the coordinates of the centroidM can be got by thefollowing equation More detailed information about thepinhole model can be found in the literature [26]

hp 1n

1113944

n

i1mihi

wp 1n

1113944

n

i1miwi

⎧⎪⎪⎪⎪⎪⎪⎪⎨

⎪⎪⎪⎪⎪⎪⎪⎩

(16)

where n is the number of pixels in current head region wp

and hp are the horizontal and vertical coordinates of thecentroid M and wi and hi are the horizontal and verticalcoordinates of the ith pixel respectively mi is the mass of theith pixel which is defined as mi 1 in this paper

Finally the pedestrian height H is calculated by thefollowing equation

H Htof minus Dco (17)

where Htof is the distance between the TOF camera and theground

33 Kalman Estimation of Real-Time Height In the exper-iments we found that the fluctuations of the pedestrianheights all approximately conform to the Gaussian distri-bution with variance 256 (unit mm2) and the variance didnot change with the state of the system +erefore Kalmanfiltering is further introduced to estimate the pedestrianheights got by (17) to achieve the more accurate real-timeheights Kalman filtering is a highly efficient recursive filterthat can estimate the state of a dynamic system from a seriesof measurements containing redundant noise [40] It cangenerate estimates of unknown variables which have provento be more accurate than those only based on a singlemeasurement [4 41] +e Kalman filter can be implementedin two stages time update stage and measurement updatestage [42]

+e time update stage is dedicated to predicting thecurrently a priori estimates through past state and the errorcovariance Equations (18) and (19) are responsible forpredicting the a priori state estimate 1113954xk and the a priori errorcovariance estimate 1113954Pk in current (kth) frame respectively

1113954xk Akminus 1xkminus 1 + Bukminus 1 (18)

1113954Pk Akminus 1Pkminus 1ATkminus 1 + Q (19)

where xkminus 1 and Pkminus 1 are respectively the state and the errorcovariance of the previous step Akminus 1 is the transfer matrixthat relates the state of the previous step to the state of the

Input S-initial head region extracted by the MSER-based segmentationProcedure

(1) n count (S)(2) pave (1n) 1113936

ni1 pi pi isin S

(3) pmax Max(pi) i 1 2 n(4) pmin Min(pi) i 1 2 n(5) while Pave minus ((pmin + pmax)2)gt δ do(7) T(pave) min (pmax minus pave pave minus pmin)(8) S pi | |pi minus pave|leT(pave) i 1 2 n1113864 1113865(9) n count (S)(10) pave (1n) 1113936

ni1 pi pi isin S

(11) pmax max(pi) i 1 2 n(12) pmin min(pi) i 1 2 n(13) end while

Output pave-the average of the pixels in the head region

ALGORITHM 1 Multilayer iterative average algorithm (MLIA)

Complexity 7

current step B is the control matrix that relates the previousinput ukminus 1 and Q is the variance of the Gaussian processnoise Based on the actual situation of pedestrians during themovement (no external input Gaussian distribution of theheight fluctuation and continuity of the height change) theparameters in time update stage are defined as followsukminus 1 equiv 0 Q equiv 256 Akminus 1 equiv 1 1113954xk is the a priori height estimatefrom the current depth image

+e measurement update stage is devoted to combiningactual measurements with a priori estimates to get theimproved posteriori estimates [42] It can be achieved by thefollowing equations

Kk 1113954PkHTk Hk

1113954PkHTk + R1113872 1113873

minus 1 (20)

xk 1113954xk + K Zk minus Hk1113954xk( 1113857 (21)

Pk I minus KkHk( 11138571113954Pk (22)

where xk and Pk are the posteriori state estimate and theposteriori error covariance estimate in current (kth) step Kk

is the Kalman gain in current step Hk is the matrix thatrelates the state to the measurement Zk I is a unit matrixand R is the variance of the Gaussian measurement noiseBased on the actual situation of measurements (cameraaccuracy and measurement process) the parameters inmeasurement update stage are defined as follows R equiv 144Hk equiv 1 xk is the posteriori height estimate from the currentdepth image and Zk is the pedestrian heights got by (17) Inaddition the initialization is defined as x1 Z1 and P1 10

4 Experiments and Analysis

41 Experimental Setup In this paper an EPC660 is used asthe TOF chip to offer a fully digital interface for the controlcircuitry and the communication between computer andcamera is realized through Gigabit network In addition theexperiment is completed with the support of the computerwith Windows 10 OS Intelreg Coretrade i3-8100 360GHz CPUand 8GB RAM +e campus corridor is selected as the firsttest site and the experimental scene is shown in Figure 5(a)+en considering the fluctuation of pedestrian height indynamic situations the research room is chosen as thesecond test site and the VICON system fixed in this site isadopted as the ground truth to confirm the feasibility of theproposed method +e experimental scene in research roomis shown in Figure 5(b) where a portion of the VICONsystem two of the 12 infrared cameras is shown While theVICON is running four lightweight reflective balls are stuckto the pedestrianrsquos head the placement layout of the balls isshown in Figure 5(c) And the average height of the four ballsis adopted as the real-time height of the pedestrian

42 Comparison with Other Popular Algorithms Before thePSO algorithm is adopted to process the images with un-wanted noise other popular algorithms are deployed toprocess the same images for a comparison More specificallythree algorithms are implemented for comparison here

(1) Maximum Connected Region (MCR) As the nameimplies MCR refers to the method of extracting thelargest connected region in an image When only asingle person appears in the field of view such as inFigure 2(e) MCR is more likely to get desirableresults than PSO In the actual situation however wedo not know in advance how many people will gothrough the test site Take Figure 6(a) as an examplewhen two people go through the test site at the sametime MCR may get a wrong result as shown inFigures 6(b) and 6(c)

(2) Edge 9reshold Method (ETM) In ETM the edgeoperators such as Canny is firstly used to obtain thepossible target contours and the number of pixels inthese contours is then calculated respectively Oncethe number is bigger than a specific threshold theregion enclosed by the corresponding contour isconsidered as the useful region and is retainedotherwise this region is considered as the uselessregion and is removed In the paper the boundarybetween the target person and the redundant noise isusually solid which makes it possible to split thetarget from the background with the ETM Moreimportantly the ETM can also get good results inmultipedestrian images with appropriate parame-ters However it is a very difficult task for the ETM toadaptively select parameters Once the test envi-ronment changes the parameters of ETM need to bereselected which limits the application of the ETM

(3) Reaction Diffusion-Level Set Evolution (RD-LSE) +eRD-LES proposed by Zhang et al [43] is an im-proved level set algorithm which is widely used inthe field of image segmentation Figure 6(d) showsthe search process using the RD-LSE algorithm forthe Figure 6(a) in which the yellow curves show theevolution processes the green curve represents theinitial contour and the red curve represents the finalcontour +is algorithm can achieve a better resultthan PSO algorithm even in the case of multiplepedestrians as shown in Figures 6(e) and 6(f ) In thepaper we take 4 different types of pictures as ex-amples to compare the performance of RD-FLS andPSO in terms of converged iterations and CPU time+e experimental results are shown in Table 1 whereimages 1ndash4 represent Figures 2(e) 6(a) 7(a) and7(g) respectively +e values in table are the averageof 100 experiments Table 1 shows that the com-putational efficiency of the PSO algorithm far ex-ceeds the RD-FLS which is the main reason why wechoose PSO

43 Experimental Results Apart from the multipedestriancases such as in Figure 6 many other cases with the pedestrianin different states are studied to verify the effectiveness androbustness of the proposed method In Figure 7(a) the pe-destrian raised his left hand above his head Figures 7(c) 7(e)and 7(f) show the experimental process and result of adoptingthe proposed method for Figure 7(a) For clarity the 3D

8 Complexity

(a) (b)

Front

Back

Mark 3

Mark 4

Mark 1

Mark 2

(c)

Figure 5 Experimental setup (a) Site campus corridor (b) Site research room (c) Placement layout of the lightweight reflective balls

(a) (b) (c)

(d) (e) (f )

Figure 6 Experiments with two-pedestrian image (a) Original image with two pedestrians (b) Image obtained by theMCR algorithm alongwith the original image (c) Image obtained by theMSER-based segmentation along with (b) (d e)+e processes and result images obtainedby the RD-LSE along with the original image (f ) Image obtained by the MSER-based segmentation along with that in (e)

Table 1 Iterations (Iter) and CPU time (Time) by FRFLS and PSO methods

MethodsImage 1 Image 2 Image 3 Image 4

Time (s) Iter Time (s) Iter Time (s) Iter Time (s) IterFRFLS 521 643 601 800 532 665 487 611PSO 0049 129 0057 86 0053 185 0050 147Image size 320 lowast 240 pixels

Complexity 9

representations of Figures 7(a) and 7(c) are shown inFigures 7(b) and 7(d) respectively Although the height of thehead is lower than that of the left hand the proposed methodcan still get the correct result Figures 7(i) 7(k) and 7(l) showthe experimental process and result of adopting the proposed

method for Figure 7(g) in which a pedestrian is kneelingAlthough the proposed D-PSO algorithm does not eliminateall redundant noises as shown in Figure 7(j) it also yieldsideal experimental results due to MSERrsquos insensitivity to asmall amount of the sporadic noise All the above experiments

(a)

200

150

100

50

0

Pixe

l

050

100150

200HeightWidth

050100150200250300

(b) (c)

200

150

100

50

0

Pixe

l

050

100150

200HeightWidth

050100150200250300

(d) (e) (f )

(g)

10080604020

0

Pixe

l

250200

150100

500Height

Width

300

200

0100

(h) (i)

150

100

50

0

Pixe

l

200150

10050

0Height

Width

200100

0

300

(j) (k) (l)

Figure 7 Experiments with the pedestrian in different states (a) Original image with the pedestrian raising his left hand (c) Image obtainedby the PSO algorithm along with that in (a) (e f ) Images obtained by the MSER-based segmentation along with that in (c) (b d) +e 3Drepresentation of images in (a c) respectively (g) Original image with the pedestrian who is kneeling (i) Image obtained by the PSOalgorithm along with that in (g) (k l) Images obtained by theMSER-based segmentation along with those in (i) (h j)+e 3D representationof images in (g i) respectively

10 Complexity

show that the performance of our method is very stable andreliable

To further verify the accuracy of the proposed method alot of experiments are conducted based on 6 subjects fourmen and two women who are asked to walk through the testsites at the usual speed Here we take a set of data obtainedfrom the research room as an example to analyse the resultsFigure 8 shows the height results obtained from the sixsubjects using the VICON alone in several continuousseconds the sex and static height of the six subjects arepresented in the legend It explains that it is unrealistic to

keep the height on the static level when the pedestrian iswalking +us it is essential to study the pedestrian height inthe dynamic situation

Due to the high speed of pictures taken by VICON andTOF cameras and the slowness of pedestrian movement(07ndash12 meters per second) we only select 5 height data persecond to show a real-time height comparison between theVICON and the proposedmethod Every fifth of one secondan image is collected with the TOF camera +e pedestrianheight in the image is obtained by the proposed method andcompared with the height collected with VICON at the same

0 100 200 300 400 500 600 700 800 900 1000 1100 1200Number

160016101620163016401650166016701680169017001710172017301740175017601770178017901800

Hei

ght (

mm

)

Men1760167617611728

Women16481629

Figure 8 +e height results got from the six subjects using the VICON alone in several continuous seconds

1800179017801770176017501740173017201710170016901680167016601650

Mea

sure

d he

ight

(mm

)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30Number

Our algorithm without Kalman filtering1760167617611728

Our algorithm with Kalman filtering1760167617611728

VICON (ground truth)1760167617611728

Figure 9 Experimental results of men with different heights in the six consecutive seconds

Complexity 11

1700

1690

1680

1670

1660

1650

1640

1630

1620

1610

1600

1590

1580

1570

Mea

sure

d he

ight

(mm

)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30Number

Our algorithm without Kalman filtering16481629

Our algorithm with Kalman filtering16481629

VICON (ground truth)16481629

Figure 10 Experimental results of women with different heights in the six consecutive seconds

28272625242322211011121314151617181920 29308765432 91

Number

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

(a)

2 3 4 5 6 7 8 91 111213141516171819202122232425262728293010

Number

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

(b)

43 61 2 7 8 95 111213141516171819202122232425262728293010

Number

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

(c)

43 61 2 7 8 95 111213141516171819202122232425262728293010

Number

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

(d)

Figure 11 +e error plot of men in the six consecutive seconds (andashd) +e men with static heights of 1760 1676 1761 and 1728

12 Complexity

time Figures 9 and 10 show the experimental results of fourmen and two women in six consecutive seconds In thefigures the dotted line represents our algorithm withoutKalman filtering the solid line represents our algorithmwithout Kalman filtering and the dotted line with the markldquo+rdquo indicates the VICON+e waveforms show the real-timeheight value in 6 consecutive seconds the static heights ofmen are 1760mm 1676mm 1761mm and 1728mm asshown in the legend of Figure 9 while the static heights ofwomen are 1648mm and 1629mm as shown in Figure 10

It can be seen from the curves that the height datameasured by our algorithm is almost consistent with the dataobtained by VICON In order to analyse the error of ouralgorithm we sort out the errors of all the data in the sixconsecutive seconds the results are shown in Figures 11 and12 +e figures show that Kalman filtering can effectivelyimprove the accuracy of height measurement which indi-cates the pedestrian height at the preceding moment facil-itates the estimate of the pedestrian height in the lattermoment

Also the sums of errors per second of the algorithmswith and without Kalman filtering are given in Table 2where the subscript ldquolowastrdquo represents male and ldquordquo representsfemale Table 2 shows that our algorithm with Kalmanfiltering has a smaller cumulative error and can moreaccurately measure the real-time height of the movingpedestrians which proves the feasibility and validity of theproposed method

5 Conclusion and Future Work

In this paper a real-time height measurement based onthe TOF camera is proposed for moving pedestrians Toget the target region a new D-PSO denoising algorithmand a segmentation algorithm based on MSER are de-veloped in the paper In addition a novel multilayer it-erative average algorithm is designed for calculating thepedestrian height Also the Kalman filtering is used toimprove the measurement accuracy +e experimentalresults demonstrate the effectiveness and practicability of

2 3 4 5 6 7 8 9 1011121314151617181920212223242526272829301Number

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2Er

ror (

)

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

(a)

2 3 4 5 6 7 8 9 1011121314151617181920212223242526272829301Number

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

(b)

Figure 12 +e error plot of women in the six consecutive seconds (a) +e woman with static height of 1648 (b) +e woman with staticheight of 1629

Table 2 +e sum of errors per second of the algorithms with and without Kalman filtering

Heights (mm) Kalman filteringSum of errors per second ()

Sum1st second 2nd second 3rd second 4th second 5th second 6th second

1760lowast Yes 1202 0956 1836 1242 1611 1525 8372No 1868 1003 2013 1362 1898 1758 9902

1676lowast Yes 2002 1799 1977 0863 1648 2137 10426No 2249 1968 2087 1602 1827 3261 12994

1761lowast Yes 1282 1483 0963 1132 0632 1487 6979No 1562 1702 1333 1617 1234 1714 9162

1728lowast Yes 1629 1652 1354 1453 1224 0902 8214No 2201 2159 1912 1592 1984 1336 11184

1648 Yes 2006 1194 1818 1014 1585 1693 9310No 2488 1245 2152 1906 2078 2087 11956

1629 Yes 1509 1838 0652 2344 1398 1109 8850No 1632 2536 1328 2508 1497 1340 10841

lowastMale female

Complexity 13

the proposed method Our future work is going to furtherimprove the measurement accuracy and focus on trackingpedestrians in real time by using the real-time height ofmoving pedestrians

Data Availability

+e data used to support the findings of this study areavailable from the corresponding author upon request

Conflicts of Interest

+e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

+e authors are grateful to the financial support from theNatural Science Foundation of China (61877065) the NationalKey Research and Development Program of China(2019YFB1405500) the National Natural Science Foundationof Guangdong (2016A030313177) Guangdong Frontier andKey Technological Innovation (2017B090910013) and theScience and Technology Innovation Commission of Shenzhen(JCYJ20170818153048647 and JCYJ20180507182239617)

References

[1] J Li X Liang S Shen et al ldquoScale-aware fast R-CNN forpedestrian detectionrdquo IEEE Transactions on Multimediavol 20 no 4 pp 985ndash996 2017

[2] F P An ldquoPedestrian re-recognition algorithm based onoptimization deep learning-sequence memory modelrdquoComplexity vol 2019 Article ID 5069026 16 pages 2019

[3] J Cao Y Pang and X Li ldquoLearning multilayer channelfeatures for pedestrian detectionrdquo IEEE Transactions on ImageProcessing vol 26 no 7 pp 3210ndash3220 2017

[4] M Ji J Liu X Xu Y Guo and Z Lu ldquoImproved pedestrianpositioning with inertial sensor based on adaptive gradientdescent and double-constrained extended kalman filterrdquoComplexity vol 2020 Article ID 4361812 11 pages 2020

[5] C Li Z Su Q Li and H Zhao ldquoAn indoor positioning errorcorrection method of pedestrian multi-motions recognized byhybrid-orders fraction domain transformationrdquo IEEE Accessvol 7 pp 11360ndash11377 2019

[6] H Zhao W Cheng N Yang et al ldquoSmartphone-based 3Dindoor pedestrian positioning through multi-modal datafusionrdquo Sensors vol 19 no 20 Article ID s19204554 2019

[7] B Wang T Su X Jin J Kong and Y Bai ldquo3D reconstructionof pedestrian trajectory with moving direction learning andoptimal gait recognitionrdquo Complexity vol 2018 Article ID8735846 10 pages 2018

[8] Y Jiang Z Li and J B Wang ldquoPtrack enhancing the ap-plicability of pedestrian tracking with wearablesrdquo IEEETransactions on Mobile Computing vol 18 no 2 pp 431ndash4432018

[9] W Xu L Liu S Zlatanova W Penard and Q Xiong ldquoApedestrian tracking algorithm using grid-based indoormodelrdquo Automation in Construction vol 92 pp 173ndash1872018

[10] L Bozgeyikli A Raij S Katkoori and R Alqasemi ldquoA surveyon virtual reality for individuals with autism spectrum

disorder design considerationsrdquo IEEE Transactions onLearning Technologies vol 11 no 2 pp 133ndash151 2017

[11] J O Nilsson D Zachariah I Skog and P Handel ldquoCoop-erative localization by dual foot-mounted inertial sensors andinter-agent rangingrdquo EURASIP Journal on Advances in SignalProcessing vol 2013 no 1 p 164 2013

[12] I Skog J-O Nilsson D Zachariah and P Handel ldquoFusingthe information from two navigation systems using an upperbound on their maximum spatial separationrdquo in Proceedingsof the 2012 International Conference on Indoor Positioning andIndoor Navigation Article ID 6418862 Sydney AustraliaNovember 2012

[13] S-B Chen Y Xin and B Luo ldquoAction-based pedestrianidentification via hierarchical matching pursuit and orderpreserving sparse codingrdquo Cognitive Computation vol 8no 5 pp 797ndash805 2016

[14] B Shin C Kim J Kim et al ldquoMotion recognition based 3Dpedestrian navigation system using smartphonerdquo IEEE Sen-sors Journal vol 16 no 18 pp 6977ndash6989 2016

[15] M Romanovas V Goridko A Al-Jawad et al ldquoA study onindoor pedestrian localization algorithms with foot-mountedsensorsrdquo in Proceedings of the International Conference onIndoor Positioning and Indoor Navigation pp 1ndash10 SydneyAustralia November 2012

[16] A Azaman ldquoComparative study on gait kinematics betweenmicrosoft kinect and vicon across different anthropometricmeasurementsrdquo Journal of Tomography System and SensorApplication vol 2 no 2 pp 12ndash17 2019

[17] W Sheng A +obbi and Y Gu ldquoAn integrated frameworkfor human-robot collaborative manipulationrdquo IEEE Trans-actions on Cybernetics vol 45 no 10 pp 2030ndash2041 2014

[18] S Tsuji and T Kohama ldquoProximity skin sensor using time-of-flight sensor for human collaborative robotrdquo IEEE SensorsJournal vol 19 no 14 pp 5859ndash5864 2019

[19] C Oprea I Pirnog I Marcu and M Udrea ldquoRobust poseestimation using Time-of-Flight imagingrdquo in Proceedings ofthe IEEE International Semiconductor Conference pp 301ndash304 Sinaia Romania January 2019

[20] A Vysocky R Pastor and P Novak ldquoInteraction with col-laborative robot using 2D and TOF camerardquo in InternationalConference on Modelling and Simulation for AutonomousSystems pp 477ndash489 Springer Cham Switzerland 2018

[21] M Gao Y Du Y Yang and J Zhang ldquoAdaptive anchor boxmechanism to improve the accuracy in the object detectionsystemrdquo Multimedia Tools and Applications vol 78 no 19pp 27383ndash27402 2019

[22] A Anwer S S Azhar Ali A Khan and F MeriaudeauldquoUnderwater 3-d scene reconstruction using kinect v2 basedon physical models for refraction and time of flight correc-tionrdquo IEEE Access vol 5 pp 15960ndash15970 2017

[23] A R Garcıa L R Miller C F Andres and P J N LorenteldquoObstacle detection using a time of flight range camerardquo inProceedings of the 2018 IEEE International Conference onVehicular Electronics and Safety (ICVES) pp 1ndash6 MadridSpain September 2018

[24] N Zengeler T Kopinski and U Handmann ldquoHand gesturerecognition in automotive humanndashmachine interaction usingdepth camerasrdquo Sensors vol 19 no 1 Article ID s190100592019

[25] M A Garduntildeo-Ramon I R Terol-Villalobos R A Osornio-Rios and L A Morales-Hernandez ldquoA new method forinpainting of depthmaps from time-of-flight sensors based ona modified closing by reconstruction algorithmrdquo Journal of

14 Complexity

Visual Communication and Image Representation vol 47pp 36ndash47 2019

[26] L Wang Y Luo H Wang and M Fei ldquoMeasurement errorcorrection model of TOF depth camerardquo Chinese Journal ofSystem Simulation vol 29 no 10 pp 2323ndash2329 2017

[27] VICON ldquoOfficial website of oxford metrics companyrdquo 2020httpswwwviconcom

[28] L Zhang W Dong D Zhang and G Shi ldquoTwo-stage imagedenoising by principal component analysis with local pixelgroupingrdquo Pattern Recognition vol 43 no 4 pp 1531ndash15492010

[29] J Kennedy and R Eberhart ldquoParticle swarm optimizationrdquo inProceedings of ICNNrsquo95-International Conference on NeuralNetworks (ICW) vol 4 pp 1942ndash1948 Perth AustraliaNovember 1995

[30] M A M De Oca T Stutzle M Birattari and M DorigoldquoFrankensteinrsquos PSO a composite particle swarm optimiza-tion algorithmrdquo IEEE Transactions on Evolutionary Compu-tation vol 13 no 5 pp 1120ndash1132 2009

[31] Z Zhen S Pang F Wang et al ldquoPattern classification andPSO optimal weights based sky images cloud motion speedcalculation method for solar PV power forecastingrdquo IEEETransactions on Industry Applications vol 55 no 4pp 3331ndash3342 2019

[32] X Wang J-S Pan and S-C Chu ldquoA parallel multi-verseoptimizer for application in multilevel image segmentationrdquoIEEE Access vol 8 pp 32018ndash32030 2020

[33] Z A Bashir and M E El-Hawary ldquoApplying wavelets toshort-term load forecasting using PSO-based neural net-worksrdquo IEEE Transactions on Power Systems vol 24 no 1pp 20ndash27 2009

[34] L Liu Y Wang F Xie and J Gao ldquoLegendre cooperativePSO strategies for trajectory optimizationrdquo Complexityvol 2018 Article ID 5036791 13 pages 2018

[35] Y Shi and R C Eberhart ldquoA modified particle swarm op-timizerrdquo in Proceedings of the 1998 IEEE InternationalConference on Evolutionary Computation Proceedingspp 69ndash73 Anchorage AK USA May 1998

[36] Y Shi and R C Eberhart ldquoParameter selection in particleswarm optimizationrdquo in International Conference on Evolu-tionary Programming pp 591ndash600 Springer Berlin Ger-many 1998

[37] X Lv D Zhou Y Tang and L Ma ldquoAn improved test se-lection optimization model based on fault ambiguity groupisolation and chaotic discrete PSOrdquo Complexity vol 2018Article ID 3942723 10 pages 2018

[38] J Matas O Chum M Urban and T Pajdla ldquoRobust wide-baseline stereo from maximally stable extremal regionsrdquoImage and Vision Computing vol 22 no 10 pp 761ndash7672004

[39] H Shim and S Lee ldquoRecovering translucent objects using asingle time-of-flight depth camerardquo IEEE Transactions onCircuits and Systems for Video Technology vol 26 no 5pp 841ndash854 2015

[40] C K Chui and G ChenKalman Filtering pp 19ndash26 SpringerInternational Publishing Berlin Germany 2017

[41] L Cui X Wang Y Xu H Jiang and J Zhou ldquoA novelswitching unscented Kalman filter method for remaininguseful life prediction of rolling bearingrdquo Measurementvol 135 pp 678ndash684 2019

[42] GWelch and G BishopAn Introduction to the Kalman FilterMacmillan New York NY USA 1995

[43] K Zhang L Zhang H Song and D Zhang ldquoReinitialization-free level set evolution via reaction diffusionrdquo IEEE Trans-actions on Image Processing vol 22 no 1 pp 258ndash271 2012

Complexity 15

vk+1i w

kv

ki + c1r1 pbesti minus x

ki1113872 1113873 + c2r2 gbest minus x

ki1113872 1113873 (3)

xk+1i x

ki + v

ki (4)

where k is the current number of iterations xki and vk

i arerespectively the position and velocity of the ith particle inthe particle swarm during the kth iteration r1 and r2 are tworandom numbers in [0 1] respectively wk is the inertiaweight in the kth iteration pbesti is the optimal solutionavailable for the ith particle gbest is the optimal solutioncurrently available for all particles and c1 and c2 are indi-vidual learning factors and social learning factors respec-tively which are generally constant As recommended by DrKenney and Dr Eberhart [29] we define learning factorsc1 c2 2 In this case r1 or r2 multiplied by 2 to give it amean of 1 PSO can well take into account both sociallearning and individual learning [35] +e scale of theparticle swarm called M is directly related to the optimi-zation result and time consumption A small scale may causethe PSO to fail to find the optimal solution and a large scalewill cause unnecessary time costs [36] Consider the twopoints the particle swarm scale is defined as M 20

+e larger the inertia weight w is the stronger the globaloptimization ability is and the weaker the local optimizationability is [37] Otherwise the local optimization ability isstronger In order to strike a balance between search speedand search accuracy w should not be a fixed constant Anonlinear decreasing function for w is adopted in the paperas shown in the following equation

wk

wmax minus wmax minus wmin( 1113857lowast1

1 + alowast bmlowast kkmax( )

(5)

where wmax and wmin are the predefined maximum andminimum inertia weights respectively k and kmax are thecurrent and maximum number of iterations and m isin Nlowastagt 0 and 0lt blt 1 are adjustment factors of the polynomialAfter trial and error we define wmax 09 wmin 03kmax 100 a 2 b 06 and m 10 +e inertia weightcurve corresponding to the above parameters is shown inFigure 3 It guarantees that PSO has a high global search-ability in the early stage to get the appropriate seed and hashigher local searchability in the later stage to improve theconvergence accuracy

Besides we adopted the maximum interclass varianceequation (6) as the fitness function in this paper +e largerthe value of the fitness function is the closer to the optimalsolution it will be

f v0 lowast v1 lowast u0 minus u1( 11138572 (6)

where v0 and v1 are respectively the proportion of theforeground and background images to the image u0 and u1represent respectively the average grayscale of the fore-ground and background images

Figures 2(l) and 2(m) show the denoising results of PSOalgorithm in 3D and 2D perspective view respectivelyCompared with other denoising algorithms this algorithmcan achieve better denoising effect without blurring the

target contour In this section a D-PSO is introduced toremove the complicated background Compared with usingthe difference algorithm alone D-PSO can not only removethe complex background in surroundings but it can alsoreduce the noises that appear after applying the differencealgorithm

23 Head Segmentation Based on Maximally Stable ExtremalRegions (MSER) When the TOF camera is used the depthvalue for different parts of the pedestrian body varies greatlyIn order to extract the head region the maximally stableextremal regions (MSER) algorithm is used in the paper +eMSER algorithm refers to performing successive binariza-tion operations on a picture the binarization threshold iscontinuously increased from 0 to 255 [38] If a connectedregion in the image is changed a little or even is not changedwithin a wide range of the binarization threshold this regionis called the maximum stable extreme region Figure 2(n)shows the result obtained by the MSER along withFigure 2(m) In the figure different connected regions aremarked with different colours for clarity It is obvious thatMSER can separate different levels of pedestrian body parts

Fortunately regardless of the height and position ofpedestrians the head shapes of pedestrians are relativelystable ellipse even for pedestrians without hair +us thecircularity is used as a constraint to get the head region +ecircularity of each region is calculated by the followingequation

C 4π lowastA

l2 (7)

where C represents the circularity of the connected region lrepresents the number of pixels in the boundary of theconnected region and A represents the number of pixelswithin the connected region

+e standard circularity is 1 and the circularity of othernoncircular objects is less than 1 According to the exper-imental equipment and environment we had an empiricalconclusion that the circularity of head region is better be-tween 06 and 10 If a connected regionrsquos circularity isbeyond this range it would be remarked as the nonndashheadregion and deleted Due to the size of the pedestrian head inpractice the number of pixelsA is used as another constraintcondition After repeated tests we conclude that the A ofhead region should be during (300 900) In other words it ispossible to be a head region only if the A of the connectedregion is within the range As stated above the constraintscan be summarized in the following equation

300leAle 900

C 4π lowastA

l2

06leCle 10

⎧⎪⎪⎪⎪⎪⎪⎪⎪⎨

⎪⎪⎪⎪⎪⎪⎪⎪⎩

(8)

By calculating and comparing the above two parametersof each connected region in Figure 2(n) the head region is

Complexity 5

extracted as shown in the yellow part of Figure 2(o) Figure 4is the pixel distribution map of the extracted head regionwhere the black dots represent pixel points and the coor-dinates represent the positions of the pixels in the imageFrom this figure we can discover another advantage of theproposed MSER-based segmentation algorithm which canremove the notable noises in the head region such as salt-and-pepper noise Since the notable noise is very differentfrom its neighbour pixels it will not be incorporated into thehead region when the MSER algorithm is used to obtain thestable region +erefore the MSER-based segmentation caneffectively filter out notable noises in the head region asshown in the red rectangles in Figure 4 Note that the redrectangles are the manual markers for easy viewing

3 Real-Time Calculation for Pedestrian Height

31 Multilayer Iterative Average Algorithm for Pixel ValueAlthough the MSER algorithm can filter out the notablenoises there will still be some noises in the head region asshown in the 3D representation of the head region inFigure 2(f ) +e typical height measurement of only usingthe head top is not accurate +us a novel multilayer it-erative average algorithm (MLIA) is proposed to get thepixel average for getting the pedestrian height +e MLIAalgorithm not only can improve accuracy but also can ef-fectively remove some outliers that MSER cannot filter out+e MLIA can be broken down into the following steps

(1) Calculating the average of pixel value adopting thefollowing equation to get the average of pixel value inthe head region as

pave 1n

1113944

n

i1pi (9)

where pave is the pixel value average n is the numberof pixels in current head region and pi representsi minus th pixel value in current head region

(2) Updating the head region traverse all the pixels inthe head region and delete the pixels that do notmeet the following equation +e remaining pixelsare combined to update the head region

pi minus pave1113868111386811138681113868

1113868111386811138681113868leT pave( 1113857 (10)

where T(pave) is a threshold function related to thecurrent average pave and it is defined as follows

T pave( 1113857 Min pmax minus pave pave minus pmin( 1113857 (11)

where pmax and pmin are the maximum pixel valueand the minimum pixel value in the head regionrespectively

(3) Repeat step (1) and step (2) above until pave satisfythe following equation

Pave minuspmin + pmax

2le δ (12)

where δ is the empirical constant In this paper δ isselected as 20 according to the actual situation

+e above steps can be summarized as the followingpseudocode (Algorithm 1)

By the way the MLIA algorithm can also be applied tothe multipedestrian situation When the image containsmore than one pedestrian the MSER-based segmentationcan get more than one head region Meanwhile the pixelvalue average of each head region needs to be calculated bythe MLIA algorithm

32HeightCalculation Once pave is obtained the average ofthe head region in original pedestrian grey image (such as inFigure 2(d)) defined as pavg can be obtained through thedeformation of (2)

+en substituting pavg into (1) to replace pi we canobtain the following equation

10 807040 60 90 1003020 500Number of iterations

04

045

05

055

06

065

07

075

Iner

tia w

eigh

ts

Figure 3 +e curve of inertia weight in PSO

175 180 185 190 195170Width

120

125

130

135

140

145

150

Hei

ght

Figure 4 +e pixel distribution map of the extracted head region

6 Complexity

davg pavg dmax minus dmin( 1113857

255+ dmin (13)

where davg is the depth value corresponding to pavg and dmaxand dmin are the maximum and minimum depth values inthe pedestrian depth image

According to the physical properties of the TOF camerathe following conversion equation can be used to recover thephysical distance from the depth data davg[39]

Ddis Ktofdavg + E Ktofpavg dmax minus dmin( 1113857

255+ dmin1113888 1113889 + E

(14)

where Ddis represents the physical distance between the TOFcamera and the pedestrian head (unit mm) E is the de-viation constant associated with the physical structure andplacement height of the TOF camera while Ktof (512) isthe conversion coefficient only associated with the physicalstructure of the TOF camera

To allow our method to work for pedestrians who are notvertically below the TOF camera the pinhole model pro-posed in our previous work [26] is adopted to correct Ddis

Dco Ddis times cos arctanOMf

1113888 11138891113888 1113889 (15)

where Dco is the corrected physical distance f is the focallength and OM is the distance between the centroid of thehead region in the grey image M and the centre of the greyimage O the coordinates of the centroidM can be got by thefollowing equation More detailed information about thepinhole model can be found in the literature [26]

hp 1n

1113944

n

i1mihi

wp 1n

1113944

n

i1miwi

⎧⎪⎪⎪⎪⎪⎪⎪⎨

⎪⎪⎪⎪⎪⎪⎪⎩

(16)

where n is the number of pixels in current head region wp

and hp are the horizontal and vertical coordinates of thecentroid M and wi and hi are the horizontal and verticalcoordinates of the ith pixel respectively mi is the mass of theith pixel which is defined as mi 1 in this paper

Finally the pedestrian height H is calculated by thefollowing equation

H Htof minus Dco (17)

where Htof is the distance between the TOF camera and theground

33 Kalman Estimation of Real-Time Height In the exper-iments we found that the fluctuations of the pedestrianheights all approximately conform to the Gaussian distri-bution with variance 256 (unit mm2) and the variance didnot change with the state of the system +erefore Kalmanfiltering is further introduced to estimate the pedestrianheights got by (17) to achieve the more accurate real-timeheights Kalman filtering is a highly efficient recursive filterthat can estimate the state of a dynamic system from a seriesof measurements containing redundant noise [40] It cangenerate estimates of unknown variables which have provento be more accurate than those only based on a singlemeasurement [4 41] +e Kalman filter can be implementedin two stages time update stage and measurement updatestage [42]

+e time update stage is dedicated to predicting thecurrently a priori estimates through past state and the errorcovariance Equations (18) and (19) are responsible forpredicting the a priori state estimate 1113954xk and the a priori errorcovariance estimate 1113954Pk in current (kth) frame respectively

1113954xk Akminus 1xkminus 1 + Bukminus 1 (18)

1113954Pk Akminus 1Pkminus 1ATkminus 1 + Q (19)

where xkminus 1 and Pkminus 1 are respectively the state and the errorcovariance of the previous step Akminus 1 is the transfer matrixthat relates the state of the previous step to the state of the

Input S-initial head region extracted by the MSER-based segmentationProcedure

(1) n count (S)(2) pave (1n) 1113936

ni1 pi pi isin S

(3) pmax Max(pi) i 1 2 n(4) pmin Min(pi) i 1 2 n(5) while Pave minus ((pmin + pmax)2)gt δ do(7) T(pave) min (pmax minus pave pave minus pmin)(8) S pi | |pi minus pave|leT(pave) i 1 2 n1113864 1113865(9) n count (S)(10) pave (1n) 1113936

ni1 pi pi isin S

(11) pmax max(pi) i 1 2 n(12) pmin min(pi) i 1 2 n(13) end while

Output pave-the average of the pixels in the head region

ALGORITHM 1 Multilayer iterative average algorithm (MLIA)

Complexity 7

current step B is the control matrix that relates the previousinput ukminus 1 and Q is the variance of the Gaussian processnoise Based on the actual situation of pedestrians during themovement (no external input Gaussian distribution of theheight fluctuation and continuity of the height change) theparameters in time update stage are defined as followsukminus 1 equiv 0 Q equiv 256 Akminus 1 equiv 1 1113954xk is the a priori height estimatefrom the current depth image

+e measurement update stage is devoted to combiningactual measurements with a priori estimates to get theimproved posteriori estimates [42] It can be achieved by thefollowing equations

Kk 1113954PkHTk Hk

1113954PkHTk + R1113872 1113873

minus 1 (20)

xk 1113954xk + K Zk minus Hk1113954xk( 1113857 (21)

Pk I minus KkHk( 11138571113954Pk (22)

where xk and Pk are the posteriori state estimate and theposteriori error covariance estimate in current (kth) step Kk

is the Kalman gain in current step Hk is the matrix thatrelates the state to the measurement Zk I is a unit matrixand R is the variance of the Gaussian measurement noiseBased on the actual situation of measurements (cameraaccuracy and measurement process) the parameters inmeasurement update stage are defined as follows R equiv 144Hk equiv 1 xk is the posteriori height estimate from the currentdepth image and Zk is the pedestrian heights got by (17) Inaddition the initialization is defined as x1 Z1 and P1 10

4 Experiments and Analysis

41 Experimental Setup In this paper an EPC660 is used asthe TOF chip to offer a fully digital interface for the controlcircuitry and the communication between computer andcamera is realized through Gigabit network In addition theexperiment is completed with the support of the computerwith Windows 10 OS Intelreg Coretrade i3-8100 360GHz CPUand 8GB RAM +e campus corridor is selected as the firsttest site and the experimental scene is shown in Figure 5(a)+en considering the fluctuation of pedestrian height indynamic situations the research room is chosen as thesecond test site and the VICON system fixed in this site isadopted as the ground truth to confirm the feasibility of theproposed method +e experimental scene in research roomis shown in Figure 5(b) where a portion of the VICONsystem two of the 12 infrared cameras is shown While theVICON is running four lightweight reflective balls are stuckto the pedestrianrsquos head the placement layout of the balls isshown in Figure 5(c) And the average height of the four ballsis adopted as the real-time height of the pedestrian

42 Comparison with Other Popular Algorithms Before thePSO algorithm is adopted to process the images with un-wanted noise other popular algorithms are deployed toprocess the same images for a comparison More specificallythree algorithms are implemented for comparison here

(1) Maximum Connected Region (MCR) As the nameimplies MCR refers to the method of extracting thelargest connected region in an image When only asingle person appears in the field of view such as inFigure 2(e) MCR is more likely to get desirableresults than PSO In the actual situation however wedo not know in advance how many people will gothrough the test site Take Figure 6(a) as an examplewhen two people go through the test site at the sametime MCR may get a wrong result as shown inFigures 6(b) and 6(c)

(2) Edge 9reshold Method (ETM) In ETM the edgeoperators such as Canny is firstly used to obtain thepossible target contours and the number of pixels inthese contours is then calculated respectively Oncethe number is bigger than a specific threshold theregion enclosed by the corresponding contour isconsidered as the useful region and is retainedotherwise this region is considered as the uselessregion and is removed In the paper the boundarybetween the target person and the redundant noise isusually solid which makes it possible to split thetarget from the background with the ETM Moreimportantly the ETM can also get good results inmultipedestrian images with appropriate parame-ters However it is a very difficult task for the ETM toadaptively select parameters Once the test envi-ronment changes the parameters of ETM need to bereselected which limits the application of the ETM

(3) Reaction Diffusion-Level Set Evolution (RD-LSE) +eRD-LES proposed by Zhang et al [43] is an im-proved level set algorithm which is widely used inthe field of image segmentation Figure 6(d) showsthe search process using the RD-LSE algorithm forthe Figure 6(a) in which the yellow curves show theevolution processes the green curve represents theinitial contour and the red curve represents the finalcontour +is algorithm can achieve a better resultthan PSO algorithm even in the case of multiplepedestrians as shown in Figures 6(e) and 6(f ) In thepaper we take 4 different types of pictures as ex-amples to compare the performance of RD-FLS andPSO in terms of converged iterations and CPU time+e experimental results are shown in Table 1 whereimages 1ndash4 represent Figures 2(e) 6(a) 7(a) and7(g) respectively +e values in table are the averageof 100 experiments Table 1 shows that the com-putational efficiency of the PSO algorithm far ex-ceeds the RD-FLS which is the main reason why wechoose PSO

43 Experimental Results Apart from the multipedestriancases such as in Figure 6 many other cases with the pedestrianin different states are studied to verify the effectiveness androbustness of the proposed method In Figure 7(a) the pe-destrian raised his left hand above his head Figures 7(c) 7(e)and 7(f) show the experimental process and result of adoptingthe proposed method for Figure 7(a) For clarity the 3D

8 Complexity

(a) (b)

Front

Back

Mark 3

Mark 4

Mark 1

Mark 2

(c)

Figure 5 Experimental setup (a) Site campus corridor (b) Site research room (c) Placement layout of the lightweight reflective balls

(a) (b) (c)

(d) (e) (f )

Figure 6 Experiments with two-pedestrian image (a) Original image with two pedestrians (b) Image obtained by theMCR algorithm alongwith the original image (c) Image obtained by theMSER-based segmentation along with (b) (d e)+e processes and result images obtainedby the RD-LSE along with the original image (f ) Image obtained by the MSER-based segmentation along with that in (e)

Table 1 Iterations (Iter) and CPU time (Time) by FRFLS and PSO methods

MethodsImage 1 Image 2 Image 3 Image 4

Time (s) Iter Time (s) Iter Time (s) Iter Time (s) IterFRFLS 521 643 601 800 532 665 487 611PSO 0049 129 0057 86 0053 185 0050 147Image size 320 lowast 240 pixels

Complexity 9

representations of Figures 7(a) and 7(c) are shown inFigures 7(b) and 7(d) respectively Although the height of thehead is lower than that of the left hand the proposed methodcan still get the correct result Figures 7(i) 7(k) and 7(l) showthe experimental process and result of adopting the proposed

method for Figure 7(g) in which a pedestrian is kneelingAlthough the proposed D-PSO algorithm does not eliminateall redundant noises as shown in Figure 7(j) it also yieldsideal experimental results due to MSERrsquos insensitivity to asmall amount of the sporadic noise All the above experiments

(a)

200

150

100

50

0

Pixe

l

050

100150

200HeightWidth

050100150200250300

(b) (c)

200

150

100

50

0

Pixe

l

050

100150

200HeightWidth

050100150200250300

(d) (e) (f )

(g)

10080604020

0

Pixe

l

250200

150100

500Height

Width

300

200

0100

(h) (i)

150

100

50

0

Pixe

l

200150

10050

0Height

Width

200100

0

300

(j) (k) (l)

Figure 7 Experiments with the pedestrian in different states (a) Original image with the pedestrian raising his left hand (c) Image obtainedby the PSO algorithm along with that in (a) (e f ) Images obtained by the MSER-based segmentation along with that in (c) (b d) +e 3Drepresentation of images in (a c) respectively (g) Original image with the pedestrian who is kneeling (i) Image obtained by the PSOalgorithm along with that in (g) (k l) Images obtained by theMSER-based segmentation along with those in (i) (h j)+e 3D representationof images in (g i) respectively

10 Complexity

show that the performance of our method is very stable andreliable

To further verify the accuracy of the proposed method alot of experiments are conducted based on 6 subjects fourmen and two women who are asked to walk through the testsites at the usual speed Here we take a set of data obtainedfrom the research room as an example to analyse the resultsFigure 8 shows the height results obtained from the sixsubjects using the VICON alone in several continuousseconds the sex and static height of the six subjects arepresented in the legend It explains that it is unrealistic to

keep the height on the static level when the pedestrian iswalking +us it is essential to study the pedestrian height inthe dynamic situation

Due to the high speed of pictures taken by VICON andTOF cameras and the slowness of pedestrian movement(07ndash12 meters per second) we only select 5 height data persecond to show a real-time height comparison between theVICON and the proposedmethod Every fifth of one secondan image is collected with the TOF camera +e pedestrianheight in the image is obtained by the proposed method andcompared with the height collected with VICON at the same

0 100 200 300 400 500 600 700 800 900 1000 1100 1200Number

160016101620163016401650166016701680169017001710172017301740175017601770178017901800

Hei

ght (

mm

)

Men1760167617611728

Women16481629

Figure 8 +e height results got from the six subjects using the VICON alone in several continuous seconds

1800179017801770176017501740173017201710170016901680167016601650

Mea

sure

d he

ight

(mm

)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30Number

Our algorithm without Kalman filtering1760167617611728

Our algorithm with Kalman filtering1760167617611728

VICON (ground truth)1760167617611728

Figure 9 Experimental results of men with different heights in the six consecutive seconds

Complexity 11

1700

1690

1680

1670

1660

1650

1640

1630

1620

1610

1600

1590

1580

1570

Mea

sure

d he

ight

(mm

)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30Number

Our algorithm without Kalman filtering16481629

Our algorithm with Kalman filtering16481629

VICON (ground truth)16481629

Figure 10 Experimental results of women with different heights in the six consecutive seconds

28272625242322211011121314151617181920 29308765432 91

Number

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

(a)

2 3 4 5 6 7 8 91 111213141516171819202122232425262728293010

Number

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

(b)

43 61 2 7 8 95 111213141516171819202122232425262728293010

Number

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

(c)

43 61 2 7 8 95 111213141516171819202122232425262728293010

Number

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

(d)

Figure 11 +e error plot of men in the six consecutive seconds (andashd) +e men with static heights of 1760 1676 1761 and 1728

12 Complexity

time Figures 9 and 10 show the experimental results of fourmen and two women in six consecutive seconds In thefigures the dotted line represents our algorithm withoutKalman filtering the solid line represents our algorithmwithout Kalman filtering and the dotted line with the markldquo+rdquo indicates the VICON+e waveforms show the real-timeheight value in 6 consecutive seconds the static heights ofmen are 1760mm 1676mm 1761mm and 1728mm asshown in the legend of Figure 9 while the static heights ofwomen are 1648mm and 1629mm as shown in Figure 10

It can be seen from the curves that the height datameasured by our algorithm is almost consistent with the dataobtained by VICON In order to analyse the error of ouralgorithm we sort out the errors of all the data in the sixconsecutive seconds the results are shown in Figures 11 and12 +e figures show that Kalman filtering can effectivelyimprove the accuracy of height measurement which indi-cates the pedestrian height at the preceding moment facil-itates the estimate of the pedestrian height in the lattermoment

Also the sums of errors per second of the algorithmswith and without Kalman filtering are given in Table 2where the subscript ldquolowastrdquo represents male and ldquordquo representsfemale Table 2 shows that our algorithm with Kalmanfiltering has a smaller cumulative error and can moreaccurately measure the real-time height of the movingpedestrians which proves the feasibility and validity of theproposed method

5 Conclusion and Future Work

In this paper a real-time height measurement based onthe TOF camera is proposed for moving pedestrians Toget the target region a new D-PSO denoising algorithmand a segmentation algorithm based on MSER are de-veloped in the paper In addition a novel multilayer it-erative average algorithm is designed for calculating thepedestrian height Also the Kalman filtering is used toimprove the measurement accuracy +e experimentalresults demonstrate the effectiveness and practicability of

2 3 4 5 6 7 8 9 1011121314151617181920212223242526272829301Number

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2Er

ror (

)

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

(a)

2 3 4 5 6 7 8 9 1011121314151617181920212223242526272829301Number

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

(b)

Figure 12 +e error plot of women in the six consecutive seconds (a) +e woman with static height of 1648 (b) +e woman with staticheight of 1629

Table 2 +e sum of errors per second of the algorithms with and without Kalman filtering

Heights (mm) Kalman filteringSum of errors per second ()

Sum1st second 2nd second 3rd second 4th second 5th second 6th second

1760lowast Yes 1202 0956 1836 1242 1611 1525 8372No 1868 1003 2013 1362 1898 1758 9902

1676lowast Yes 2002 1799 1977 0863 1648 2137 10426No 2249 1968 2087 1602 1827 3261 12994

1761lowast Yes 1282 1483 0963 1132 0632 1487 6979No 1562 1702 1333 1617 1234 1714 9162

1728lowast Yes 1629 1652 1354 1453 1224 0902 8214No 2201 2159 1912 1592 1984 1336 11184

1648 Yes 2006 1194 1818 1014 1585 1693 9310No 2488 1245 2152 1906 2078 2087 11956

1629 Yes 1509 1838 0652 2344 1398 1109 8850No 1632 2536 1328 2508 1497 1340 10841

lowastMale female

Complexity 13

the proposed method Our future work is going to furtherimprove the measurement accuracy and focus on trackingpedestrians in real time by using the real-time height ofmoving pedestrians

Data Availability

+e data used to support the findings of this study areavailable from the corresponding author upon request

Conflicts of Interest

+e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

+e authors are grateful to the financial support from theNatural Science Foundation of China (61877065) the NationalKey Research and Development Program of China(2019YFB1405500) the National Natural Science Foundationof Guangdong (2016A030313177) Guangdong Frontier andKey Technological Innovation (2017B090910013) and theScience and Technology Innovation Commission of Shenzhen(JCYJ20170818153048647 and JCYJ20180507182239617)

References

[1] J Li X Liang S Shen et al ldquoScale-aware fast R-CNN forpedestrian detectionrdquo IEEE Transactions on Multimediavol 20 no 4 pp 985ndash996 2017

[2] F P An ldquoPedestrian re-recognition algorithm based onoptimization deep learning-sequence memory modelrdquoComplexity vol 2019 Article ID 5069026 16 pages 2019

[3] J Cao Y Pang and X Li ldquoLearning multilayer channelfeatures for pedestrian detectionrdquo IEEE Transactions on ImageProcessing vol 26 no 7 pp 3210ndash3220 2017

[4] M Ji J Liu X Xu Y Guo and Z Lu ldquoImproved pedestrianpositioning with inertial sensor based on adaptive gradientdescent and double-constrained extended kalman filterrdquoComplexity vol 2020 Article ID 4361812 11 pages 2020

[5] C Li Z Su Q Li and H Zhao ldquoAn indoor positioning errorcorrection method of pedestrian multi-motions recognized byhybrid-orders fraction domain transformationrdquo IEEE Accessvol 7 pp 11360ndash11377 2019

[6] H Zhao W Cheng N Yang et al ldquoSmartphone-based 3Dindoor pedestrian positioning through multi-modal datafusionrdquo Sensors vol 19 no 20 Article ID s19204554 2019

[7] B Wang T Su X Jin J Kong and Y Bai ldquo3D reconstructionof pedestrian trajectory with moving direction learning andoptimal gait recognitionrdquo Complexity vol 2018 Article ID8735846 10 pages 2018

[8] Y Jiang Z Li and J B Wang ldquoPtrack enhancing the ap-plicability of pedestrian tracking with wearablesrdquo IEEETransactions on Mobile Computing vol 18 no 2 pp 431ndash4432018

[9] W Xu L Liu S Zlatanova W Penard and Q Xiong ldquoApedestrian tracking algorithm using grid-based indoormodelrdquo Automation in Construction vol 92 pp 173ndash1872018

[10] L Bozgeyikli A Raij S Katkoori and R Alqasemi ldquoA surveyon virtual reality for individuals with autism spectrum

disorder design considerationsrdquo IEEE Transactions onLearning Technologies vol 11 no 2 pp 133ndash151 2017

[11] J O Nilsson D Zachariah I Skog and P Handel ldquoCoop-erative localization by dual foot-mounted inertial sensors andinter-agent rangingrdquo EURASIP Journal on Advances in SignalProcessing vol 2013 no 1 p 164 2013

[12] I Skog J-O Nilsson D Zachariah and P Handel ldquoFusingthe information from two navigation systems using an upperbound on their maximum spatial separationrdquo in Proceedingsof the 2012 International Conference on Indoor Positioning andIndoor Navigation Article ID 6418862 Sydney AustraliaNovember 2012

[13] S-B Chen Y Xin and B Luo ldquoAction-based pedestrianidentification via hierarchical matching pursuit and orderpreserving sparse codingrdquo Cognitive Computation vol 8no 5 pp 797ndash805 2016

[14] B Shin C Kim J Kim et al ldquoMotion recognition based 3Dpedestrian navigation system using smartphonerdquo IEEE Sen-sors Journal vol 16 no 18 pp 6977ndash6989 2016

[15] M Romanovas V Goridko A Al-Jawad et al ldquoA study onindoor pedestrian localization algorithms with foot-mountedsensorsrdquo in Proceedings of the International Conference onIndoor Positioning and Indoor Navigation pp 1ndash10 SydneyAustralia November 2012

[16] A Azaman ldquoComparative study on gait kinematics betweenmicrosoft kinect and vicon across different anthropometricmeasurementsrdquo Journal of Tomography System and SensorApplication vol 2 no 2 pp 12ndash17 2019

[17] W Sheng A +obbi and Y Gu ldquoAn integrated frameworkfor human-robot collaborative manipulationrdquo IEEE Trans-actions on Cybernetics vol 45 no 10 pp 2030ndash2041 2014

[18] S Tsuji and T Kohama ldquoProximity skin sensor using time-of-flight sensor for human collaborative robotrdquo IEEE SensorsJournal vol 19 no 14 pp 5859ndash5864 2019

[19] C Oprea I Pirnog I Marcu and M Udrea ldquoRobust poseestimation using Time-of-Flight imagingrdquo in Proceedings ofthe IEEE International Semiconductor Conference pp 301ndash304 Sinaia Romania January 2019

[20] A Vysocky R Pastor and P Novak ldquoInteraction with col-laborative robot using 2D and TOF camerardquo in InternationalConference on Modelling and Simulation for AutonomousSystems pp 477ndash489 Springer Cham Switzerland 2018

[21] M Gao Y Du Y Yang and J Zhang ldquoAdaptive anchor boxmechanism to improve the accuracy in the object detectionsystemrdquo Multimedia Tools and Applications vol 78 no 19pp 27383ndash27402 2019

[22] A Anwer S S Azhar Ali A Khan and F MeriaudeauldquoUnderwater 3-d scene reconstruction using kinect v2 basedon physical models for refraction and time of flight correc-tionrdquo IEEE Access vol 5 pp 15960ndash15970 2017

[23] A R Garcıa L R Miller C F Andres and P J N LorenteldquoObstacle detection using a time of flight range camerardquo inProceedings of the 2018 IEEE International Conference onVehicular Electronics and Safety (ICVES) pp 1ndash6 MadridSpain September 2018

[24] N Zengeler T Kopinski and U Handmann ldquoHand gesturerecognition in automotive humanndashmachine interaction usingdepth camerasrdquo Sensors vol 19 no 1 Article ID s190100592019

[25] M A Garduntildeo-Ramon I R Terol-Villalobos R A Osornio-Rios and L A Morales-Hernandez ldquoA new method forinpainting of depthmaps from time-of-flight sensors based ona modified closing by reconstruction algorithmrdquo Journal of

14 Complexity

Visual Communication and Image Representation vol 47pp 36ndash47 2019

[26] L Wang Y Luo H Wang and M Fei ldquoMeasurement errorcorrection model of TOF depth camerardquo Chinese Journal ofSystem Simulation vol 29 no 10 pp 2323ndash2329 2017

[27] VICON ldquoOfficial website of oxford metrics companyrdquo 2020httpswwwviconcom

[28] L Zhang W Dong D Zhang and G Shi ldquoTwo-stage imagedenoising by principal component analysis with local pixelgroupingrdquo Pattern Recognition vol 43 no 4 pp 1531ndash15492010

[29] J Kennedy and R Eberhart ldquoParticle swarm optimizationrdquo inProceedings of ICNNrsquo95-International Conference on NeuralNetworks (ICW) vol 4 pp 1942ndash1948 Perth AustraliaNovember 1995

[30] M A M De Oca T Stutzle M Birattari and M DorigoldquoFrankensteinrsquos PSO a composite particle swarm optimiza-tion algorithmrdquo IEEE Transactions on Evolutionary Compu-tation vol 13 no 5 pp 1120ndash1132 2009

[31] Z Zhen S Pang F Wang et al ldquoPattern classification andPSO optimal weights based sky images cloud motion speedcalculation method for solar PV power forecastingrdquo IEEETransactions on Industry Applications vol 55 no 4pp 3331ndash3342 2019

[32] X Wang J-S Pan and S-C Chu ldquoA parallel multi-verseoptimizer for application in multilevel image segmentationrdquoIEEE Access vol 8 pp 32018ndash32030 2020

[33] Z A Bashir and M E El-Hawary ldquoApplying wavelets toshort-term load forecasting using PSO-based neural net-worksrdquo IEEE Transactions on Power Systems vol 24 no 1pp 20ndash27 2009

[34] L Liu Y Wang F Xie and J Gao ldquoLegendre cooperativePSO strategies for trajectory optimizationrdquo Complexityvol 2018 Article ID 5036791 13 pages 2018

[35] Y Shi and R C Eberhart ldquoA modified particle swarm op-timizerrdquo in Proceedings of the 1998 IEEE InternationalConference on Evolutionary Computation Proceedingspp 69ndash73 Anchorage AK USA May 1998

[36] Y Shi and R C Eberhart ldquoParameter selection in particleswarm optimizationrdquo in International Conference on Evolu-tionary Programming pp 591ndash600 Springer Berlin Ger-many 1998

[37] X Lv D Zhou Y Tang and L Ma ldquoAn improved test se-lection optimization model based on fault ambiguity groupisolation and chaotic discrete PSOrdquo Complexity vol 2018Article ID 3942723 10 pages 2018

[38] J Matas O Chum M Urban and T Pajdla ldquoRobust wide-baseline stereo from maximally stable extremal regionsrdquoImage and Vision Computing vol 22 no 10 pp 761ndash7672004

[39] H Shim and S Lee ldquoRecovering translucent objects using asingle time-of-flight depth camerardquo IEEE Transactions onCircuits and Systems for Video Technology vol 26 no 5pp 841ndash854 2015

[40] C K Chui and G ChenKalman Filtering pp 19ndash26 SpringerInternational Publishing Berlin Germany 2017

[41] L Cui X Wang Y Xu H Jiang and J Zhou ldquoA novelswitching unscented Kalman filter method for remaininguseful life prediction of rolling bearingrdquo Measurementvol 135 pp 678ndash684 2019

[42] GWelch and G BishopAn Introduction to the Kalman FilterMacmillan New York NY USA 1995

[43] K Zhang L Zhang H Song and D Zhang ldquoReinitialization-free level set evolution via reaction diffusionrdquo IEEE Trans-actions on Image Processing vol 22 no 1 pp 258ndash271 2012

Complexity 15

extracted as shown in the yellow part of Figure 2(o) Figure 4is the pixel distribution map of the extracted head regionwhere the black dots represent pixel points and the coor-dinates represent the positions of the pixels in the imageFrom this figure we can discover another advantage of theproposed MSER-based segmentation algorithm which canremove the notable noises in the head region such as salt-and-pepper noise Since the notable noise is very differentfrom its neighbour pixels it will not be incorporated into thehead region when the MSER algorithm is used to obtain thestable region +erefore the MSER-based segmentation caneffectively filter out notable noises in the head region asshown in the red rectangles in Figure 4 Note that the redrectangles are the manual markers for easy viewing

3 Real-Time Calculation for Pedestrian Height

31 Multilayer Iterative Average Algorithm for Pixel ValueAlthough the MSER algorithm can filter out the notablenoises there will still be some noises in the head region asshown in the 3D representation of the head region inFigure 2(f ) +e typical height measurement of only usingthe head top is not accurate +us a novel multilayer it-erative average algorithm (MLIA) is proposed to get thepixel average for getting the pedestrian height +e MLIAalgorithm not only can improve accuracy but also can ef-fectively remove some outliers that MSER cannot filter out+e MLIA can be broken down into the following steps

(1) Calculating the average of pixel value adopting thefollowing equation to get the average of pixel value inthe head region as

pave 1n

1113944

n

i1pi (9)

where pave is the pixel value average n is the numberof pixels in current head region and pi representsi minus th pixel value in current head region

(2) Updating the head region traverse all the pixels inthe head region and delete the pixels that do notmeet the following equation +e remaining pixelsare combined to update the head region

pi minus pave1113868111386811138681113868

1113868111386811138681113868leT pave( 1113857 (10)

where T(pave) is a threshold function related to thecurrent average pave and it is defined as follows

T pave( 1113857 Min pmax minus pave pave minus pmin( 1113857 (11)

where pmax and pmin are the maximum pixel valueand the minimum pixel value in the head regionrespectively

(3) Repeat step (1) and step (2) above until pave satisfythe following equation

Pave minuspmin + pmax

2le δ (12)

where δ is the empirical constant In this paper δ isselected as 20 according to the actual situation

+e above steps can be summarized as the followingpseudocode (Algorithm 1)

By the way the MLIA algorithm can also be applied tothe multipedestrian situation When the image containsmore than one pedestrian the MSER-based segmentationcan get more than one head region Meanwhile the pixelvalue average of each head region needs to be calculated bythe MLIA algorithm

32HeightCalculation Once pave is obtained the average ofthe head region in original pedestrian grey image (such as inFigure 2(d)) defined as pavg can be obtained through thedeformation of (2)

+en substituting pavg into (1) to replace pi we canobtain the following equation

10 807040 60 90 1003020 500Number of iterations

04

045

05

055

06

065

07

075

Iner

tia w

eigh

ts

Figure 3 +e curve of inertia weight in PSO

175 180 185 190 195170Width

120

125

130

135

140

145

150

Hei

ght

Figure 4 +e pixel distribution map of the extracted head region

6 Complexity

davg pavg dmax minus dmin( 1113857

255+ dmin (13)

where davg is the depth value corresponding to pavg and dmaxand dmin are the maximum and minimum depth values inthe pedestrian depth image

According to the physical properties of the TOF camerathe following conversion equation can be used to recover thephysical distance from the depth data davg[39]

Ddis Ktofdavg + E Ktofpavg dmax minus dmin( 1113857

255+ dmin1113888 1113889 + E

(14)

where Ddis represents the physical distance between the TOFcamera and the pedestrian head (unit mm) E is the de-viation constant associated with the physical structure andplacement height of the TOF camera while Ktof (512) isthe conversion coefficient only associated with the physicalstructure of the TOF camera

To allow our method to work for pedestrians who are notvertically below the TOF camera the pinhole model pro-posed in our previous work [26] is adopted to correct Ddis

Dco Ddis times cos arctanOMf

1113888 11138891113888 1113889 (15)

where Dco is the corrected physical distance f is the focallength and OM is the distance between the centroid of thehead region in the grey image M and the centre of the greyimage O the coordinates of the centroidM can be got by thefollowing equation More detailed information about thepinhole model can be found in the literature [26]

hp 1n

1113944

n

i1mihi

wp 1n

1113944

n

i1miwi

⎧⎪⎪⎪⎪⎪⎪⎪⎨

⎪⎪⎪⎪⎪⎪⎪⎩

(16)

where n is the number of pixels in current head region wp

and hp are the horizontal and vertical coordinates of thecentroid M and wi and hi are the horizontal and verticalcoordinates of the ith pixel respectively mi is the mass of theith pixel which is defined as mi 1 in this paper

Finally the pedestrian height H is calculated by thefollowing equation

H Htof minus Dco (17)

where Htof is the distance between the TOF camera and theground

33 Kalman Estimation of Real-Time Height In the exper-iments we found that the fluctuations of the pedestrianheights all approximately conform to the Gaussian distri-bution with variance 256 (unit mm2) and the variance didnot change with the state of the system +erefore Kalmanfiltering is further introduced to estimate the pedestrianheights got by (17) to achieve the more accurate real-timeheights Kalman filtering is a highly efficient recursive filterthat can estimate the state of a dynamic system from a seriesof measurements containing redundant noise [40] It cangenerate estimates of unknown variables which have provento be more accurate than those only based on a singlemeasurement [4 41] +e Kalman filter can be implementedin two stages time update stage and measurement updatestage [42]

+e time update stage is dedicated to predicting thecurrently a priori estimates through past state and the errorcovariance Equations (18) and (19) are responsible forpredicting the a priori state estimate 1113954xk and the a priori errorcovariance estimate 1113954Pk in current (kth) frame respectively

1113954xk Akminus 1xkminus 1 + Bukminus 1 (18)

1113954Pk Akminus 1Pkminus 1ATkminus 1 + Q (19)

where xkminus 1 and Pkminus 1 are respectively the state and the errorcovariance of the previous step Akminus 1 is the transfer matrixthat relates the state of the previous step to the state of the

Input S-initial head region extracted by the MSER-based segmentationProcedure

(1) n count (S)(2) pave (1n) 1113936

ni1 pi pi isin S

(3) pmax Max(pi) i 1 2 n(4) pmin Min(pi) i 1 2 n(5) while Pave minus ((pmin + pmax)2)gt δ do(7) T(pave) min (pmax minus pave pave minus pmin)(8) S pi | |pi minus pave|leT(pave) i 1 2 n1113864 1113865(9) n count (S)(10) pave (1n) 1113936

ni1 pi pi isin S

(11) pmax max(pi) i 1 2 n(12) pmin min(pi) i 1 2 n(13) end while

Output pave-the average of the pixels in the head region

ALGORITHM 1 Multilayer iterative average algorithm (MLIA)

Complexity 7

current step B is the control matrix that relates the previousinput ukminus 1 and Q is the variance of the Gaussian processnoise Based on the actual situation of pedestrians during themovement (no external input Gaussian distribution of theheight fluctuation and continuity of the height change) theparameters in time update stage are defined as followsukminus 1 equiv 0 Q equiv 256 Akminus 1 equiv 1 1113954xk is the a priori height estimatefrom the current depth image

+e measurement update stage is devoted to combiningactual measurements with a priori estimates to get theimproved posteriori estimates [42] It can be achieved by thefollowing equations

Kk 1113954PkHTk Hk

1113954PkHTk + R1113872 1113873

minus 1 (20)

xk 1113954xk + K Zk minus Hk1113954xk( 1113857 (21)

Pk I minus KkHk( 11138571113954Pk (22)

where xk and Pk are the posteriori state estimate and theposteriori error covariance estimate in current (kth) step Kk

is the Kalman gain in current step Hk is the matrix thatrelates the state to the measurement Zk I is a unit matrixand R is the variance of the Gaussian measurement noiseBased on the actual situation of measurements (cameraaccuracy and measurement process) the parameters inmeasurement update stage are defined as follows R equiv 144Hk equiv 1 xk is the posteriori height estimate from the currentdepth image and Zk is the pedestrian heights got by (17) Inaddition the initialization is defined as x1 Z1 and P1 10

4 Experiments and Analysis

41 Experimental Setup In this paper an EPC660 is used asthe TOF chip to offer a fully digital interface for the controlcircuitry and the communication between computer andcamera is realized through Gigabit network In addition theexperiment is completed with the support of the computerwith Windows 10 OS Intelreg Coretrade i3-8100 360GHz CPUand 8GB RAM +e campus corridor is selected as the firsttest site and the experimental scene is shown in Figure 5(a)+en considering the fluctuation of pedestrian height indynamic situations the research room is chosen as thesecond test site and the VICON system fixed in this site isadopted as the ground truth to confirm the feasibility of theproposed method +e experimental scene in research roomis shown in Figure 5(b) where a portion of the VICONsystem two of the 12 infrared cameras is shown While theVICON is running four lightweight reflective balls are stuckto the pedestrianrsquos head the placement layout of the balls isshown in Figure 5(c) And the average height of the four ballsis adopted as the real-time height of the pedestrian

42 Comparison with Other Popular Algorithms Before thePSO algorithm is adopted to process the images with un-wanted noise other popular algorithms are deployed toprocess the same images for a comparison More specificallythree algorithms are implemented for comparison here

(1) Maximum Connected Region (MCR) As the nameimplies MCR refers to the method of extracting thelargest connected region in an image When only asingle person appears in the field of view such as inFigure 2(e) MCR is more likely to get desirableresults than PSO In the actual situation however wedo not know in advance how many people will gothrough the test site Take Figure 6(a) as an examplewhen two people go through the test site at the sametime MCR may get a wrong result as shown inFigures 6(b) and 6(c)

(2) Edge 9reshold Method (ETM) In ETM the edgeoperators such as Canny is firstly used to obtain thepossible target contours and the number of pixels inthese contours is then calculated respectively Oncethe number is bigger than a specific threshold theregion enclosed by the corresponding contour isconsidered as the useful region and is retainedotherwise this region is considered as the uselessregion and is removed In the paper the boundarybetween the target person and the redundant noise isusually solid which makes it possible to split thetarget from the background with the ETM Moreimportantly the ETM can also get good results inmultipedestrian images with appropriate parame-ters However it is a very difficult task for the ETM toadaptively select parameters Once the test envi-ronment changes the parameters of ETM need to bereselected which limits the application of the ETM

(3) Reaction Diffusion-Level Set Evolution (RD-LSE) +eRD-LES proposed by Zhang et al [43] is an im-proved level set algorithm which is widely used inthe field of image segmentation Figure 6(d) showsthe search process using the RD-LSE algorithm forthe Figure 6(a) in which the yellow curves show theevolution processes the green curve represents theinitial contour and the red curve represents the finalcontour +is algorithm can achieve a better resultthan PSO algorithm even in the case of multiplepedestrians as shown in Figures 6(e) and 6(f ) In thepaper we take 4 different types of pictures as ex-amples to compare the performance of RD-FLS andPSO in terms of converged iterations and CPU time+e experimental results are shown in Table 1 whereimages 1ndash4 represent Figures 2(e) 6(a) 7(a) and7(g) respectively +e values in table are the averageof 100 experiments Table 1 shows that the com-putational efficiency of the PSO algorithm far ex-ceeds the RD-FLS which is the main reason why wechoose PSO

43 Experimental Results Apart from the multipedestriancases such as in Figure 6 many other cases with the pedestrianin different states are studied to verify the effectiveness androbustness of the proposed method In Figure 7(a) the pe-destrian raised his left hand above his head Figures 7(c) 7(e)and 7(f) show the experimental process and result of adoptingthe proposed method for Figure 7(a) For clarity the 3D

8 Complexity

(a) (b)

Front

Back

Mark 3

Mark 4

Mark 1

Mark 2

(c)

Figure 5 Experimental setup (a) Site campus corridor (b) Site research room (c) Placement layout of the lightweight reflective balls

(a) (b) (c)

(d) (e) (f )

Figure 6 Experiments with two-pedestrian image (a) Original image with two pedestrians (b) Image obtained by theMCR algorithm alongwith the original image (c) Image obtained by theMSER-based segmentation along with (b) (d e)+e processes and result images obtainedby the RD-LSE along with the original image (f ) Image obtained by the MSER-based segmentation along with that in (e)

Table 1 Iterations (Iter) and CPU time (Time) by FRFLS and PSO methods

MethodsImage 1 Image 2 Image 3 Image 4

Time (s) Iter Time (s) Iter Time (s) Iter Time (s) IterFRFLS 521 643 601 800 532 665 487 611PSO 0049 129 0057 86 0053 185 0050 147Image size 320 lowast 240 pixels

Complexity 9

representations of Figures 7(a) and 7(c) are shown inFigures 7(b) and 7(d) respectively Although the height of thehead is lower than that of the left hand the proposed methodcan still get the correct result Figures 7(i) 7(k) and 7(l) showthe experimental process and result of adopting the proposed

method for Figure 7(g) in which a pedestrian is kneelingAlthough the proposed D-PSO algorithm does not eliminateall redundant noises as shown in Figure 7(j) it also yieldsideal experimental results due to MSERrsquos insensitivity to asmall amount of the sporadic noise All the above experiments

(a)

200

150

100

50

0

Pixe

l

050

100150

200HeightWidth

050100150200250300

(b) (c)

200

150

100

50

0

Pixe

l

050

100150

200HeightWidth

050100150200250300

(d) (e) (f )

(g)

10080604020

0

Pixe

l

250200

150100

500Height

Width

300

200

0100

(h) (i)

150

100

50

0

Pixe

l

200150

10050

0Height

Width

200100

0

300

(j) (k) (l)

Figure 7 Experiments with the pedestrian in different states (a) Original image with the pedestrian raising his left hand (c) Image obtainedby the PSO algorithm along with that in (a) (e f ) Images obtained by the MSER-based segmentation along with that in (c) (b d) +e 3Drepresentation of images in (a c) respectively (g) Original image with the pedestrian who is kneeling (i) Image obtained by the PSOalgorithm along with that in (g) (k l) Images obtained by theMSER-based segmentation along with those in (i) (h j)+e 3D representationof images in (g i) respectively

10 Complexity

show that the performance of our method is very stable andreliable

To further verify the accuracy of the proposed method alot of experiments are conducted based on 6 subjects fourmen and two women who are asked to walk through the testsites at the usual speed Here we take a set of data obtainedfrom the research room as an example to analyse the resultsFigure 8 shows the height results obtained from the sixsubjects using the VICON alone in several continuousseconds the sex and static height of the six subjects arepresented in the legend It explains that it is unrealistic to

keep the height on the static level when the pedestrian iswalking +us it is essential to study the pedestrian height inthe dynamic situation

Due to the high speed of pictures taken by VICON andTOF cameras and the slowness of pedestrian movement(07ndash12 meters per second) we only select 5 height data persecond to show a real-time height comparison between theVICON and the proposedmethod Every fifth of one secondan image is collected with the TOF camera +e pedestrianheight in the image is obtained by the proposed method andcompared with the height collected with VICON at the same

0 100 200 300 400 500 600 700 800 900 1000 1100 1200Number

160016101620163016401650166016701680169017001710172017301740175017601770178017901800

Hei

ght (

mm

)

Men1760167617611728

Women16481629

Figure 8 +e height results got from the six subjects using the VICON alone in several continuous seconds

1800179017801770176017501740173017201710170016901680167016601650

Mea

sure

d he

ight

(mm

)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30Number

Our algorithm without Kalman filtering1760167617611728

Our algorithm with Kalman filtering1760167617611728

VICON (ground truth)1760167617611728

Figure 9 Experimental results of men with different heights in the six consecutive seconds

Complexity 11

1700

1690

1680

1670

1660

1650

1640

1630

1620

1610

1600

1590

1580

1570

Mea

sure

d he

ight

(mm

)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30Number

Our algorithm without Kalman filtering16481629

Our algorithm with Kalman filtering16481629

VICON (ground truth)16481629

Figure 10 Experimental results of women with different heights in the six consecutive seconds

28272625242322211011121314151617181920 29308765432 91

Number

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

(a)

2 3 4 5 6 7 8 91 111213141516171819202122232425262728293010

Number

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

(b)

43 61 2 7 8 95 111213141516171819202122232425262728293010

Number

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

(c)

43 61 2 7 8 95 111213141516171819202122232425262728293010

Number

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

(d)

Figure 11 +e error plot of men in the six consecutive seconds (andashd) +e men with static heights of 1760 1676 1761 and 1728

12 Complexity

time Figures 9 and 10 show the experimental results of fourmen and two women in six consecutive seconds In thefigures the dotted line represents our algorithm withoutKalman filtering the solid line represents our algorithmwithout Kalman filtering and the dotted line with the markldquo+rdquo indicates the VICON+e waveforms show the real-timeheight value in 6 consecutive seconds the static heights ofmen are 1760mm 1676mm 1761mm and 1728mm asshown in the legend of Figure 9 while the static heights ofwomen are 1648mm and 1629mm as shown in Figure 10

It can be seen from the curves that the height datameasured by our algorithm is almost consistent with the dataobtained by VICON In order to analyse the error of ouralgorithm we sort out the errors of all the data in the sixconsecutive seconds the results are shown in Figures 11 and12 +e figures show that Kalman filtering can effectivelyimprove the accuracy of height measurement which indi-cates the pedestrian height at the preceding moment facil-itates the estimate of the pedestrian height in the lattermoment

Also the sums of errors per second of the algorithmswith and without Kalman filtering are given in Table 2where the subscript ldquolowastrdquo represents male and ldquordquo representsfemale Table 2 shows that our algorithm with Kalmanfiltering has a smaller cumulative error and can moreaccurately measure the real-time height of the movingpedestrians which proves the feasibility and validity of theproposed method

5 Conclusion and Future Work

In this paper a real-time height measurement based onthe TOF camera is proposed for moving pedestrians Toget the target region a new D-PSO denoising algorithmand a segmentation algorithm based on MSER are de-veloped in the paper In addition a novel multilayer it-erative average algorithm is designed for calculating thepedestrian height Also the Kalman filtering is used toimprove the measurement accuracy +e experimentalresults demonstrate the effectiveness and practicability of

2 3 4 5 6 7 8 9 1011121314151617181920212223242526272829301Number

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2Er

ror (

)

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

(a)

2 3 4 5 6 7 8 9 1011121314151617181920212223242526272829301Number

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

(b)

Figure 12 +e error plot of women in the six consecutive seconds (a) +e woman with static height of 1648 (b) +e woman with staticheight of 1629

Table 2 +e sum of errors per second of the algorithms with and without Kalman filtering

Heights (mm) Kalman filteringSum of errors per second ()

Sum1st second 2nd second 3rd second 4th second 5th second 6th second

1760lowast Yes 1202 0956 1836 1242 1611 1525 8372No 1868 1003 2013 1362 1898 1758 9902

1676lowast Yes 2002 1799 1977 0863 1648 2137 10426No 2249 1968 2087 1602 1827 3261 12994

1761lowast Yes 1282 1483 0963 1132 0632 1487 6979No 1562 1702 1333 1617 1234 1714 9162

1728lowast Yes 1629 1652 1354 1453 1224 0902 8214No 2201 2159 1912 1592 1984 1336 11184

1648 Yes 2006 1194 1818 1014 1585 1693 9310No 2488 1245 2152 1906 2078 2087 11956

1629 Yes 1509 1838 0652 2344 1398 1109 8850No 1632 2536 1328 2508 1497 1340 10841

lowastMale female

Complexity 13

the proposed method Our future work is going to furtherimprove the measurement accuracy and focus on trackingpedestrians in real time by using the real-time height ofmoving pedestrians

Data Availability

+e data used to support the findings of this study areavailable from the corresponding author upon request

Conflicts of Interest

+e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

+e authors are grateful to the financial support from theNatural Science Foundation of China (61877065) the NationalKey Research and Development Program of China(2019YFB1405500) the National Natural Science Foundationof Guangdong (2016A030313177) Guangdong Frontier andKey Technological Innovation (2017B090910013) and theScience and Technology Innovation Commission of Shenzhen(JCYJ20170818153048647 and JCYJ20180507182239617)

References

[1] J Li X Liang S Shen et al ldquoScale-aware fast R-CNN forpedestrian detectionrdquo IEEE Transactions on Multimediavol 20 no 4 pp 985ndash996 2017

[2] F P An ldquoPedestrian re-recognition algorithm based onoptimization deep learning-sequence memory modelrdquoComplexity vol 2019 Article ID 5069026 16 pages 2019

[3] J Cao Y Pang and X Li ldquoLearning multilayer channelfeatures for pedestrian detectionrdquo IEEE Transactions on ImageProcessing vol 26 no 7 pp 3210ndash3220 2017

[4] M Ji J Liu X Xu Y Guo and Z Lu ldquoImproved pedestrianpositioning with inertial sensor based on adaptive gradientdescent and double-constrained extended kalman filterrdquoComplexity vol 2020 Article ID 4361812 11 pages 2020

[5] C Li Z Su Q Li and H Zhao ldquoAn indoor positioning errorcorrection method of pedestrian multi-motions recognized byhybrid-orders fraction domain transformationrdquo IEEE Accessvol 7 pp 11360ndash11377 2019

[6] H Zhao W Cheng N Yang et al ldquoSmartphone-based 3Dindoor pedestrian positioning through multi-modal datafusionrdquo Sensors vol 19 no 20 Article ID s19204554 2019

[7] B Wang T Su X Jin J Kong and Y Bai ldquo3D reconstructionof pedestrian trajectory with moving direction learning andoptimal gait recognitionrdquo Complexity vol 2018 Article ID8735846 10 pages 2018

[8] Y Jiang Z Li and J B Wang ldquoPtrack enhancing the ap-plicability of pedestrian tracking with wearablesrdquo IEEETransactions on Mobile Computing vol 18 no 2 pp 431ndash4432018

[9] W Xu L Liu S Zlatanova W Penard and Q Xiong ldquoApedestrian tracking algorithm using grid-based indoormodelrdquo Automation in Construction vol 92 pp 173ndash1872018

[10] L Bozgeyikli A Raij S Katkoori and R Alqasemi ldquoA surveyon virtual reality for individuals with autism spectrum

disorder design considerationsrdquo IEEE Transactions onLearning Technologies vol 11 no 2 pp 133ndash151 2017

[11] J O Nilsson D Zachariah I Skog and P Handel ldquoCoop-erative localization by dual foot-mounted inertial sensors andinter-agent rangingrdquo EURASIP Journal on Advances in SignalProcessing vol 2013 no 1 p 164 2013

[12] I Skog J-O Nilsson D Zachariah and P Handel ldquoFusingthe information from two navigation systems using an upperbound on their maximum spatial separationrdquo in Proceedingsof the 2012 International Conference on Indoor Positioning andIndoor Navigation Article ID 6418862 Sydney AustraliaNovember 2012

[13] S-B Chen Y Xin and B Luo ldquoAction-based pedestrianidentification via hierarchical matching pursuit and orderpreserving sparse codingrdquo Cognitive Computation vol 8no 5 pp 797ndash805 2016

[14] B Shin C Kim J Kim et al ldquoMotion recognition based 3Dpedestrian navigation system using smartphonerdquo IEEE Sen-sors Journal vol 16 no 18 pp 6977ndash6989 2016

[15] M Romanovas V Goridko A Al-Jawad et al ldquoA study onindoor pedestrian localization algorithms with foot-mountedsensorsrdquo in Proceedings of the International Conference onIndoor Positioning and Indoor Navigation pp 1ndash10 SydneyAustralia November 2012

[16] A Azaman ldquoComparative study on gait kinematics betweenmicrosoft kinect and vicon across different anthropometricmeasurementsrdquo Journal of Tomography System and SensorApplication vol 2 no 2 pp 12ndash17 2019

[17] W Sheng A +obbi and Y Gu ldquoAn integrated frameworkfor human-robot collaborative manipulationrdquo IEEE Trans-actions on Cybernetics vol 45 no 10 pp 2030ndash2041 2014

[18] S Tsuji and T Kohama ldquoProximity skin sensor using time-of-flight sensor for human collaborative robotrdquo IEEE SensorsJournal vol 19 no 14 pp 5859ndash5864 2019

[19] C Oprea I Pirnog I Marcu and M Udrea ldquoRobust poseestimation using Time-of-Flight imagingrdquo in Proceedings ofthe IEEE International Semiconductor Conference pp 301ndash304 Sinaia Romania January 2019

[20] A Vysocky R Pastor and P Novak ldquoInteraction with col-laborative robot using 2D and TOF camerardquo in InternationalConference on Modelling and Simulation for AutonomousSystems pp 477ndash489 Springer Cham Switzerland 2018

[21] M Gao Y Du Y Yang and J Zhang ldquoAdaptive anchor boxmechanism to improve the accuracy in the object detectionsystemrdquo Multimedia Tools and Applications vol 78 no 19pp 27383ndash27402 2019

[22] A Anwer S S Azhar Ali A Khan and F MeriaudeauldquoUnderwater 3-d scene reconstruction using kinect v2 basedon physical models for refraction and time of flight correc-tionrdquo IEEE Access vol 5 pp 15960ndash15970 2017

[23] A R Garcıa L R Miller C F Andres and P J N LorenteldquoObstacle detection using a time of flight range camerardquo inProceedings of the 2018 IEEE International Conference onVehicular Electronics and Safety (ICVES) pp 1ndash6 MadridSpain September 2018

[24] N Zengeler T Kopinski and U Handmann ldquoHand gesturerecognition in automotive humanndashmachine interaction usingdepth camerasrdquo Sensors vol 19 no 1 Article ID s190100592019

[25] M A Garduntildeo-Ramon I R Terol-Villalobos R A Osornio-Rios and L A Morales-Hernandez ldquoA new method forinpainting of depthmaps from time-of-flight sensors based ona modified closing by reconstruction algorithmrdquo Journal of

14 Complexity

Visual Communication and Image Representation vol 47pp 36ndash47 2019

[26] L Wang Y Luo H Wang and M Fei ldquoMeasurement errorcorrection model of TOF depth camerardquo Chinese Journal ofSystem Simulation vol 29 no 10 pp 2323ndash2329 2017

[27] VICON ldquoOfficial website of oxford metrics companyrdquo 2020httpswwwviconcom

[28] L Zhang W Dong D Zhang and G Shi ldquoTwo-stage imagedenoising by principal component analysis with local pixelgroupingrdquo Pattern Recognition vol 43 no 4 pp 1531ndash15492010

[29] J Kennedy and R Eberhart ldquoParticle swarm optimizationrdquo inProceedings of ICNNrsquo95-International Conference on NeuralNetworks (ICW) vol 4 pp 1942ndash1948 Perth AustraliaNovember 1995

[30] M A M De Oca T Stutzle M Birattari and M DorigoldquoFrankensteinrsquos PSO a composite particle swarm optimiza-tion algorithmrdquo IEEE Transactions on Evolutionary Compu-tation vol 13 no 5 pp 1120ndash1132 2009

[31] Z Zhen S Pang F Wang et al ldquoPattern classification andPSO optimal weights based sky images cloud motion speedcalculation method for solar PV power forecastingrdquo IEEETransactions on Industry Applications vol 55 no 4pp 3331ndash3342 2019

[32] X Wang J-S Pan and S-C Chu ldquoA parallel multi-verseoptimizer for application in multilevel image segmentationrdquoIEEE Access vol 8 pp 32018ndash32030 2020

[33] Z A Bashir and M E El-Hawary ldquoApplying wavelets toshort-term load forecasting using PSO-based neural net-worksrdquo IEEE Transactions on Power Systems vol 24 no 1pp 20ndash27 2009

[34] L Liu Y Wang F Xie and J Gao ldquoLegendre cooperativePSO strategies for trajectory optimizationrdquo Complexityvol 2018 Article ID 5036791 13 pages 2018

[35] Y Shi and R C Eberhart ldquoA modified particle swarm op-timizerrdquo in Proceedings of the 1998 IEEE InternationalConference on Evolutionary Computation Proceedingspp 69ndash73 Anchorage AK USA May 1998

[36] Y Shi and R C Eberhart ldquoParameter selection in particleswarm optimizationrdquo in International Conference on Evolu-tionary Programming pp 591ndash600 Springer Berlin Ger-many 1998

[37] X Lv D Zhou Y Tang and L Ma ldquoAn improved test se-lection optimization model based on fault ambiguity groupisolation and chaotic discrete PSOrdquo Complexity vol 2018Article ID 3942723 10 pages 2018

[38] J Matas O Chum M Urban and T Pajdla ldquoRobust wide-baseline stereo from maximally stable extremal regionsrdquoImage and Vision Computing vol 22 no 10 pp 761ndash7672004

[39] H Shim and S Lee ldquoRecovering translucent objects using asingle time-of-flight depth camerardquo IEEE Transactions onCircuits and Systems for Video Technology vol 26 no 5pp 841ndash854 2015

[40] C K Chui and G ChenKalman Filtering pp 19ndash26 SpringerInternational Publishing Berlin Germany 2017

[41] L Cui X Wang Y Xu H Jiang and J Zhou ldquoA novelswitching unscented Kalman filter method for remaininguseful life prediction of rolling bearingrdquo Measurementvol 135 pp 678ndash684 2019

[42] GWelch and G BishopAn Introduction to the Kalman FilterMacmillan New York NY USA 1995

[43] K Zhang L Zhang H Song and D Zhang ldquoReinitialization-free level set evolution via reaction diffusionrdquo IEEE Trans-actions on Image Processing vol 22 no 1 pp 258ndash271 2012

Complexity 15

davg pavg dmax minus dmin( 1113857

255+ dmin (13)

where davg is the depth value corresponding to pavg and dmaxand dmin are the maximum and minimum depth values inthe pedestrian depth image

According to the physical properties of the TOF camerathe following conversion equation can be used to recover thephysical distance from the depth data davg[39]

Ddis Ktofdavg + E Ktofpavg dmax minus dmin( 1113857

255+ dmin1113888 1113889 + E

(14)

where Ddis represents the physical distance between the TOFcamera and the pedestrian head (unit mm) E is the de-viation constant associated with the physical structure andplacement height of the TOF camera while Ktof (512) isthe conversion coefficient only associated with the physicalstructure of the TOF camera

To allow our method to work for pedestrians who are notvertically below the TOF camera the pinhole model pro-posed in our previous work [26] is adopted to correct Ddis

Dco Ddis times cos arctanOMf

1113888 11138891113888 1113889 (15)

where Dco is the corrected physical distance f is the focallength and OM is the distance between the centroid of thehead region in the grey image M and the centre of the greyimage O the coordinates of the centroidM can be got by thefollowing equation More detailed information about thepinhole model can be found in the literature [26]

hp 1n

1113944

n

i1mihi

wp 1n

1113944

n

i1miwi

⎧⎪⎪⎪⎪⎪⎪⎪⎨

⎪⎪⎪⎪⎪⎪⎪⎩

(16)

where n is the number of pixels in current head region wp

and hp are the horizontal and vertical coordinates of thecentroid M and wi and hi are the horizontal and verticalcoordinates of the ith pixel respectively mi is the mass of theith pixel which is defined as mi 1 in this paper

Finally the pedestrian height H is calculated by thefollowing equation

H Htof minus Dco (17)

where Htof is the distance between the TOF camera and theground

33 Kalman Estimation of Real-Time Height In the exper-iments we found that the fluctuations of the pedestrianheights all approximately conform to the Gaussian distri-bution with variance 256 (unit mm2) and the variance didnot change with the state of the system +erefore Kalmanfiltering is further introduced to estimate the pedestrianheights got by (17) to achieve the more accurate real-timeheights Kalman filtering is a highly efficient recursive filterthat can estimate the state of a dynamic system from a seriesof measurements containing redundant noise [40] It cangenerate estimates of unknown variables which have provento be more accurate than those only based on a singlemeasurement [4 41] +e Kalman filter can be implementedin two stages time update stage and measurement updatestage [42]

+e time update stage is dedicated to predicting thecurrently a priori estimates through past state and the errorcovariance Equations (18) and (19) are responsible forpredicting the a priori state estimate 1113954xk and the a priori errorcovariance estimate 1113954Pk in current (kth) frame respectively

1113954xk Akminus 1xkminus 1 + Bukminus 1 (18)

1113954Pk Akminus 1Pkminus 1ATkminus 1 + Q (19)

where xkminus 1 and Pkminus 1 are respectively the state and the errorcovariance of the previous step Akminus 1 is the transfer matrixthat relates the state of the previous step to the state of the

Input S-initial head region extracted by the MSER-based segmentationProcedure

(1) n count (S)(2) pave (1n) 1113936

ni1 pi pi isin S

(3) pmax Max(pi) i 1 2 n(4) pmin Min(pi) i 1 2 n(5) while Pave minus ((pmin + pmax)2)gt δ do(7) T(pave) min (pmax minus pave pave minus pmin)(8) S pi | |pi minus pave|leT(pave) i 1 2 n1113864 1113865(9) n count (S)(10) pave (1n) 1113936

ni1 pi pi isin S

(11) pmax max(pi) i 1 2 n(12) pmin min(pi) i 1 2 n(13) end while

Output pave-the average of the pixels in the head region

ALGORITHM 1 Multilayer iterative average algorithm (MLIA)

Complexity 7

current step B is the control matrix that relates the previousinput ukminus 1 and Q is the variance of the Gaussian processnoise Based on the actual situation of pedestrians during themovement (no external input Gaussian distribution of theheight fluctuation and continuity of the height change) theparameters in time update stage are defined as followsukminus 1 equiv 0 Q equiv 256 Akminus 1 equiv 1 1113954xk is the a priori height estimatefrom the current depth image

+e measurement update stage is devoted to combiningactual measurements with a priori estimates to get theimproved posteriori estimates [42] It can be achieved by thefollowing equations

Kk 1113954PkHTk Hk

1113954PkHTk + R1113872 1113873

minus 1 (20)

xk 1113954xk + K Zk minus Hk1113954xk( 1113857 (21)

Pk I minus KkHk( 11138571113954Pk (22)

where xk and Pk are the posteriori state estimate and theposteriori error covariance estimate in current (kth) step Kk

is the Kalman gain in current step Hk is the matrix thatrelates the state to the measurement Zk I is a unit matrixand R is the variance of the Gaussian measurement noiseBased on the actual situation of measurements (cameraaccuracy and measurement process) the parameters inmeasurement update stage are defined as follows R equiv 144Hk equiv 1 xk is the posteriori height estimate from the currentdepth image and Zk is the pedestrian heights got by (17) Inaddition the initialization is defined as x1 Z1 and P1 10

4 Experiments and Analysis

41 Experimental Setup In this paper an EPC660 is used asthe TOF chip to offer a fully digital interface for the controlcircuitry and the communication between computer andcamera is realized through Gigabit network In addition theexperiment is completed with the support of the computerwith Windows 10 OS Intelreg Coretrade i3-8100 360GHz CPUand 8GB RAM +e campus corridor is selected as the firsttest site and the experimental scene is shown in Figure 5(a)+en considering the fluctuation of pedestrian height indynamic situations the research room is chosen as thesecond test site and the VICON system fixed in this site isadopted as the ground truth to confirm the feasibility of theproposed method +e experimental scene in research roomis shown in Figure 5(b) where a portion of the VICONsystem two of the 12 infrared cameras is shown While theVICON is running four lightweight reflective balls are stuckto the pedestrianrsquos head the placement layout of the balls isshown in Figure 5(c) And the average height of the four ballsis adopted as the real-time height of the pedestrian

42 Comparison with Other Popular Algorithms Before thePSO algorithm is adopted to process the images with un-wanted noise other popular algorithms are deployed toprocess the same images for a comparison More specificallythree algorithms are implemented for comparison here

(1) Maximum Connected Region (MCR) As the nameimplies MCR refers to the method of extracting thelargest connected region in an image When only asingle person appears in the field of view such as inFigure 2(e) MCR is more likely to get desirableresults than PSO In the actual situation however wedo not know in advance how many people will gothrough the test site Take Figure 6(a) as an examplewhen two people go through the test site at the sametime MCR may get a wrong result as shown inFigures 6(b) and 6(c)

(2) Edge 9reshold Method (ETM) In ETM the edgeoperators such as Canny is firstly used to obtain thepossible target contours and the number of pixels inthese contours is then calculated respectively Oncethe number is bigger than a specific threshold theregion enclosed by the corresponding contour isconsidered as the useful region and is retainedotherwise this region is considered as the uselessregion and is removed In the paper the boundarybetween the target person and the redundant noise isusually solid which makes it possible to split thetarget from the background with the ETM Moreimportantly the ETM can also get good results inmultipedestrian images with appropriate parame-ters However it is a very difficult task for the ETM toadaptively select parameters Once the test envi-ronment changes the parameters of ETM need to bereselected which limits the application of the ETM

(3) Reaction Diffusion-Level Set Evolution (RD-LSE) +eRD-LES proposed by Zhang et al [43] is an im-proved level set algorithm which is widely used inthe field of image segmentation Figure 6(d) showsthe search process using the RD-LSE algorithm forthe Figure 6(a) in which the yellow curves show theevolution processes the green curve represents theinitial contour and the red curve represents the finalcontour +is algorithm can achieve a better resultthan PSO algorithm even in the case of multiplepedestrians as shown in Figures 6(e) and 6(f ) In thepaper we take 4 different types of pictures as ex-amples to compare the performance of RD-FLS andPSO in terms of converged iterations and CPU time+e experimental results are shown in Table 1 whereimages 1ndash4 represent Figures 2(e) 6(a) 7(a) and7(g) respectively +e values in table are the averageof 100 experiments Table 1 shows that the com-putational efficiency of the PSO algorithm far ex-ceeds the RD-FLS which is the main reason why wechoose PSO

43 Experimental Results Apart from the multipedestriancases such as in Figure 6 many other cases with the pedestrianin different states are studied to verify the effectiveness androbustness of the proposed method In Figure 7(a) the pe-destrian raised his left hand above his head Figures 7(c) 7(e)and 7(f) show the experimental process and result of adoptingthe proposed method for Figure 7(a) For clarity the 3D

8 Complexity

(a) (b)

Front

Back

Mark 3

Mark 4

Mark 1

Mark 2

(c)

Figure 5 Experimental setup (a) Site campus corridor (b) Site research room (c) Placement layout of the lightweight reflective balls

(a) (b) (c)

(d) (e) (f )

Figure 6 Experiments with two-pedestrian image (a) Original image with two pedestrians (b) Image obtained by theMCR algorithm alongwith the original image (c) Image obtained by theMSER-based segmentation along with (b) (d e)+e processes and result images obtainedby the RD-LSE along with the original image (f ) Image obtained by the MSER-based segmentation along with that in (e)

Table 1 Iterations (Iter) and CPU time (Time) by FRFLS and PSO methods

MethodsImage 1 Image 2 Image 3 Image 4

Time (s) Iter Time (s) Iter Time (s) Iter Time (s) IterFRFLS 521 643 601 800 532 665 487 611PSO 0049 129 0057 86 0053 185 0050 147Image size 320 lowast 240 pixels

Complexity 9

representations of Figures 7(a) and 7(c) are shown inFigures 7(b) and 7(d) respectively Although the height of thehead is lower than that of the left hand the proposed methodcan still get the correct result Figures 7(i) 7(k) and 7(l) showthe experimental process and result of adopting the proposed

method for Figure 7(g) in which a pedestrian is kneelingAlthough the proposed D-PSO algorithm does not eliminateall redundant noises as shown in Figure 7(j) it also yieldsideal experimental results due to MSERrsquos insensitivity to asmall amount of the sporadic noise All the above experiments

(a)

200

150

100

50

0

Pixe

l

050

100150

200HeightWidth

050100150200250300

(b) (c)

200

150

100

50

0

Pixe

l

050

100150

200HeightWidth

050100150200250300

(d) (e) (f )

(g)

10080604020

0

Pixe

l

250200

150100

500Height

Width

300

200

0100

(h) (i)

150

100

50

0

Pixe

l

200150

10050

0Height

Width

200100

0

300

(j) (k) (l)

Figure 7 Experiments with the pedestrian in different states (a) Original image with the pedestrian raising his left hand (c) Image obtainedby the PSO algorithm along with that in (a) (e f ) Images obtained by the MSER-based segmentation along with that in (c) (b d) +e 3Drepresentation of images in (a c) respectively (g) Original image with the pedestrian who is kneeling (i) Image obtained by the PSOalgorithm along with that in (g) (k l) Images obtained by theMSER-based segmentation along with those in (i) (h j)+e 3D representationof images in (g i) respectively

10 Complexity

show that the performance of our method is very stable andreliable

To further verify the accuracy of the proposed method alot of experiments are conducted based on 6 subjects fourmen and two women who are asked to walk through the testsites at the usual speed Here we take a set of data obtainedfrom the research room as an example to analyse the resultsFigure 8 shows the height results obtained from the sixsubjects using the VICON alone in several continuousseconds the sex and static height of the six subjects arepresented in the legend It explains that it is unrealistic to

keep the height on the static level when the pedestrian iswalking +us it is essential to study the pedestrian height inthe dynamic situation

Due to the high speed of pictures taken by VICON andTOF cameras and the slowness of pedestrian movement(07ndash12 meters per second) we only select 5 height data persecond to show a real-time height comparison between theVICON and the proposedmethod Every fifth of one secondan image is collected with the TOF camera +e pedestrianheight in the image is obtained by the proposed method andcompared with the height collected with VICON at the same

0 100 200 300 400 500 600 700 800 900 1000 1100 1200Number

160016101620163016401650166016701680169017001710172017301740175017601770178017901800

Hei

ght (

mm

)

Men1760167617611728

Women16481629

Figure 8 +e height results got from the six subjects using the VICON alone in several continuous seconds

1800179017801770176017501740173017201710170016901680167016601650

Mea

sure

d he

ight

(mm

)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30Number

Our algorithm without Kalman filtering1760167617611728

Our algorithm with Kalman filtering1760167617611728

VICON (ground truth)1760167617611728

Figure 9 Experimental results of men with different heights in the six consecutive seconds

Complexity 11

1700

1690

1680

1670

1660

1650

1640

1630

1620

1610

1600

1590

1580

1570

Mea

sure

d he

ight

(mm

)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30Number

Our algorithm without Kalman filtering16481629

Our algorithm with Kalman filtering16481629

VICON (ground truth)16481629

Figure 10 Experimental results of women with different heights in the six consecutive seconds

28272625242322211011121314151617181920 29308765432 91

Number

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

(a)

2 3 4 5 6 7 8 91 111213141516171819202122232425262728293010

Number

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

(b)

43 61 2 7 8 95 111213141516171819202122232425262728293010

Number

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

(c)

43 61 2 7 8 95 111213141516171819202122232425262728293010

Number

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

(d)

Figure 11 +e error plot of men in the six consecutive seconds (andashd) +e men with static heights of 1760 1676 1761 and 1728

12 Complexity

time Figures 9 and 10 show the experimental results of fourmen and two women in six consecutive seconds In thefigures the dotted line represents our algorithm withoutKalman filtering the solid line represents our algorithmwithout Kalman filtering and the dotted line with the markldquo+rdquo indicates the VICON+e waveforms show the real-timeheight value in 6 consecutive seconds the static heights ofmen are 1760mm 1676mm 1761mm and 1728mm asshown in the legend of Figure 9 while the static heights ofwomen are 1648mm and 1629mm as shown in Figure 10

It can be seen from the curves that the height datameasured by our algorithm is almost consistent with the dataobtained by VICON In order to analyse the error of ouralgorithm we sort out the errors of all the data in the sixconsecutive seconds the results are shown in Figures 11 and12 +e figures show that Kalman filtering can effectivelyimprove the accuracy of height measurement which indi-cates the pedestrian height at the preceding moment facil-itates the estimate of the pedestrian height in the lattermoment

Also the sums of errors per second of the algorithmswith and without Kalman filtering are given in Table 2where the subscript ldquolowastrdquo represents male and ldquordquo representsfemale Table 2 shows that our algorithm with Kalmanfiltering has a smaller cumulative error and can moreaccurately measure the real-time height of the movingpedestrians which proves the feasibility and validity of theproposed method

5 Conclusion and Future Work

In this paper a real-time height measurement based onthe TOF camera is proposed for moving pedestrians Toget the target region a new D-PSO denoising algorithmand a segmentation algorithm based on MSER are de-veloped in the paper In addition a novel multilayer it-erative average algorithm is designed for calculating thepedestrian height Also the Kalman filtering is used toimprove the measurement accuracy +e experimentalresults demonstrate the effectiveness and practicability of

2 3 4 5 6 7 8 9 1011121314151617181920212223242526272829301Number

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2Er

ror (

)

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

(a)

2 3 4 5 6 7 8 9 1011121314151617181920212223242526272829301Number

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

(b)

Figure 12 +e error plot of women in the six consecutive seconds (a) +e woman with static height of 1648 (b) +e woman with staticheight of 1629

Table 2 +e sum of errors per second of the algorithms with and without Kalman filtering

Heights (mm) Kalman filteringSum of errors per second ()

Sum1st second 2nd second 3rd second 4th second 5th second 6th second

1760lowast Yes 1202 0956 1836 1242 1611 1525 8372No 1868 1003 2013 1362 1898 1758 9902

1676lowast Yes 2002 1799 1977 0863 1648 2137 10426No 2249 1968 2087 1602 1827 3261 12994

1761lowast Yes 1282 1483 0963 1132 0632 1487 6979No 1562 1702 1333 1617 1234 1714 9162

1728lowast Yes 1629 1652 1354 1453 1224 0902 8214No 2201 2159 1912 1592 1984 1336 11184

1648 Yes 2006 1194 1818 1014 1585 1693 9310No 2488 1245 2152 1906 2078 2087 11956

1629 Yes 1509 1838 0652 2344 1398 1109 8850No 1632 2536 1328 2508 1497 1340 10841

lowastMale female

Complexity 13

the proposed method Our future work is going to furtherimprove the measurement accuracy and focus on trackingpedestrians in real time by using the real-time height ofmoving pedestrians

Data Availability

+e data used to support the findings of this study areavailable from the corresponding author upon request

Conflicts of Interest

+e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

+e authors are grateful to the financial support from theNatural Science Foundation of China (61877065) the NationalKey Research and Development Program of China(2019YFB1405500) the National Natural Science Foundationof Guangdong (2016A030313177) Guangdong Frontier andKey Technological Innovation (2017B090910013) and theScience and Technology Innovation Commission of Shenzhen(JCYJ20170818153048647 and JCYJ20180507182239617)

References

[1] J Li X Liang S Shen et al ldquoScale-aware fast R-CNN forpedestrian detectionrdquo IEEE Transactions on Multimediavol 20 no 4 pp 985ndash996 2017

[2] F P An ldquoPedestrian re-recognition algorithm based onoptimization deep learning-sequence memory modelrdquoComplexity vol 2019 Article ID 5069026 16 pages 2019

[3] J Cao Y Pang and X Li ldquoLearning multilayer channelfeatures for pedestrian detectionrdquo IEEE Transactions on ImageProcessing vol 26 no 7 pp 3210ndash3220 2017

[4] M Ji J Liu X Xu Y Guo and Z Lu ldquoImproved pedestrianpositioning with inertial sensor based on adaptive gradientdescent and double-constrained extended kalman filterrdquoComplexity vol 2020 Article ID 4361812 11 pages 2020

[5] C Li Z Su Q Li and H Zhao ldquoAn indoor positioning errorcorrection method of pedestrian multi-motions recognized byhybrid-orders fraction domain transformationrdquo IEEE Accessvol 7 pp 11360ndash11377 2019

[6] H Zhao W Cheng N Yang et al ldquoSmartphone-based 3Dindoor pedestrian positioning through multi-modal datafusionrdquo Sensors vol 19 no 20 Article ID s19204554 2019

[7] B Wang T Su X Jin J Kong and Y Bai ldquo3D reconstructionof pedestrian trajectory with moving direction learning andoptimal gait recognitionrdquo Complexity vol 2018 Article ID8735846 10 pages 2018

[8] Y Jiang Z Li and J B Wang ldquoPtrack enhancing the ap-plicability of pedestrian tracking with wearablesrdquo IEEETransactions on Mobile Computing vol 18 no 2 pp 431ndash4432018

[9] W Xu L Liu S Zlatanova W Penard and Q Xiong ldquoApedestrian tracking algorithm using grid-based indoormodelrdquo Automation in Construction vol 92 pp 173ndash1872018

[10] L Bozgeyikli A Raij S Katkoori and R Alqasemi ldquoA surveyon virtual reality for individuals with autism spectrum

disorder design considerationsrdquo IEEE Transactions onLearning Technologies vol 11 no 2 pp 133ndash151 2017

[11] J O Nilsson D Zachariah I Skog and P Handel ldquoCoop-erative localization by dual foot-mounted inertial sensors andinter-agent rangingrdquo EURASIP Journal on Advances in SignalProcessing vol 2013 no 1 p 164 2013

[12] I Skog J-O Nilsson D Zachariah and P Handel ldquoFusingthe information from two navigation systems using an upperbound on their maximum spatial separationrdquo in Proceedingsof the 2012 International Conference on Indoor Positioning andIndoor Navigation Article ID 6418862 Sydney AustraliaNovember 2012

[13] S-B Chen Y Xin and B Luo ldquoAction-based pedestrianidentification via hierarchical matching pursuit and orderpreserving sparse codingrdquo Cognitive Computation vol 8no 5 pp 797ndash805 2016

[14] B Shin C Kim J Kim et al ldquoMotion recognition based 3Dpedestrian navigation system using smartphonerdquo IEEE Sen-sors Journal vol 16 no 18 pp 6977ndash6989 2016

[15] M Romanovas V Goridko A Al-Jawad et al ldquoA study onindoor pedestrian localization algorithms with foot-mountedsensorsrdquo in Proceedings of the International Conference onIndoor Positioning and Indoor Navigation pp 1ndash10 SydneyAustralia November 2012

[16] A Azaman ldquoComparative study on gait kinematics betweenmicrosoft kinect and vicon across different anthropometricmeasurementsrdquo Journal of Tomography System and SensorApplication vol 2 no 2 pp 12ndash17 2019

[17] W Sheng A +obbi and Y Gu ldquoAn integrated frameworkfor human-robot collaborative manipulationrdquo IEEE Trans-actions on Cybernetics vol 45 no 10 pp 2030ndash2041 2014

[18] S Tsuji and T Kohama ldquoProximity skin sensor using time-of-flight sensor for human collaborative robotrdquo IEEE SensorsJournal vol 19 no 14 pp 5859ndash5864 2019

[19] C Oprea I Pirnog I Marcu and M Udrea ldquoRobust poseestimation using Time-of-Flight imagingrdquo in Proceedings ofthe IEEE International Semiconductor Conference pp 301ndash304 Sinaia Romania January 2019

[20] A Vysocky R Pastor and P Novak ldquoInteraction with col-laborative robot using 2D and TOF camerardquo in InternationalConference on Modelling and Simulation for AutonomousSystems pp 477ndash489 Springer Cham Switzerland 2018

[21] M Gao Y Du Y Yang and J Zhang ldquoAdaptive anchor boxmechanism to improve the accuracy in the object detectionsystemrdquo Multimedia Tools and Applications vol 78 no 19pp 27383ndash27402 2019

[22] A Anwer S S Azhar Ali A Khan and F MeriaudeauldquoUnderwater 3-d scene reconstruction using kinect v2 basedon physical models for refraction and time of flight correc-tionrdquo IEEE Access vol 5 pp 15960ndash15970 2017

[23] A R Garcıa L R Miller C F Andres and P J N LorenteldquoObstacle detection using a time of flight range camerardquo inProceedings of the 2018 IEEE International Conference onVehicular Electronics and Safety (ICVES) pp 1ndash6 MadridSpain September 2018

[24] N Zengeler T Kopinski and U Handmann ldquoHand gesturerecognition in automotive humanndashmachine interaction usingdepth camerasrdquo Sensors vol 19 no 1 Article ID s190100592019

[25] M A Garduntildeo-Ramon I R Terol-Villalobos R A Osornio-Rios and L A Morales-Hernandez ldquoA new method forinpainting of depthmaps from time-of-flight sensors based ona modified closing by reconstruction algorithmrdquo Journal of

14 Complexity

Visual Communication and Image Representation vol 47pp 36ndash47 2019

[26] L Wang Y Luo H Wang and M Fei ldquoMeasurement errorcorrection model of TOF depth camerardquo Chinese Journal ofSystem Simulation vol 29 no 10 pp 2323ndash2329 2017

[27] VICON ldquoOfficial website of oxford metrics companyrdquo 2020httpswwwviconcom

[28] L Zhang W Dong D Zhang and G Shi ldquoTwo-stage imagedenoising by principal component analysis with local pixelgroupingrdquo Pattern Recognition vol 43 no 4 pp 1531ndash15492010

[29] J Kennedy and R Eberhart ldquoParticle swarm optimizationrdquo inProceedings of ICNNrsquo95-International Conference on NeuralNetworks (ICW) vol 4 pp 1942ndash1948 Perth AustraliaNovember 1995

[30] M A M De Oca T Stutzle M Birattari and M DorigoldquoFrankensteinrsquos PSO a composite particle swarm optimiza-tion algorithmrdquo IEEE Transactions on Evolutionary Compu-tation vol 13 no 5 pp 1120ndash1132 2009

[31] Z Zhen S Pang F Wang et al ldquoPattern classification andPSO optimal weights based sky images cloud motion speedcalculation method for solar PV power forecastingrdquo IEEETransactions on Industry Applications vol 55 no 4pp 3331ndash3342 2019

[32] X Wang J-S Pan and S-C Chu ldquoA parallel multi-verseoptimizer for application in multilevel image segmentationrdquoIEEE Access vol 8 pp 32018ndash32030 2020

[33] Z A Bashir and M E El-Hawary ldquoApplying wavelets toshort-term load forecasting using PSO-based neural net-worksrdquo IEEE Transactions on Power Systems vol 24 no 1pp 20ndash27 2009

[34] L Liu Y Wang F Xie and J Gao ldquoLegendre cooperativePSO strategies for trajectory optimizationrdquo Complexityvol 2018 Article ID 5036791 13 pages 2018

[35] Y Shi and R C Eberhart ldquoA modified particle swarm op-timizerrdquo in Proceedings of the 1998 IEEE InternationalConference on Evolutionary Computation Proceedingspp 69ndash73 Anchorage AK USA May 1998

[36] Y Shi and R C Eberhart ldquoParameter selection in particleswarm optimizationrdquo in International Conference on Evolu-tionary Programming pp 591ndash600 Springer Berlin Ger-many 1998

[37] X Lv D Zhou Y Tang and L Ma ldquoAn improved test se-lection optimization model based on fault ambiguity groupisolation and chaotic discrete PSOrdquo Complexity vol 2018Article ID 3942723 10 pages 2018

[38] J Matas O Chum M Urban and T Pajdla ldquoRobust wide-baseline stereo from maximally stable extremal regionsrdquoImage and Vision Computing vol 22 no 10 pp 761ndash7672004

[39] H Shim and S Lee ldquoRecovering translucent objects using asingle time-of-flight depth camerardquo IEEE Transactions onCircuits and Systems for Video Technology vol 26 no 5pp 841ndash854 2015

[40] C K Chui and G ChenKalman Filtering pp 19ndash26 SpringerInternational Publishing Berlin Germany 2017

[41] L Cui X Wang Y Xu H Jiang and J Zhou ldquoA novelswitching unscented Kalman filter method for remaininguseful life prediction of rolling bearingrdquo Measurementvol 135 pp 678ndash684 2019

[42] GWelch and G BishopAn Introduction to the Kalman FilterMacmillan New York NY USA 1995

[43] K Zhang L Zhang H Song and D Zhang ldquoReinitialization-free level set evolution via reaction diffusionrdquo IEEE Trans-actions on Image Processing vol 22 no 1 pp 258ndash271 2012

Complexity 15

current step B is the control matrix that relates the previousinput ukminus 1 and Q is the variance of the Gaussian processnoise Based on the actual situation of pedestrians during themovement (no external input Gaussian distribution of theheight fluctuation and continuity of the height change) theparameters in time update stage are defined as followsukminus 1 equiv 0 Q equiv 256 Akminus 1 equiv 1 1113954xk is the a priori height estimatefrom the current depth image

+e measurement update stage is devoted to combiningactual measurements with a priori estimates to get theimproved posteriori estimates [42] It can be achieved by thefollowing equations

Kk 1113954PkHTk Hk

1113954PkHTk + R1113872 1113873

minus 1 (20)

xk 1113954xk + K Zk minus Hk1113954xk( 1113857 (21)

Pk I minus KkHk( 11138571113954Pk (22)

where xk and Pk are the posteriori state estimate and theposteriori error covariance estimate in current (kth) step Kk

is the Kalman gain in current step Hk is the matrix thatrelates the state to the measurement Zk I is a unit matrixand R is the variance of the Gaussian measurement noiseBased on the actual situation of measurements (cameraaccuracy and measurement process) the parameters inmeasurement update stage are defined as follows R equiv 144Hk equiv 1 xk is the posteriori height estimate from the currentdepth image and Zk is the pedestrian heights got by (17) Inaddition the initialization is defined as x1 Z1 and P1 10

4 Experiments and Analysis

41 Experimental Setup In this paper an EPC660 is used asthe TOF chip to offer a fully digital interface for the controlcircuitry and the communication between computer andcamera is realized through Gigabit network In addition theexperiment is completed with the support of the computerwith Windows 10 OS Intelreg Coretrade i3-8100 360GHz CPUand 8GB RAM +e campus corridor is selected as the firsttest site and the experimental scene is shown in Figure 5(a)+en considering the fluctuation of pedestrian height indynamic situations the research room is chosen as thesecond test site and the VICON system fixed in this site isadopted as the ground truth to confirm the feasibility of theproposed method +e experimental scene in research roomis shown in Figure 5(b) where a portion of the VICONsystem two of the 12 infrared cameras is shown While theVICON is running four lightweight reflective balls are stuckto the pedestrianrsquos head the placement layout of the balls isshown in Figure 5(c) And the average height of the four ballsis adopted as the real-time height of the pedestrian

42 Comparison with Other Popular Algorithms Before thePSO algorithm is adopted to process the images with un-wanted noise other popular algorithms are deployed toprocess the same images for a comparison More specificallythree algorithms are implemented for comparison here

(1) Maximum Connected Region (MCR) As the nameimplies MCR refers to the method of extracting thelargest connected region in an image When only asingle person appears in the field of view such as inFigure 2(e) MCR is more likely to get desirableresults than PSO In the actual situation however wedo not know in advance how many people will gothrough the test site Take Figure 6(a) as an examplewhen two people go through the test site at the sametime MCR may get a wrong result as shown inFigures 6(b) and 6(c)

(2) Edge 9reshold Method (ETM) In ETM the edgeoperators such as Canny is firstly used to obtain thepossible target contours and the number of pixels inthese contours is then calculated respectively Oncethe number is bigger than a specific threshold theregion enclosed by the corresponding contour isconsidered as the useful region and is retainedotherwise this region is considered as the uselessregion and is removed In the paper the boundarybetween the target person and the redundant noise isusually solid which makes it possible to split thetarget from the background with the ETM Moreimportantly the ETM can also get good results inmultipedestrian images with appropriate parame-ters However it is a very difficult task for the ETM toadaptively select parameters Once the test envi-ronment changes the parameters of ETM need to bereselected which limits the application of the ETM

(3) Reaction Diffusion-Level Set Evolution (RD-LSE) +eRD-LES proposed by Zhang et al [43] is an im-proved level set algorithm which is widely used inthe field of image segmentation Figure 6(d) showsthe search process using the RD-LSE algorithm forthe Figure 6(a) in which the yellow curves show theevolution processes the green curve represents theinitial contour and the red curve represents the finalcontour +is algorithm can achieve a better resultthan PSO algorithm even in the case of multiplepedestrians as shown in Figures 6(e) and 6(f ) In thepaper we take 4 different types of pictures as ex-amples to compare the performance of RD-FLS andPSO in terms of converged iterations and CPU time+e experimental results are shown in Table 1 whereimages 1ndash4 represent Figures 2(e) 6(a) 7(a) and7(g) respectively +e values in table are the averageof 100 experiments Table 1 shows that the com-putational efficiency of the PSO algorithm far ex-ceeds the RD-FLS which is the main reason why wechoose PSO

43 Experimental Results Apart from the multipedestriancases such as in Figure 6 many other cases with the pedestrianin different states are studied to verify the effectiveness androbustness of the proposed method In Figure 7(a) the pe-destrian raised his left hand above his head Figures 7(c) 7(e)and 7(f) show the experimental process and result of adoptingthe proposed method for Figure 7(a) For clarity the 3D

8 Complexity

(a) (b)

Front

Back

Mark 3

Mark 4

Mark 1

Mark 2

(c)

Figure 5 Experimental setup (a) Site campus corridor (b) Site research room (c) Placement layout of the lightweight reflective balls

(a) (b) (c)

(d) (e) (f )

Figure 6 Experiments with two-pedestrian image (a) Original image with two pedestrians (b) Image obtained by theMCR algorithm alongwith the original image (c) Image obtained by theMSER-based segmentation along with (b) (d e)+e processes and result images obtainedby the RD-LSE along with the original image (f ) Image obtained by the MSER-based segmentation along with that in (e)

Table 1 Iterations (Iter) and CPU time (Time) by FRFLS and PSO methods

MethodsImage 1 Image 2 Image 3 Image 4

Time (s) Iter Time (s) Iter Time (s) Iter Time (s) IterFRFLS 521 643 601 800 532 665 487 611PSO 0049 129 0057 86 0053 185 0050 147Image size 320 lowast 240 pixels

Complexity 9

representations of Figures 7(a) and 7(c) are shown inFigures 7(b) and 7(d) respectively Although the height of thehead is lower than that of the left hand the proposed methodcan still get the correct result Figures 7(i) 7(k) and 7(l) showthe experimental process and result of adopting the proposed

method for Figure 7(g) in which a pedestrian is kneelingAlthough the proposed D-PSO algorithm does not eliminateall redundant noises as shown in Figure 7(j) it also yieldsideal experimental results due to MSERrsquos insensitivity to asmall amount of the sporadic noise All the above experiments

(a)

200

150

100

50

0

Pixe

l

050

100150

200HeightWidth

050100150200250300

(b) (c)

200

150

100

50

0

Pixe

l

050

100150

200HeightWidth

050100150200250300

(d) (e) (f )

(g)

10080604020

0

Pixe

l

250200

150100

500Height

Width

300

200

0100

(h) (i)

150

100

50

0

Pixe

l

200150

10050

0Height

Width

200100

0

300

(j) (k) (l)

Figure 7 Experiments with the pedestrian in different states (a) Original image with the pedestrian raising his left hand (c) Image obtainedby the PSO algorithm along with that in (a) (e f ) Images obtained by the MSER-based segmentation along with that in (c) (b d) +e 3Drepresentation of images in (a c) respectively (g) Original image with the pedestrian who is kneeling (i) Image obtained by the PSOalgorithm along with that in (g) (k l) Images obtained by theMSER-based segmentation along with those in (i) (h j)+e 3D representationof images in (g i) respectively

10 Complexity

show that the performance of our method is very stable andreliable

To further verify the accuracy of the proposed method alot of experiments are conducted based on 6 subjects fourmen and two women who are asked to walk through the testsites at the usual speed Here we take a set of data obtainedfrom the research room as an example to analyse the resultsFigure 8 shows the height results obtained from the sixsubjects using the VICON alone in several continuousseconds the sex and static height of the six subjects arepresented in the legend It explains that it is unrealistic to

keep the height on the static level when the pedestrian iswalking +us it is essential to study the pedestrian height inthe dynamic situation

Due to the high speed of pictures taken by VICON andTOF cameras and the slowness of pedestrian movement(07ndash12 meters per second) we only select 5 height data persecond to show a real-time height comparison between theVICON and the proposedmethod Every fifth of one secondan image is collected with the TOF camera +e pedestrianheight in the image is obtained by the proposed method andcompared with the height collected with VICON at the same

0 100 200 300 400 500 600 700 800 900 1000 1100 1200Number

160016101620163016401650166016701680169017001710172017301740175017601770178017901800

Hei

ght (

mm

)

Men1760167617611728

Women16481629

Figure 8 +e height results got from the six subjects using the VICON alone in several continuous seconds

1800179017801770176017501740173017201710170016901680167016601650

Mea

sure

d he

ight

(mm

)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30Number

Our algorithm without Kalman filtering1760167617611728

Our algorithm with Kalman filtering1760167617611728

VICON (ground truth)1760167617611728

Figure 9 Experimental results of men with different heights in the six consecutive seconds

Complexity 11

1700

1690

1680

1670

1660

1650

1640

1630

1620

1610

1600

1590

1580

1570

Mea

sure

d he

ight

(mm

)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30Number

Our algorithm without Kalman filtering16481629

Our algorithm with Kalman filtering16481629

VICON (ground truth)16481629

Figure 10 Experimental results of women with different heights in the six consecutive seconds

28272625242322211011121314151617181920 29308765432 91

Number

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

(a)

2 3 4 5 6 7 8 91 111213141516171819202122232425262728293010

Number

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

(b)

43 61 2 7 8 95 111213141516171819202122232425262728293010

Number

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

(c)

43 61 2 7 8 95 111213141516171819202122232425262728293010

Number

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

(d)

Figure 11 +e error plot of men in the six consecutive seconds (andashd) +e men with static heights of 1760 1676 1761 and 1728

12 Complexity

time Figures 9 and 10 show the experimental results of fourmen and two women in six consecutive seconds In thefigures the dotted line represents our algorithm withoutKalman filtering the solid line represents our algorithmwithout Kalman filtering and the dotted line with the markldquo+rdquo indicates the VICON+e waveforms show the real-timeheight value in 6 consecutive seconds the static heights ofmen are 1760mm 1676mm 1761mm and 1728mm asshown in the legend of Figure 9 while the static heights ofwomen are 1648mm and 1629mm as shown in Figure 10

It can be seen from the curves that the height datameasured by our algorithm is almost consistent with the dataobtained by VICON In order to analyse the error of ouralgorithm we sort out the errors of all the data in the sixconsecutive seconds the results are shown in Figures 11 and12 +e figures show that Kalman filtering can effectivelyimprove the accuracy of height measurement which indi-cates the pedestrian height at the preceding moment facil-itates the estimate of the pedestrian height in the lattermoment

Also the sums of errors per second of the algorithmswith and without Kalman filtering are given in Table 2where the subscript ldquolowastrdquo represents male and ldquordquo representsfemale Table 2 shows that our algorithm with Kalmanfiltering has a smaller cumulative error and can moreaccurately measure the real-time height of the movingpedestrians which proves the feasibility and validity of theproposed method

5 Conclusion and Future Work

In this paper a real-time height measurement based onthe TOF camera is proposed for moving pedestrians Toget the target region a new D-PSO denoising algorithmand a segmentation algorithm based on MSER are de-veloped in the paper In addition a novel multilayer it-erative average algorithm is designed for calculating thepedestrian height Also the Kalman filtering is used toimprove the measurement accuracy +e experimentalresults demonstrate the effectiveness and practicability of

2 3 4 5 6 7 8 9 1011121314151617181920212223242526272829301Number

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2Er

ror (

)

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

(a)

2 3 4 5 6 7 8 9 1011121314151617181920212223242526272829301Number

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

(b)

Figure 12 +e error plot of women in the six consecutive seconds (a) +e woman with static height of 1648 (b) +e woman with staticheight of 1629

Table 2 +e sum of errors per second of the algorithms with and without Kalman filtering

Heights (mm) Kalman filteringSum of errors per second ()

Sum1st second 2nd second 3rd second 4th second 5th second 6th second

1760lowast Yes 1202 0956 1836 1242 1611 1525 8372No 1868 1003 2013 1362 1898 1758 9902

1676lowast Yes 2002 1799 1977 0863 1648 2137 10426No 2249 1968 2087 1602 1827 3261 12994

1761lowast Yes 1282 1483 0963 1132 0632 1487 6979No 1562 1702 1333 1617 1234 1714 9162

1728lowast Yes 1629 1652 1354 1453 1224 0902 8214No 2201 2159 1912 1592 1984 1336 11184

1648 Yes 2006 1194 1818 1014 1585 1693 9310No 2488 1245 2152 1906 2078 2087 11956

1629 Yes 1509 1838 0652 2344 1398 1109 8850No 1632 2536 1328 2508 1497 1340 10841

lowastMale female

Complexity 13

the proposed method Our future work is going to furtherimprove the measurement accuracy and focus on trackingpedestrians in real time by using the real-time height ofmoving pedestrians

Data Availability

+e data used to support the findings of this study areavailable from the corresponding author upon request

Conflicts of Interest

+e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

+e authors are grateful to the financial support from theNatural Science Foundation of China (61877065) the NationalKey Research and Development Program of China(2019YFB1405500) the National Natural Science Foundationof Guangdong (2016A030313177) Guangdong Frontier andKey Technological Innovation (2017B090910013) and theScience and Technology Innovation Commission of Shenzhen(JCYJ20170818153048647 and JCYJ20180507182239617)

References

[1] J Li X Liang S Shen et al ldquoScale-aware fast R-CNN forpedestrian detectionrdquo IEEE Transactions on Multimediavol 20 no 4 pp 985ndash996 2017

[2] F P An ldquoPedestrian re-recognition algorithm based onoptimization deep learning-sequence memory modelrdquoComplexity vol 2019 Article ID 5069026 16 pages 2019

[3] J Cao Y Pang and X Li ldquoLearning multilayer channelfeatures for pedestrian detectionrdquo IEEE Transactions on ImageProcessing vol 26 no 7 pp 3210ndash3220 2017

[4] M Ji J Liu X Xu Y Guo and Z Lu ldquoImproved pedestrianpositioning with inertial sensor based on adaptive gradientdescent and double-constrained extended kalman filterrdquoComplexity vol 2020 Article ID 4361812 11 pages 2020

[5] C Li Z Su Q Li and H Zhao ldquoAn indoor positioning errorcorrection method of pedestrian multi-motions recognized byhybrid-orders fraction domain transformationrdquo IEEE Accessvol 7 pp 11360ndash11377 2019

[6] H Zhao W Cheng N Yang et al ldquoSmartphone-based 3Dindoor pedestrian positioning through multi-modal datafusionrdquo Sensors vol 19 no 20 Article ID s19204554 2019

[7] B Wang T Su X Jin J Kong and Y Bai ldquo3D reconstructionof pedestrian trajectory with moving direction learning andoptimal gait recognitionrdquo Complexity vol 2018 Article ID8735846 10 pages 2018

[8] Y Jiang Z Li and J B Wang ldquoPtrack enhancing the ap-plicability of pedestrian tracking with wearablesrdquo IEEETransactions on Mobile Computing vol 18 no 2 pp 431ndash4432018

[9] W Xu L Liu S Zlatanova W Penard and Q Xiong ldquoApedestrian tracking algorithm using grid-based indoormodelrdquo Automation in Construction vol 92 pp 173ndash1872018

[10] L Bozgeyikli A Raij S Katkoori and R Alqasemi ldquoA surveyon virtual reality for individuals with autism spectrum

disorder design considerationsrdquo IEEE Transactions onLearning Technologies vol 11 no 2 pp 133ndash151 2017

[11] J O Nilsson D Zachariah I Skog and P Handel ldquoCoop-erative localization by dual foot-mounted inertial sensors andinter-agent rangingrdquo EURASIP Journal on Advances in SignalProcessing vol 2013 no 1 p 164 2013

[12] I Skog J-O Nilsson D Zachariah and P Handel ldquoFusingthe information from two navigation systems using an upperbound on their maximum spatial separationrdquo in Proceedingsof the 2012 International Conference on Indoor Positioning andIndoor Navigation Article ID 6418862 Sydney AustraliaNovember 2012

[13] S-B Chen Y Xin and B Luo ldquoAction-based pedestrianidentification via hierarchical matching pursuit and orderpreserving sparse codingrdquo Cognitive Computation vol 8no 5 pp 797ndash805 2016

[14] B Shin C Kim J Kim et al ldquoMotion recognition based 3Dpedestrian navigation system using smartphonerdquo IEEE Sen-sors Journal vol 16 no 18 pp 6977ndash6989 2016

[15] M Romanovas V Goridko A Al-Jawad et al ldquoA study onindoor pedestrian localization algorithms with foot-mountedsensorsrdquo in Proceedings of the International Conference onIndoor Positioning and Indoor Navigation pp 1ndash10 SydneyAustralia November 2012

[16] A Azaman ldquoComparative study on gait kinematics betweenmicrosoft kinect and vicon across different anthropometricmeasurementsrdquo Journal of Tomography System and SensorApplication vol 2 no 2 pp 12ndash17 2019

[17] W Sheng A +obbi and Y Gu ldquoAn integrated frameworkfor human-robot collaborative manipulationrdquo IEEE Trans-actions on Cybernetics vol 45 no 10 pp 2030ndash2041 2014

[18] S Tsuji and T Kohama ldquoProximity skin sensor using time-of-flight sensor for human collaborative robotrdquo IEEE SensorsJournal vol 19 no 14 pp 5859ndash5864 2019

[19] C Oprea I Pirnog I Marcu and M Udrea ldquoRobust poseestimation using Time-of-Flight imagingrdquo in Proceedings ofthe IEEE International Semiconductor Conference pp 301ndash304 Sinaia Romania January 2019

[20] A Vysocky R Pastor and P Novak ldquoInteraction with col-laborative robot using 2D and TOF camerardquo in InternationalConference on Modelling and Simulation for AutonomousSystems pp 477ndash489 Springer Cham Switzerland 2018

[21] M Gao Y Du Y Yang and J Zhang ldquoAdaptive anchor boxmechanism to improve the accuracy in the object detectionsystemrdquo Multimedia Tools and Applications vol 78 no 19pp 27383ndash27402 2019

[22] A Anwer S S Azhar Ali A Khan and F MeriaudeauldquoUnderwater 3-d scene reconstruction using kinect v2 basedon physical models for refraction and time of flight correc-tionrdquo IEEE Access vol 5 pp 15960ndash15970 2017

[23] A R Garcıa L R Miller C F Andres and P J N LorenteldquoObstacle detection using a time of flight range camerardquo inProceedings of the 2018 IEEE International Conference onVehicular Electronics and Safety (ICVES) pp 1ndash6 MadridSpain September 2018

[24] N Zengeler T Kopinski and U Handmann ldquoHand gesturerecognition in automotive humanndashmachine interaction usingdepth camerasrdquo Sensors vol 19 no 1 Article ID s190100592019

[25] M A Garduntildeo-Ramon I R Terol-Villalobos R A Osornio-Rios and L A Morales-Hernandez ldquoA new method forinpainting of depthmaps from time-of-flight sensors based ona modified closing by reconstruction algorithmrdquo Journal of

14 Complexity

Visual Communication and Image Representation vol 47pp 36ndash47 2019

[26] L Wang Y Luo H Wang and M Fei ldquoMeasurement errorcorrection model of TOF depth camerardquo Chinese Journal ofSystem Simulation vol 29 no 10 pp 2323ndash2329 2017

[27] VICON ldquoOfficial website of oxford metrics companyrdquo 2020httpswwwviconcom

[28] L Zhang W Dong D Zhang and G Shi ldquoTwo-stage imagedenoising by principal component analysis with local pixelgroupingrdquo Pattern Recognition vol 43 no 4 pp 1531ndash15492010

[29] J Kennedy and R Eberhart ldquoParticle swarm optimizationrdquo inProceedings of ICNNrsquo95-International Conference on NeuralNetworks (ICW) vol 4 pp 1942ndash1948 Perth AustraliaNovember 1995

[30] M A M De Oca T Stutzle M Birattari and M DorigoldquoFrankensteinrsquos PSO a composite particle swarm optimiza-tion algorithmrdquo IEEE Transactions on Evolutionary Compu-tation vol 13 no 5 pp 1120ndash1132 2009

[31] Z Zhen S Pang F Wang et al ldquoPattern classification andPSO optimal weights based sky images cloud motion speedcalculation method for solar PV power forecastingrdquo IEEETransactions on Industry Applications vol 55 no 4pp 3331ndash3342 2019

[32] X Wang J-S Pan and S-C Chu ldquoA parallel multi-verseoptimizer for application in multilevel image segmentationrdquoIEEE Access vol 8 pp 32018ndash32030 2020

[33] Z A Bashir and M E El-Hawary ldquoApplying wavelets toshort-term load forecasting using PSO-based neural net-worksrdquo IEEE Transactions on Power Systems vol 24 no 1pp 20ndash27 2009

[34] L Liu Y Wang F Xie and J Gao ldquoLegendre cooperativePSO strategies for trajectory optimizationrdquo Complexityvol 2018 Article ID 5036791 13 pages 2018

[35] Y Shi and R C Eberhart ldquoA modified particle swarm op-timizerrdquo in Proceedings of the 1998 IEEE InternationalConference on Evolutionary Computation Proceedingspp 69ndash73 Anchorage AK USA May 1998

[36] Y Shi and R C Eberhart ldquoParameter selection in particleswarm optimizationrdquo in International Conference on Evolu-tionary Programming pp 591ndash600 Springer Berlin Ger-many 1998

[37] X Lv D Zhou Y Tang and L Ma ldquoAn improved test se-lection optimization model based on fault ambiguity groupisolation and chaotic discrete PSOrdquo Complexity vol 2018Article ID 3942723 10 pages 2018

[38] J Matas O Chum M Urban and T Pajdla ldquoRobust wide-baseline stereo from maximally stable extremal regionsrdquoImage and Vision Computing vol 22 no 10 pp 761ndash7672004

[39] H Shim and S Lee ldquoRecovering translucent objects using asingle time-of-flight depth camerardquo IEEE Transactions onCircuits and Systems for Video Technology vol 26 no 5pp 841ndash854 2015

[40] C K Chui and G ChenKalman Filtering pp 19ndash26 SpringerInternational Publishing Berlin Germany 2017

[41] L Cui X Wang Y Xu H Jiang and J Zhou ldquoA novelswitching unscented Kalman filter method for remaininguseful life prediction of rolling bearingrdquo Measurementvol 135 pp 678ndash684 2019

[42] GWelch and G BishopAn Introduction to the Kalman FilterMacmillan New York NY USA 1995

[43] K Zhang L Zhang H Song and D Zhang ldquoReinitialization-free level set evolution via reaction diffusionrdquo IEEE Trans-actions on Image Processing vol 22 no 1 pp 258ndash271 2012

Complexity 15

(a) (b)

Front

Back

Mark 3

Mark 4

Mark 1

Mark 2

(c)

Figure 5 Experimental setup (a) Site campus corridor (b) Site research room (c) Placement layout of the lightweight reflective balls

(a) (b) (c)

(d) (e) (f )

Figure 6 Experiments with two-pedestrian image (a) Original image with two pedestrians (b) Image obtained by theMCR algorithm alongwith the original image (c) Image obtained by theMSER-based segmentation along with (b) (d e)+e processes and result images obtainedby the RD-LSE along with the original image (f ) Image obtained by the MSER-based segmentation along with that in (e)

Table 1 Iterations (Iter) and CPU time (Time) by FRFLS and PSO methods

MethodsImage 1 Image 2 Image 3 Image 4

Time (s) Iter Time (s) Iter Time (s) Iter Time (s) IterFRFLS 521 643 601 800 532 665 487 611PSO 0049 129 0057 86 0053 185 0050 147Image size 320 lowast 240 pixels

Complexity 9

representations of Figures 7(a) and 7(c) are shown inFigures 7(b) and 7(d) respectively Although the height of thehead is lower than that of the left hand the proposed methodcan still get the correct result Figures 7(i) 7(k) and 7(l) showthe experimental process and result of adopting the proposed

method for Figure 7(g) in which a pedestrian is kneelingAlthough the proposed D-PSO algorithm does not eliminateall redundant noises as shown in Figure 7(j) it also yieldsideal experimental results due to MSERrsquos insensitivity to asmall amount of the sporadic noise All the above experiments

(a)

200

150

100

50

0

Pixe

l

050

100150

200HeightWidth

050100150200250300

(b) (c)

200

150

100

50

0

Pixe

l

050

100150

200HeightWidth

050100150200250300

(d) (e) (f )

(g)

10080604020

0

Pixe

l

250200

150100

500Height

Width

300

200

0100

(h) (i)

150

100

50

0

Pixe

l

200150

10050

0Height

Width

200100

0

300

(j) (k) (l)

Figure 7 Experiments with the pedestrian in different states (a) Original image with the pedestrian raising his left hand (c) Image obtainedby the PSO algorithm along with that in (a) (e f ) Images obtained by the MSER-based segmentation along with that in (c) (b d) +e 3Drepresentation of images in (a c) respectively (g) Original image with the pedestrian who is kneeling (i) Image obtained by the PSOalgorithm along with that in (g) (k l) Images obtained by theMSER-based segmentation along with those in (i) (h j)+e 3D representationof images in (g i) respectively

10 Complexity

show that the performance of our method is very stable andreliable

To further verify the accuracy of the proposed method alot of experiments are conducted based on 6 subjects fourmen and two women who are asked to walk through the testsites at the usual speed Here we take a set of data obtainedfrom the research room as an example to analyse the resultsFigure 8 shows the height results obtained from the sixsubjects using the VICON alone in several continuousseconds the sex and static height of the six subjects arepresented in the legend It explains that it is unrealistic to

keep the height on the static level when the pedestrian iswalking +us it is essential to study the pedestrian height inthe dynamic situation

Due to the high speed of pictures taken by VICON andTOF cameras and the slowness of pedestrian movement(07ndash12 meters per second) we only select 5 height data persecond to show a real-time height comparison between theVICON and the proposedmethod Every fifth of one secondan image is collected with the TOF camera +e pedestrianheight in the image is obtained by the proposed method andcompared with the height collected with VICON at the same

0 100 200 300 400 500 600 700 800 900 1000 1100 1200Number

160016101620163016401650166016701680169017001710172017301740175017601770178017901800

Hei

ght (

mm

)

Men1760167617611728

Women16481629

Figure 8 +e height results got from the six subjects using the VICON alone in several continuous seconds

1800179017801770176017501740173017201710170016901680167016601650

Mea

sure

d he

ight

(mm

)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30Number

Our algorithm without Kalman filtering1760167617611728

Our algorithm with Kalman filtering1760167617611728

VICON (ground truth)1760167617611728

Figure 9 Experimental results of men with different heights in the six consecutive seconds

Complexity 11

1700

1690

1680

1670

1660

1650

1640

1630

1620

1610

1600

1590

1580

1570

Mea

sure

d he

ight

(mm

)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30Number

Our algorithm without Kalman filtering16481629

Our algorithm with Kalman filtering16481629

VICON (ground truth)16481629

Figure 10 Experimental results of women with different heights in the six consecutive seconds

28272625242322211011121314151617181920 29308765432 91

Number

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

(a)

2 3 4 5 6 7 8 91 111213141516171819202122232425262728293010

Number

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

(b)

43 61 2 7 8 95 111213141516171819202122232425262728293010

Number

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

(c)

43 61 2 7 8 95 111213141516171819202122232425262728293010

Number

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

(d)

Figure 11 +e error plot of men in the six consecutive seconds (andashd) +e men with static heights of 1760 1676 1761 and 1728

12 Complexity

time Figures 9 and 10 show the experimental results of fourmen and two women in six consecutive seconds In thefigures the dotted line represents our algorithm withoutKalman filtering the solid line represents our algorithmwithout Kalman filtering and the dotted line with the markldquo+rdquo indicates the VICON+e waveforms show the real-timeheight value in 6 consecutive seconds the static heights ofmen are 1760mm 1676mm 1761mm and 1728mm asshown in the legend of Figure 9 while the static heights ofwomen are 1648mm and 1629mm as shown in Figure 10

It can be seen from the curves that the height datameasured by our algorithm is almost consistent with the dataobtained by VICON In order to analyse the error of ouralgorithm we sort out the errors of all the data in the sixconsecutive seconds the results are shown in Figures 11 and12 +e figures show that Kalman filtering can effectivelyimprove the accuracy of height measurement which indi-cates the pedestrian height at the preceding moment facil-itates the estimate of the pedestrian height in the lattermoment

Also the sums of errors per second of the algorithmswith and without Kalman filtering are given in Table 2where the subscript ldquolowastrdquo represents male and ldquordquo representsfemale Table 2 shows that our algorithm with Kalmanfiltering has a smaller cumulative error and can moreaccurately measure the real-time height of the movingpedestrians which proves the feasibility and validity of theproposed method

5 Conclusion and Future Work

In this paper a real-time height measurement based onthe TOF camera is proposed for moving pedestrians Toget the target region a new D-PSO denoising algorithmand a segmentation algorithm based on MSER are de-veloped in the paper In addition a novel multilayer it-erative average algorithm is designed for calculating thepedestrian height Also the Kalman filtering is used toimprove the measurement accuracy +e experimentalresults demonstrate the effectiveness and practicability of

2 3 4 5 6 7 8 9 1011121314151617181920212223242526272829301Number

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2Er

ror (

)

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

(a)

2 3 4 5 6 7 8 9 1011121314151617181920212223242526272829301Number

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

(b)

Figure 12 +e error plot of women in the six consecutive seconds (a) +e woman with static height of 1648 (b) +e woman with staticheight of 1629

Table 2 +e sum of errors per second of the algorithms with and without Kalman filtering

Heights (mm) Kalman filteringSum of errors per second ()

Sum1st second 2nd second 3rd second 4th second 5th second 6th second

1760lowast Yes 1202 0956 1836 1242 1611 1525 8372No 1868 1003 2013 1362 1898 1758 9902

1676lowast Yes 2002 1799 1977 0863 1648 2137 10426No 2249 1968 2087 1602 1827 3261 12994

1761lowast Yes 1282 1483 0963 1132 0632 1487 6979No 1562 1702 1333 1617 1234 1714 9162

1728lowast Yes 1629 1652 1354 1453 1224 0902 8214No 2201 2159 1912 1592 1984 1336 11184

1648 Yes 2006 1194 1818 1014 1585 1693 9310No 2488 1245 2152 1906 2078 2087 11956

1629 Yes 1509 1838 0652 2344 1398 1109 8850No 1632 2536 1328 2508 1497 1340 10841

lowastMale female

Complexity 13

the proposed method Our future work is going to furtherimprove the measurement accuracy and focus on trackingpedestrians in real time by using the real-time height ofmoving pedestrians

Data Availability

+e data used to support the findings of this study areavailable from the corresponding author upon request

Conflicts of Interest

+e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

+e authors are grateful to the financial support from theNatural Science Foundation of China (61877065) the NationalKey Research and Development Program of China(2019YFB1405500) the National Natural Science Foundationof Guangdong (2016A030313177) Guangdong Frontier andKey Technological Innovation (2017B090910013) and theScience and Technology Innovation Commission of Shenzhen(JCYJ20170818153048647 and JCYJ20180507182239617)

References

[1] J Li X Liang S Shen et al ldquoScale-aware fast R-CNN forpedestrian detectionrdquo IEEE Transactions on Multimediavol 20 no 4 pp 985ndash996 2017

[2] F P An ldquoPedestrian re-recognition algorithm based onoptimization deep learning-sequence memory modelrdquoComplexity vol 2019 Article ID 5069026 16 pages 2019

[3] J Cao Y Pang and X Li ldquoLearning multilayer channelfeatures for pedestrian detectionrdquo IEEE Transactions on ImageProcessing vol 26 no 7 pp 3210ndash3220 2017

[4] M Ji J Liu X Xu Y Guo and Z Lu ldquoImproved pedestrianpositioning with inertial sensor based on adaptive gradientdescent and double-constrained extended kalman filterrdquoComplexity vol 2020 Article ID 4361812 11 pages 2020

[5] C Li Z Su Q Li and H Zhao ldquoAn indoor positioning errorcorrection method of pedestrian multi-motions recognized byhybrid-orders fraction domain transformationrdquo IEEE Accessvol 7 pp 11360ndash11377 2019

[6] H Zhao W Cheng N Yang et al ldquoSmartphone-based 3Dindoor pedestrian positioning through multi-modal datafusionrdquo Sensors vol 19 no 20 Article ID s19204554 2019

[7] B Wang T Su X Jin J Kong and Y Bai ldquo3D reconstructionof pedestrian trajectory with moving direction learning andoptimal gait recognitionrdquo Complexity vol 2018 Article ID8735846 10 pages 2018

[8] Y Jiang Z Li and J B Wang ldquoPtrack enhancing the ap-plicability of pedestrian tracking with wearablesrdquo IEEETransactions on Mobile Computing vol 18 no 2 pp 431ndash4432018

[9] W Xu L Liu S Zlatanova W Penard and Q Xiong ldquoApedestrian tracking algorithm using grid-based indoormodelrdquo Automation in Construction vol 92 pp 173ndash1872018

[10] L Bozgeyikli A Raij S Katkoori and R Alqasemi ldquoA surveyon virtual reality for individuals with autism spectrum

disorder design considerationsrdquo IEEE Transactions onLearning Technologies vol 11 no 2 pp 133ndash151 2017

[11] J O Nilsson D Zachariah I Skog and P Handel ldquoCoop-erative localization by dual foot-mounted inertial sensors andinter-agent rangingrdquo EURASIP Journal on Advances in SignalProcessing vol 2013 no 1 p 164 2013

[12] I Skog J-O Nilsson D Zachariah and P Handel ldquoFusingthe information from two navigation systems using an upperbound on their maximum spatial separationrdquo in Proceedingsof the 2012 International Conference on Indoor Positioning andIndoor Navigation Article ID 6418862 Sydney AustraliaNovember 2012

[13] S-B Chen Y Xin and B Luo ldquoAction-based pedestrianidentification via hierarchical matching pursuit and orderpreserving sparse codingrdquo Cognitive Computation vol 8no 5 pp 797ndash805 2016

[14] B Shin C Kim J Kim et al ldquoMotion recognition based 3Dpedestrian navigation system using smartphonerdquo IEEE Sen-sors Journal vol 16 no 18 pp 6977ndash6989 2016

[15] M Romanovas V Goridko A Al-Jawad et al ldquoA study onindoor pedestrian localization algorithms with foot-mountedsensorsrdquo in Proceedings of the International Conference onIndoor Positioning and Indoor Navigation pp 1ndash10 SydneyAustralia November 2012

[16] A Azaman ldquoComparative study on gait kinematics betweenmicrosoft kinect and vicon across different anthropometricmeasurementsrdquo Journal of Tomography System and SensorApplication vol 2 no 2 pp 12ndash17 2019

[17] W Sheng A +obbi and Y Gu ldquoAn integrated frameworkfor human-robot collaborative manipulationrdquo IEEE Trans-actions on Cybernetics vol 45 no 10 pp 2030ndash2041 2014

[18] S Tsuji and T Kohama ldquoProximity skin sensor using time-of-flight sensor for human collaborative robotrdquo IEEE SensorsJournal vol 19 no 14 pp 5859ndash5864 2019

[19] C Oprea I Pirnog I Marcu and M Udrea ldquoRobust poseestimation using Time-of-Flight imagingrdquo in Proceedings ofthe IEEE International Semiconductor Conference pp 301ndash304 Sinaia Romania January 2019

[20] A Vysocky R Pastor and P Novak ldquoInteraction with col-laborative robot using 2D and TOF camerardquo in InternationalConference on Modelling and Simulation for AutonomousSystems pp 477ndash489 Springer Cham Switzerland 2018

[21] M Gao Y Du Y Yang and J Zhang ldquoAdaptive anchor boxmechanism to improve the accuracy in the object detectionsystemrdquo Multimedia Tools and Applications vol 78 no 19pp 27383ndash27402 2019

[22] A Anwer S S Azhar Ali A Khan and F MeriaudeauldquoUnderwater 3-d scene reconstruction using kinect v2 basedon physical models for refraction and time of flight correc-tionrdquo IEEE Access vol 5 pp 15960ndash15970 2017

[23] A R Garcıa L R Miller C F Andres and P J N LorenteldquoObstacle detection using a time of flight range camerardquo inProceedings of the 2018 IEEE International Conference onVehicular Electronics and Safety (ICVES) pp 1ndash6 MadridSpain September 2018

[24] N Zengeler T Kopinski and U Handmann ldquoHand gesturerecognition in automotive humanndashmachine interaction usingdepth camerasrdquo Sensors vol 19 no 1 Article ID s190100592019

[25] M A Garduntildeo-Ramon I R Terol-Villalobos R A Osornio-Rios and L A Morales-Hernandez ldquoA new method forinpainting of depthmaps from time-of-flight sensors based ona modified closing by reconstruction algorithmrdquo Journal of

14 Complexity

Visual Communication and Image Representation vol 47pp 36ndash47 2019

[26] L Wang Y Luo H Wang and M Fei ldquoMeasurement errorcorrection model of TOF depth camerardquo Chinese Journal ofSystem Simulation vol 29 no 10 pp 2323ndash2329 2017

[27] VICON ldquoOfficial website of oxford metrics companyrdquo 2020httpswwwviconcom

[28] L Zhang W Dong D Zhang and G Shi ldquoTwo-stage imagedenoising by principal component analysis with local pixelgroupingrdquo Pattern Recognition vol 43 no 4 pp 1531ndash15492010

[29] J Kennedy and R Eberhart ldquoParticle swarm optimizationrdquo inProceedings of ICNNrsquo95-International Conference on NeuralNetworks (ICW) vol 4 pp 1942ndash1948 Perth AustraliaNovember 1995

[30] M A M De Oca T Stutzle M Birattari and M DorigoldquoFrankensteinrsquos PSO a composite particle swarm optimiza-tion algorithmrdquo IEEE Transactions on Evolutionary Compu-tation vol 13 no 5 pp 1120ndash1132 2009

[31] Z Zhen S Pang F Wang et al ldquoPattern classification andPSO optimal weights based sky images cloud motion speedcalculation method for solar PV power forecastingrdquo IEEETransactions on Industry Applications vol 55 no 4pp 3331ndash3342 2019

[32] X Wang J-S Pan and S-C Chu ldquoA parallel multi-verseoptimizer for application in multilevel image segmentationrdquoIEEE Access vol 8 pp 32018ndash32030 2020

[33] Z A Bashir and M E El-Hawary ldquoApplying wavelets toshort-term load forecasting using PSO-based neural net-worksrdquo IEEE Transactions on Power Systems vol 24 no 1pp 20ndash27 2009

[34] L Liu Y Wang F Xie and J Gao ldquoLegendre cooperativePSO strategies for trajectory optimizationrdquo Complexityvol 2018 Article ID 5036791 13 pages 2018

[35] Y Shi and R C Eberhart ldquoA modified particle swarm op-timizerrdquo in Proceedings of the 1998 IEEE InternationalConference on Evolutionary Computation Proceedingspp 69ndash73 Anchorage AK USA May 1998

[36] Y Shi and R C Eberhart ldquoParameter selection in particleswarm optimizationrdquo in International Conference on Evolu-tionary Programming pp 591ndash600 Springer Berlin Ger-many 1998

[37] X Lv D Zhou Y Tang and L Ma ldquoAn improved test se-lection optimization model based on fault ambiguity groupisolation and chaotic discrete PSOrdquo Complexity vol 2018Article ID 3942723 10 pages 2018

[38] J Matas O Chum M Urban and T Pajdla ldquoRobust wide-baseline stereo from maximally stable extremal regionsrdquoImage and Vision Computing vol 22 no 10 pp 761ndash7672004

[39] H Shim and S Lee ldquoRecovering translucent objects using asingle time-of-flight depth camerardquo IEEE Transactions onCircuits and Systems for Video Technology vol 26 no 5pp 841ndash854 2015

[40] C K Chui and G ChenKalman Filtering pp 19ndash26 SpringerInternational Publishing Berlin Germany 2017

[41] L Cui X Wang Y Xu H Jiang and J Zhou ldquoA novelswitching unscented Kalman filter method for remaininguseful life prediction of rolling bearingrdquo Measurementvol 135 pp 678ndash684 2019

[42] GWelch and G BishopAn Introduction to the Kalman FilterMacmillan New York NY USA 1995

[43] K Zhang L Zhang H Song and D Zhang ldquoReinitialization-free level set evolution via reaction diffusionrdquo IEEE Trans-actions on Image Processing vol 22 no 1 pp 258ndash271 2012

Complexity 15

representations of Figures 7(a) and 7(c) are shown inFigures 7(b) and 7(d) respectively Although the height of thehead is lower than that of the left hand the proposed methodcan still get the correct result Figures 7(i) 7(k) and 7(l) showthe experimental process and result of adopting the proposed

method for Figure 7(g) in which a pedestrian is kneelingAlthough the proposed D-PSO algorithm does not eliminateall redundant noises as shown in Figure 7(j) it also yieldsideal experimental results due to MSERrsquos insensitivity to asmall amount of the sporadic noise All the above experiments

(a)

200

150

100

50

0

Pixe

l

050

100150

200HeightWidth

050100150200250300

(b) (c)

200

150

100

50

0

Pixe

l

050

100150

200HeightWidth

050100150200250300

(d) (e) (f )

(g)

10080604020

0

Pixe

l

250200

150100

500Height

Width

300

200

0100

(h) (i)

150

100

50

0

Pixe

l

200150

10050

0Height

Width

200100

0

300

(j) (k) (l)

Figure 7 Experiments with the pedestrian in different states (a) Original image with the pedestrian raising his left hand (c) Image obtainedby the PSO algorithm along with that in (a) (e f ) Images obtained by the MSER-based segmentation along with that in (c) (b d) +e 3Drepresentation of images in (a c) respectively (g) Original image with the pedestrian who is kneeling (i) Image obtained by the PSOalgorithm along with that in (g) (k l) Images obtained by theMSER-based segmentation along with those in (i) (h j)+e 3D representationof images in (g i) respectively

10 Complexity

show that the performance of our method is very stable andreliable

To further verify the accuracy of the proposed method alot of experiments are conducted based on 6 subjects fourmen and two women who are asked to walk through the testsites at the usual speed Here we take a set of data obtainedfrom the research room as an example to analyse the resultsFigure 8 shows the height results obtained from the sixsubjects using the VICON alone in several continuousseconds the sex and static height of the six subjects arepresented in the legend It explains that it is unrealistic to

keep the height on the static level when the pedestrian iswalking +us it is essential to study the pedestrian height inthe dynamic situation

Due to the high speed of pictures taken by VICON andTOF cameras and the slowness of pedestrian movement(07ndash12 meters per second) we only select 5 height data persecond to show a real-time height comparison between theVICON and the proposedmethod Every fifth of one secondan image is collected with the TOF camera +e pedestrianheight in the image is obtained by the proposed method andcompared with the height collected with VICON at the same

0 100 200 300 400 500 600 700 800 900 1000 1100 1200Number

160016101620163016401650166016701680169017001710172017301740175017601770178017901800

Hei

ght (

mm

)

Men1760167617611728

Women16481629

Figure 8 +e height results got from the six subjects using the VICON alone in several continuous seconds

1800179017801770176017501740173017201710170016901680167016601650

Mea

sure

d he

ight

(mm

)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30Number

Our algorithm without Kalman filtering1760167617611728

Our algorithm with Kalman filtering1760167617611728

VICON (ground truth)1760167617611728

Figure 9 Experimental results of men with different heights in the six consecutive seconds

Complexity 11

1700

1690

1680

1670

1660

1650

1640

1630

1620

1610

1600

1590

1580

1570

Mea

sure

d he

ight

(mm

)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30Number

Our algorithm without Kalman filtering16481629

Our algorithm with Kalman filtering16481629

VICON (ground truth)16481629

Figure 10 Experimental results of women with different heights in the six consecutive seconds

28272625242322211011121314151617181920 29308765432 91

Number

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

(a)

2 3 4 5 6 7 8 91 111213141516171819202122232425262728293010

Number

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

(b)

43 61 2 7 8 95 111213141516171819202122232425262728293010

Number

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

(c)

43 61 2 7 8 95 111213141516171819202122232425262728293010

Number

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

(d)

Figure 11 +e error plot of men in the six consecutive seconds (andashd) +e men with static heights of 1760 1676 1761 and 1728

12 Complexity

time Figures 9 and 10 show the experimental results of fourmen and two women in six consecutive seconds In thefigures the dotted line represents our algorithm withoutKalman filtering the solid line represents our algorithmwithout Kalman filtering and the dotted line with the markldquo+rdquo indicates the VICON+e waveforms show the real-timeheight value in 6 consecutive seconds the static heights ofmen are 1760mm 1676mm 1761mm and 1728mm asshown in the legend of Figure 9 while the static heights ofwomen are 1648mm and 1629mm as shown in Figure 10

It can be seen from the curves that the height datameasured by our algorithm is almost consistent with the dataobtained by VICON In order to analyse the error of ouralgorithm we sort out the errors of all the data in the sixconsecutive seconds the results are shown in Figures 11 and12 +e figures show that Kalman filtering can effectivelyimprove the accuracy of height measurement which indi-cates the pedestrian height at the preceding moment facil-itates the estimate of the pedestrian height in the lattermoment

Also the sums of errors per second of the algorithmswith and without Kalman filtering are given in Table 2where the subscript ldquolowastrdquo represents male and ldquordquo representsfemale Table 2 shows that our algorithm with Kalmanfiltering has a smaller cumulative error and can moreaccurately measure the real-time height of the movingpedestrians which proves the feasibility and validity of theproposed method

5 Conclusion and Future Work

In this paper a real-time height measurement based onthe TOF camera is proposed for moving pedestrians Toget the target region a new D-PSO denoising algorithmand a segmentation algorithm based on MSER are de-veloped in the paper In addition a novel multilayer it-erative average algorithm is designed for calculating thepedestrian height Also the Kalman filtering is used toimprove the measurement accuracy +e experimentalresults demonstrate the effectiveness and practicability of

2 3 4 5 6 7 8 9 1011121314151617181920212223242526272829301Number

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2Er

ror (

)

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

(a)

2 3 4 5 6 7 8 9 1011121314151617181920212223242526272829301Number

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

(b)

Figure 12 +e error plot of women in the six consecutive seconds (a) +e woman with static height of 1648 (b) +e woman with staticheight of 1629

Table 2 +e sum of errors per second of the algorithms with and without Kalman filtering

Heights (mm) Kalman filteringSum of errors per second ()

Sum1st second 2nd second 3rd second 4th second 5th second 6th second

1760lowast Yes 1202 0956 1836 1242 1611 1525 8372No 1868 1003 2013 1362 1898 1758 9902

1676lowast Yes 2002 1799 1977 0863 1648 2137 10426No 2249 1968 2087 1602 1827 3261 12994

1761lowast Yes 1282 1483 0963 1132 0632 1487 6979No 1562 1702 1333 1617 1234 1714 9162

1728lowast Yes 1629 1652 1354 1453 1224 0902 8214No 2201 2159 1912 1592 1984 1336 11184

1648 Yes 2006 1194 1818 1014 1585 1693 9310No 2488 1245 2152 1906 2078 2087 11956

1629 Yes 1509 1838 0652 2344 1398 1109 8850No 1632 2536 1328 2508 1497 1340 10841

lowastMale female

Complexity 13

the proposed method Our future work is going to furtherimprove the measurement accuracy and focus on trackingpedestrians in real time by using the real-time height ofmoving pedestrians

Data Availability

+e data used to support the findings of this study areavailable from the corresponding author upon request

Conflicts of Interest

+e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

+e authors are grateful to the financial support from theNatural Science Foundation of China (61877065) the NationalKey Research and Development Program of China(2019YFB1405500) the National Natural Science Foundationof Guangdong (2016A030313177) Guangdong Frontier andKey Technological Innovation (2017B090910013) and theScience and Technology Innovation Commission of Shenzhen(JCYJ20170818153048647 and JCYJ20180507182239617)

References

[1] J Li X Liang S Shen et al ldquoScale-aware fast R-CNN forpedestrian detectionrdquo IEEE Transactions on Multimediavol 20 no 4 pp 985ndash996 2017

[2] F P An ldquoPedestrian re-recognition algorithm based onoptimization deep learning-sequence memory modelrdquoComplexity vol 2019 Article ID 5069026 16 pages 2019

[3] J Cao Y Pang and X Li ldquoLearning multilayer channelfeatures for pedestrian detectionrdquo IEEE Transactions on ImageProcessing vol 26 no 7 pp 3210ndash3220 2017

[4] M Ji J Liu X Xu Y Guo and Z Lu ldquoImproved pedestrianpositioning with inertial sensor based on adaptive gradientdescent and double-constrained extended kalman filterrdquoComplexity vol 2020 Article ID 4361812 11 pages 2020

[5] C Li Z Su Q Li and H Zhao ldquoAn indoor positioning errorcorrection method of pedestrian multi-motions recognized byhybrid-orders fraction domain transformationrdquo IEEE Accessvol 7 pp 11360ndash11377 2019

[6] H Zhao W Cheng N Yang et al ldquoSmartphone-based 3Dindoor pedestrian positioning through multi-modal datafusionrdquo Sensors vol 19 no 20 Article ID s19204554 2019

[7] B Wang T Su X Jin J Kong and Y Bai ldquo3D reconstructionof pedestrian trajectory with moving direction learning andoptimal gait recognitionrdquo Complexity vol 2018 Article ID8735846 10 pages 2018

[8] Y Jiang Z Li and J B Wang ldquoPtrack enhancing the ap-plicability of pedestrian tracking with wearablesrdquo IEEETransactions on Mobile Computing vol 18 no 2 pp 431ndash4432018

[9] W Xu L Liu S Zlatanova W Penard and Q Xiong ldquoApedestrian tracking algorithm using grid-based indoormodelrdquo Automation in Construction vol 92 pp 173ndash1872018

[10] L Bozgeyikli A Raij S Katkoori and R Alqasemi ldquoA surveyon virtual reality for individuals with autism spectrum

disorder design considerationsrdquo IEEE Transactions onLearning Technologies vol 11 no 2 pp 133ndash151 2017

[11] J O Nilsson D Zachariah I Skog and P Handel ldquoCoop-erative localization by dual foot-mounted inertial sensors andinter-agent rangingrdquo EURASIP Journal on Advances in SignalProcessing vol 2013 no 1 p 164 2013

[12] I Skog J-O Nilsson D Zachariah and P Handel ldquoFusingthe information from two navigation systems using an upperbound on their maximum spatial separationrdquo in Proceedingsof the 2012 International Conference on Indoor Positioning andIndoor Navigation Article ID 6418862 Sydney AustraliaNovember 2012

[13] S-B Chen Y Xin and B Luo ldquoAction-based pedestrianidentification via hierarchical matching pursuit and orderpreserving sparse codingrdquo Cognitive Computation vol 8no 5 pp 797ndash805 2016

[14] B Shin C Kim J Kim et al ldquoMotion recognition based 3Dpedestrian navigation system using smartphonerdquo IEEE Sen-sors Journal vol 16 no 18 pp 6977ndash6989 2016

[15] M Romanovas V Goridko A Al-Jawad et al ldquoA study onindoor pedestrian localization algorithms with foot-mountedsensorsrdquo in Proceedings of the International Conference onIndoor Positioning and Indoor Navigation pp 1ndash10 SydneyAustralia November 2012

[16] A Azaman ldquoComparative study on gait kinematics betweenmicrosoft kinect and vicon across different anthropometricmeasurementsrdquo Journal of Tomography System and SensorApplication vol 2 no 2 pp 12ndash17 2019

[17] W Sheng A +obbi and Y Gu ldquoAn integrated frameworkfor human-robot collaborative manipulationrdquo IEEE Trans-actions on Cybernetics vol 45 no 10 pp 2030ndash2041 2014

[18] S Tsuji and T Kohama ldquoProximity skin sensor using time-of-flight sensor for human collaborative robotrdquo IEEE SensorsJournal vol 19 no 14 pp 5859ndash5864 2019

[19] C Oprea I Pirnog I Marcu and M Udrea ldquoRobust poseestimation using Time-of-Flight imagingrdquo in Proceedings ofthe IEEE International Semiconductor Conference pp 301ndash304 Sinaia Romania January 2019

[20] A Vysocky R Pastor and P Novak ldquoInteraction with col-laborative robot using 2D and TOF camerardquo in InternationalConference on Modelling and Simulation for AutonomousSystems pp 477ndash489 Springer Cham Switzerland 2018

[21] M Gao Y Du Y Yang and J Zhang ldquoAdaptive anchor boxmechanism to improve the accuracy in the object detectionsystemrdquo Multimedia Tools and Applications vol 78 no 19pp 27383ndash27402 2019

[22] A Anwer S S Azhar Ali A Khan and F MeriaudeauldquoUnderwater 3-d scene reconstruction using kinect v2 basedon physical models for refraction and time of flight correc-tionrdquo IEEE Access vol 5 pp 15960ndash15970 2017

[23] A R Garcıa L R Miller C F Andres and P J N LorenteldquoObstacle detection using a time of flight range camerardquo inProceedings of the 2018 IEEE International Conference onVehicular Electronics and Safety (ICVES) pp 1ndash6 MadridSpain September 2018

[24] N Zengeler T Kopinski and U Handmann ldquoHand gesturerecognition in automotive humanndashmachine interaction usingdepth camerasrdquo Sensors vol 19 no 1 Article ID s190100592019

[25] M A Garduntildeo-Ramon I R Terol-Villalobos R A Osornio-Rios and L A Morales-Hernandez ldquoA new method forinpainting of depthmaps from time-of-flight sensors based ona modified closing by reconstruction algorithmrdquo Journal of

14 Complexity

Visual Communication and Image Representation vol 47pp 36ndash47 2019

[26] L Wang Y Luo H Wang and M Fei ldquoMeasurement errorcorrection model of TOF depth camerardquo Chinese Journal ofSystem Simulation vol 29 no 10 pp 2323ndash2329 2017

[27] VICON ldquoOfficial website of oxford metrics companyrdquo 2020httpswwwviconcom

[28] L Zhang W Dong D Zhang and G Shi ldquoTwo-stage imagedenoising by principal component analysis with local pixelgroupingrdquo Pattern Recognition vol 43 no 4 pp 1531ndash15492010

[29] J Kennedy and R Eberhart ldquoParticle swarm optimizationrdquo inProceedings of ICNNrsquo95-International Conference on NeuralNetworks (ICW) vol 4 pp 1942ndash1948 Perth AustraliaNovember 1995

[30] M A M De Oca T Stutzle M Birattari and M DorigoldquoFrankensteinrsquos PSO a composite particle swarm optimiza-tion algorithmrdquo IEEE Transactions on Evolutionary Compu-tation vol 13 no 5 pp 1120ndash1132 2009

[31] Z Zhen S Pang F Wang et al ldquoPattern classification andPSO optimal weights based sky images cloud motion speedcalculation method for solar PV power forecastingrdquo IEEETransactions on Industry Applications vol 55 no 4pp 3331ndash3342 2019

[32] X Wang J-S Pan and S-C Chu ldquoA parallel multi-verseoptimizer for application in multilevel image segmentationrdquoIEEE Access vol 8 pp 32018ndash32030 2020

[33] Z A Bashir and M E El-Hawary ldquoApplying wavelets toshort-term load forecasting using PSO-based neural net-worksrdquo IEEE Transactions on Power Systems vol 24 no 1pp 20ndash27 2009

[34] L Liu Y Wang F Xie and J Gao ldquoLegendre cooperativePSO strategies for trajectory optimizationrdquo Complexityvol 2018 Article ID 5036791 13 pages 2018

[35] Y Shi and R C Eberhart ldquoA modified particle swarm op-timizerrdquo in Proceedings of the 1998 IEEE InternationalConference on Evolutionary Computation Proceedingspp 69ndash73 Anchorage AK USA May 1998

[36] Y Shi and R C Eberhart ldquoParameter selection in particleswarm optimizationrdquo in International Conference on Evolu-tionary Programming pp 591ndash600 Springer Berlin Ger-many 1998

[37] X Lv D Zhou Y Tang and L Ma ldquoAn improved test se-lection optimization model based on fault ambiguity groupisolation and chaotic discrete PSOrdquo Complexity vol 2018Article ID 3942723 10 pages 2018

[38] J Matas O Chum M Urban and T Pajdla ldquoRobust wide-baseline stereo from maximally stable extremal regionsrdquoImage and Vision Computing vol 22 no 10 pp 761ndash7672004

[39] H Shim and S Lee ldquoRecovering translucent objects using asingle time-of-flight depth camerardquo IEEE Transactions onCircuits and Systems for Video Technology vol 26 no 5pp 841ndash854 2015

[40] C K Chui and G ChenKalman Filtering pp 19ndash26 SpringerInternational Publishing Berlin Germany 2017

[41] L Cui X Wang Y Xu H Jiang and J Zhou ldquoA novelswitching unscented Kalman filter method for remaininguseful life prediction of rolling bearingrdquo Measurementvol 135 pp 678ndash684 2019

[42] GWelch and G BishopAn Introduction to the Kalman FilterMacmillan New York NY USA 1995

[43] K Zhang L Zhang H Song and D Zhang ldquoReinitialization-free level set evolution via reaction diffusionrdquo IEEE Trans-actions on Image Processing vol 22 no 1 pp 258ndash271 2012

Complexity 15

show that the performance of our method is very stable andreliable

To further verify the accuracy of the proposed method alot of experiments are conducted based on 6 subjects fourmen and two women who are asked to walk through the testsites at the usual speed Here we take a set of data obtainedfrom the research room as an example to analyse the resultsFigure 8 shows the height results obtained from the sixsubjects using the VICON alone in several continuousseconds the sex and static height of the six subjects arepresented in the legend It explains that it is unrealistic to

keep the height on the static level when the pedestrian iswalking +us it is essential to study the pedestrian height inthe dynamic situation

Due to the high speed of pictures taken by VICON andTOF cameras and the slowness of pedestrian movement(07ndash12 meters per second) we only select 5 height data persecond to show a real-time height comparison between theVICON and the proposedmethod Every fifth of one secondan image is collected with the TOF camera +e pedestrianheight in the image is obtained by the proposed method andcompared with the height collected with VICON at the same

0 100 200 300 400 500 600 700 800 900 1000 1100 1200Number

160016101620163016401650166016701680169017001710172017301740175017601770178017901800

Hei

ght (

mm

)

Men1760167617611728

Women16481629

Figure 8 +e height results got from the six subjects using the VICON alone in several continuous seconds

1800179017801770176017501740173017201710170016901680167016601650

Mea

sure

d he

ight

(mm

)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30Number

Our algorithm without Kalman filtering1760167617611728

Our algorithm with Kalman filtering1760167617611728

VICON (ground truth)1760167617611728

Figure 9 Experimental results of men with different heights in the six consecutive seconds

Complexity 11

1700

1690

1680

1670

1660

1650

1640

1630

1620

1610

1600

1590

1580

1570

Mea

sure

d he

ight

(mm

)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30Number

Our algorithm without Kalman filtering16481629

Our algorithm with Kalman filtering16481629

VICON (ground truth)16481629

Figure 10 Experimental results of women with different heights in the six consecutive seconds

28272625242322211011121314151617181920 29308765432 91

Number

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

(a)

2 3 4 5 6 7 8 91 111213141516171819202122232425262728293010

Number

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

(b)

43 61 2 7 8 95 111213141516171819202122232425262728293010

Number

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

(c)

43 61 2 7 8 95 111213141516171819202122232425262728293010

Number

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

(d)

Figure 11 +e error plot of men in the six consecutive seconds (andashd) +e men with static heights of 1760 1676 1761 and 1728

12 Complexity

time Figures 9 and 10 show the experimental results of fourmen and two women in six consecutive seconds In thefigures the dotted line represents our algorithm withoutKalman filtering the solid line represents our algorithmwithout Kalman filtering and the dotted line with the markldquo+rdquo indicates the VICON+e waveforms show the real-timeheight value in 6 consecutive seconds the static heights ofmen are 1760mm 1676mm 1761mm and 1728mm asshown in the legend of Figure 9 while the static heights ofwomen are 1648mm and 1629mm as shown in Figure 10

It can be seen from the curves that the height datameasured by our algorithm is almost consistent with the dataobtained by VICON In order to analyse the error of ouralgorithm we sort out the errors of all the data in the sixconsecutive seconds the results are shown in Figures 11 and12 +e figures show that Kalman filtering can effectivelyimprove the accuracy of height measurement which indi-cates the pedestrian height at the preceding moment facil-itates the estimate of the pedestrian height in the lattermoment

Also the sums of errors per second of the algorithmswith and without Kalman filtering are given in Table 2where the subscript ldquolowastrdquo represents male and ldquordquo representsfemale Table 2 shows that our algorithm with Kalmanfiltering has a smaller cumulative error and can moreaccurately measure the real-time height of the movingpedestrians which proves the feasibility and validity of theproposed method

5 Conclusion and Future Work

In this paper a real-time height measurement based onthe TOF camera is proposed for moving pedestrians Toget the target region a new D-PSO denoising algorithmand a segmentation algorithm based on MSER are de-veloped in the paper In addition a novel multilayer it-erative average algorithm is designed for calculating thepedestrian height Also the Kalman filtering is used toimprove the measurement accuracy +e experimentalresults demonstrate the effectiveness and practicability of

2 3 4 5 6 7 8 9 1011121314151617181920212223242526272829301Number

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2Er

ror (

)

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

(a)

2 3 4 5 6 7 8 9 1011121314151617181920212223242526272829301Number

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

(b)

Figure 12 +e error plot of women in the six consecutive seconds (a) +e woman with static height of 1648 (b) +e woman with staticheight of 1629

Table 2 +e sum of errors per second of the algorithms with and without Kalman filtering

Heights (mm) Kalman filteringSum of errors per second ()

Sum1st second 2nd second 3rd second 4th second 5th second 6th second

1760lowast Yes 1202 0956 1836 1242 1611 1525 8372No 1868 1003 2013 1362 1898 1758 9902

1676lowast Yes 2002 1799 1977 0863 1648 2137 10426No 2249 1968 2087 1602 1827 3261 12994

1761lowast Yes 1282 1483 0963 1132 0632 1487 6979No 1562 1702 1333 1617 1234 1714 9162

1728lowast Yes 1629 1652 1354 1453 1224 0902 8214No 2201 2159 1912 1592 1984 1336 11184

1648 Yes 2006 1194 1818 1014 1585 1693 9310No 2488 1245 2152 1906 2078 2087 11956

1629 Yes 1509 1838 0652 2344 1398 1109 8850No 1632 2536 1328 2508 1497 1340 10841

lowastMale female

Complexity 13

the proposed method Our future work is going to furtherimprove the measurement accuracy and focus on trackingpedestrians in real time by using the real-time height ofmoving pedestrians

Data Availability

+e data used to support the findings of this study areavailable from the corresponding author upon request

Conflicts of Interest

+e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

+e authors are grateful to the financial support from theNatural Science Foundation of China (61877065) the NationalKey Research and Development Program of China(2019YFB1405500) the National Natural Science Foundationof Guangdong (2016A030313177) Guangdong Frontier andKey Technological Innovation (2017B090910013) and theScience and Technology Innovation Commission of Shenzhen(JCYJ20170818153048647 and JCYJ20180507182239617)

References

[1] J Li X Liang S Shen et al ldquoScale-aware fast R-CNN forpedestrian detectionrdquo IEEE Transactions on Multimediavol 20 no 4 pp 985ndash996 2017

[2] F P An ldquoPedestrian re-recognition algorithm based onoptimization deep learning-sequence memory modelrdquoComplexity vol 2019 Article ID 5069026 16 pages 2019

[3] J Cao Y Pang and X Li ldquoLearning multilayer channelfeatures for pedestrian detectionrdquo IEEE Transactions on ImageProcessing vol 26 no 7 pp 3210ndash3220 2017

[4] M Ji J Liu X Xu Y Guo and Z Lu ldquoImproved pedestrianpositioning with inertial sensor based on adaptive gradientdescent and double-constrained extended kalman filterrdquoComplexity vol 2020 Article ID 4361812 11 pages 2020

[5] C Li Z Su Q Li and H Zhao ldquoAn indoor positioning errorcorrection method of pedestrian multi-motions recognized byhybrid-orders fraction domain transformationrdquo IEEE Accessvol 7 pp 11360ndash11377 2019

[6] H Zhao W Cheng N Yang et al ldquoSmartphone-based 3Dindoor pedestrian positioning through multi-modal datafusionrdquo Sensors vol 19 no 20 Article ID s19204554 2019

[7] B Wang T Su X Jin J Kong and Y Bai ldquo3D reconstructionof pedestrian trajectory with moving direction learning andoptimal gait recognitionrdquo Complexity vol 2018 Article ID8735846 10 pages 2018

[8] Y Jiang Z Li and J B Wang ldquoPtrack enhancing the ap-plicability of pedestrian tracking with wearablesrdquo IEEETransactions on Mobile Computing vol 18 no 2 pp 431ndash4432018

[9] W Xu L Liu S Zlatanova W Penard and Q Xiong ldquoApedestrian tracking algorithm using grid-based indoormodelrdquo Automation in Construction vol 92 pp 173ndash1872018

[10] L Bozgeyikli A Raij S Katkoori and R Alqasemi ldquoA surveyon virtual reality for individuals with autism spectrum

disorder design considerationsrdquo IEEE Transactions onLearning Technologies vol 11 no 2 pp 133ndash151 2017

[11] J O Nilsson D Zachariah I Skog and P Handel ldquoCoop-erative localization by dual foot-mounted inertial sensors andinter-agent rangingrdquo EURASIP Journal on Advances in SignalProcessing vol 2013 no 1 p 164 2013

[12] I Skog J-O Nilsson D Zachariah and P Handel ldquoFusingthe information from two navigation systems using an upperbound on their maximum spatial separationrdquo in Proceedingsof the 2012 International Conference on Indoor Positioning andIndoor Navigation Article ID 6418862 Sydney AustraliaNovember 2012

[13] S-B Chen Y Xin and B Luo ldquoAction-based pedestrianidentification via hierarchical matching pursuit and orderpreserving sparse codingrdquo Cognitive Computation vol 8no 5 pp 797ndash805 2016

[14] B Shin C Kim J Kim et al ldquoMotion recognition based 3Dpedestrian navigation system using smartphonerdquo IEEE Sen-sors Journal vol 16 no 18 pp 6977ndash6989 2016

[15] M Romanovas V Goridko A Al-Jawad et al ldquoA study onindoor pedestrian localization algorithms with foot-mountedsensorsrdquo in Proceedings of the International Conference onIndoor Positioning and Indoor Navigation pp 1ndash10 SydneyAustralia November 2012

[16] A Azaman ldquoComparative study on gait kinematics betweenmicrosoft kinect and vicon across different anthropometricmeasurementsrdquo Journal of Tomography System and SensorApplication vol 2 no 2 pp 12ndash17 2019

[17] W Sheng A +obbi and Y Gu ldquoAn integrated frameworkfor human-robot collaborative manipulationrdquo IEEE Trans-actions on Cybernetics vol 45 no 10 pp 2030ndash2041 2014

[18] S Tsuji and T Kohama ldquoProximity skin sensor using time-of-flight sensor for human collaborative robotrdquo IEEE SensorsJournal vol 19 no 14 pp 5859ndash5864 2019

[19] C Oprea I Pirnog I Marcu and M Udrea ldquoRobust poseestimation using Time-of-Flight imagingrdquo in Proceedings ofthe IEEE International Semiconductor Conference pp 301ndash304 Sinaia Romania January 2019

[20] A Vysocky R Pastor and P Novak ldquoInteraction with col-laborative robot using 2D and TOF camerardquo in InternationalConference on Modelling and Simulation for AutonomousSystems pp 477ndash489 Springer Cham Switzerland 2018

[21] M Gao Y Du Y Yang and J Zhang ldquoAdaptive anchor boxmechanism to improve the accuracy in the object detectionsystemrdquo Multimedia Tools and Applications vol 78 no 19pp 27383ndash27402 2019

[22] A Anwer S S Azhar Ali A Khan and F MeriaudeauldquoUnderwater 3-d scene reconstruction using kinect v2 basedon physical models for refraction and time of flight correc-tionrdquo IEEE Access vol 5 pp 15960ndash15970 2017

[23] A R Garcıa L R Miller C F Andres and P J N LorenteldquoObstacle detection using a time of flight range camerardquo inProceedings of the 2018 IEEE International Conference onVehicular Electronics and Safety (ICVES) pp 1ndash6 MadridSpain September 2018

[24] N Zengeler T Kopinski and U Handmann ldquoHand gesturerecognition in automotive humanndashmachine interaction usingdepth camerasrdquo Sensors vol 19 no 1 Article ID s190100592019

[25] M A Garduntildeo-Ramon I R Terol-Villalobos R A Osornio-Rios and L A Morales-Hernandez ldquoA new method forinpainting of depthmaps from time-of-flight sensors based ona modified closing by reconstruction algorithmrdquo Journal of

14 Complexity

Visual Communication and Image Representation vol 47pp 36ndash47 2019

[26] L Wang Y Luo H Wang and M Fei ldquoMeasurement errorcorrection model of TOF depth camerardquo Chinese Journal ofSystem Simulation vol 29 no 10 pp 2323ndash2329 2017

[27] VICON ldquoOfficial website of oxford metrics companyrdquo 2020httpswwwviconcom

[28] L Zhang W Dong D Zhang and G Shi ldquoTwo-stage imagedenoising by principal component analysis with local pixelgroupingrdquo Pattern Recognition vol 43 no 4 pp 1531ndash15492010

[29] J Kennedy and R Eberhart ldquoParticle swarm optimizationrdquo inProceedings of ICNNrsquo95-International Conference on NeuralNetworks (ICW) vol 4 pp 1942ndash1948 Perth AustraliaNovember 1995

[30] M A M De Oca T Stutzle M Birattari and M DorigoldquoFrankensteinrsquos PSO a composite particle swarm optimiza-tion algorithmrdquo IEEE Transactions on Evolutionary Compu-tation vol 13 no 5 pp 1120ndash1132 2009

[31] Z Zhen S Pang F Wang et al ldquoPattern classification andPSO optimal weights based sky images cloud motion speedcalculation method for solar PV power forecastingrdquo IEEETransactions on Industry Applications vol 55 no 4pp 3331ndash3342 2019

[32] X Wang J-S Pan and S-C Chu ldquoA parallel multi-verseoptimizer for application in multilevel image segmentationrdquoIEEE Access vol 8 pp 32018ndash32030 2020

[33] Z A Bashir and M E El-Hawary ldquoApplying wavelets toshort-term load forecasting using PSO-based neural net-worksrdquo IEEE Transactions on Power Systems vol 24 no 1pp 20ndash27 2009

[34] L Liu Y Wang F Xie and J Gao ldquoLegendre cooperativePSO strategies for trajectory optimizationrdquo Complexityvol 2018 Article ID 5036791 13 pages 2018

[35] Y Shi and R C Eberhart ldquoA modified particle swarm op-timizerrdquo in Proceedings of the 1998 IEEE InternationalConference on Evolutionary Computation Proceedingspp 69ndash73 Anchorage AK USA May 1998

[36] Y Shi and R C Eberhart ldquoParameter selection in particleswarm optimizationrdquo in International Conference on Evolu-tionary Programming pp 591ndash600 Springer Berlin Ger-many 1998

[37] X Lv D Zhou Y Tang and L Ma ldquoAn improved test se-lection optimization model based on fault ambiguity groupisolation and chaotic discrete PSOrdquo Complexity vol 2018Article ID 3942723 10 pages 2018

[38] J Matas O Chum M Urban and T Pajdla ldquoRobust wide-baseline stereo from maximally stable extremal regionsrdquoImage and Vision Computing vol 22 no 10 pp 761ndash7672004

[39] H Shim and S Lee ldquoRecovering translucent objects using asingle time-of-flight depth camerardquo IEEE Transactions onCircuits and Systems for Video Technology vol 26 no 5pp 841ndash854 2015

[40] C K Chui and G ChenKalman Filtering pp 19ndash26 SpringerInternational Publishing Berlin Germany 2017

[41] L Cui X Wang Y Xu H Jiang and J Zhou ldquoA novelswitching unscented Kalman filter method for remaininguseful life prediction of rolling bearingrdquo Measurementvol 135 pp 678ndash684 2019

[42] GWelch and G BishopAn Introduction to the Kalman FilterMacmillan New York NY USA 1995

[43] K Zhang L Zhang H Song and D Zhang ldquoReinitialization-free level set evolution via reaction diffusionrdquo IEEE Trans-actions on Image Processing vol 22 no 1 pp 258ndash271 2012

Complexity 15

1700

1690

1680

1670

1660

1650

1640

1630

1620

1610

1600

1590

1580

1570

Mea

sure

d he

ight

(mm

)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30Number

Our algorithm without Kalman filtering16481629

Our algorithm with Kalman filtering16481629

VICON (ground truth)16481629

Figure 10 Experimental results of women with different heights in the six consecutive seconds

28272625242322211011121314151617181920 29308765432 91

Number

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

(a)

2 3 4 5 6 7 8 91 111213141516171819202122232425262728293010

Number

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

(b)

43 61 2 7 8 95 111213141516171819202122232425262728293010

Number

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

(c)

43 61 2 7 8 95 111213141516171819202122232425262728293010

Number

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

(d)

Figure 11 +e error plot of men in the six consecutive seconds (andashd) +e men with static heights of 1760 1676 1761 and 1728

12 Complexity

time Figures 9 and 10 show the experimental results of fourmen and two women in six consecutive seconds In thefigures the dotted line represents our algorithm withoutKalman filtering the solid line represents our algorithmwithout Kalman filtering and the dotted line with the markldquo+rdquo indicates the VICON+e waveforms show the real-timeheight value in 6 consecutive seconds the static heights ofmen are 1760mm 1676mm 1761mm and 1728mm asshown in the legend of Figure 9 while the static heights ofwomen are 1648mm and 1629mm as shown in Figure 10

It can be seen from the curves that the height datameasured by our algorithm is almost consistent with the dataobtained by VICON In order to analyse the error of ouralgorithm we sort out the errors of all the data in the sixconsecutive seconds the results are shown in Figures 11 and12 +e figures show that Kalman filtering can effectivelyimprove the accuracy of height measurement which indi-cates the pedestrian height at the preceding moment facil-itates the estimate of the pedestrian height in the lattermoment

Also the sums of errors per second of the algorithmswith and without Kalman filtering are given in Table 2where the subscript ldquolowastrdquo represents male and ldquordquo representsfemale Table 2 shows that our algorithm with Kalmanfiltering has a smaller cumulative error and can moreaccurately measure the real-time height of the movingpedestrians which proves the feasibility and validity of theproposed method

5 Conclusion and Future Work

In this paper a real-time height measurement based onthe TOF camera is proposed for moving pedestrians Toget the target region a new D-PSO denoising algorithmand a segmentation algorithm based on MSER are de-veloped in the paper In addition a novel multilayer it-erative average algorithm is designed for calculating thepedestrian height Also the Kalman filtering is used toimprove the measurement accuracy +e experimentalresults demonstrate the effectiveness and practicability of

2 3 4 5 6 7 8 9 1011121314151617181920212223242526272829301Number

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2Er

ror (

)

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

(a)

2 3 4 5 6 7 8 9 1011121314151617181920212223242526272829301Number

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

(b)

Figure 12 +e error plot of women in the six consecutive seconds (a) +e woman with static height of 1648 (b) +e woman with staticheight of 1629

Table 2 +e sum of errors per second of the algorithms with and without Kalman filtering

Heights (mm) Kalman filteringSum of errors per second ()

Sum1st second 2nd second 3rd second 4th second 5th second 6th second

1760lowast Yes 1202 0956 1836 1242 1611 1525 8372No 1868 1003 2013 1362 1898 1758 9902

1676lowast Yes 2002 1799 1977 0863 1648 2137 10426No 2249 1968 2087 1602 1827 3261 12994

1761lowast Yes 1282 1483 0963 1132 0632 1487 6979No 1562 1702 1333 1617 1234 1714 9162

1728lowast Yes 1629 1652 1354 1453 1224 0902 8214No 2201 2159 1912 1592 1984 1336 11184

1648 Yes 2006 1194 1818 1014 1585 1693 9310No 2488 1245 2152 1906 2078 2087 11956

1629 Yes 1509 1838 0652 2344 1398 1109 8850No 1632 2536 1328 2508 1497 1340 10841

lowastMale female

Complexity 13

the proposed method Our future work is going to furtherimprove the measurement accuracy and focus on trackingpedestrians in real time by using the real-time height ofmoving pedestrians

Data Availability

+e data used to support the findings of this study areavailable from the corresponding author upon request

Conflicts of Interest

+e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

+e authors are grateful to the financial support from theNatural Science Foundation of China (61877065) the NationalKey Research and Development Program of China(2019YFB1405500) the National Natural Science Foundationof Guangdong (2016A030313177) Guangdong Frontier andKey Technological Innovation (2017B090910013) and theScience and Technology Innovation Commission of Shenzhen(JCYJ20170818153048647 and JCYJ20180507182239617)

References

[1] J Li X Liang S Shen et al ldquoScale-aware fast R-CNN forpedestrian detectionrdquo IEEE Transactions on Multimediavol 20 no 4 pp 985ndash996 2017

[2] F P An ldquoPedestrian re-recognition algorithm based onoptimization deep learning-sequence memory modelrdquoComplexity vol 2019 Article ID 5069026 16 pages 2019

[3] J Cao Y Pang and X Li ldquoLearning multilayer channelfeatures for pedestrian detectionrdquo IEEE Transactions on ImageProcessing vol 26 no 7 pp 3210ndash3220 2017

[4] M Ji J Liu X Xu Y Guo and Z Lu ldquoImproved pedestrianpositioning with inertial sensor based on adaptive gradientdescent and double-constrained extended kalman filterrdquoComplexity vol 2020 Article ID 4361812 11 pages 2020

[5] C Li Z Su Q Li and H Zhao ldquoAn indoor positioning errorcorrection method of pedestrian multi-motions recognized byhybrid-orders fraction domain transformationrdquo IEEE Accessvol 7 pp 11360ndash11377 2019

[6] H Zhao W Cheng N Yang et al ldquoSmartphone-based 3Dindoor pedestrian positioning through multi-modal datafusionrdquo Sensors vol 19 no 20 Article ID s19204554 2019

[7] B Wang T Su X Jin J Kong and Y Bai ldquo3D reconstructionof pedestrian trajectory with moving direction learning andoptimal gait recognitionrdquo Complexity vol 2018 Article ID8735846 10 pages 2018

[8] Y Jiang Z Li and J B Wang ldquoPtrack enhancing the ap-plicability of pedestrian tracking with wearablesrdquo IEEETransactions on Mobile Computing vol 18 no 2 pp 431ndash4432018

[9] W Xu L Liu S Zlatanova W Penard and Q Xiong ldquoApedestrian tracking algorithm using grid-based indoormodelrdquo Automation in Construction vol 92 pp 173ndash1872018

[10] L Bozgeyikli A Raij S Katkoori and R Alqasemi ldquoA surveyon virtual reality for individuals with autism spectrum

disorder design considerationsrdquo IEEE Transactions onLearning Technologies vol 11 no 2 pp 133ndash151 2017

[11] J O Nilsson D Zachariah I Skog and P Handel ldquoCoop-erative localization by dual foot-mounted inertial sensors andinter-agent rangingrdquo EURASIP Journal on Advances in SignalProcessing vol 2013 no 1 p 164 2013

[12] I Skog J-O Nilsson D Zachariah and P Handel ldquoFusingthe information from two navigation systems using an upperbound on their maximum spatial separationrdquo in Proceedingsof the 2012 International Conference on Indoor Positioning andIndoor Navigation Article ID 6418862 Sydney AustraliaNovember 2012

[13] S-B Chen Y Xin and B Luo ldquoAction-based pedestrianidentification via hierarchical matching pursuit and orderpreserving sparse codingrdquo Cognitive Computation vol 8no 5 pp 797ndash805 2016

[14] B Shin C Kim J Kim et al ldquoMotion recognition based 3Dpedestrian navigation system using smartphonerdquo IEEE Sen-sors Journal vol 16 no 18 pp 6977ndash6989 2016

[15] M Romanovas V Goridko A Al-Jawad et al ldquoA study onindoor pedestrian localization algorithms with foot-mountedsensorsrdquo in Proceedings of the International Conference onIndoor Positioning and Indoor Navigation pp 1ndash10 SydneyAustralia November 2012

[16] A Azaman ldquoComparative study on gait kinematics betweenmicrosoft kinect and vicon across different anthropometricmeasurementsrdquo Journal of Tomography System and SensorApplication vol 2 no 2 pp 12ndash17 2019

[17] W Sheng A +obbi and Y Gu ldquoAn integrated frameworkfor human-robot collaborative manipulationrdquo IEEE Trans-actions on Cybernetics vol 45 no 10 pp 2030ndash2041 2014

[18] S Tsuji and T Kohama ldquoProximity skin sensor using time-of-flight sensor for human collaborative robotrdquo IEEE SensorsJournal vol 19 no 14 pp 5859ndash5864 2019

[19] C Oprea I Pirnog I Marcu and M Udrea ldquoRobust poseestimation using Time-of-Flight imagingrdquo in Proceedings ofthe IEEE International Semiconductor Conference pp 301ndash304 Sinaia Romania January 2019

[20] A Vysocky R Pastor and P Novak ldquoInteraction with col-laborative robot using 2D and TOF camerardquo in InternationalConference on Modelling and Simulation for AutonomousSystems pp 477ndash489 Springer Cham Switzerland 2018

[21] M Gao Y Du Y Yang and J Zhang ldquoAdaptive anchor boxmechanism to improve the accuracy in the object detectionsystemrdquo Multimedia Tools and Applications vol 78 no 19pp 27383ndash27402 2019

[22] A Anwer S S Azhar Ali A Khan and F MeriaudeauldquoUnderwater 3-d scene reconstruction using kinect v2 basedon physical models for refraction and time of flight correc-tionrdquo IEEE Access vol 5 pp 15960ndash15970 2017

[23] A R Garcıa L R Miller C F Andres and P J N LorenteldquoObstacle detection using a time of flight range camerardquo inProceedings of the 2018 IEEE International Conference onVehicular Electronics and Safety (ICVES) pp 1ndash6 MadridSpain September 2018

[24] N Zengeler T Kopinski and U Handmann ldquoHand gesturerecognition in automotive humanndashmachine interaction usingdepth camerasrdquo Sensors vol 19 no 1 Article ID s190100592019

[25] M A Garduntildeo-Ramon I R Terol-Villalobos R A Osornio-Rios and L A Morales-Hernandez ldquoA new method forinpainting of depthmaps from time-of-flight sensors based ona modified closing by reconstruction algorithmrdquo Journal of

14 Complexity

Visual Communication and Image Representation vol 47pp 36ndash47 2019

[26] L Wang Y Luo H Wang and M Fei ldquoMeasurement errorcorrection model of TOF depth camerardquo Chinese Journal ofSystem Simulation vol 29 no 10 pp 2323ndash2329 2017

[27] VICON ldquoOfficial website of oxford metrics companyrdquo 2020httpswwwviconcom

[28] L Zhang W Dong D Zhang and G Shi ldquoTwo-stage imagedenoising by principal component analysis with local pixelgroupingrdquo Pattern Recognition vol 43 no 4 pp 1531ndash15492010

[29] J Kennedy and R Eberhart ldquoParticle swarm optimizationrdquo inProceedings of ICNNrsquo95-International Conference on NeuralNetworks (ICW) vol 4 pp 1942ndash1948 Perth AustraliaNovember 1995

[30] M A M De Oca T Stutzle M Birattari and M DorigoldquoFrankensteinrsquos PSO a composite particle swarm optimiza-tion algorithmrdquo IEEE Transactions on Evolutionary Compu-tation vol 13 no 5 pp 1120ndash1132 2009

[31] Z Zhen S Pang F Wang et al ldquoPattern classification andPSO optimal weights based sky images cloud motion speedcalculation method for solar PV power forecastingrdquo IEEETransactions on Industry Applications vol 55 no 4pp 3331ndash3342 2019

[32] X Wang J-S Pan and S-C Chu ldquoA parallel multi-verseoptimizer for application in multilevel image segmentationrdquoIEEE Access vol 8 pp 32018ndash32030 2020

[33] Z A Bashir and M E El-Hawary ldquoApplying wavelets toshort-term load forecasting using PSO-based neural net-worksrdquo IEEE Transactions on Power Systems vol 24 no 1pp 20ndash27 2009

[34] L Liu Y Wang F Xie and J Gao ldquoLegendre cooperativePSO strategies for trajectory optimizationrdquo Complexityvol 2018 Article ID 5036791 13 pages 2018

[35] Y Shi and R C Eberhart ldquoA modified particle swarm op-timizerrdquo in Proceedings of the 1998 IEEE InternationalConference on Evolutionary Computation Proceedingspp 69ndash73 Anchorage AK USA May 1998

[36] Y Shi and R C Eberhart ldquoParameter selection in particleswarm optimizationrdquo in International Conference on Evolu-tionary Programming pp 591ndash600 Springer Berlin Ger-many 1998

[37] X Lv D Zhou Y Tang and L Ma ldquoAn improved test se-lection optimization model based on fault ambiguity groupisolation and chaotic discrete PSOrdquo Complexity vol 2018Article ID 3942723 10 pages 2018

[38] J Matas O Chum M Urban and T Pajdla ldquoRobust wide-baseline stereo from maximally stable extremal regionsrdquoImage and Vision Computing vol 22 no 10 pp 761ndash7672004

[39] H Shim and S Lee ldquoRecovering translucent objects using asingle time-of-flight depth camerardquo IEEE Transactions onCircuits and Systems for Video Technology vol 26 no 5pp 841ndash854 2015

[40] C K Chui and G ChenKalman Filtering pp 19ndash26 SpringerInternational Publishing Berlin Germany 2017

[41] L Cui X Wang Y Xu H Jiang and J Zhou ldquoA novelswitching unscented Kalman filter method for remaininguseful life prediction of rolling bearingrdquo Measurementvol 135 pp 678ndash684 2019

[42] GWelch and G BishopAn Introduction to the Kalman FilterMacmillan New York NY USA 1995

[43] K Zhang L Zhang H Song and D Zhang ldquoReinitialization-free level set evolution via reaction diffusionrdquo IEEE Trans-actions on Image Processing vol 22 no 1 pp 258ndash271 2012

Complexity 15

time Figures 9 and 10 show the experimental results of fourmen and two women in six consecutive seconds In thefigures the dotted line represents our algorithm withoutKalman filtering the solid line represents our algorithmwithout Kalman filtering and the dotted line with the markldquo+rdquo indicates the VICON+e waveforms show the real-timeheight value in 6 consecutive seconds the static heights ofmen are 1760mm 1676mm 1761mm and 1728mm asshown in the legend of Figure 9 while the static heights ofwomen are 1648mm and 1629mm as shown in Figure 10

It can be seen from the curves that the height datameasured by our algorithm is almost consistent with the dataobtained by VICON In order to analyse the error of ouralgorithm we sort out the errors of all the data in the sixconsecutive seconds the results are shown in Figures 11 and12 +e figures show that Kalman filtering can effectivelyimprove the accuracy of height measurement which indi-cates the pedestrian height at the preceding moment facil-itates the estimate of the pedestrian height in the lattermoment

Also the sums of errors per second of the algorithmswith and without Kalman filtering are given in Table 2where the subscript ldquolowastrdquo represents male and ldquordquo representsfemale Table 2 shows that our algorithm with Kalmanfiltering has a smaller cumulative error and can moreaccurately measure the real-time height of the movingpedestrians which proves the feasibility and validity of theproposed method

5 Conclusion and Future Work

In this paper a real-time height measurement based onthe TOF camera is proposed for moving pedestrians Toget the target region a new D-PSO denoising algorithmand a segmentation algorithm based on MSER are de-veloped in the paper In addition a novel multilayer it-erative average algorithm is designed for calculating thepedestrian height Also the Kalman filtering is used toimprove the measurement accuracy +e experimentalresults demonstrate the effectiveness and practicability of

2 3 4 5 6 7 8 9 1011121314151617181920212223242526272829301Number

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2Er

ror (

)

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

(a)

2 3 4 5 6 7 8 9 1011121314151617181920212223242526272829301Number

ndash2

ndash15

ndash1

ndash05

0

05

1

15

2

Erro

r (

)

Our algorithm without Kalman filteringOur algorithm with Kalman filtering

(b)

Figure 12 +e error plot of women in the six consecutive seconds (a) +e woman with static height of 1648 (b) +e woman with staticheight of 1629

Table 2 +e sum of errors per second of the algorithms with and without Kalman filtering

Heights (mm) Kalman filteringSum of errors per second ()

Sum1st second 2nd second 3rd second 4th second 5th second 6th second

1760lowast Yes 1202 0956 1836 1242 1611 1525 8372No 1868 1003 2013 1362 1898 1758 9902

1676lowast Yes 2002 1799 1977 0863 1648 2137 10426No 2249 1968 2087 1602 1827 3261 12994

1761lowast Yes 1282 1483 0963 1132 0632 1487 6979No 1562 1702 1333 1617 1234 1714 9162

1728lowast Yes 1629 1652 1354 1453 1224 0902 8214No 2201 2159 1912 1592 1984 1336 11184

1648 Yes 2006 1194 1818 1014 1585 1693 9310No 2488 1245 2152 1906 2078 2087 11956

1629 Yes 1509 1838 0652 2344 1398 1109 8850No 1632 2536 1328 2508 1497 1340 10841

lowastMale female

Complexity 13

the proposed method Our future work is going to furtherimprove the measurement accuracy and focus on trackingpedestrians in real time by using the real-time height ofmoving pedestrians

Data Availability

+e data used to support the findings of this study areavailable from the corresponding author upon request

Conflicts of Interest

+e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

+e authors are grateful to the financial support from theNatural Science Foundation of China (61877065) the NationalKey Research and Development Program of China(2019YFB1405500) the National Natural Science Foundationof Guangdong (2016A030313177) Guangdong Frontier andKey Technological Innovation (2017B090910013) and theScience and Technology Innovation Commission of Shenzhen(JCYJ20170818153048647 and JCYJ20180507182239617)

References

[1] J Li X Liang S Shen et al ldquoScale-aware fast R-CNN forpedestrian detectionrdquo IEEE Transactions on Multimediavol 20 no 4 pp 985ndash996 2017

[2] F P An ldquoPedestrian re-recognition algorithm based onoptimization deep learning-sequence memory modelrdquoComplexity vol 2019 Article ID 5069026 16 pages 2019

[3] J Cao Y Pang and X Li ldquoLearning multilayer channelfeatures for pedestrian detectionrdquo IEEE Transactions on ImageProcessing vol 26 no 7 pp 3210ndash3220 2017

[4] M Ji J Liu X Xu Y Guo and Z Lu ldquoImproved pedestrianpositioning with inertial sensor based on adaptive gradientdescent and double-constrained extended kalman filterrdquoComplexity vol 2020 Article ID 4361812 11 pages 2020

[5] C Li Z Su Q Li and H Zhao ldquoAn indoor positioning errorcorrection method of pedestrian multi-motions recognized byhybrid-orders fraction domain transformationrdquo IEEE Accessvol 7 pp 11360ndash11377 2019

[6] H Zhao W Cheng N Yang et al ldquoSmartphone-based 3Dindoor pedestrian positioning through multi-modal datafusionrdquo Sensors vol 19 no 20 Article ID s19204554 2019

[7] B Wang T Su X Jin J Kong and Y Bai ldquo3D reconstructionof pedestrian trajectory with moving direction learning andoptimal gait recognitionrdquo Complexity vol 2018 Article ID8735846 10 pages 2018

[8] Y Jiang Z Li and J B Wang ldquoPtrack enhancing the ap-plicability of pedestrian tracking with wearablesrdquo IEEETransactions on Mobile Computing vol 18 no 2 pp 431ndash4432018

[9] W Xu L Liu S Zlatanova W Penard and Q Xiong ldquoApedestrian tracking algorithm using grid-based indoormodelrdquo Automation in Construction vol 92 pp 173ndash1872018

[10] L Bozgeyikli A Raij S Katkoori and R Alqasemi ldquoA surveyon virtual reality for individuals with autism spectrum

disorder design considerationsrdquo IEEE Transactions onLearning Technologies vol 11 no 2 pp 133ndash151 2017

[11] J O Nilsson D Zachariah I Skog and P Handel ldquoCoop-erative localization by dual foot-mounted inertial sensors andinter-agent rangingrdquo EURASIP Journal on Advances in SignalProcessing vol 2013 no 1 p 164 2013

[12] I Skog J-O Nilsson D Zachariah and P Handel ldquoFusingthe information from two navigation systems using an upperbound on their maximum spatial separationrdquo in Proceedingsof the 2012 International Conference on Indoor Positioning andIndoor Navigation Article ID 6418862 Sydney AustraliaNovember 2012

[13] S-B Chen Y Xin and B Luo ldquoAction-based pedestrianidentification via hierarchical matching pursuit and orderpreserving sparse codingrdquo Cognitive Computation vol 8no 5 pp 797ndash805 2016

[14] B Shin C Kim J Kim et al ldquoMotion recognition based 3Dpedestrian navigation system using smartphonerdquo IEEE Sen-sors Journal vol 16 no 18 pp 6977ndash6989 2016

[15] M Romanovas V Goridko A Al-Jawad et al ldquoA study onindoor pedestrian localization algorithms with foot-mountedsensorsrdquo in Proceedings of the International Conference onIndoor Positioning and Indoor Navigation pp 1ndash10 SydneyAustralia November 2012

[16] A Azaman ldquoComparative study on gait kinematics betweenmicrosoft kinect and vicon across different anthropometricmeasurementsrdquo Journal of Tomography System and SensorApplication vol 2 no 2 pp 12ndash17 2019

[17] W Sheng A +obbi and Y Gu ldquoAn integrated frameworkfor human-robot collaborative manipulationrdquo IEEE Trans-actions on Cybernetics vol 45 no 10 pp 2030ndash2041 2014

[18] S Tsuji and T Kohama ldquoProximity skin sensor using time-of-flight sensor for human collaborative robotrdquo IEEE SensorsJournal vol 19 no 14 pp 5859ndash5864 2019

[19] C Oprea I Pirnog I Marcu and M Udrea ldquoRobust poseestimation using Time-of-Flight imagingrdquo in Proceedings ofthe IEEE International Semiconductor Conference pp 301ndash304 Sinaia Romania January 2019

[20] A Vysocky R Pastor and P Novak ldquoInteraction with col-laborative robot using 2D and TOF camerardquo in InternationalConference on Modelling and Simulation for AutonomousSystems pp 477ndash489 Springer Cham Switzerland 2018

[21] M Gao Y Du Y Yang and J Zhang ldquoAdaptive anchor boxmechanism to improve the accuracy in the object detectionsystemrdquo Multimedia Tools and Applications vol 78 no 19pp 27383ndash27402 2019

[22] A Anwer S S Azhar Ali A Khan and F MeriaudeauldquoUnderwater 3-d scene reconstruction using kinect v2 basedon physical models for refraction and time of flight correc-tionrdquo IEEE Access vol 5 pp 15960ndash15970 2017

[23] A R Garcıa L R Miller C F Andres and P J N LorenteldquoObstacle detection using a time of flight range camerardquo inProceedings of the 2018 IEEE International Conference onVehicular Electronics and Safety (ICVES) pp 1ndash6 MadridSpain September 2018

[24] N Zengeler T Kopinski and U Handmann ldquoHand gesturerecognition in automotive humanndashmachine interaction usingdepth camerasrdquo Sensors vol 19 no 1 Article ID s190100592019

[25] M A Garduntildeo-Ramon I R Terol-Villalobos R A Osornio-Rios and L A Morales-Hernandez ldquoA new method forinpainting of depthmaps from time-of-flight sensors based ona modified closing by reconstruction algorithmrdquo Journal of

14 Complexity

Visual Communication and Image Representation vol 47pp 36ndash47 2019

[26] L Wang Y Luo H Wang and M Fei ldquoMeasurement errorcorrection model of TOF depth camerardquo Chinese Journal ofSystem Simulation vol 29 no 10 pp 2323ndash2329 2017

[27] VICON ldquoOfficial website of oxford metrics companyrdquo 2020httpswwwviconcom

[28] L Zhang W Dong D Zhang and G Shi ldquoTwo-stage imagedenoising by principal component analysis with local pixelgroupingrdquo Pattern Recognition vol 43 no 4 pp 1531ndash15492010

[29] J Kennedy and R Eberhart ldquoParticle swarm optimizationrdquo inProceedings of ICNNrsquo95-International Conference on NeuralNetworks (ICW) vol 4 pp 1942ndash1948 Perth AustraliaNovember 1995

[30] M A M De Oca T Stutzle M Birattari and M DorigoldquoFrankensteinrsquos PSO a composite particle swarm optimiza-tion algorithmrdquo IEEE Transactions on Evolutionary Compu-tation vol 13 no 5 pp 1120ndash1132 2009

[31] Z Zhen S Pang F Wang et al ldquoPattern classification andPSO optimal weights based sky images cloud motion speedcalculation method for solar PV power forecastingrdquo IEEETransactions on Industry Applications vol 55 no 4pp 3331ndash3342 2019

[32] X Wang J-S Pan and S-C Chu ldquoA parallel multi-verseoptimizer for application in multilevel image segmentationrdquoIEEE Access vol 8 pp 32018ndash32030 2020

[33] Z A Bashir and M E El-Hawary ldquoApplying wavelets toshort-term load forecasting using PSO-based neural net-worksrdquo IEEE Transactions on Power Systems vol 24 no 1pp 20ndash27 2009

[34] L Liu Y Wang F Xie and J Gao ldquoLegendre cooperativePSO strategies for trajectory optimizationrdquo Complexityvol 2018 Article ID 5036791 13 pages 2018

[35] Y Shi and R C Eberhart ldquoA modified particle swarm op-timizerrdquo in Proceedings of the 1998 IEEE InternationalConference on Evolutionary Computation Proceedingspp 69ndash73 Anchorage AK USA May 1998

[36] Y Shi and R C Eberhart ldquoParameter selection in particleswarm optimizationrdquo in International Conference on Evolu-tionary Programming pp 591ndash600 Springer Berlin Ger-many 1998

[37] X Lv D Zhou Y Tang and L Ma ldquoAn improved test se-lection optimization model based on fault ambiguity groupisolation and chaotic discrete PSOrdquo Complexity vol 2018Article ID 3942723 10 pages 2018

[38] J Matas O Chum M Urban and T Pajdla ldquoRobust wide-baseline stereo from maximally stable extremal regionsrdquoImage and Vision Computing vol 22 no 10 pp 761ndash7672004

[39] H Shim and S Lee ldquoRecovering translucent objects using asingle time-of-flight depth camerardquo IEEE Transactions onCircuits and Systems for Video Technology vol 26 no 5pp 841ndash854 2015

[40] C K Chui and G ChenKalman Filtering pp 19ndash26 SpringerInternational Publishing Berlin Germany 2017

[41] L Cui X Wang Y Xu H Jiang and J Zhou ldquoA novelswitching unscented Kalman filter method for remaininguseful life prediction of rolling bearingrdquo Measurementvol 135 pp 678ndash684 2019

[42] GWelch and G BishopAn Introduction to the Kalman FilterMacmillan New York NY USA 1995

[43] K Zhang L Zhang H Song and D Zhang ldquoReinitialization-free level set evolution via reaction diffusionrdquo IEEE Trans-actions on Image Processing vol 22 no 1 pp 258ndash271 2012

Complexity 15

the proposed method Our future work is going to furtherimprove the measurement accuracy and focus on trackingpedestrians in real time by using the real-time height ofmoving pedestrians

Data Availability

+e data used to support the findings of this study areavailable from the corresponding author upon request

Conflicts of Interest

+e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

+e authors are grateful to the financial support from theNatural Science Foundation of China (61877065) the NationalKey Research and Development Program of China(2019YFB1405500) the National Natural Science Foundationof Guangdong (2016A030313177) Guangdong Frontier andKey Technological Innovation (2017B090910013) and theScience and Technology Innovation Commission of Shenzhen(JCYJ20170818153048647 and JCYJ20180507182239617)

References

[1] J Li X Liang S Shen et al ldquoScale-aware fast R-CNN forpedestrian detectionrdquo IEEE Transactions on Multimediavol 20 no 4 pp 985ndash996 2017

[2] F P An ldquoPedestrian re-recognition algorithm based onoptimization deep learning-sequence memory modelrdquoComplexity vol 2019 Article ID 5069026 16 pages 2019

[3] J Cao Y Pang and X Li ldquoLearning multilayer channelfeatures for pedestrian detectionrdquo IEEE Transactions on ImageProcessing vol 26 no 7 pp 3210ndash3220 2017

[4] M Ji J Liu X Xu Y Guo and Z Lu ldquoImproved pedestrianpositioning with inertial sensor based on adaptive gradientdescent and double-constrained extended kalman filterrdquoComplexity vol 2020 Article ID 4361812 11 pages 2020

[5] C Li Z Su Q Li and H Zhao ldquoAn indoor positioning errorcorrection method of pedestrian multi-motions recognized byhybrid-orders fraction domain transformationrdquo IEEE Accessvol 7 pp 11360ndash11377 2019

[6] H Zhao W Cheng N Yang et al ldquoSmartphone-based 3Dindoor pedestrian positioning through multi-modal datafusionrdquo Sensors vol 19 no 20 Article ID s19204554 2019

[7] B Wang T Su X Jin J Kong and Y Bai ldquo3D reconstructionof pedestrian trajectory with moving direction learning andoptimal gait recognitionrdquo Complexity vol 2018 Article ID8735846 10 pages 2018

[8] Y Jiang Z Li and J B Wang ldquoPtrack enhancing the ap-plicability of pedestrian tracking with wearablesrdquo IEEETransactions on Mobile Computing vol 18 no 2 pp 431ndash4432018

[9] W Xu L Liu S Zlatanova W Penard and Q Xiong ldquoApedestrian tracking algorithm using grid-based indoormodelrdquo Automation in Construction vol 92 pp 173ndash1872018

[10] L Bozgeyikli A Raij S Katkoori and R Alqasemi ldquoA surveyon virtual reality for individuals with autism spectrum

disorder design considerationsrdquo IEEE Transactions onLearning Technologies vol 11 no 2 pp 133ndash151 2017

[11] J O Nilsson D Zachariah I Skog and P Handel ldquoCoop-erative localization by dual foot-mounted inertial sensors andinter-agent rangingrdquo EURASIP Journal on Advances in SignalProcessing vol 2013 no 1 p 164 2013

[12] I Skog J-O Nilsson D Zachariah and P Handel ldquoFusingthe information from two navigation systems using an upperbound on their maximum spatial separationrdquo in Proceedingsof the 2012 International Conference on Indoor Positioning andIndoor Navigation Article ID 6418862 Sydney AustraliaNovember 2012

[13] S-B Chen Y Xin and B Luo ldquoAction-based pedestrianidentification via hierarchical matching pursuit and orderpreserving sparse codingrdquo Cognitive Computation vol 8no 5 pp 797ndash805 2016

[14] B Shin C Kim J Kim et al ldquoMotion recognition based 3Dpedestrian navigation system using smartphonerdquo IEEE Sen-sors Journal vol 16 no 18 pp 6977ndash6989 2016

[15] M Romanovas V Goridko A Al-Jawad et al ldquoA study onindoor pedestrian localization algorithms with foot-mountedsensorsrdquo in Proceedings of the International Conference onIndoor Positioning and Indoor Navigation pp 1ndash10 SydneyAustralia November 2012

[16] A Azaman ldquoComparative study on gait kinematics betweenmicrosoft kinect and vicon across different anthropometricmeasurementsrdquo Journal of Tomography System and SensorApplication vol 2 no 2 pp 12ndash17 2019

[17] W Sheng A +obbi and Y Gu ldquoAn integrated frameworkfor human-robot collaborative manipulationrdquo IEEE Trans-actions on Cybernetics vol 45 no 10 pp 2030ndash2041 2014

[18] S Tsuji and T Kohama ldquoProximity skin sensor using time-of-flight sensor for human collaborative robotrdquo IEEE SensorsJournal vol 19 no 14 pp 5859ndash5864 2019

[19] C Oprea I Pirnog I Marcu and M Udrea ldquoRobust poseestimation using Time-of-Flight imagingrdquo in Proceedings ofthe IEEE International Semiconductor Conference pp 301ndash304 Sinaia Romania January 2019

[20] A Vysocky R Pastor and P Novak ldquoInteraction with col-laborative robot using 2D and TOF camerardquo in InternationalConference on Modelling and Simulation for AutonomousSystems pp 477ndash489 Springer Cham Switzerland 2018

[21] M Gao Y Du Y Yang and J Zhang ldquoAdaptive anchor boxmechanism to improve the accuracy in the object detectionsystemrdquo Multimedia Tools and Applications vol 78 no 19pp 27383ndash27402 2019

[22] A Anwer S S Azhar Ali A Khan and F MeriaudeauldquoUnderwater 3-d scene reconstruction using kinect v2 basedon physical models for refraction and time of flight correc-tionrdquo IEEE Access vol 5 pp 15960ndash15970 2017

[23] A R Garcıa L R Miller C F Andres and P J N LorenteldquoObstacle detection using a time of flight range camerardquo inProceedings of the 2018 IEEE International Conference onVehicular Electronics and Safety (ICVES) pp 1ndash6 MadridSpain September 2018

[24] N Zengeler T Kopinski and U Handmann ldquoHand gesturerecognition in automotive humanndashmachine interaction usingdepth camerasrdquo Sensors vol 19 no 1 Article ID s190100592019

[25] M A Garduntildeo-Ramon I R Terol-Villalobos R A Osornio-Rios and L A Morales-Hernandez ldquoA new method forinpainting of depthmaps from time-of-flight sensors based ona modified closing by reconstruction algorithmrdquo Journal of

14 Complexity

Visual Communication and Image Representation vol 47pp 36ndash47 2019

[26] L Wang Y Luo H Wang and M Fei ldquoMeasurement errorcorrection model of TOF depth camerardquo Chinese Journal ofSystem Simulation vol 29 no 10 pp 2323ndash2329 2017

[27] VICON ldquoOfficial website of oxford metrics companyrdquo 2020httpswwwviconcom

[28] L Zhang W Dong D Zhang and G Shi ldquoTwo-stage imagedenoising by principal component analysis with local pixelgroupingrdquo Pattern Recognition vol 43 no 4 pp 1531ndash15492010

[29] J Kennedy and R Eberhart ldquoParticle swarm optimizationrdquo inProceedings of ICNNrsquo95-International Conference on NeuralNetworks (ICW) vol 4 pp 1942ndash1948 Perth AustraliaNovember 1995

[30] M A M De Oca T Stutzle M Birattari and M DorigoldquoFrankensteinrsquos PSO a composite particle swarm optimiza-tion algorithmrdquo IEEE Transactions on Evolutionary Compu-tation vol 13 no 5 pp 1120ndash1132 2009

[31] Z Zhen S Pang F Wang et al ldquoPattern classification andPSO optimal weights based sky images cloud motion speedcalculation method for solar PV power forecastingrdquo IEEETransactions on Industry Applications vol 55 no 4pp 3331ndash3342 2019

[32] X Wang J-S Pan and S-C Chu ldquoA parallel multi-verseoptimizer for application in multilevel image segmentationrdquoIEEE Access vol 8 pp 32018ndash32030 2020

[33] Z A Bashir and M E El-Hawary ldquoApplying wavelets toshort-term load forecasting using PSO-based neural net-worksrdquo IEEE Transactions on Power Systems vol 24 no 1pp 20ndash27 2009

[34] L Liu Y Wang F Xie and J Gao ldquoLegendre cooperativePSO strategies for trajectory optimizationrdquo Complexityvol 2018 Article ID 5036791 13 pages 2018

[35] Y Shi and R C Eberhart ldquoA modified particle swarm op-timizerrdquo in Proceedings of the 1998 IEEE InternationalConference on Evolutionary Computation Proceedingspp 69ndash73 Anchorage AK USA May 1998

[36] Y Shi and R C Eberhart ldquoParameter selection in particleswarm optimizationrdquo in International Conference on Evolu-tionary Programming pp 591ndash600 Springer Berlin Ger-many 1998

[37] X Lv D Zhou Y Tang and L Ma ldquoAn improved test se-lection optimization model based on fault ambiguity groupisolation and chaotic discrete PSOrdquo Complexity vol 2018Article ID 3942723 10 pages 2018

[38] J Matas O Chum M Urban and T Pajdla ldquoRobust wide-baseline stereo from maximally stable extremal regionsrdquoImage and Vision Computing vol 22 no 10 pp 761ndash7672004

[39] H Shim and S Lee ldquoRecovering translucent objects using asingle time-of-flight depth camerardquo IEEE Transactions onCircuits and Systems for Video Technology vol 26 no 5pp 841ndash854 2015

[40] C K Chui and G ChenKalman Filtering pp 19ndash26 SpringerInternational Publishing Berlin Germany 2017

[41] L Cui X Wang Y Xu H Jiang and J Zhou ldquoA novelswitching unscented Kalman filter method for remaininguseful life prediction of rolling bearingrdquo Measurementvol 135 pp 678ndash684 2019

[42] GWelch and G BishopAn Introduction to the Kalman FilterMacmillan New York NY USA 1995

[43] K Zhang L Zhang H Song and D Zhang ldquoReinitialization-free level set evolution via reaction diffusionrdquo IEEE Trans-actions on Image Processing vol 22 no 1 pp 258ndash271 2012

Complexity 15

Visual Communication and Image Representation vol 47pp 36ndash47 2019

[26] L Wang Y Luo H Wang and M Fei ldquoMeasurement errorcorrection model of TOF depth camerardquo Chinese Journal ofSystem Simulation vol 29 no 10 pp 2323ndash2329 2017

[27] VICON ldquoOfficial website of oxford metrics companyrdquo 2020httpswwwviconcom

[28] L Zhang W Dong D Zhang and G Shi ldquoTwo-stage imagedenoising by principal component analysis with local pixelgroupingrdquo Pattern Recognition vol 43 no 4 pp 1531ndash15492010

[29] J Kennedy and R Eberhart ldquoParticle swarm optimizationrdquo inProceedings of ICNNrsquo95-International Conference on NeuralNetworks (ICW) vol 4 pp 1942ndash1948 Perth AustraliaNovember 1995

[30] M A M De Oca T Stutzle M Birattari and M DorigoldquoFrankensteinrsquos PSO a composite particle swarm optimiza-tion algorithmrdquo IEEE Transactions on Evolutionary Compu-tation vol 13 no 5 pp 1120ndash1132 2009

[31] Z Zhen S Pang F Wang et al ldquoPattern classification andPSO optimal weights based sky images cloud motion speedcalculation method for solar PV power forecastingrdquo IEEETransactions on Industry Applications vol 55 no 4pp 3331ndash3342 2019

[32] X Wang J-S Pan and S-C Chu ldquoA parallel multi-verseoptimizer for application in multilevel image segmentationrdquoIEEE Access vol 8 pp 32018ndash32030 2020

[33] Z A Bashir and M E El-Hawary ldquoApplying wavelets toshort-term load forecasting using PSO-based neural net-worksrdquo IEEE Transactions on Power Systems vol 24 no 1pp 20ndash27 2009

[34] L Liu Y Wang F Xie and J Gao ldquoLegendre cooperativePSO strategies for trajectory optimizationrdquo Complexityvol 2018 Article ID 5036791 13 pages 2018

[35] Y Shi and R C Eberhart ldquoA modified particle swarm op-timizerrdquo in Proceedings of the 1998 IEEE InternationalConference on Evolutionary Computation Proceedingspp 69ndash73 Anchorage AK USA May 1998

[36] Y Shi and R C Eberhart ldquoParameter selection in particleswarm optimizationrdquo in International Conference on Evolu-tionary Programming pp 591ndash600 Springer Berlin Ger-many 1998

[37] X Lv D Zhou Y Tang and L Ma ldquoAn improved test se-lection optimization model based on fault ambiguity groupisolation and chaotic discrete PSOrdquo Complexity vol 2018Article ID 3942723 10 pages 2018

[38] J Matas O Chum M Urban and T Pajdla ldquoRobust wide-baseline stereo from maximally stable extremal regionsrdquoImage and Vision Computing vol 22 no 10 pp 761ndash7672004

[39] H Shim and S Lee ldquoRecovering translucent objects using asingle time-of-flight depth camerardquo IEEE Transactions onCircuits and Systems for Video Technology vol 26 no 5pp 841ndash854 2015

[40] C K Chui and G ChenKalman Filtering pp 19ndash26 SpringerInternational Publishing Berlin Germany 2017

[41] L Cui X Wang Y Xu H Jiang and J Zhou ldquoA novelswitching unscented Kalman filter method for remaininguseful life prediction of rolling bearingrdquo Measurementvol 135 pp 678ndash684 2019

[42] GWelch and G BishopAn Introduction to the Kalman FilterMacmillan New York NY USA 1995

[43] K Zhang L Zhang H Song and D Zhang ldquoReinitialization-free level set evolution via reaction diffusionrdquo IEEE Trans-actions on Image Processing vol 22 no 1 pp 258ndash271 2012

Complexity 15