Mechanics of Electronic Textiles”Deformation analysis using image processing”

M.J.M. FeronReport number MT08.20

Supervisors:Dr. P. BoutenDr. Ir. R.H.J. PeerlingsProf. Dr. Ir. M.G.D. Geers

Philips Research LaboratoriesPhotonic Materials and Devices

University of Technology, EindhovenDepartment of Mechanical EngineeringMechanics of Materials

May, 2008

ii

Abstract

In the present study mechanics of textiles are under consideration. Studying the effect ofcomponents on textile substrates is the aim of this study. Numerical models are availableto approximate textile behavior and require input parameters. These parameters are to beobtained by experimental techniques. A global experimental technique such as a tensile testerwith its global time, displacement, and force information is not sufficient for determining theseparameters.

A technique has to be devised, which can capture local effects during deformation. Therefore,an experimental set-up comprising a tensile tester and an image capturing device is created,where the image arrays are used to obtain local information. In order to obtain that infor-mation, a processing algorithm is needed, which can convert visual data into displacementinformation. Two methods are devised and described; Pattern Matching and Pattern Recog-nition.

Pattern Matching is a technique that uses a correlation algorithm to create an automatictemplate mesh of an image. In subsequent images of an image array these templates aretracked, where the centers of gravity of these templates provide displacement information, .e.a displacement field. Pattern Recognition tracks binary ‘blobs’ in a certain region of interest.In the case of Pattern Recognition several regions of interest are manually selected to createa displacement field. Guidance in using the devised tools is provided and discussions on useand results are presented.

iii

iv

Contents

Abstract iii

Contents vi

1 Introduction 11.1 Problem description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.2 Research goal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.3 Solution approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2 Image Principles and Definitions 42.1 Digital Imaging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2.1.1 Resolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.1.2 Image Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.1.3 Number of Planes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.1.4 Image array . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.2 System setup and considerations . . . . . . . . . . . . . . . . . . . . . . . . . 52.3 Image analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

3 Pattern Matching 93.1 Normalized Cross-Correlation . . . . . . . . . . . . . . . . . . . . . . . . . . . 93.2 Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

3.2.1 Block size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123.2.2 Initial Minimum Match Score . . . . . . . . . . . . . . . . . . . . . . . 133.2.3 Track Minimum Match Score . . . . . . . . . . . . . . . . . . . . . . . 133.2.4 Relearn Score . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

3.3 Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

4 Pattern Recognition 174.1 Binary morphology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

4.1.1 Structuring element . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194.1.2 Morphological functions . . . . . . . . . . . . . . . . . . . . . . . . . . 194.1.3 Connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204.1.4 Particle motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

4.2 User Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224.3 Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

v

5 Discussion 265.1 Pattern Matching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

5.1.1 Strengths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275.1.2 Weaknesses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275.1.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

5.2 Pattern Recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285.2.1 Sensitivity and accuracy . . . . . . . . . . . . . . . . . . . . . . . . . . 285.2.2 Out-of-plane deformation . . . . . . . . . . . . . . . . . . . . . . . . . 30

6 Conclusions 32

7 Recommendations 33

References 34

vi

Chapter 1

Introduction

In studying the deformation characteristics of materials numerous experimental techniquesare available, e.g. tensile testing, indentation, bending, etc. Because of the availability of atensile tester and the fact that tensile testing presents a straightforward way of experimentallyobtaining material characteristics, an Instron 5566 universal testing machine is used. The set-up as used is depicted in Fig. 1.1. The typical output of a tensile test consists of force and

Figure 1.1: Experimental set-up, comprising an Instron 5566 universal testing machine, a test sample,a camera for digital image analysis, and a computer for tensile tester software and image capturingsoftware.

displacement as a function of time, based on clamp movement and force measurement at theclamp. The constitutive response of the material obtained does not contain any informationon local effects in the sample during loading, while measurements can be influenced by theselocal effects. Several techniques and algorithms exist that can be used in studying the effect

1

of local phenomena.

1.1 Problem description

Within the field of electronic textiles, it is common to have functional islands on top of thetextile substrates. These islands can assume different functions, e.g. light emission or dif-fusion, energy supply, etc. Typically, these islands are less compliant than the substrate onwhich they are placed. In order to experimentally study the effects of these islands on thesubstrates, a technique that can provide local displacement information is a sine qua non.In determining which algorithms and methods to apply, the first step is to determine the

Figure 1.2: Typical image as obtained during tensile testing using a uniform backlight.

structure and/or texture of the material under consideration. Correlation techniques are gen-erally used for random and irregular structures or textures, whereas Pattern Recognition hasa distinct advantage in the presence of regular or specific features. In Fig. 1.2 a typical imageof a woven electronic structure is depicted, where the regular trademarks of the substrate canbe distinguished.

1.2 Research goal

Quantifying the characteristics of deformation in electronic textiles under tensile loading, inthis case uni-axial, implies the necessity to capture local deformations within a test specimen.The goal of this study is to devise a method to process image arrays and translate them intolocal displacement and local strain information. In achieving this goal the method shouldincorporate the following features:

• Sufficiently accurate

• Efficient interface

• Based on LabView and/or MatLab (standard Philips software)

• Cost a tenth of commercially available packages

2

The goal of this study is to devise a method, capable of capturing local displacement infor-mation during a tensile test, without an investment of approximately 80k for a commerciallyavailable method like Aramis [1]. The goal of this report is to provide a comprehensive guidefor using image processing, e.g. within the field of electronic textiles, in order to obtain sampledisplacement information during loading tests.

1.3 Solution approach

Two methods are devised in order to achieve the research goal. The frameworks are based onthe LabView Vision module for Pattern Matching and Pattern Recognition respectively andrely on a MatLab package for post-processing. The Pattern Matching technique is used tocreate a ”trackable mesh” over the image array, and in this way local displacement informa-tion can be obtained. The Pattern Recognition requires the user to point out specific regionsof interest which are subsequently tracked throughout an image array.

The two methods are constructed, tested, and compared. First, a general chapter presents im-age theory, experimental techniques and practicalities. Chapter 3 elaborates on the PatternMatching Technique, for which the theory, interface, output and post-processing are dis-cussed. In chapter 4 the Pattern Recognition framework is presented analogously to Chapter3. Eventually, results and limitations are compared and discussed.

3

Chapter 2

Image Principles and Definitions

This chapter contains the preliminaries for analyzing digital image arrays. Subjects coveredrange from the actual definition of a digital image to devising an effective experimental set-upin order to create high contrast image arrays.

2.1 Digital Imaging

For the sake of brevity, the term image refers to a digital image throughout this report. Animage is defined as a 2D array of values representing light intensity or

f(x, y) (2.1)

where f is the brightness of the point (x, y), and x and y represent the spatial cartesiancoordinates of a picture element, or pixel. Fig. 2.1 represents this schematically. Notice that

(0,0)

f(x,y)

X

Y

Figure 2.1: Spatial reference of a pixel.

by convention the (0, 0) coordinates are located in the left top of the image, i.e. the value ofx increases from left to right, and the value of y increases from top to bottom.

An image capturing device assigns a numeric location and gray or color level to each pixel.Each image type is governed by three properties, i.e. resolution, definition, and number ofplanes.

4

2.1.1 Resolution

Each image is composed of M columns and N rows and consequently has a spatial resolutionof M ×N . Notice that the image has M pixels along the horizontal axis and N pixels alongthe vertical axis.

2.1.2 Image Definition

The definition of an image indicates the number of shades in a grayscale or color range. Thisdefinition is governed by the bit depth, which is defined as the number of bits used to encodethe value of a pixel. For a given bit depth of n, the image has an image definition of 2n,referring to 2n different values for a pixel.

Typical image definitions are 8-bit, 16-bit, floating point, or color encoding. In this reporthowever, only 8-bit image definitions are considered for simplicity reasons. For a guide onother image definitions within the LabView Vision packages, see [2].

2.1.3 Number of Planes

The number of planes in an image corresponds to the number of arrays of pixels that composethe image. Grayscale or pseudo-color images are composed of one plane, true-color images ofthree planes (Red, Green, and Blue). In this study only one-plane grayscale bitmap imagesare considered.

2.1.4 Image array

An image array is considered as a row of images. When analyzing image arrays it is convenientto have control over the size of the image array. Analysis can be time consuming in somecases; it may therefore be efficient to test parameter settings on a smaller part of the originalimage array. Two definitions are introduced and illustrated in Fig. 2.2. Offset defines thestart of the new image array, whereas interleave refers to spacing between images of the oldarray. The new image array will consist of images indicated in grey, Fig. 2.2.

2.2 System setup and considerations

If the eventual goal of the analysis is to obtain strain fields or local strains, it is not necessaryto convert pixel coordinates and displacements into ”real world” or spatial coordinates anddisplacements, since strains are dimensionless, e.g.

εlinear =l − l0

l0(2.2)

εln = ln(

l

l0

)(2.3)

where εlinear and εln represent a linear strain and logarithmic strain respectively. The formeris generally applied in small deformation mechanics, whereas the latter is apt to large defor-mation mechanics. l and l0 represent the current and initial distance between two arbitrarymaterial points respectively. However, in the case that true displacements are to be measured,

5

Offs

et

Inte

rleav

e

image 1

2

3

4

5

6

7

8

9

10

11

12

13

14

Figure 2.2: Illustration of offset and interleave, where the gray image array is the new image arrayif an offset of 4 and an interleave of 2 is applied.

a translation from pixel coordinates into spatial coordinates is needed.

In the following paragraph a number of considerations is presented to assist in ensuringquality input for the image processing techniques. Quality input can be obtained by optimiz-ing the following parameters of the imaging setup to the experimental goal, some of whichare represented in Fig. 2.3:

• Resolution - Smallest distinguishable feature size on an object, 1 in Fig. 2.3(a)

• Pixel resolution - The minimum number of pixels needed to represent the object

• Field of view - The area of the object acquired by the camera, 2 in Fig. 2.3(a)

• Working distance - The distance from the front of the lens to the object, 3 in Fig. 2.3(a)

• Depth of field - The maximum object depth that is still in focus, 4 in Fig. 2.3(a)

• Sensor size - The typical area of the heart of the camera, typically defined by the sensor’shorizontal dimension, 5 in Fig. 2.3(a)

2.3 Image analysis

An important tool in considering image arrays is the histogram, a representation of an imagethat shows the total number of pixels at each grayscale level. A histogram can aid in identi-fying features such as background, objects, and noise. As a result, acquisition conditions canbe adjusted in order to provide a more productive setup. Two important criteria should beconsidered.

• Saturation - too much or too little light in the image sensor

6

(a) The following parameters are depicted: 1) Reso-lution 2) Field of View 3) Working distance 4) Depthof field 5) Sensor

(b) Numerical or digital image consisting of pixels asrecorded by the camera

Figure 2.3: Fundamental parameters of an imaging system

• Contrast - difference of grayscale values for object features

For example, the histograms of Fig. 2.4(b) and Fig. 2.4(d) show the distribution of grayscalevalues in the corresponding images (Figs. 2.4(a) and 2.4(c)). A number of peaks can bedistinguished. Note that in Fig. 2.4(d) a peak occurs for high intensity, due to the largewhite areas in Fig. 2.4(c), where back-light reaches the camera directly. When the sample inthe image undergoes a large deformation it does not block the back-light completely anymore.

In order to optimize the analysis, several filters or adjustments can be applied in the post-processing phase. However, this is beyond the scope of this report. For more practicalinformation, see [2, 3].

7

(a) Image of an arbitrary diagonal sample beforeloading (640x480 pixels)

0 50 100 150 200 250

0

1000

2000

3000

4000

5000

6000

(b) Histogram corresponding to 2.4(a)

(c) Image of an arbitrary diagonal sample duringloading (640x480 pixels)

0 50 100 150 200 250

0

1000

2000

3000

4000

5000

6000

(d) Histogram corresponding to 2.4(c)

Figure 2.4: Two arbitrary sample images in a typical sample array and their corresponding his-tograms

8

Chapter 3

Pattern Matching

Pattern Matching is a technique that locates regions of a grayscale image that match a knownreference pattern or template. This template acts as an idealized representation of a featurein the image. The software tool subsequently searches for instances of the template in eachacquired image, calculating a score for each match.

In the present context Pattern Matching is used to obtain local displacement fields in atime-efficient way. The technique is robust with respect to lighting variation, blur, noise,and geometric transformations such as translation, rotation or scaling. However, it also haslimitations which are discussed later in this chapter.

In this chapter analysis of an image array is considered. Examples of an initial image and anarbitrary image during deformation in the array are depicted in Fig. 3.5

(a) Initial image of a textile sample image array (b) Arbitrary image later in the same image array

Figure 3.1: A two-image array; the deformation can be seen by comparing the two imagesd

3.1 Normalized Cross-Correlation

Pattern Matching techniques are built around a certain algorithm, e.g. pyramidal match-ing, scale- and rotation-invariant matching. Here, normalized cross-correlation is employed,

9

a common method for finding a template within an image.

The basic concept of correlation is defined as follows. Consider a subimage w(x, y) of sizeK × L within an image f(x, y) of size M × N , where K ≤ M and L ≤ N . The correlationbetween w(x, y) and f(x, y) at a point (i, j) is governed by

C(i, j) =L−1∑

x=0

K−1∑

y=0

w(x, y)f(x + i, y + j) (3.1)

where i = 0, 1, ...M − 1, j = 0, 1...N − 1. The summation is taken over the region where wand f coincide.

K

L

w(x,y)

(i,j)j

i

f(x,y)

(0,0)

M

N

x

y

Figure 3.2: Schematic illustration with relevant correlation parameters

Fig. 3.2 illustrates the variables defined above. As is shown in this figure, the origin is as-sumed to be in the left upper corner of the image. Correlation is the process of translatingthe subimage through the image and computing the value for C in every particular area. Thisinvolves multiplying each pixel in the template by the image pixel that it coincides with andadding the outcome over all pixels in the template.

In the present study, the templates are square for the sake of simplicity, in other wordsK = L. Only one point of a template is stored as data output, i.e. the template center.

It is wise to extend this correlation technique to a normalized version, where the correla-tion is less sensitive to amplitude changes in the image (intensity). Note that if the intensityf in the image is doubled, so are the values of C. The normalized correlation coefficient is

10

defined as

R(i, j) =

L−1∑

x=0

K−1∑

y=0

(w(x, y)− w̄)(f(x + i, y + j)− f̄(i, j))

√√√√L−1∑

x=0

K−1∑

y=0

(w(x, y)− w̄)2

√√√√L−1∑

x=0

K−1∑

y=0

(f(x + i, y + j)− f̄(i, j))2

(3.2)

where w̄ and f̄(i, j) are average intensity values of the template w and the region coincidentwith the current location of w respectively. The value of R is consequently defined within thedomain [−1, 1].

Incorporating this algorithm implies insensitivity to the following image properties, Fig. 3.3.

• The pattern of the image

• The position of the pattern in the image

• The orientation of the pattern

• Multiple instances of the pattern in the image, if applicable

Figure 3.3: Pattern orientation and how templates are recovered from altered images. The referencepattern is located regardless of rotation or scaling.

Fig. 3.3 shows a template image, a template match shifted in the image, a template matchrotated in the image and a template match scaled in the image.

3.2 Interface

In order to create an effective and efficient user interface, a number of features have to beincorporated. Only the crucial features are discussed in this chapter. The interface andprocessing tools are all developed in LabView (Vision module). The features are discussedbelow by illustrating the interface.

Fig. 3.4 shows the initial interface when starting the Pattern Matching tool. The left partof the interface deals with data management issues which are not of primary interest andare essentially self-explanatory. The right part of the interface deals which analysis options,which are now discussed in further detail.

11

Figure 3.4: Initial interface of the Pattern Matching tool. Analysis parameters are used as input forfurther analysis.

Several parameters have to be set: Block size, Minimum Initial Match Score, Minimum TrackMatch Score, Relearn Pattern Score limit, and Maximum Track distance. These parametersare discussed in the following paragraphs.

3.2.1 Block size

The block size, or template size, is defined as the total number of pixels to be covered bya subimage or, in other words, the area covered by w, K × L. For simplicity reasons thetemplate is always square, which implies the unique definition by only stating the number ofpixels in a side of the square. An input of “20” corresponds to a subimage of 20× 20 = 400pixels.

The ideal block size setting depends on considerations such as density of the displacementfield (since templates do not overlap), accuracy of the displacement field, and effectiveness ofthe created grid. The following considerations are noteworthy

• A dense displacement field will yield a lot of displacement information, but will alsoincrease the likelihood of non-uniqueness of the templates.

• In the particular case of tracking large deformations in textile, large block sizes can havethe problem of significant block deformation, in which case displacement informationbecomes inaccurate

• Non-uniqueness of images is detrimental to the effectiveness of the analysis

In every image templates are searched only around the last known template position, i.e. notin the complete image. This measure decreases the impact of non-uniqueness in the overallimage.

12

3.2.2 Initial Minimum Match Score

Match scores are defined as a binary judgement on the correlation coefficient. If c, the localanalogy of C or r, the local analogy of R in the case of a normalized coefficient is largerthan or equal to the threshold or match score, the subimage or template is trackable andrecognized in the subsequent image. However, if the calculated value is smaller than thethreshold, the template is not tracked anymore and displacement data will not be availablefor this subimage. If the Initial Match Score is defined as IMS and the variable statingwhether a tracking is successful is introduced as p it is possible to write

r ≥ IMS ⇒ p = 1 (3.3)r < IMS ⇒ p = 0 (3.4)

The normalized correlation coefficient is compared to the score, which will determine whetherthe match is successful. The initial minimum match score essentially defines the level requiredfor uniqueness of the templates. Scores are given in the range of [0 − 1000], while the toolinternally uses uses values in the range [0 − 1] in order to compare with the correlationcoefficient. A score of 980 equals a requirement of r ≥ 0.98. Setting this value high restrictsthe tracking of templates in time. Templates can be discarded in the first step because ofnon-uniqueness. The following considerations are noteworthy

• Too high a match score will not yield any usable data, whereas setting it too low maycause multiple matches or ineffectiveness of the method during analysis, causing datato be discarded at a later stage.

• As a rule of thumb, this match score should at least be larger than two other scores:track match score, and relearn score.

The initial image is divided into templates and those are ”learned”. This implies attachinga certain pixel combination to each part of the image. This process can be time consuming,hence the interface provides an indication of processing time, as illustrated by Fig. 3.5(a).The interface shows the template being learned, the uniqueness of the template ”Number ofMatches”, and an indication of learning progress.

3.2.3 Track Minimum Match Score

The Track Match Score is defined as TMS and analogous to IMS for every new image inthe image array. For IMS it is then possible to write

IMS = (TMS)0 (3.5)

Setting this value high increases the likelihood of data being discarded during analysis becausethe criteria for TMS, analogous to (3.3), are not met. Setting it too low implies increasingnon-uniqueness. The following considerations are noteworthy

• Too high a match score will not yield any data, whereas setting it too low will yieldlarge amounts of ineffective data.

• As a rule of thumb, the track match score should at least be smaller than the initialmatch score.

13

(a) Interface during ”learning”; the templateunder consideration is depicted, the unique-ness is presented by ”Number of Matches” andlearning progress is indicated

(b) Processing pane duringanalysis; From left to right, topto bottom: number of matches,number of templates in theimage, location progress (startsat zero for every new image),and image or overall progressbar and percentage.

Figure 3.5: Process indicators for learning and tracking

During analysis, a processing pane shows the overall progress of the analysis, as is illustrated inFig. 3.5(b). The left upper number indicates uniqueness of the template under considerationand is equal to 1 in the case of a unique template. The right upper number refers to thetotal number of templates in one image. The location and image progress bars indicate theprogress of template analysis in one image and overall or image progress respectively. Thelatter is also indicated as a percentage. Note that the algorithm matches all templates beforemoving on to the next image.

3.2.4 Relearn Score

The relearn feature offers the possibility to redefine a template at the position of an oldtemplate. This feature can be used to reactivate areas that could not be tracked during acertain deformation within an image array. If the score is set higher than the TMS, thetool will change the template definition into a new one before the old one is discarded. Fig.3.6 illustrates the relearn process. Whenever the correlation coefficient does not match therelearn score, the Pattern Matching tool will define a new template at the last known location.

3.3 Output

Output is restricted to the locations of centers of mass of the matrix of templates throughoutthe image array. Whether tracking of a template is successful is indicated by the trackingparameter p in the data output file.

14

Deformation

Template New template Deformed area

Figure 3.6: The influence of deformation on the match of a template in the subsequent image. Whendeformations are large and the match scores are high, the correlation coefficient can be lower than therelearn score, a new template at that location will be ”learned”.

An output visualization is depicted in Fig. 3.7. In this visualization the center of massof a template in an image is marked by a green dot (light equals successful, dark equals un-successful). As a result, an image array describing a deformation will yield lines or dottedlines. Again, light green if successful, and dark green if unsuccessful.

Figure 3.7: Output visualization of the Pattern Tracking tool, where successfully tracked templatepaths are light green and unsuccessful ones are dark green.

In this particular case, the total deformation was already quite large and a high quantity oftemplates could not be tracked. In general, this visualization illustrates in an effective waywhether or not to change analysis parameters and conduct the analysis again. Considerationspresented in the previous section can act as a guideline.

The most important output is a data-file containing three important parameters of eachtemplate. The output file can be considered as a matrix of size m× n, where m refers to thenumber of templates and n refers to the number of images in the array. Each component in

15

this matrix is composed of three numbers:

• x-coordinate of the center of mass of the template

• y-coordinate of the center of mass of the template

• the tracking parameter p, where p = 1 for a successful track and p = 0 for an unsuccessfultrack

This file is schematically represented by

Output.dat =x11 y11 p11 · · · x1j y1j p1j

.... . .

...xi1 yi1 pi1 · · · xij yij pij

(3.6)

where x and y refer to the coordinates of a template in an image, p represents the trackingparameter, and i and j refer to the template and image respectively. In other words, if thestring of a template, consisting of coordinates xij and yij and parameter pij is considered asa single matrix component, an analysis of an image array of n = 300 images using m = 250templates would yield and output matrix of 250× 300.

Having analyzed the image array and knowing the locations of the templates throughoutan image array, any post-processor can be used to calculate displacements, strains, etc. Usingsuch a post-processor also enables one to apply data filtering techniques. Post-processing isbeyond the scope of this section and will be discussed elsewhere.

16

Chapter 4

Pattern Recognition

Pattern Recognition is a technique that is essentially different from Pattern Matching in thesense that instead of correlating templates within an image array, only certain characteristicfeatures are tracked throughout an image array. This chapter discusses image theory andpractice involved in using such a technique.

Only regions of interest, indicated by manual input, are considered for analysis and thereforeno large amounts of data are stored. Within an arbitrary region of interest an image featurecan be distinguished, i.e. a collection of pixels which have a significant contrast with thesurrounding area. Consider an image array such as the one depicted in Fig. 4.1.

(a) Initial image of a textile sample image array (b) Arbitrary image later in the same image array

Figure 4.1: A two-image array, where the deformation can be seen when the two images are compared

Features within this image can be distinguished, e.g. different yarns and a structure on top.On the structure black markers are used for recognition purposes.

4.1 Binary morphology

The first step of the algorithm behind Pattern Recognition is defining the actual feature.Within an image, a region of interest (ROI) is selected for analysis. If within an arbitrary

17

ROI a grayscale threshold is set, this region can be converted into a binary image, so that afeature can be distinguished, see Fig. 4.2. The binary image is obtained by a thresholding

Figure 4.2: Schematic representation of selecting a region of interest, which is in turn converted intoa binary image

technique. In this case simple thresholding is used, but automatic and more sophisticatedtechniques are available within commercial software packages such as LabView. The principleof thresholding is illustrated in Fig. 4.3. This thresholding process is a subjective process

ki

h(i)

Fre

qu

en

cy

Gray Level Value

Feature

ClassBackground

Class

Figure 4.3: Graphical representation of thresholding, where k is the threshold value, i represents theintensity or gray level value, and h(i) corresponds to the number of pixels in the image at each graylevel value

and it can cause undesired information such as noise particles, particles touching the borderof images, particles touching each other, etc. By affecting the actual shape of particles, byusing morphological functions, or setting restrictions to the analysis used, the informationstored in a binary image can be improved.

18

4.1.1 Structuring element

Morphological functions transform a pixel based on the number of neighboring pixels and theintensity values of those pixels. A neighbor is defined as a pixel whose value affects the valuesof nearby pixels during certain image processing functions. A structuring element is a two-dimensional binary mask that defines the size and effect of the neighborhood on each pixel,controlling the effect of the binary morphological functions on the shape and the boundaryof a particle.

It is common to use a binary structuring element of 3× 3, where the values of a structuringelement govern neighborhood influence as follows:

• If the element component is 1, the value of the corresponding source image pixel affectsthe central pixel’s value during a transformation.

• If the element component is 0, the value of the corresponding source image pixel doesnot affect the central pixel’s value during a transformation.

A structuring element alters a pixel P0 so that it becomes a function of its neighboring pixelvalues, as is illustrated by Fig. 4.4.

0

0

0

00

1

1 1

1

x

Structuring element Source Image Transform Image

neighbors used

to calculate the

new P0 value

new P0 value

Figure 4.4: Graphical representation of a structuring element and its effect on a morphologicalfunction

4.1.2 Morphological functions

Many morphological functions exist, of which only dilation is explained here. Dilation caneliminate holes isolated within particles and expands its contour according to the templatedefined by the structuring element.

For any given P0, the structuring element is centered on P0. The pixels masked by a co-

19

efficient of the structuring element equal to 1 are then referred to as Pi.

if Pi = 1 → P0 = 1else P0 = 0

If there is one pixel in the neighborhood, Pi, that has value 1, P0 also assumes the value of 1.This function can be applied several times to account for bigger isolated holes within particles.

Another morphological function is the border function. Border functions can remove parti-cles that touch the border of the region of interest. These particles may have been truncatedduring the digitization of the image, and their elimination helps to avoid erroneous particlemeasurements.

4.1.3 Connectivity

After the thresholding process, binary shapes appear within a region of interest. These shapescan be grouped into particles, thereby introducing the concept of connectivity. The selectedconnectivity determines whether two adjoining pixels are included in the same particle. Thereare two possibilities, see Fig. 4.5.

• Connectivity-4, two pixels are considered part of the same particle if they are eitherhorizontally or vertically adjacent.

• Connectivity-8, two pixels are considered part of the same particle if they are eitherhorizontally, vertically, or diagonally adjacent.

Connectivity - 4 Connectivity - 8

Figure 4.5: Graphical representation of the concept of connectivity

In a rectangular pixel frame, which is the most common, each pixel P0 has eight neighbors.From a mathematical point of view, the pixels P1, P3, P5, and P7 are closer to P0 than thepixels P2, P4, P6, and P8 if the pixels are numbered as follows.

P8 P1 P2

P7 P0 P3

P6 P5 P4

(4.1)

If D is then defined as the distance from P0 to P1, the distance of P0 to Pi, for i = 1, 2...8, is√

2D D√

2DD 0 D√2D D

√2D

(4.2)

20

Accordingly, particle count for connectivity-4 is equal to or larger than the particle count forconnectivity-8, as is illustrated in Fig. 4.6.

X

X

X

O O

+

I

I

Connectivity - 4

Part. Count = 4

X

X

X

X X

X

I

I

Connectivity - 8

Part. Count = 2

Figure 4.6: Particle count implication due to connectivity, where a different symbol represents apixel being part of a different particle.

4.1.4 Particle motion

In order to keep a particle within its region of interest even during large deformations, analgorithm is used to translate the ROI. Fig. 4.7 provides the framework of analysis.

K

L

w(q,r)

(0,0)

i

j

g(x,y)

(0,0)

M

N

x

yb

a

P

Figure 4.7: Definition of particle coordinates

Within an M ×N image, a K×L region of interest is selected, which encloses a particle withcenter of gravity P . Two Cartesian coordinate systems are defined:

• a global coordinate system, at image level, (x, y)

21

• a local coordinate system, at ROI level, (q, r)

Global coordinates (0, 0) correspond with the upper left corner of an image, whereas localcoordinates (0, 0) correspond with the upper left corner of the ROI. The location of the centerof gravity of a particle is defined as the summation of two sets of coordinates.

xp = iroi + ap (4.3)yp = jroi + bp (4.4)

where i and j represent the x- and y-coordinate of the upper left corner of the region of inter-est, and a and b represent the q- and r-coordinate of the particle within the ROI respectively.

In order to account for large translations of particles, the region of interest should followthe particle movement. A rational way of doing this is depicted in Fig. 4.8.

Pf-1

Pf

ROIf ROIf

Pf

ROIf+1

Pf+1

f f+1

Figure 4.8: The definition of particle movement and ROI movement by two subsequent images fand f + 1. As the center of particle P moves in time, the region of interest follows its movement.

Consider a particle with center of mass P and a region of interest ROI. At any point intime, denoted by an index f , P (x, y)f is defined. If Pf 6= Pf−1, but lies within the region ofinterest at time f , the region of interest is shifted according to the displacement of P duringthe previous increment. In this way the motion of the region of interest always lags oneincrement behind, implying the necessity to align the image interleave and size of the regionof interest, i.e. if the displacement of the particle between two subsequent images is too largeor the region of interest two small and the particle falls outside the ROI, the particle is nottracked.

4.2 User Interface

The first tab of the user interface of the Pattern Recognition tool constructed in LabView isdepicted in Fig. 4.9(a). This part of the interface has two distinct purposes:

1. Image array management or data management; parameters such as offset and interleavegovern the image array definition.

2. Region of interest selection; a rectangular selection tool allows one to point out regionsof interest on the first image in an array.

22

In the left bottom corner the input regarding connectivity is provided.

The second tab of the interface deals with the analysis of a specific region of interest, aspointed out in the first tab of the interface. A screen shot of this tab is presented in Fig.4.9(b). The analysis tab has one main input pane and three result panes. The purpose ofthis tab is listed as:

1. Input pane, on which the thresholding process is defined, as well as some basic morpho-logical functions such as dilation

2. Region of interest image; an image of the ROI (real-time)

3. Region of interest; a binary image of the ROI (real-time)

4. Coordinate graph: x- and y-coordinates of the center of gravity of the ROI during theanalysis process (available after completion of analysis)

23

(a) The interface of the Pattern Recognition tool (first tab); data management

(b) The interface of the Pattern Recognition tool (second tab); ROI analysis. 1) Inputpane 2) ROI 3) Binary ROI 4) Coordinate graph

Figure 4.9: User Interface as constructed in the Pattern Recognition tool

24

4.3 Output

The output of the PR tool consists of three components:

• ROI Analysis output; real-time or direct output, e.g. coordinate graph and the binaryimage, used for analyzing the quality of ROI settings

• PR movie output; an avi-file of the image array with superposed ROI’s

• PR data file; containing all analysis information needed for post-processing

Figure 4.10: Result of ROI analysis; the coordinate graph (the images in Fig. 4.1 were image 1 andimage 300 in this image array respectively)

For an arbitrary image array, the location graph is depicted in Fig. 4.10. The motion ofthe center of gravity of the particle within the ROI can be observed in the coordinate graph.Moreover, ROI analysis quality can be judged. The subjective process of thresholding cancause failure of recognizing the particle, which in turn can be observed from this graph. Ifthe particle is not found in a particular image in the array, the values for the coordinates areset to zero.

The PR data output file is important for post-processing, i.e. determination of displace-ments, strains, etc. Apart from storage of analysis options and the definitions of the regionsof interest, a matrix is stored which looks like

data =

p1 x11 y11 p11 · · · x1j y1j p1j

p2. . .

...

.... . .

pi xi1 yi1 pi1 · · · xij yij pij

(4.5)

where pi = 1 indicates a successful complete tracking procedure for a region of interest with nosingle value for pij equal to zero. i and j indicate the region of interest and image respectively,and x and y represent the particle center of gravity coordinates as recognized by the PR-tool.

25

Chapter 5

Discussion

This chapter will present and compare the results, possibilities and restrictions of the twomethods described in the previous chapters, i.e. Pattern Matching and Pattern Recognition.

5.1 Pattern Matching

A visualization of an ”initial template grid” for the Pattern Matching technique can be ob-tained by using Matlab as a post-processor for superposing the locations of the centers ofgravity of all templates and the initial image in an array, see Fig. 5.1. From this information

Figure 5.1: An initial template distribution of the Pattern Matching tool on a textile sample, wherethe markers represent the centers of gravity of templates.

it can be concluded that the displacement field will be dense, although it can be seen thatalready in this initial image, not all templates satisfy the Initial Match Score (IMS), see 3.2.2.This can be concluded from the fact that there are no markers in image areas with pooruniqueness characteristics.

When the markers are plotted again further in the image array, this time without a back-ground image, a representation like Fig. 5.2 is obtained. Fig. 5.2 is a slightly deformedor stretched version of Fig. 5.1. What can be observed, is that in middle regions, whereuniqueness conditions are poor, the Pattern Matching tool has trouble successfully tracking

26

Figure 5.2: A template distribution in an image array of the Pattern Matching tool, where themarkers represent the centers of gravity of templates.

the templates. Some templates are not recognized anymore, pij = 0, others are useless, be-cause they ”jump” to a neighboring template, which in turn is a uniqueness problem.

In post-processing filtering can be applied to filter out erroneous data points. This is notwithin the scope of this report and therefore left out of consideration.

5.1.1 Strengths

The biggest advantage of an automatic template generation is implemented within the Pat-tern Matching environment, is that is easily generates a complete grid of templates, i.e. alocation field (displacement field). Uniqueness of features within the image is important foreffectiveness, in other words, an irregular high contrast image would result in the best dataoutput. Unfortunately, one cannot choose the image to be analyzed, but instead should matchthe technique to the image under consideration.

5.1.2 Weaknesses

Weaknesses of this method surface when it is applied to the examples in this chapter andthey can be categorized

• Demands on uniqueness, if the structure is too repetitive at the template scale, effec-tiveness will be low

• Analysis time, large amounts of data are calculated and stored and consequently anessentially slow process is created (minutes)

• Fine tuning analysis parameters, iterative optimization of parameters is only possibleupon completion of the analysis

• Resolution of the method equals one pixel

27

Figure 5.3: The link between deformation - relearning - data erosion

5.1.3 Discussion

Another point of concern lies within analysis of large deformations. A prerequisite for makinglarge deformation analysis effective, is that a relearn algorithm is used. At a certain pointin time the correlation coefficient will not meet the track matching score anymore, or evenbefore that time, a new template is ”relearned”. This relearn implies that data of the originaltemplate is discarded and new data based on neighboring pixels are then used within thenew template. This can cause the center of gravity of a template to shift as a result of largedeformation. This fact is simply illustrated by Fig. 5.3. From the demands on the tool itis clear that another tool is needed, e.g. Pattern Recognition, that can cope with the pooruniqueness characteristics of the images. Furthermore, optimized analysis time, resolution aswell as the efficiency in tuning analysis parameters are all part of this second tool.

5.2 Pattern Recognition

Analogous to section 5.1 an image and the regions of interest on that image can be superposed.In this section another image is used and depicted in Fig. 5.4. Here, two different examplesof ROI selection are shown. Any combination of ROI selection is possible and all regionscan be analyzed and optimized separately. Note that the selection of regions with this tooloccurs manually, therefore, this Pattern Recognition tool is not efficient in obtaining a densedisplacement field. The process of thresholding can be time-consuming, especially whenimage quality (e.g. contrast) is poor. Deformation characteristics can be derived from thedisplacements of these regions of interest together with the force information from the tensiletesting equipment 1.

5.2.1 Sensitivity and accuracy

The Pattern Recognition tool is capable of sub-pixel accuracy as opposed to the PatternMatching technique. The PR tool calculates the center of gravity in hundredths of a pixel,implying the influence of the binary morphology on the center of gravity calculation. Ifthe particle is represented by few pixels, or if the particle is approximately rectangular, theaccuracy of the method decreases. This is illustrated by Fig. 5.5. If the particle underconsideration undergoes a certain movement, an image array can represent that movement.Suppose that the particle is represented by four pixels, the center of gravity calculation has anaccuracy of the order of a pixel. In achieving high accuracy and sensitivity, the morphologyof the features and the movement they experience is crucial. For example, consider a particle

28

(a) Rectangular region of interest set-up (b) Horizontal region of interest set-up

Figure 5.4: Region of interest on an initial image in an array, where the markers indicate the centerof gravity of the particle(s) within a region of interest

(a) The movement of a particle

(b) Representation of the movement of a particle, where the particle is described by fourpixels

Figure 5.5: Implication of rectangular representation of a particle on the determination of the centerof gravity.

as the one in Fig. 5.5(a). A simple rotation of the image capturing device will significantlyimprove the accuracy of the center of gravity calculation, since the pixel frame of the camerais no longer aligned with the movement of the particle.

Several alignment problems can occur when clamping samples:

• clamp rotation misalignment

• clamp translation misalignment

• sample rotation misalignment

• sample translation misalignment

The first two problems concern the tensile tester, or clamps, and their position with respect toeach other. The other problems have to do with how one clamps the samples. The rotation

29

and translation misalignments of the sample are defined as two perpendicular horizontallines at the sample top and sample bottom. As a result, a rotation misalignment causesan inhomogeneous out-of-plane bending of the sample before loading. Discussing all thesealignment problems and how they can be recognized from local displacement data is beyondthe scope of this report.

5.2.2 Out-of-plane deformation

It is possible to compare local information, obtained through the Pattern Recognition toolwith global displacement information, which is determined by a tensile tester. When consid-ering the sample in Fig. 5.4(a), the logarithmic strain graphs of Fig. 5.6 can be constructedfor the elongation in tensile direction and the perpendicular contraction respectively. Forbackground on strain definitions [4]. This sample shows an out-of-plane deformation due tomisalignment of the clamps in the tensile tester or by bending of the sample before loading.

0 10 20 30 40 500

0.01

0.02

0.03

0.04

Time [s]

Loga

rithm

ic s

trai

n, ε

ln [−

]

TTMeanStrain 1Strain 2

(a) Elongational (tensile direction) strain history cal-culated by the Tensile Tester, and two local strainswith their mean, where ”Strain 1” is determined bythe left regions and ”Strain 2” is based on the rightregions of Fig. 5.4(a).

0 10 20 30 40 50−6

−4

−2

0

2

4

6x 10

−3

Time [s]

Loga

rithm

ic s

trai

n, ε

ln [−

]

MeanStrain 1Strain 2

(b) Contractive (perpendicular to tensile direction)strain history calculated by the Tensile Tester, andtwo local strains with their mean, where ”Strain 1”is determined by the lower regions and ”Strain 2” isbased on the upper regions of Fig. 5.4(a).

Figure 5.6: Tensile elongation and perpendicular contraction with out-of-plane deformation due toclamp misalignment

The sample aligns with the loading direction during its deformation. From Fig. 5.6(a) it canbe concluded that the slopes of the local and Tensile Tester data do not agree. Furthermore,from Fig. 5.6(b) the strains are positive in the first part of the deformation, an elongationwhich is due to out-of-plane movement in the sample. The deviation is interpreted as eithera rotation misalignment of the clamps or out-of-plane bending of the sample edges prior toloading. This hypothesis is tested by observing the elongation strain closer to the centerline of the tensile direction. This verification is presented in Fig. 5.7 and the hypothesis isreinforced by the fact that the slopes of the tensile tester and the local strain measurementalign.

30

(a) Location of the regions or particles of interest usedfor a strain calculation.

0 10 20 30 40 500

0.01

0.02

0.03

0.04

Time [s]

Loga

rithm

ic s

trai

n, ε

ln [−

]TTMeanStrain 1Strain 2

(b) Elongational (tensile direction) strain history cal-culated by the Tensile Tester, and two local strainswith their mean, where ”Strain 1” is determined bythe left regions and ”Strain 2” is based on the rightregions of Fig. 5.7(a).

Figure 5.7: Strain determination close the sample center line in order to decrease the influence ofout-of-plane deformation

31

Chapter 6

Conclusions

Global experiments such as tensile tests cannot account for local influences on deformationbehavior. Furthermore, the effect of clamping textile, especially in a diagonal direction, isnot trivial. An uniaxial stress state cannot be assumed throughout the complete sample andthus material parameters have to be determined from a part of the sample that actually ex-periences a uniaxial stress state.

The devised Pattern Matching tool is particularly apt to creating displacement fields. Tem-plates are learned and matched in subsequent images, from which the displacements arederived. This method becomes ineffective when uniqueness of the images is poor. Moreover,the maximum accuracy is restricted due to the fact that a template moves in steps of a pixel,i.e. the accuracy equals one pixel. The method is tested and is efficient in determining dis-placement fields of image arrays with good uniqueness characteristics.

The Pattern Recognition tool solves the problem of coping with poor uniqueness charac-teristics of the images and increases the accuracy to sub-pixel level. Regions of interest areindicated manually and a certain feature or ‘blob’ is followed throughout an image array. Themanual selection of regions of interest makes the generation of a displacement field inefficient.The Pattern Recognition tool is successful in obtaining large local displacements from imagearrays.

Application of these tools has proved crucial in the process of parameter identification [4]. ThePattern Recognition is clearly the tool of choice for further local experimental deformationsanalysis on textiles.

32

Chapter 7

Recommendations

Long term

• Expand the method to three dimensions by adding an extra camera, enabling out-of-plane deformation analysis

Short term

• Increase the resolution of the camera to increase accuracy of local displacement analysis

• Improve the uniqueness characteristics of the textile samples

• Study the method in its application to components on textile substrates

Acknowledgements

• Martin Vernhout, supported LabView programming

• Adrie Raaijmakers, supported LabView programming

33

References

[1] Trilion. Aramis. Technical report, http://www.trilion.com/Products/aramis.html, 2008.

[2] IMAQ Vision Concepts Manual. National Instruments, 2004. www.ni.com.

[3] Matlab Image Processing User’s Guide. The MathWorks Inc., 2005. www.matlab.com.

[4] M.J.M. Feron. Mechanics of textiles - experimental analysis and deformation modeling.Technical report, Philips Research, Eindhoven, 2008.

34