automation guidebook

8/4/2019 Automation Guidebook

1/68

IR Automation Guidebook:Temperature Monitoring and Controlwith IR Cameras

$29.95


2/68


3/68

IR Automation Guidebook:Temperature Monitoring and Controlwith IR Cameras


4/68

ii

The thoughts, ideas, opinions, and recommendations

expressed in this book are intended for informationalpurposes only. FLIR accepts no liability for actions taken byreaders in their individual businesses or circumstances.

Published by FLIR Systems IncorporatedThis booklet may not be reproduced in any orm withoutthe permission in writing rom FLIR Systems Incorporated.www.goinfrared.com 1 800 GO-INFRA Copyright 2008. All rights reserved.

USA, Canada and Latin America

FLIR Systems IncorporatedAmericas Main Oce, USABoston, MA1-800-GO-INFRA (464-6372) or1-978-901-8000

Europe, Middle East, Asia and AricaFLIR SystemsInternational Main Oce, SwedenTel: +32 3 287 87 10


5/68

iii

ContentsPreface iv

Chapter 1 Typical Monitoring and Control Applications 1

Chapter 2

Remote IR Monitoring 5

Chapter 3Temperature Measurementfor Automated Processes 17

Chapter 4Combining Machine Vision

and Temperature Measurement 25Chapter 5Real-Time Control Issues 32

Appendix AGlossary 40

Appendix B Thermographic Measurement Techniques 43

Appendix CHistory and Theory of Infrared Technology 45

Appendix DCommand Syntax Examplesfor A320 Resource Socket Services 58

Appendix EQuick Summary ofFLIR IR Cameras Inside Back Cover


6/68

iv

PreaceManuacturing and process engineersare under constant pressure to makeproduction systems and processes moreecient and less costly. Frequently, theirsolutions use automation techniques toimprove throughput and product quality.Automated IR (infrared) radiation imagingoers the potential for improving a host

of industrial production applications,including process monitoring andcontrol, quality assurance, assetmanagement, and machine conditionmonitoring.

This handbook is intended to help thoseconsidering the creation or improvemento production automation or monitoring

systems with IR cameras. Numerousapplication examples will be presentedwith explanations of how these IR visionsystems can best be implemented.

Some o the major topics that will be

covered include:

Integration o IR cameras intoautomation systems

Data communications interaces

Command and control othermographic cameras

Principles o thermographic

measurementsInteracing with a PC or PLC controller

Standard sotware packages or IRcamera systems

These complex matters require attentionto many details; therefore, this handbookcannot answer every question a systemdesigner will have about the use of

IR cameras in automated systems. Itis meant to serve only as a roadmapthrough the major issues that must befaced in IR vision system design.


7/68

1

Typical Monitoring andControl Applications

Typical Monitoring andControl Applications

Temperature Measurementswith IR Cameras

Infrared (IR) radiation is not detectableby the human eye, but an IR cameracan convert it into a visual image that

depicts thermal variations across anobject or scene. IR covers a portion ofthe electromagnetic spectrum fromapproximately 900 to 14,000 nanometers(0.914 m). IR is emitted by all objects attemperatures above absolute zero, andthe amount of radiation increases withtemperature. A properly calibrated IRcamera can capture thermographic images

of target objects and can provide accuratenon-contact temperature measurementsof those objects. These quantitativemeasurements can be used in a variety ofmonitoring and control applications.

In contrast, other types of IR imagersprovide only relative temperaturedierences across an object or scene.

Hence, they are used to make qualitativeassessments of the target objects,primarily in monitoring applicationswhere thermal images are interpretedbased on temperature contrast. Oneexample is to identiy image areas thatcorrelate to physical anomalies, such asconstruction or sub-surface details, liquidlevels, etc.

In some cases, an IR camera is justiablyreerred to as a smart sensor. In thesecases the IR camera has built-in logicand analytics that allows the comparisonof measured temperatures with user-supplied temperature data. It also has adigital I/O interface so that a dierential

temperature can be used or alarm and

control functions. In addition, a smartIR camera is a calibrated thermographicinstrument capable of accurate non-contact temperature measurements.

IR cameras with these capabilitiesoperate much like other types of smarttemperature sensors. They have fast,high-resolution A/D (Analog to Digital)

converters that sample incoming data,pass it through a calibration function, andprovide temperature readouts. They mayalso have other communication interfacesthat provide an output stream of analogor digital data. This allows thermographicimages and temperature data to betransmitted to remote locations forprocess monitoring and control.

Generally, smart IR cameras are usedin quantitative applications thatrequire accurate measurements o thetemperature dierence between atarget object and its surroundings. Sincetemperature changes in most processesare relatively slow, the near-real-time datacommunications o smart IR cameras are

adequate or many process control loopsand machine vision systems.

Automation Applications

Typical automated applications usingIR cameras or process temperaturemonitoring and control include:

Continuous casting, extrusion, and roll

ormingDiscrete parts manuacturing

Production where contact temperaturemeasurements pose problems

Inspection and quality control

Packaging production and operations

Chapter 1


8/68

Chapter 1

Environmental, machine, and safetymonitoring

Temperature monitoring as a proxy orother variables

The examples below demonstrate a widerange of applications that can be servedwith IR cameras. Potential applicationsare limited only by the imagination o thesystem designer.

Plywood Mill Machine Monitoring

Problem: Steam from open vats of hotwater obscures the machinery operatorsview of the logs as they are maneuveredfor proper alignment in the log vat.

Solution: An IR camera can present animage to the operator that makes the

cloud of steam virtually transparent,thereby allowing logs to be properlyaligned in the log vat. This example ofa qualitative application is illustrated inFigure 1.

Production Testing o Car Seat Heaters

Problem: Using contact temperaturesensors to assure proper operation ooptional car seat heaters slows downproduction and is inaccurate i sensorsare not properly placed.

Solution: An IR camera can detectthermal radiation rom the heater

elements inside the seats and providean accurate non-contact temperaturemeasurement.

This quantitative measurement can bemade with a camera that is permanentlymounted on a xture that is swunginto measurement position whenthe car reaches a designated point

on the assembly line. A monitor nearthat position provides an image witha temperature scale that reveals thetemperature o the car seat heaterelements, as shown in Figure 2.

Figure 1. Plywood mill application


9/68

3

Typical Monitoring and Control Applications

Packaging Operations

Problem: On a high-speed packagingline, ecient methods for non-destructive testing of a glued box sealare scarce, and most tend to be verycumbersome. In addition, the glue

application method has a good deal o

variability that must be monitored andrecorded with statistical quality controlroutines.

Solution: Since the glue is heated priorto application, its temperature and

Figure 2. Production testing o car seat heater elements

Figure 3. Machine vision box seal quality control


10/68

4

Chapter 1

locations on the box lid can be monitored

with an IR camera. Moreover, the imagecan be digitized in a way that allows thisinormation to be stored in a statisticalquality control database or trend analysisand equipment monitoring as shown in

Figure 3.

This is an example o using dierentialtemperature as a proxy or another

variable. In this case, temperaturereplaces mechanical methods oinspection/testing.

Summary

The automation examples presentedin this chapter have barely scratchedthe surace o the application space

that smart IR cameras can serve. Inthe following chapters, more detailedexamples will be presented alongwith practical inormation on theimplementation o automated systemsthat exploit the advantages of IR cameras.These chapters are organized accordingto the major types o applications thattypically use IR cameras:

Remote thermographic monitoringNon-contact temperaturemeasurement or automated processes

Combining IR machine vision withtemperature measurement

Real-time control and monitoring issues and answers


11/68

Remote IR MonitoringChapter

Remote IR MonitoringOverview

Inrared radiation is emitted by all objectsat temperatures above absolute zeroand is detectable by IR cameras. Sincethese cameras have various means ofcommunicating thermographic imagesand temperatures to remote locations,

they are ideal or remote and unattendedmonitoring. Moreover, smart IR cameras(those with built-in logic, analytics, anddata communications), can comparethe temperatures obtained rom theirthermographic images with user-denedsettings. This allows the camera tooutput a digital signal or alarm andcontrol purposes, while also providing

live images.

IR Camera Operation

IR camera construction is similar toa digital video camera. The maincomponents are a lens that ocuses IRonto a detector, plus electronics andsotware or processing and displaying

thermographic images and temperatureson an LCD or CRT monitor (Figure 1).Instead of a charge coupled devicethat video and digital still cameras use,the IR camera detector is a ocal planearray (FPA) of micrometer size pixelsmade of various materials sensitive to

IR wavelengths. FPA resolution rangesfrom about 8080 pixels up to 10241024pixels. In some IR cameras, the videoprocessing electronics include the logicand analytical unctions mentionedearlier. Camera rmware allows theuser to focus on a specic area of theFPA or use the entire detector areafor calculating minimum, maximum,and average temperatures. Typically,temperature measurement precision isC or better.

The camera lens and distance to thetarget object results in a eld of view(FOV) that determines the spot sizecovered by each pixel. The pixels analogoutput represents the intensity o heat

energy received from the spot it coverson the target object. In FLIR IR cameras,the A/D converters that digitize the pixeloutput have resolutions that range from8 bits (28 or 0255 pixels) up to 14 bits(214 or 016383 pixels). The thermographicimage seen on the monitor screen isthe result o a microprocessor mappingthese pixel output values to a color or

gray scale scheme representing relativetemperatures. In addition, radiometricinormation associated with the heatenergy impinging on a pixel is storedor use in calculating the precisetemperature of the spot covered bythat pixel.

IR In

Optics

NIR

MWIR

LWIR

Video

Processing

Electronics

Detector Cooling

Digitization

User InterfaceUser ControlVideo Output

Digital Output

Synchronization In/Out

System Status

Figure 1. Simplied block diagram o an IR camera


12/68

6

Chapter

Hence, IR cameras with these capabilitiesoperate much like other types o smarttemperature sensors. Their calibratedoutputs can be accessed via one or morecommunication interaces and monitoredat a remote location. Images saved fromthese cameras are ully radiometric1 andcan be analyzed o-line with standardsoftware packages, such as thoseavailable from FLIR.

Important Criteria in RemoteMonitoring Systems

When considering an IR camera or aremote monitoring system, some of theimportant variables to consider are:

Spot size the smallest feature in a

scene that can be measuredFOV (Field of View) the area that thecamera sees

Working distance distance from theront o the camera lens to the nearesttarget object

Depth of eld the maximum depth ofa scene that stays in ocus

Resolution the number of pixels andsize of the sensors active area

NETD (Noise Equivalent TemperatureDierence) the lowest level of heatenergy that can be measured

Spectral sensitivity portion ofthe IR spectrum that the camera issensitive to

Temperature measurement range,precision, and repeatability afunction of overall camera design

1 Radiometry is a measure of how much energy isradiating from an object, as opposed to thermography,which is a measure o how hot an object is; the two arerelated but not the same.

Another fundamental considerationis which portion of a cameras FOVcontains the critical inormationrequired or monitoring purposes. Theobjects within the FOV must provide anaccurate indication o the situation beingmonitored, based on the temperatureo those objects. Depending on thesituation, the target objects may needto be in the same position consistently

within the cameras FOV. Otherapplication variables related to themonitored scene include:

Emissivity of the target objects

Reected temperatures within the FOV

Atmospheric temperature andhumidity

These topics will be covered in moredetail in a subsequent chapter.

Remote Asset Monitoring

One type of application where IR camerasare very useful is in remote monitoringof property, inventory, and other assetsto help prevent loss and improve safety.

Frequently, this involves storage facilities,such as warehouses or open areas orbulk materials. The ollowing examplecan serve as a general model for settingup an IR camera monitoring system orthis type o application.

Hazardous Waste Storage Monitoring. Inthis application barrels o chemical wasteproducts are stored in a covered facility,but one in which they cannot be totallyprotected from moisture. Thus, there isthe possibility o leaks or barrel contentsbecoming contaminated by air andmoisture, causing a rise in temperaturedue to a chemical reaction. Ultimately,there is a risk of re, or even an explosion.


13/68

7

Remote IR Monitoring

While visible light cameras might beused in such an application, there often

is a line-of-sight problem where manyof the barrels cannot be seen, even withmultiple cameras positioned throughoutthe storage area. In addition, smoke orames would have to be present beforea visible light camera could detect aproblem. This might be too late orpreventative measures to be taken.In contrast, stand-alone IR camerasmonitoring the acility can detect atemperature rise within their FOV beforere occurs (Figures 2a and 2b).

Depending on the camera manufacturer,several monitoring options are available.For instance, the FLIR A320 cameraallows a threshold temperature value tobe set internally or alarm purposes. In

addition, the cameras logic and clockfunctions can be congured so that a risein temperature must be maintained or acertain period o time beore an alarm issent. This allows the system to ignore atemporary temperature rise in a camerasFOV caused by a forklift entering the area

to add or remove barrels. Furthermore,a hysteresis unction can also be used to

prevent an alarm from turning o untilthe detected temperature alls well belowthe setpoint (Figure 3).

Cameras with a digital I/O interfacetypically provide an OFF/ON type ofoutput or alarm purposes. The digitalI/O output is either o or on; when on, itis typically a DC voltage or current. For

example, the digital I/O output from aFLIR A320 camera is 1030VDC for loadsof 100mA or less. Typically, the digitalI/O output is sent to a PLC (ProgrambleLogic Controller) that controls the portiono an alarm system associated with themonitored area.

A good way to set up the alarm systemis to have all cameras congured so theyhave a high level digital output when thetemperature is below the alarm conditionthat holds a PLC in its non-alarm state.When the alarm setpoint temperature isdetected, the cameras digital I/O outputgoes low (typically zero volts) after anappropriate time delay, causing the PLC

Figure 2a. IR image o a hazardous waste storagearea showing two spot temperature readings(26.4F and 16.8F) that are in the sae range, plusone reading (98.8F) that is abnormally high.

Figure 2b. A subsequent image o the samearea shows that the abnormal reading in 2ahas increased urther, causing an alarm togo o.


14/68

8

Chapter

to go into its alarm state. This creates afail-safe system. If power to the camera islost, then there is no high level output tothe PLC, which treats that event just as ifa temperature had reached the setpoint,thereby causing an alarm. This alertspersonnel that they have either lost themonitoring unction or there is indeed atemperature rise.

Image monitoring. Receiving a warningbased on temperature measurements isvery useful, but the real power of IR-based asset monitoring is in the camerasimage processing capabilities. Controlroom personnel can get live images fromIR cameras that visible light cameras

and other temperature detectorscannot provide. Again, cameras vary bymanufacturer, but the most versatile onesoer a variety of data communicationormats or sending thermographicimages to remote locations. Increasingly,web-enabled cameras are used to allow

monitoring rom any location where a PCis available.

Figure 4 illustrates a system usingthe FLIR A320s Ethernet and TCP/IPcommunication protocols in conjunctionwith its alarm setpoint capabilities. TheEthernet portion o the system allowscable runs of up to 100 meters in length.

By communicating a digital alarm directlyto the PLC, it can immediately activatea visual and/or audible alarm. The visualalarm can appear on an annunciatorpanel telling the operator where thealarm originated; the operator then goesto the PC to look at live image(s) of thatlocation. Images and temperature datacan be stored or uture reerence and

analysis.A320 cameras can also be congured toautomatically send temperature dataand images to a PC via e-mail (SMTP) orFTP protocol whenever the temperaturesetpoint is reached, thereby creating arecord for subsequent review.

Time

Temperature

ThresholdTemperature(Warning On)

Warning OTemperature

Deadband

O Temp = On Temp Deadband

Hysteresis Also known as deadband

Can be thought of as another threshold setting wherethe smart sensor resets the alarm that was generatedwhen the original setpoint was compromised

Used to prevent signal chatter

Figure 3. Hysteresis is an important signal processing characteristic o smart IR cameras, whichmakes monitoring and control unctions much more eective.


15/68

9


In conjunction with a host controllerrunning FLIRs IR MONITOR (or othersuitable software), temperature data canbe captured for trend analysis. The A320can also supply a digital compressionof the cameras analog video signal,which can be sent as MPEG-4 streamingdigital video over an Ethernet link to a PCmonitor. IR MONITOR can be used to set

up temperature measurements, imagecapture, and camera display functions.This application allows the PC to displayup to nine camera images at a timeand switch between additional cameragroups as needed. The FLIR IP CONFIGsotware can be used to set up eachcameras IP address.

After the cameras are congured, thePC used or monitoring does not needto remain on the network continually.By using the FTP and SMTP protocolswithin the camera, the user can receiveradiometric images upon alarm eventsor on a time based schedule. Also, any

available PC with a web browser can beused to access the cameras web serverfor live video and basic control. This webinterace is password protected.

Most IR cameras have an analogvideo output in a PAL or NTSC format.Therefore, another image monitoringpossibility is to use a TV monitor to

display thermographic video. A singlecontrol room monitor can be used witha switch to view live images from eachcamera sequentially. When the camerasare properly congured, control roompersonnel can view scaled temperaturereadings for any point or area (minimum,maximum, and average) in that image.(See color scales in the screen capture

images depicted in Figure 2.) Not onlywill the operator know when thereis excessive heat, he or she can seewhere it is.

Another example of the innovativefunctions available in camera rmwareor external sotware is a eature called

Figure 4. An example o one type o system conguration or remote IR camera monitoring. Thesystem uses a digital alarm output or annunciating an over-temperature condition and transmitsstreaming MPEG-4 compressed video that allows the scene to be viewed on a PC monitor.


16/68

10

Chapter

image masking. This enables the user

to pre-select specic areas of interestor analysis o temperature data. Thisis illustrated in Figure 5, which showscontinuous monitoring o substationhotspots that indicate problem areas.

A similar type of pattern recognitionsotware can be used or automatedinspection in metal soldering andwelding and in laser welding oplastic parts. IR cameras can see heatconducting through the nished partsto check the temperature o the areaswhere parts are joined together againsta stored value. In addition, the softwarecan learn a weld path to make sure thispath is correct, which is accomplished

by programming the specic pixels inan image to be used by the sotware orthis purpose. Alternatively, the program

developer can save an image of aperfect part and then have the softwarelook for minimum, maximum, or deltavalues that tells the equipment operatori a part passes inspection. The car seatheater inspection described in Chapter 1can be an example of this, and the sameprinciple is used in the inspection o carwindow heater elements by applyingpower to them and looking at theirthermographic image.

Power over Ethernet. It should be notedthat a camera with Ethernet connectivitycan be powered from a variety of sources,depending on its design. Typically, aconnection or an external DC supply isused, or where available, the camera is

powered via PoE (Power over Ethernet).PoE uses a power supply connected tothe network with spare signal leads not

Figure 5. Masking unctionality o the FLIR A320 IR camera, which is also available in some thirdparty sotware programs.


17/68

11


otherwise used in 10/100baseT Ethernetsystems. Various PoE congurations arepossible. Figure 6 depicts one in whichthe power source is located at one endof the network. (Gigabit Ethernet uses allavailable data pairs, so PoE is not possiblewith these systems.)

PoE eliminates the need or a separate

power source and conduit run oreach camera on the network. Theonly additional cost is or some minorelectrical hardware associated with PoE.

Many applications encompass areas thatexceed the maximum Ethernet cablerun of 100m. In those cases, there arewireless and beroptic converter options

that provide o-the-shelf solutionsfor communicating over much greaterdistances. These are requently used inthe bulk material storage applicationsdescribed below.

Additional Asset Monitoring Situations

Bulk Material Storage. Many bulkmaterials are stored in open yards whereair and moisture can help promotedecomposition and other exothermicreactions that raise the temperature othe pile. This brings with it the threatof re, direct monetary loss, and safetyissues for personnel. In addition, thereis the risk o consequential damagescaused by res, including loss of nearbyproperty, water damage resulting fromre-ghting, and production shutdowns.Materials that are especially prone tospontaneous combustion include organicwastes (compost, etc.), scrap paperfor recycling, wood, coal, and variousinorganic chemicals, such as cement and

chlorine hydrates. Even in the absenceof spontaneous combustion, manybulk materials like plastics pose a rehazard due to sparks or other externalignition sources.

1

5

Spare Pair

Signal Pair

Signal Pair

4

2TX+48V

RX

DC/DCConverter3

6

1

5

4

2

3

6RX TX

5

Spare Pair

4

5

4

Power SourcingEquipment (PSE)

PoweredDevice (PD)

Figure 6. Schematic depicting spare-pair PoE delivery using the endpoint PSE arrangement.


18/68

1

Chapter

In most cases, prevention is less costlythan a cure, and the best prevention iscontinuous monitoring o the materials.The cost o an automated temperaturemonitoring system using IR cameras isa modest and worthwhile investment.System design can take the same orm asthe one described earlier for hazardouswaste barrels. Cameras are conguredto generate a direct alarm output to anoperator when user-dened maximumtemperature thresholds are exceeded.Audible and visual alarms in a controlroom draw the operators attention to apossible spontaneous re development.Various types of software have beendeveloped to isolate trouble spots, suchas the waste pile zone monitoring systemdepicted in Figure 7.

Although self-ignition usually startswithin the bottom layers of a stock pile,continuous monitoring o the suracereveals hot spots at an early stage (Figure8), so measures can be taken to preventa major re from breaking out. Largestorage yards generally require multiplecameras for total coverage, with thecameras mounted on metal masts abovethe stock piles. This calls or cameras withhousings and other eatures designed

or reliable operation in harsh industrialenvironments.

Critical Vessel Monitoring (CVM).Thereare several applications where thetemperature of a vessel and its contents

are critical. The vessels could be usedfor chemical reactions, liquid heating,or merely storage. For large vessels,the use o contact temperature sensorsposes problems. One reason could benon-uniform temperatures throughout avessel and across its surface. This wouldrequire a large number o contact typesensors, whose installations can become

quite costly.

For most CVM applications, a few IRcameras can image nearly 100% of avessels surface (Figure 9). Moreover, theycan measure the surace temperature othe CVM to trend and predict when theinternal reractory will break down andcompromise the mechanical integrity o

the system. If specic regions of interest(ROIs) must be focused on, IR camerarmware (or external PC software) allowsthe selection o spot temperature pointsor areas or measurement.

Again, some variation of the systemsdescribed earlier can be used. Depending

Figure 7. Control room or waste pile processing, and screen capture o the zone monitoring layout,which uses a FLIR IR camera on a pan-tilt mount or re hazard warning.


19/68

13


Figure 8. Visible light and IR images o a coal pile the thermographic image clearly identies a hotspot that is a re about to erupt.

on the application environment, anexplosion proo housing or the cameramay be a requirement. HMI (human-machine interface) software, such as

SCADACAM iAlert from Pivotal Vision,can be used to provide a monitoringoverview. This has the ability to combineall o the camera images into a singlespatial representation o the monitoredarea in this case, a attened-out viewof the vessel. This view can be updatedcontinuously for a near-real-timethermographic representation.

Electrical Substation Monitoring. Reliableoperation o substations is crucial oruninterrupted electrical service. Besideslightning strikes and large overloads,aging equipment and connections area major cause o inrastructure ailuresand service interruptions. Many of thesefailures can be avoided with eectivepreventative maintenance monitoring.Often, the temperatures of transformers,breakers, connections, etc. will begin tocreep up beore a catastrophic ailureoccurs. Detection o these temperatureincreases with IR cameras allowspreventative maintenance operations

Figure 9. CVM monitoring example showingcamera locations, network connections, and PC.

1 Computer2 CAT-6 Ethernet cable with RJ45

connectors

3 Industrial Ethernet switch with PoE4 ThermoVision A320 cameras5 Industrial process to be monitored,

e.g., a gasier


20/68

14

Chapter

beore an unplanned outage happens.(See Figure 10.)

The cameras can be installed on a pan/tilt mounting mechanism to continuallysurvey large areas of a substation (Figure11). A few cameras can provide real-timecoverage of all the critical equipmentthat should be monitored. In additionto preventative maintenance functions,these cameras also serve as securitymonitors or intrusion detection around

the clock.

By combining the cameras Ethernetand/or wireless connectivity with aweb-enabled operator interface, liveimages can be transmitted to utilitycontrol rooms miles away. In addition,trending sotware can be used to detectdangerous temperature excursions and

notify maintenance personnel via emailand snapshot images o the aectedequipment.

These eatures and unctions are alreadyin place at leading utility companies inthe U.S., such as Exel Energys Substationo the Future. Companies such as

Exel consider IR monitoring a strategicinvestment in automation, which ispart of a common SCADA (Supervisory

Control And Data Acquisition) platformor maintenance and security operations.The most advanced systems providetime-stamped 3-D thermal modelingof critical equipment and areas, plustemperature trending and analysis. Acompany-wide system of alerts providesalarms on high, low, dierential, andambient temperatures within or between

zones in real time.

The previous examples represent just afew applications that can benet fromremote IR camera monitoring. A fewother applications where IR temperaturemonitoring is being used include:

Oil and gas industries (explorationrigs, reneries, are gas ues, naturalgas processing, pipelines, and storagefacilities)

Electric utilities (power generationplants, distribution lines, substations,and transformers)

Figure 10. Visible light and IR images o a substation showing a transormer with excessivetemperature.


21/68

1


Figure 11. Example pan/tilt mounting system.

Smarter surveillance for a smarter gridMeet ScadaCam Intelligent Surveillance, the only system in its price rangethat can automatically perform site patrols, monitor equipment temperature,

and scan for security breaches without human supervision.

By combining visual, thermal imaging, and thermographic cameras into a

multifunctional operations and security automation tool, ScadaCam candetect, validate, and alarm you of problems that could otherwise result in a

major outage before they occur.

See it in action at www.pivotal-vision.com/tryit


22/68

16

Chapter

Predictive and preventativemaintenance (continuous/xedposition monitoring o criticalequipment)

Besides these, there are many qualitativeremote monitoring applications whereimaging is the predominant eature. Forexample, IR cameras can be used as parto an early warning system or orest

res (Figure 12), detecting blazes beforesignicant amounts of smoke appear.Another example is using IR imaging tolook through condensation vapor thatwould otherwise obscure an operatorsview of equipment and processes. This isbeing used in coking plants, veneer mills,and plywood log handling operations,among others (see Chapter 1, Figure 1).

Summary

As noted in the text, IR cameratemperature data may be used or

qualitative monitoring or for quantitativetemperature measurement and control.

In the former, thermal images areobtained and interpreted based on

temperature contrast. It can be used

to identiy image areas that correlate

to sub-surface details, liquid levels,refractory, etc.

Quantitative measurements generallyrequire the IR camera to accurately

determine the temperature dierence

between the target object and its

surroundings. In remote monitoring, thisallows the temperature data to be used

for alarm purposes or to even shut downequipment. Since temperature changes

slowly in many situations, the near-real-time data communications o smart IR

cameras are more than adequate or

alarm and control systems.

Figure 12. Ngaros IRIS Watchman orest re early warning system uses a FLIR IR camera.


23/68

17

Temperature Measurementfor Automated Processes

Temperature Measurementfor Automated Processes

Background

In Chapter the emphasis was onspecic applications where a singletemperature threshold is programmedinto an IR camera, and when the

threshold is reached an alarm istriggered through a PLC. Multiplecameras are often required, but viewingan IR cameras thermographic imageis a secondary consideration toverify an alarm condition. Chapter 3ocuses on applications where multipletemperatures within a single camerasFOV are important, and that information

is used or some sort o process controlfunction. In these applications, thecamera is typically integrated with otherprocess control elements, such as a PC orPLC using third party sotware and moresophisticated communication schemes.

Typical Camera MeasurementFunctions

Many IR cameras provide the user withdierent operating modes that supportcorrect temperature measurementsunder various application conditions.Typical measurement unctions include:

Spotmeter

Area

Image mask

Delta T

Isotherm

Temperature range

Color or gray scale settings

The last two are used with the others to

provide a visual indication of the rangeof temperatures in the cameras FOV.Generally, spot and area temperaturestend to be the most useul in monitoringand control applications, and mostcameras allow multiple spots or areas tobe set within the thermographic image.For example, the FLIR A320 camerasupports up to our spots and our areas.

Cursor unctions allow easy selection oan area of interest, such as the crosshairsof the spot readings in Figure 1. Inaddition, the cursor may be able to selectcirclular, square, and irregularly shapedpolygon areas.

Figure 1. IR image o a printed circuit board

indicating three spot temperature readings.Image colors correspond to the temperaturescale on the right.

The spotmeter nds the temperatureat a particular point. The area unctionisolates a selected area o an object orscene and may provide the maximum,minimum, and average temperaturesinside that area. The temperaturemeasurement range typically isselectable by the user. This is a valuableeature when a scene has a temperaturerange narrower than a cameras full-scale range. Setting a narrower rangeallows better resolution o the imagesand higher accuracy in the measured

Chapter 3


24/68

18

Chapter 3

Figure 2. Gray scale images o a car engine the let view has white as the hottest temperature andthe right view shows black as the hottest.

temperatures. Therefore, images willbetter illustrate smaller temperaturedierences. On the other hand, abroader scale and/or higher maximum

temperature range may be needed toprevent saturation of the portion of theimage at the highest temperature.

As an adjunct to the temperature rangeselection, most cameras allow a userto set up a color scale or gray scale tooptimize the camera image. Figure 2illustrates two gray scale possibilities.

In Figure 1, a so-called iron scale wasused or a color rendering. In a mannersimilar to the gray scale above, thehottest temperatures can be renderedas either lighter colors or darker colors.Another possibility is rendering imageswith what is known as a rainbow scale

(Figure 3).

While choice o color scale is oten amatter of personal preference, there maybe times when one type o scale is betterthan another or illustrating the range otemperatures in a scene.

Figure 3. Rainbow scale showing lowertemperatures towards the blue end o thespectrum.

Application Examples

Go/No-Go. In these applications, one ormore temperatures are monitored to

make sure they meet process criteria,and machinery is shut down or product

rejected when a measured temperaturegoes above or below the setpoint. Agood example o this is a manuacturer

of automotive door panels that usesIR cameras to monitor and measure

part temperatures prior to a molding

procedure.


25/68

19

Temperature Measurement for Automated Processes

This process starts with reinorcing partsthat have been stored in a warehouse.

In either the warehouse or duringtransport to the molding line, theseparts can become wet due to moisturecondensation or exposure to inclementweather. If that happens, they may notreach a high enough temperature in themolding press and nished panels will beo poor quality.

The parts go into the press two at atime from a conveyor where they aresealed together and the nished doorpanel is molded into the required shapefor a specic car model. If the parts arewet, this creates steam in the press andcauses mold temperature to be too low.

However, it was found that movement ofwet parts on the conveyor causes their

temperature to be lower than normal. So,just before the parts go into the press,the conveyor stops and an IR cameramakes a non-contact measuremento their temperature. The diagram in

Figure 4 is typical for this type of qualitycontrol application.

The IR cameras area tools are applied tothe thermographic image to check orthe minimum allowable temperature o

the two parts. I either temperature is

below the setpoint (typically, the ambienttemperature), then a digital I/O output toa PLC causes an alarm to be sounded and

Figure 4. Typical Go/No-Go QC inspection system using IR cameras.

1 Computer or PLC

2 CAT-6 Ethernet cable withRJ45 connectors

3 Industrial Ethernet switcheswith ber optic ports

4 Fiber optic cable

5 ThermoVision A320 or A325cameras

6 Industrial process to bemonitored, e.g., items on aconveyor belt


26/68

0

Chapter 3

the molding line is halted so the parts can

be removed.

For OEMs, preventing bad panels fromgetting to the end product avoids apotential loss o business. Warrantyreplacement o a door panel ater an endcustomer takes possession o the car is anexpensive proposition for the OEM.

The trick is to make sure the camera ismeasuring the temperature o the partsand not the oor beneath the conveyor,which is within the cameras FOV andtypically much cooler. This occurs whenthe parts are not in the proper position. Aphotoelectric detector tells the PLC whenthe parts enter the press area; otherwiseits ladder logic ignores the alarm outputrom the camera.

Continuous Process Monitoring.Temperature is an important variablein many processes. It can either be anintegral part o a process or act as aproxy or something else. The ollowingdescribes an example that encompassesboth o these situations.

Articial ber production typicallyinvolves a continuous extrusion process.Multiple strands may be extrudedsimultaneously or, in the case of non-woven sheets, a web process may beinvolved. In either case, monitoringthe temperature o the material as itcomes out o the extruder can detectstrand breakage or material blockageand backup in the process. Using an IRcamera or unattended monitoring cancatch these malfunctions early, beforea huge mess is created that causesa long machinery outage and costlyproduction losses. In addition, the actual

temperature readings can be used ortrend analysis.

Depending on the application, either thespot or area measurement unctions othe camera can be used. In the latter case,it is likely that the application would takeadvantage of all the area measurementcapabilities minimum, maximum, andaverage temperatures of the dened

area. I any o these were to all outsidethe user-dened limits, the applicationprogram running on a PC or PLCcould instantly shut down the processmachinery.

In one such application, FLIRcustomized the camera rmware toallow simultaneous monitoring o up

to 10 dierent areas. Figure 5 shows amonitored area covering six ber strandscoming out of the extruder, along with analarm setpoint temperature in the upperlet corner.

Figure 5. Monitoring o articial bers comingout o an extruder.

As in the case of many remote monitoringapplications, the user may choose toroute the cameras analog video to acontrol room monitor. For cameraswith an Ethernet connection, digitally


27/68

1


compressed (MPEG-4) streaming videocan be available for monitoring on a PCscreen. With FLIRs A320 camera, imagesand alarms can be sent to a remote PC viaTCP/IP and SMTP (email) protocols.

While a visible light camera may be ableto detect broken ber strands, an IRcamera can also provide temperaturemeasurements or trending and statistical

process control (SPC) purposes. Inaddition, some textile processes createsteam or condensation vapors that avisible light camera cannot see through,but an IR camera can. Thus, an IR cameraprovides multiple functions and is morecost eective.

Data Communications and

Sotware Considerations

Dierent cameras have dierent videoframe rates. The frame rate governs howrequently the thermographic imageand its temperature data are updated.A typical rate might be every 200ms orso. The cameras digital communicationsprotocol could create a small amount o

additional latency in the update process.Still, because process temperatures tendto change slowly, collecting temperaturedata at this rate provides a wealth ofinormation or quality control purposes.

In many IR cameras there is some sorto serial/socket interace that can beused or communications with the PCor PLC that is running a control script,or application. When a system designeror user is most familiar with PLCs, thecontrol algorithm can be built arounda virtual PLC created on a PC, whichemulates actual PLC hardware and logic.In any case, a human-machine interface(HMI) is created to monitor data coming

rom the camera. The details described

below are based on FLIRs A320 camera,but should be representative of mostcameras that transmit data over anEthernet link.

The only physical interace or digitaldata transfer from the FLIR A320 is theEthernet port. Only TCP/IP is supported,but the camera should work seamlessly

on any LAN when the proper IP address,netmask, and possibly a gateway is setup in the camera. The two main wayso controlling the camera are throughthe command control interace and theresource control interace. Digital imagestreaming, data le transfer, and otherfunctionality is provided through theIP services interface. A lot of software

unctionality is exposed throughsotware resources. These resourcescan be reached through the FLIR IPResource Socket Service. This is thecameras resource control (serial/socket)interace. Independent o the physicalEthernet interface, it is possible to accessthe camera system using TCP/IP withtelnet, ftp, http, and FLIR Resource Socket

Services (among others).Most PLCs provide serial/socket interfacesfor Ethernet. One example is Allen-Bradleys EtherNet/IP Web Server Module(EWEB for short). Another example is HMSIndustrial Networks Anybus X-GatewayEthernet interface module, which canconvert this serial socket interface to

many industrial network protocols, suchas EtherNet I/P, Modbus-TCP, Pronet,Ethernet Powerlink, EtherCAT, FLNet., etc.

Camera setup and data acquisition isnormally done directly through the FLIR IRMONITOR and IP CONFIG software runningon a PC. Afterward, the camera can be


28/68

Chapter 3

connected on the network for continuousmonitoring and data logging via PC orPLC control. Typically, the telnet protocol,accessed by the Windows PC runningthe application program, is used to querythe camera for data. This protocol is alsoavailable for most PLCs. In either case, thistakes place though the cameras ResourceSocket Services. (Command syntax iscontained in the cameras ICD manual; afew examples are listed in Appendix D.)

The system designer or FLIR wouldcreate the message instructions thatallow the PLC to query the camera ortemperature data and thermographic

images in the same way it is done with PCcontrol. Alternatively, the PLC can holdthe Ethernet port open and call or thecamera to continuously output data tothis port at the maximum rate possible. Ineither case, alarm functions and decision-making is perormed by the applicationprogram running on the PLC (or PC if

applicable). (See Figure 6.) Typically,temperatures and images collected ortrend analysis and statistical processcontrol purposes are stored on a separateserver connected to the network, whichis running transaction manager sotwareor downloading and storing data.

1 Computer, PLC, and/or transaction managerserver

2 CAT-6 Ethernet cablewith RJ45 connectors

3 Industrial Ethernetswitches with ber optic

ports4 Fiber optic cable

5 Wireless access points

6 CAT-6 Ethernet cablewith RJ45 connectors.Powering the camerausing PoE (Power overEthernet)

7 Industrial Ethernetswitch

8 ThermoVision A320cameras monitoring aprocess or other targetobjects

Figure 6. Generalized IR machine vision system and its communications network


29/68

3


Figure 7. Example o a control and data acquisition option or IR cameras


30/68

4

Chapter 3

For system developers who are writingor modifying code with Visual Basic, C++,etc. for customized applications runningon a PC, there are a few options. FLIRsResearcher package supports OLE-2,the Microsot standard or linking andembedding data between applications.Image and temperature data can belinked rom Researcher into othercompliant applications, such as Excel. Thelinked data updates automatically, so if atemperature value changes in Researcherit will automatically change in the linkedapplication. In addition, Researcherprovides an automation interface thatcan be used to control the sotware usingVisual Basic or VBA. Other o-the-shelfoptions for OLE control include NationalInstruments MATLAB and LabVIEW.However, none of the aforementioned areOPC (OLE for Process Control) compatible.

There are other out-of-the-box solutionsthat do not require the writing oapplication source code. One of these isIRControl from Automation Technology,GmbH. IRControl simplies automated

processing o complex tasks with its

built in Automation Interface based onMicrosot COM/DCOM. All essentialmeasurement, analysis, and controlunctions or FLIR IR cameras are directlyprogrammable using macro commands.This allows the execution o controlscripts automatically based on digitalinput events. In addition, IRControlaccepts remote control commands sent

over an RS-232 link. Therefore, remotecontrol o IRControl by other computersor PLCs is greatly simplied. The softwarealso includes a comprehensive reportgenerator.

Summary

A variety of control and data acquisition

options are available for IR cameras(see Figure 7). They are similar to thoseused with visible light cameras thatare employed in machine vision andautomation systems. IR cameras providethe added advantage of accurate non-contact temperature measurementswithin a single instrument.


31/68

Combining Machine Vision andTemperature Measurement

Combining MachineVision and TemperatureMeasurement

Background

Traditionally, visible light cameras havebeen a mainstay in machine visionsystems used or automated inspection

and process control. Many o thesesystems also require temperaturemeasurements to assure product quality.In numerous cases, an IR camera cansupply both an image o the productand critical temperature data. I theapplication will not benet fromthermographic images and non-contacttemperature measurements, then a

visible light camera is certainly lessexpensive. If the opposite is true, then anIR camera should be considered by thesystem designer.

As the sophistication of IR camerascontinues to increase, along withassociated hardware and software, theiruse in automated systems is growingrapidly. Because o their combinedimaging and temperature measurementcapabilities, they can be very costeective. The main impediment to theirwider usage is system designers lacko amiliarity with IR camera eaturesand the related standards, systems, andsotware that support them. This chaptersupplies a good deal o that inormation.

Machine Vision Applications

As in the case of visible light cameras,thermographic cameras and theirassociated software can recognize thesize, shape, and relative location oftarget objects (i.e., they can do pattern

matching). Moreover, the electronicsin newer IR cameras provide fast signalprocessing that allows high video framesrates (60Hz or higher) to capture relativelyfast-moving parts on a production line.Their A/D converters combine shortintegration times with 14- to 16-bitresolution, which is critical for properlycharacterizing moving targets or targetswhose temperatures change rapidly.

Figure 1. Results o automated inspection o ICson a circuit board

One example of the latter is automatedinspection o operating ICs on a circuitboard (Figure 1). In some cases, thisinvolves overload testing in which anIC is subjected to a current pulse so itsheat loading can be characterized. In onesuch case the IC is forward and reversebiased with current levels outside ofdesign limits using a pulse that lasts

800ms. The IR camera captures imagesduring and ater the current pulse tocharacterize temperature rise and fall.With a 60Hz frame rate, a new frame canbe captured about every 17ms. In such asystem nearly 50 frames can be capturedduring the 800ms pulse, and many more

Chapter 4


32/68

6

Chapter 4

are typically captured afterward to revealheat dissipation characteristics.

In other applications of this sort, a goodimage can be stored and compared tothe inspection image by using pixel-by-pixel subtraction. Ideally, the resultingimage would be entirely black, indicatingno dierence and a good part. Areaswith excessive temperature dierences

indicate a bad part, making it very easy todiscern unwanted dierences.

There are many other applicationswhere the combination of non-contacttemperature measurements and imagingat high frame rates is extremely valuable.Some automated systems where IRcameras are already being used include:

Automotive part production andassembly lines

Steel mill operations, such as slagmonitoring and ladle inspection

Casting, soldering, and welding ofmetals and plastics

Food processing lines

Product packaging

Non-destructive testing, like sub-surface detection of voids in moldedparts

Electric utility equipment monitoring

R&D, prototyping, and production inthe electronics industry

An interesting automotive example ismonitoring the temperature distribution

of a pressure casting mold for a safety-critical part (Figure 2). Prior to installationof the IR machine vision system, themanufacturer was doing 100% inspectionusing an X-ray system to revealsubsurace imperections. It was notpractical to do this as an inline procedure,

so the X-rays were taken a few hours afterpart production. If the X-rays showed asignicant problem in parts coming froma particular mold, this information wasrelayed to the production area so thatmold temperatures could be adjusted.This was a lengthy and costly process thatoten resulted in high scrap rates. Withthe IR camera system, the mold operatorcan immediately check and adjust thetemperature distribution o the mold.

Figure 2. Pressure casting mold and itstemperature distribution an IR cameraimage is used by the operator to adjust themold temperatures as required to producegood parts.

Enabling Technology

Data communications are the backboneof modern industrial SCADA, PLC, HMIs,and Machine Vision systems. Ethernet hasbecome the de acto standard or suchsystems. Considering this, the features of


33/68

7

Combining Machine Vision and Temperature Measurement

IR cameras that make or practical use in

machine vision applications are GigabitEthernet (GigE) connectivity, GigE Visioncompliance, a GenICam interface, anda wide range o third party sotwarethat supports these cameras. There areother hardware eatures that are alsoimportant.

Generally, ultra-high detector resolutions

are not needed in the targetedapplications, so a typical focal planearray (FPA) would be 320x240 pixels.Nevertheless, outputting a 16-bit imagestream of these 76,800 pixels at a 60Hzframe rate amounts to about 74Mb/sec. While this is much slower than a1000baseT Ethernet system is capable of,multiple cameras may be connected and

there may be a lot of other trac on thenetwork between image transmissions.

To speed up image transfers, dataanalysis and decision-making must takeplace outside the camera and is one othe reasons why there is a good marketfor third-party thermographic software.The other reason is that most machine

vision systems are custom designedfor specic production processes. Ofcourse, IR camera manufacturers supplyvarious types of software to support theirproducts and acilitate application inthese systems.

The goal of the GigE Vision technicalstandard is to provide a version of GigEthat meets the requirements o the

machine vision industry. One of theindustry objectives is the ability to mixand match components from variousmanuacturers that meet the standard.Another is relatively inexpensiveaccessories, such as cabling, switches,and network interface cards (NICs) as well

as the ability to use relatively long cableruns where required.

The GigE Vision standard, which is basedon UDP/IP, has four main elements:

A mechanism that allows theapplication to detect and enumeratedevices and denes how the devicesobtain a valid IP address.

GigE Vision Control Protocol (GVCP) that allows the conguration ofdetected devices and guaranteestransmission reliability.

GigE Vision Streaming Protocol (GVSP)that allows applications to receiveinformation from devices.

Bootstrap registers that describe thedevice itself (current IP address, serialnumber, manufacturer, etc.).

With GigE capabilities and appropriatesoftware, an IR machine vision systemdoes not require a separate ramegrabber, which was typically the casewith visible light cameras in the past.In eect, the GigE port on the PC is theframe grabber. Older visible light cameras

that have only analog video outputs(NTSC and PAL) are limited to muchlower frame rates and video monitorobservations. By using GigE, an IR visionsystem not only has higher frame rates,but can be monitored remotely overmuch greater distances compared tolocal processing and transmitting dataover USB, Firewire, CameraLink, etc.

In addition, Ethernet components areinexpensive compared to frame-grabbercards and related hardware.

A GigE Vision camera typically usesan NIC, and multiple cameras can beconnected on the network. However, thedrivers supplied by NIC manufacturers


34/68

8

Chapter 4

use the Windows or Linux IP stack, whichmay lead to unpredictable behavior,such as data transmission delays. Byusing more ecient dedicated driverscompatible with the GigE Visionstandard, the IP stack can be bypassedand data streamed directly to memory atthe kernel level of the PC system. In otherwords, Direct Memory Access (DMA)transfers are negotiated, which alsoeliminates most CPU intervention. Thus anear-real-time IR vision system is createdin which almost all of the CPU time isdedicated to processing images.

To make sure a camera is GigE Visioncompliant, look for the ocial stamp(shown in Figure 3) that can only beapplied i the camera conorms to the

standard.

Figure 3. Ocial trademark or GigE compliantproducts

GenICam compliance should alsobe considered or an IR camera.GenICam compliance makes it easierfor developers to integrate camerasinto their IR vision system. The goal ofthe GenICam standard is to providea generic programming interace orall kinds of cameras. No matter whatinterface technology (GigE Vision,Camera Link, 1394, etc.) is used, or whatcamera features are being implemented,the application programming interace(API) should be the same. The GenICamstandard consists o multiple modulesand the main tasks each perorms are:

GenApi: conguring the camera

Standard Feature Names:recommended names and types orcommon eatures

GenTL: transport layer interface,grabbing images

The GenApi and Standard FeatureNames modules are currently part of the

standard module only. GenTL should benished soon.

Common tasks associated with IRcameras in machine vision systemsinclude conguration settings, commandand control, processing the image, andappending temperature measurementresults to the image data stream. Inaddition, the cameras digital I/O canbe used to control other hardware, andthere are triggering and synchronizationfunctions associated with real-time dataacquisition. GigE Vision makes hardwareindependence possible, while GenICamcreates sotware independence. Forexample, in a system with IR camerascompliant in both and connected to aGigE network, virtually any applicationprogram can command a camera tosend a 60Hz stream of images that canbe easily captured without droppingrames and losing important data. Thisinormation can be processed or alarmfunctions, trend analysis and statisticalprocess control.

Third Party Sotware ExpandsApplications

By adhering to the standards describedabove, IR camera manufacturers aremaking it easier for developers tointegrate their cameras into visionsystems with a broad array o unctions


35/68

9


(Figure 4). Camera manufacturers alsosupply a variety of software products to

ease integration tasks. For example, theFLIR A325 comes with three packages thatrun on a PC controller:

IP Conguration utility nds camerason the network and congures them

IR Monitor displays images andtemperature data on up to nine

cameras simultaneously

AXXX Control and Image interface low-level descriptions of how tocommunicate with the camera,including image formats and C-codeexamples

In addition, optional software developertoolkits are available (FLIR SDK, LabVIEW

SDK, Active GigE SDK from A&B Software,etc.) for those creating source code forcustom applications within programmingenvironments such as Visual Basic, C++,Delphi, etc. However, the strength ofa camera like the A325 is its ability tointerace with third party sotware thateliminates or minimizes the need towrite source code. For example, National

Instruments Vision Builder for AutomatedInspection is a congurable package forbuilding, benchmarking, and deployingmachine vision applications (Figure5). It does not require the user to writeprogram code. A built-in deploymentinterace acilitates system installation

Figure 4. IR cameras can be used in a broad array o applications


36/68

30

Chapter 4

and includes the ability to denecomplex pass/fail decisions, controldigital I/O, and communicate with serialor Ethernet devices, such as PLCs, PCs,and HMIs. Similar features are available inCommon Vision Blox, a Stemmer Imagingproduct that contains hardware- and

language-independent tools and librariesor imaging proessionals.

By using third party sotware to get muchof the analytics, command, and controlunctions out o the camera and onto aPC, application possibilities are greatlyexpanded. One possibility is creating amixed camera system. For instance, IR

cameras could be used to supply thermalimages and temperature data, whilevisible light cameras could provide whitelight color recognition.

The ood processing industry is one inwhich higher level analytics are used withIR cameras for automated machine visionapplications. A broad area of applicationswhere IR vision systems excel is in 100%inspection o cooked ood items comingout of a continuous conveyor oven. Aprimary concern is making sure theitems have been thoroughly cooked,which can be determined by havingthe camera measure their temperature,which is illustrated in Figure 6 for

hamburger patties. This can be done by

dening measurement spots or areascorresponding to the locations o burgersas they exit the oven. If the temperatureof a burger is too low, the machinevision program logic not only providesan alarm, but also displays an image tothe oven operator to show the specicburger that should be removed from theline. As in other applications, minimum,maximum, and average temperatures canbe collected for specic burgers or theFOV as a whole and used for trending andSPC purposes.

Figure 6. IR machine vision image orchecking hamburger doneness by measuring

temperature

In another example involving chickentenders, temperature is again used tocheck or proper cooking. The piecescome out of the oven and drop ontoanother conveyor in more or less randomlocations (Figure 7). The operator canuse the thermographic image to locate

undercooked items within the randomlyspaced parts and then remove them fromthe conveyor.

In the production of frozen entres,IR machine vision can use patternrecognition sotware to check or properlling of food tray compartments.

Find any number ofedges

Set up coordinatesystems

Find and matchpatterns

Acquire with IEEE 1394and GigE cameras

Detect and measureobjects

Calibrate measurementsto real-world units

Perform advancedgeometric analysis

Find and measurestraight edges

Find circular edges

Make caliper distancemeasurements

Make pass/faildecisions

Read text (OCR)

Communicate withexternal devices suchas PLCs

Measure intensity

Read 1D and 2Dbar codes

Figure 3. Examples o the many unctionsavailable in Vision Builder or automated

inspection


37/68

31


Similarly, it can be used for 100%inspection of the heat-sealed cellophane

cover over the nished entre. An addedunction could be laser marking o abad item so it can be removed at theinspection station.

Summary

IR machine vision and temperaturemeasurements can be applied to an

innite number of automated processes.In many cases, they provide imagesand information that are not availablewith visible light cameras, and theyalso complement white light images

where the latter are required. IR cameras

like the FLIR A325 provide a stream ofdigitized IR images at fast frame rates for

relatively high-speed processes, whichcan be transmitted over GigE networks toremote locations. Compliance with GigEVision and GenICam standards meansthat such cameras can be integrated

with a wide variety of similarly compliantequipment and supported by a broad

range o third party sotware. Trigger andsynchronization capabilities allow themto control, or be controlled by, a host ofother types of equipment. The availabilityof wireless and beroptic line adaptersallow these cameras to be used almost

anywhere, including over long distances.

Figure 7. An IR temperature measurementand thermographic image are used to locateundercooked chicken tenders and stop the lineso bad parts can be removed.


38/68

3

Chapter 5

Real-Time Control IssuesBackgroundReal-time control is an important issue inmost IR machine vision systems used forautomated temperature monitoring andinspection. Having said that, it should benoted that real time tends to be a relativeterm, the measure of which varies with

the application and user requirements. Insome applications, users would considera response time of 100 milliseconds tomeet their denition of real-time. Onthe other hand, many electronic eventsare extremely fast or short-lived, and aone-microsecond response might beneeded. As mentioned in earlier chapters,process temperatures tend to change

relatively slowly, so an IR machine visionsystem that can update images andtemperatures every 10-100ms, or evenless frequently, may be adequate.

Hardware and Sotware PlatormConsiderations

In most cases, a PC with a MicrosoftWindows operating system (OS) isntwell suited for controlling fast, real-timeapplications. Windows is reerred to as anon-deterministic OS because it typicallycannot provide predictable responsetimes in critical measurement and control

situations. Therefore, the solution is tolink the PC to a system that can operateautonomously and provide rapid,predictable responses to external stimuli.

Deterministic applications (thoseintended to be event driven) arecontrolled better with systems basedon an embedded microprocessor and/

or digital signal processor (DSP) thathas a dierent type of OS or perhapsa special version of Windows other thanthe ones typically ound on a home or

Figure 1. PLCs are a good choice or creating deterministic (event-driven) systems, supported by aPC that is used or data trending.


39/68

33

Real-Time Control Issues

oce PC. Often, a system based on a PLCwith 115VAC control I/O is much moreappropriate for real-time applications(Figure 1). PLC processors are designedto operate with deterministic controlloops, and the 115VAC control signals areinherently immune to noisy industrialenvironments.

I the required response time is long

enough, and there are other reasonsto use one o the amiliar Windowsoperating systems, keep in mind thatspecial steps are needed to improve itsdata polling methodology. In a polledsystem, the PC checks many devices tosee if theyre ready to send or receivedata. In the context o data acquisitionfrom an Ethernet-based IR camera, this

typically involves reading values froma data stream. In a Windows-basedPC, the time between polled readingsis scheduled by Windows, so its non-deterministic. In other words, the time atwhich Windows will initiate an operationcannot be known precisely. Its operationdepends on any number o systemfactors, such as computer speed, the OS,

programming languages, and applicationcode optimization.

Polling can be appropriate with slower,less time-sensitive operations. In contrast,event-driven programming schemes areless dependent on OS timing and tendto reduce latency problems. They canbe used to create more deterministic

systems that collect discrete data valuesthat are closely related to the physicalphenomena being represented.

Creating such a system within aWindows OS environment generallyrequires writing program code usingVisual C/C++ , Visual Basic, etc. Using

these tools, a programmer can takeadvantage of Windows events andmessaging unctionality to create amore deterministic application thatruns relatively fast and provides tightcontrol. Rather than constantly polling todetermine if data is ready for collection,such programs can use the PCs CPUfor additional tasks, such as databaseor network access, until interruptedby the automation system hardware.As discussed in Chapter 4, there aresoftware developer kits that take someof the work out of these tasks, and thirdparty sotware packages can eliminateor minimize the need to write programcode. An example is illustrated in Figure 2.

Figure 2. Third party sotware providespowerul control and analytic tools or IRmachine vision systems without writingprogram code.

Data Communication Latencies

Hardware and data communications

have signicant eects on systemresponse time. The Ethernet interaces onmany IR cameras allow communicationdistances of 4000 feet or more. Wirelessand beroptic adapters and hubs canextend the scope of the network evenmore. Networked systems require the


40/68

34

Chapter 5

installation of one or more NICs in thePC and conguring its OS for networksupport. These requirements are easilyand economically met with Ethernet,TCP/IP, and Windows, as described inChapter 3.

A functional drawback of Ethernet-basedsystems concerns real-time control. LikeWindows, Ethernet is a non-deterministicsystem that in many applicationsprecludes fast, real-time processcontrol. This can become even more ofan issue when the World Wide Web isinvolved. Again, there are work-aroundsto minimize inherent weaknesses. As

mentioned in Chapter 4, drivers suppliedby NIC manufacturers use the Windowsor Linux IP stack, which may result in datatransmission delays. By using dedicateddrivers compatible with the GigE Visionstandard, data can be streamed directlyto memory using a DMA transfer.

Since older communication protocols

(RS-232, 422, 485, etc.) are even slower,Ethernet is still the protocol o choicein most IR machine vision systems. Thedigitized streaming video from FLIRsA325 camera allows near-real-timedata acquisition o thermal imagesand temperature data provided the

1 Computer, PLC, and/or transaction managerserver

2 CAT-6 Ethernet cablewith RJ45 connectors

3 Industrial Ethernetswitches with ber opticports

4 Fiber optic cable

5 Wireless access points

6 Industrial Ethernetswitch

7 ThermoVision A320 or

A325 cameras monitoringa process or other targetobjects

Figure 3. Generalized IR machine vision system and its communications network


41/68

3


PC has the appropriate NIC driver andapplication program. Network adaptersfor beroptic and wireless connectivitycan extend Ethernets scope (Figure 3).Other hardware timing issues can beminimized by using direct-wired digitalI/O and triggering between individualcameras, PLCs, etc. Analog video(NTSC and PAL) for conventional imagemonitoring is probably most applicable

to qualitative applications where timingis not critical.

IR Camera Hardware andFirmware Issues

Thermal Time Constants or Cooled andUncooled IR Cameras. In general, timeconstant reers to the time it takes or asensing element to respond to within63.2% of a step change in the state ofa target that is being sensed (Figure4). In IR sensing and thermography,the thermal time constant o an IRcameras detector is a limiting factor ininstrument perormance as it relates toresponse time.

100%

80%

63%

0 1 2

Thermal Time Constants

StepC

hange

inT

argetState

Figure 4. Thermal time constant conceptshowing an integral number o time constantson the X-axis.

Older IR cameras have response timessimilar to the human eye, so they areunsuitable or capturing thermal imagesof fast moving objects or those withrapidly changing temperatures. NewerIR cameras have detectors and digitalelectronics with response times in thesub-millisecond region. Cooled quantumdetectors are very sensitive and veryfast (sub-microsecond response times),but their bulkiness and cost tends torule them out o many automationapplications. In addition, quantumdetectors have response curveswith detectivity that varies stronglywith IR wavelength. FLIR has maderecent improvements to its uncooledbroadband microbolometer detectorsand associated A/D converters so theycan continuously output images withembedded temperature data at a60Hz rate. This is satisfactory for mosttemperature monitoring and IR machinevision applications.

Temperature Measurement Range.The overall temperature range of anIR camera is primarily a unction o

its detector and calibration. Cameraelectronics, which include calibrationfunctions, can handle wide variationsin absolute detector sensitivities.For example, the FLIR A325s overallmeasurement range is divided into user-selectable temperature scales that have ameasurement accuracy of 2C (3.6F) or2% of reading:

20C to + 120C (4F to +248F)0C to +350C (32F to +662F)

Optionally, 250C to +1200C (482F to2192F)

This is a valuable feature when a scenehas a temperature range narrower than


42/68

36

Chapter 5

a cameras overall range. Selecting anarrower scale allows better resolutiono the images and higher accuracy inthe measured temperatures. Therefore,the images will better illustrate smallertemperature dierences. On the otherhand, a broader scale and/or highermaximum temperature range may beneeded to prevent saturation of theportion o the image at the highesttemperature.

Its important to understand how thecameras calibration and temperaturemeasurement processes aect itsresponse time. IR cameras measureirradiance, not temperature, but thetwo are related. When an IR camerais thermographically calibrated, itcan measure temperatures based onstandard blackbody radiances at specictemperatures. As will be discussed later,the emissivity of the target object beingmeasured is vital to achieving accuratetemperature readings. (Emissivity oremittance is the radiative properties of anobject relative to a perfect blackbody.)

When an IR camera is calibrated at thefactory, calibration factors are storedinternally as a table of values basedon the cameras A/D counts from thetemperature/radiance measurements oa standard blackbody. When the systemmakes a measurement in an application,it takes the digital value of the signal at agiven moment, goes into the appropriatecalibration table, and calculatestemperature. Before the nal result ispresented, due consideration is givento other factors, like emissivity of thetarget objects, atmospheric attenuation,reected ambient temperature, and thecameras ambient temperature drift.

Figure 5. Creating a temperature scale narrowerthan the cameras ull range improves imageresolution and may improve deect detection.

As an adjunct to major temperature scaleselections, most IR cameras allow a userto set up a color scale or gray scale or atemperature range thats even narrower(Figure 5). This should be done where

practical, not only because of improvedimage resolution, but also because ofresponse time considerations. A narrowertemperature range can reduce the A/Dconverters processing load and overallresponse time o the system.

Another complexity is the fact thateach individual pixel in the cameras

ocal plane array has a slightly dierentgain and zero oset. To create a usefulthermographic image, the dierentgains and osets must be correctedto a normalized value. This multi-stepcalibration process is perormed by thecamera rmware (Figure 6). The non-uniformity correction (NUC) factors arealso stored in a table.

IR cameras also have dierentmeasurement modes: spotmeter andarea measurements in the case o theFLIR A320 Series. The spotmeter nds thetemperature at a particular point whereasthe area unction isolates a selected areaof an object or scene. In the latter case,


43/68

37


camera rmware nds the minimum andmaximum temperatures and calculatesthe average temperature inside thearea selected. Clearly, more processingtime is required for area measurements,particularly i multiple areas are selected.

This also means that more data is beingtransmitted over a machine visionsystems communications network, alongwith more latency.

Emissivity Calibration. Earlier, it waspointed out that accurate temperaturemeasurements on a specic object

require the emissivity value for thatobject. In eect, this adjusts the factorycalibration that is based on a perectblackbody having an emissivity value of1.0. This adjustment consumes processortime. To avoid this, the FLIR A325 uses a

global emissivity value (input by the user)for the cameras entire FOV. Normally,this isnt a problem for machine visionapplications and it avoids the timerequired to apply non-global emissivityvalues on the y. Instead, the applicationprogram is set up to make decisions

Figure 6b. Final steps in IR cameras NUC process

Signal SignalWithout any correction First correction step

Radiation

20C +120C

+20C

Radiation

20C +120C

+20C

Signal Signal

Third correction,

Non-Uniformity Correction (NUC) After NUC

Radiation

20C +120C

+20C

Radiation

20C +120C

+20C

Figure 6a. First step in detector non-uniormity correction (NUC) perormed by IR camera rmware


44/68

38

Chapter 5

based on the temperature value of atarget area compared to a standard valueor compared to the targets surroundings.While this may not be accurate in anabsolute sense, it is the relative dierencethat is most important.

If the system developer wants accuratetemperature measurements on dierentobjects with dierent emissivities (e.g.,

for PC board inspections), then he/shemust create an emissivity map for thecameras FOV. This cannot be done withthe data coming out o a camera usinga global emissivity value. To create anemissivity map, the developer will needto write some program code, typicallyby using the FLIR Software DevelopersKit. The routine thats developed sets up

the system to read the FLIR proprietarystream of data coming out of the A/Dconverter and applies emissivity valuesto it. This creates an emissivity map thatcovers selected areas or spots withinthe FOV.

Important Thermographic Principles

As alluded to above, there must bea temperature dierence between atarget object and its surroundings inorder to create a useul thermographicimage. In most situations, the user alsoneeds a measurement of this relativetemperature dierence or decisionmaking, either automatically or by amachine operator. However, there are a

ew ambient conditions that may obscurethe temperature dierence.

In addition to emitting radiation, anobject reacts to incident radiation romits surroundings by absorbing andrefecting a portion o it or by allowingsome of it to pass through (as through a

lens). Therefore, the maximum radiationthat impinges on an IR camera lens aimedat an object comes rom three sources:(1) the objects inherent temperaturewithout inuence from its surroundings,(2) radiation from its surroundings thatwas reected onto the objects surface,and (3) radiation that passed through theobject rom behind. This is known as theTotal Radiation Law (see below). However,all these radiation components becomeattenuated as they pass through theatmosphere on their way to the cameralens. Since the atmosphere absorbspart of the radiation, it also radiatessome itsel.

Total Radiation Law

1 = + + .The coecients , , and

describe an objectsincident energy absorption

(), reection (), andtransmission ().

Figure 7. Total Radiation Law

The role of emissivity in distinguishingan objects temperature from itssurroundings was discussed above.As alluded to earlier, its wise to takeprecautions that prevent reected energyrom impinging on the target. Correctionsor atmospheric attenuation are normallybuilt into the camera rmware. Still,other gases and hot steam between thetarget and the camera lens can makemeasurements impossible, or at leastinaccurate. Similarly, an object that istransparent to IR wavelengths may resultin the camera measuring the backgroundbehind it or some combination o theobject and the background.


45/68

39


In the last two cases, spectral lters thatare selective at specic wavelengthscan help. Certain lters can make anotherwise opaque gas appear transparentor a transparent object appear opaqueover the appropriate IR band (Figure 8).

Summary

IR cameras used in machine vision andother automation systems are analogousto visible light cameras in similar systems.White light cameras have optical issuesthat must be managed, whereas IRcameras have thermographic issues toresolve. In both cases, achieving real-time

(or near-real-time) response requiresthoughtul selection o the controllerand careul design o the applicationprogram. Third party sotware canprovide out-of-the-box programdevelopment tools that eliminate orminimize the need to write programcode. Generally, there are no perfectsolutions developing an automatedmachine vision system, whether basedon visible light or IR, usually involvescompromises o one sort or another.The camera manuacturer can be agreat source of help in developingthese systems.

Filter Adaptation1.0

0.9

0.8

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0.0

3 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9 4.0

Wavelength, m

T

ransmission%

3.45m NBP lter

Polyethylene

transmission

Resulting

transmission

Figure 8. Application o a narrow bandpass (NBP) lter to achieve nearly complete absorption andhigh emittance (green curve) rom polyethylene lm, allowing its temperature measurement.


46/68

40

Appendix A

Glossaryabsorption (absorption actor). Theamount o radiation absorbed by anobject relative to the received radiation. Anumber between 0 and 1.

ambient. Objects and gases that emitradiation towards the object beingmeasured.

atmosphere. The gases between theobject being measured and the camera,normally air.

autoadjust. A function making a cameraperorm an internal image correction.

autopalette. The IR image is shown withan uneven spread of colors, displayingcold objects as well as hot ones at the

same time.

blackbody. Totally non-reectiveobject. All its radiation is due to its owntemperature.

blackbody radiator. An IR radiatingdevice with blackbody properties used tocalibrate IR cameras.

calculated atmospheric transmission. Atransmission value computed from thetemperature, the relative humidity of theair, and the distance to the object.

cavity radiator. A bottle shaped radiatorwith an absorbing inside, viewed throughthe bottleneck.

color temperature. The temperature or

which the color o a blackbody matches aspecic color.

conduction. The process that makes heatspread into a material.

continuous adjust. A function thatadjusts the image. The unction works

all the time, continuously adjustingbrightness and contrast according to theimage content.

convection. The process that makes hotair or liquid rise.

dierence temperature. A value that isthe result o a subtraction between twotemperature values.

dual isotherm. An isotherm with twocolor bands instead o one.

emissivity (emissivity actor). Theamount o radiation coming rom anobject compared to that o a blackbody.A number between 0 and 1.

emittance. Amount of energy emitted

rom an object per unit o time and area(W/m2).

estimated atmospheric transmission.A transmission value, supplied by a user,replacing a calculated one.

external optics. Extra lenses, lters, heatshields etc. that can be put between the

camera and the object being measured.lter. A material transparent only to someof the infrared wavelengths.

FOV. Field of view: The horizontal anglethat can be viewed through an IR lens.

FPA. Focal plane array: A type of IRdetector.

graybody. An object that emits a xedraction o the amount o energy o ablackbody for each wavelength.

IFOV. Instantaneous Field Of View: Ameasure o the geometrical resolution oan IR camera.


47/68

41

Glossary

image correction (internal or external).

A way of compensating for sensitivitydierences in various parts of live imagesand also of stabilizing the camera.

inrared. Non-visible radiation, with awavelength from about 213 m.

IR. Inrared.

isotherm. A function highlighting those

parts of an image that fall above, below,or between one or more temperatureintervals.

isothermal cavity. A bottle-shapedradiator with a uniorm temperatureviewed through the bottleneck.

Laser LocatIR. An electrically poweredlight source on the camera that emitslaser radiation in a thin, concentratedbeam to point at certain parts o theobject in ront o the camera.

laser pointer. An electrically poweredlight source on the camera that emitslaser radiation in a thin, concentratedbeam to point at certain parts o theobject in ront o the camera.

level. The center value of thetemperature scale, usually expressed as asignal value.

manual adjust. A way to adjust theimage by manually changing certainparameters.

NETD. Noise equivalent temperature

dierence: A measure of the image noiselevel of an IR camera.

noise. Undesired small disturbance in theinrared image.

object parameters. A set of valuesdescribing the circumstances under

which the measurement o an object

was made and the object itself (suchas emissivity, ambient temperature,distance, etc.)

object signal. A non-calibrated valuerelated to the amount o radiationreceived by the camera from the object.

palette. The set o colors used to display

an IR image.

pixel. A picture element. One single spotin an image.

radiance. Amount of energy emittedfrom an object per unit of time, area, andangle (W/m/sr).

radiant power. Amount of energyemitted rom an object per unit otime (W).

radiation. The process by whichelectromagnetic energy is emitted byan object or a gas.

radiator. A piece of IR radiating

equipment.

range. The current overall temperaturemeasurement limitation o an IR camera.Cameras can have several ranges,which are expressed as two blackbodytemperatures that limit the currentcalibration.

reerence temperature. A temperaturewhich the ordinary measured values canbe compared with.

refection. The amount o radiationreected by an object relative to thereceived radiation. A number between0 and 1.


48/68

4

Appendix

automation guidebook

Documents