smart sensor web: web-based exploitation of sensor fusion...

8
Physical Models Dynamic Data Bases Micro Sensors Wireless Communications Next Generation Internet Real-time Imagery Micro-Weather Moving Targets Integration Complete Situation Awareness Complete Situation Awareness DoD DoD Science & Technology Science & Technology Figure 1: Smart Sensor Web - tactical information to the Warfigher, on demand Smart Sensor Web: Web-Based Exploitation of Sensor Fusion for Visualization of the Tactical Battlefield Jeffrey L. Paul Sensor Systems Office of the Deputy Under Secretary of Defense (Science and Technology) Department of Defense Pentagon, Washington, DC, U.S.A. [email protected] Abstract - Smart Sensor Web (SSW) is a recent DUSD(S&T) initiative inspired by extraordinary technological advances in sensors and microelectronics and by the emergence of the Internet as a real time communication tool. The overall vision for SSW is an intelligent, web-centric distribution and fusion of sensor information that provides greatly enhanced situational awareness, on demand, to Warfighters at lower echelons. Emphasis is on multi- sensor fusion of large arrays of local sensors, joined with other assets, to provide real-time imagery, weather, targeting information, mission planning, and simulations for military operations on land, sea, and air. This paper gives an overview of this new initiative, highlights some of the technology challenges in Sensor/Information Fusion, and presents a program approach for near-term demonstrations and long-term solutions, involving the DoD, National Labs, commercial industry, and academia. Keywords: Microsensors, thermal imagers, multisensor fusion, situational awareness, automatic target recognition, Image web, network architecture, microrobotics, the Internet. 1 Smart Sensor Web : a new DoD Initiative for the Warfighter Extraordinary advances have occurred over the last decade in sensors, microelectronics, and communications – in both the military and commercial sectors. In particular, the emergence of the Internet has inspired the overall vision for Smart Sensor Web (SSW) as a military tactical tool to provide battlefield commanders at lower echelons with access to dynamically-updated databases, collected from large arrays of local sensors. SSW will give the Warfighter the right information at the right time…in real or near real time on the battlefield. This includes the fusion of information from high resolution digital maps, 3-D terrain features, multi-spectral imagery from sensors deployed via manned and Unmanned Ground Vehicles (UGVs), Unmanned Aerial Vehicles (UAVs), air, sea, and national assets. SSW will provide the Warfighter with critical and timely answers to the basic questions of Situational Awareness: Where am I? Where are my troops and other friendly forces and what is their status ? What is known about the enemy and what is his status ? What/when/where is my next action on the battlefield ? Smart Sensor Web will provide the necessary information to the Warfighter, on demand, tailored to his immediate battlefield needs and requirements. 1.1 It all starts with Sensors Sensors will be everywhere. The last decade has seen an enormous proliferation of sensors, both militarily and commercially. Thermal imagers were used extensively on ground combat vehicles and aircraft during the Gulf War by the U.S. and Coalition Forces and allowed us to

Upload: others

Post on 21-Aug-2020

5 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Smart Sensor Web: Web-Based Exploitation of Sensor Fusion ...isif.org/fusion/proceedings/fusion00CD/fusion2000/... · By linking local arrays of sensors with battlefield information

Physical ModelsDynamic Data BasesMicro SensorsWireless CommunicationsNext Generation Internet

Real-time ImageryMicro-WeatherMoving TargetsIntegration

Complete Situation AwarenessComplete Situation Awareness

DoD DoD Science & TechnologyScience & Technology

Figure 1: Smart Sensor Web - tactical information to the Warfigher, on demand

Smart Sensor Web:Web-Based Exploitation of Sensor Fusionfor Visualization of the Tactical Battlefield

Jeffrey L. PaulSensor Systems

Office of the Deputy Under Secretary of Defense(Science and Technology)

Department of DefensePentagon, Washington, DC, U.S.A.

[email protected]

Abstract - Smart Sensor Web (SSW) is a recentDUSD(S&T) initiative inspired by extraordinarytechnological advances in sensors andmicroelectronics and by the emergence of the Internetas a real time communication tool. The overall visionfor SSW is an intelligent, web-centric distribution andfusion of sensor information that provides greatlyenhanced situational awareness, on demand, toWarfighters at lower echelons. Emphasis is on multi-sensor fusion of large arrays of local sensors, joinedwith other assets, to provide real-time imagery,weather, targeting information, mission planning, andsimulations for military operations on land, sea, andair. This paper gives an overview of this new initiative,highlights some of the technology challenges inSensor/Information Fusion, and presents a programapproach for near-term demonstrations and long-termsolutions, involving the DoD, National Labs,commercial industry, and academia.

Keywords: Microsensors, thermal imagers,multisensor fusion, situational awareness, automatictarget recognition, Image web, network architecture,microrobotics, the Internet.

1 Smart Sensor Web : a new DoD Initiative for the Warfighter

Extraordinary advances have occurred over the lastdecade in sensors, microelectronics, andcommunications – in both the military and commercialsectors. In particular, the emergence of the Internet hasinspired the overall vision for Smart Sensor Web(SSW) as a military tactical tool to provide battlefieldcommanders at lower echelons with access todynamically-updated databases, collected from largearrays of local sensors. SSW will give the Warfighterthe right information at the right time…in real or nearreal time on the battlefield. This includes the fusion ofinformation from high resolution digital maps, 3-Dterrain features, multi-spectral imagery from sensorsdeployed via manned and Unmanned Ground Vehicles(UGVs), Unmanned Aerial Vehicles (UAVs), air, sea,and national assets.

SSW will provide the Warfighter with critical andtimely answers to the basic questions of SituationalAwareness: Where am I? Where are my troops andother friendly forces and what is their status ? What is

known about the enemy and what ishis status ? What/when/where is mynext action on the battlefield ? SmartSensor Web will provide thenecessary information to theWarfighter, on demand, tailored to hisimmediate battlefield needs andrequirements.

1.1 It all starts with Sensors

Sensors will be everywhere. Thelast decade has seen an enormousproliferation of sensors, bothmilitarily and commercially. Thermalimagers were used extensively onground combat vehicles and aircraft

during the Gulf War by the U.S. andCoalition Forces and allowed us to

Page 2: Smart Sensor Web: Web-Based Exploitation of Sensor Fusion ...isif.org/fusion/proceedings/fusion00CD/fusion2000/... · By linking local arrays of sensors with battlefield information

“Own the Night.” Second Generation, ThermalImaging technology has now become the standard formost high-performance, Intelligence, Surveillance, andReconnaissance (ISR) systems and Target Acquisitionsystems throughout the U.S., NATO, and WesternWorld Allies.

We should “own the night ;” we certainly paid enoughfor it. The development and fielding of sensor systemsrepresents a massive investment in R&D andprocurement dollars over the last 30 years. But withdeclining defense budgets over the last several years,the requirement for more affordable military productshas become a higher priority. As a result, technologyadvancements were intensified to achieveimprovements in manufacturing processes, low-costmaterials and low-cost, integrated assemblies. Dual-use technologies and commercial off-the-shelf (COTS)products are being exploited for military use. Nationaland international sales of these products for civilian usehas also increased … all adding up to dramaticreductions in the cost of military sensors.

Reduction in the size of the U.S military has provided astrong incentive for more effective, rapid deploymentforces, with greater mobility and military effectiveness.Hence, the DoD continues to invest inmicroelectronics, again with dramatic results insmaller, cheaper, lighter sensors. A notable example isthe development of the “MicroFLIR,” sponsored by theUS Army Night Vision and Electronics Directorate(NVESD), Ft. Belvoir, Virginia. This device utilizes

advances in uncooled thermal sensor technology andunique microelectronics to provide a system weighingonly 70 grams, with a volume of approximately 12cubic inches, and requiring only 540 milliwatts ofpower.

With the ever-increasing proliferation of low-costthermal sensors and related technologies, we can nolonger expect to be the only military forces to own thenight.

Tomorrow’s battlefield will be proliferated withsensors of all types that could contribute to localsituational awareness. SSW will exploit theinformation from commercial products distributedthroughout an area (local TV cameras and othercommercial devices already transmitting overcommercial airwaves and the Internet), or seed thebattlefield by rapidly deploying military sensors via airdrops, robotic vehicles, pre-positioned assets, soldierplatforms, UAVs, or overhead surveillance. Imaging

Small Areas ≤≤≤≤ 1km2Response Times ≤≤≤≤ 1 minLocations Errors ≤≤≤≤ 1 mSpatial Resolution ≤≤≤≤ .3 m

HMMWV

Sensor Technologies

TVUncooled ThermalLADARWeatherAcousticMagneticCOTS/MOTS

Sensor Technologies

SAR, IFSARGMTIHRR

Cooled ThermalLADAR

PanchromaticSpectral

Enablers

Data LinksATR

Hi Perf CompUAV

MIMCGPS

LOCAAS

UGV/ UGS

µµµµ UAV

Global Hawk

Predator JSF

NTM/DII

Unique SSW Capabilities• Urban Canyons• Subways, Sewers• Difficult Rural Terrain• Heavily Forested Areas• Track Slow and Fast Objects• Rapid Response Time

Large Areas ≤≤≤≤ 100 km2Response Times ≥≥≥≥ 1 minLocations Errors ≥≥≥≥ 1 kmSpatial Resolution .3 - 30m

LRAS

EnablersMEMSATRComputersSensor FusionDisplaysWirelessNetworksRoboticsGPS

HMD

Smart SensorWeb Current ISRResolutionTime

&Space

Figure 2: The numerous elements of SSW create a challenge for Sensor/Information Fusion

Figure 3 : Micro FLIR - to seed the battlefield

Page 3: Smart Sensor Web: Web-Based Exploitation of Sensor Fusion ...isif.org/fusion/proceedings/fusion00CD/fusion2000/... · By linking local arrays of sensors with battlefield information

sensors include : TV cameras, low light level CCDs,cooled and uncooled thermal cameras, laser radar,multi-spectral imagers, and imaging radar. Non-imaging sensors include : laser rangefinders,designators, weather sensors, chemical and biologicalagent sensors, physiological status sensors, seismic,acoustic and magnetic sensors.

1.2 Creating the Web

Having sensors everywhere on the battlefield is of verylimited value unless the information gained from themcan be accessed and shared in a timely manner by theappropriate level of military commanders. Currently,this sensor information is largely “stove-piped” tospecific users operating a specific sensor, for a specificmilitary mission. Digitization of this sensorinformation and the capability to transmit it overwireless links now make it possible to share thisinformation among several users by creating larger,more complex sensor networks and an architecture tofacilitate the linkage of numerous local networks ofsensors on the battlefield.

Advances in wireless communications and InformationTechnology in the commercial sector, worldwide, aredramatic and self-evident. Integration of discreetdevices, such as cellular phones, pagers, television,video cassette recorders, compact disc players, andvideo games, will provide consumers with muchgreater access and control of the information they needor want in their daily lives. In this area of InformationTechnology, commercial investment and competition-driven innovation dwarfs the activities within the DoD.It is therefore critical that SSW leverage and embracethese capabilities in every appropriate way.

The Internet is another important example oftechnology growth in the commercial sector. Thiscapability of real time access to unlimited amounts ofinformation to an exponentially growing number ofworldwide users is a key enabler and fundamentalconcept for the Sensor Web. Military users will accessinformation from SSW via websites.

1.3 Making it ‘‘Smart’’

The web-centric network of large arrays of sensors willprovide large amounts of data and imagery to users ondemand. However, to effectively utilize thisinformation, the web must become more intelligent, tointrepret and provide the right information in a timelymanner, without cognitive overload.

We start with smart sensors. Imaging and non-imaging sensors can collect both battlefieldenvironment information and target signatures andreadily convert this to digitized imagery andinformation. This digitized information can beprocessed at the individual sensor level to accomplishfirst order interpretation. Image/signal processing canassist in target acquisition, classification, and in somecases, recognition and identification, depending on thetype of target, unique signature data, and level of on-board processing.

Automatic Target Recognition (ATR) algorithmshave matured significantly over the last several yearsand are now capable of on-board, model-based imageprocessing, with less memory storage requirementsthan previous techniques, such as template-matching.Use of neural-net processing architectures will allowATR algorithms to train and adapt to new targetsignatures in a wider variety of backgrounds, yielding

more robust ATR capabilities with fewer false alarms.By linking local arrays of sensors with battlefieldinformation of a particular area, the sensor webnetwork can facilitate multisensor fusion processing,thereby increasing the value/confidence of the sensorinformation to the Warfighter. This could includevisible/near Infrared images (.4-1 micron), thermalimages from Long-Wavelength (8-12 microns) andMid-Wavelength (3-5 Microns) and/or otherorhthogonal signature data (acoustic, seismic,magnetic), thus allowing the sensor network torecognize/identify targets with increased confidenceand lower false alarm rates.

Local area sensor networks can then provide target andbattlefield environment information to the next higher

UAV

Recognize/Identify

Timeline … the Tactical Advantage

HumanDisplay

Sensor

DisplayProcessor

Search - Acquire - Recognize - Engage Targets before they engage you!

Scene

Figure 5 : ATR – critical to making Sensor Web Smart

Figure 4 : Creating the Web

Page 4: Smart Sensor Web: Web-Based Exploitation of Sensor Fusion ...isif.org/fusion/proceedings/fusion00CD/fusion2000/... · By linking local arrays of sensors with battlefield information

level of information fusion, adding weather data,environmental day, terrain maps, and other contextualdata from other databases. Linking these databasesthroughout the Sensor Web for smart routing ofrequests and intelligent fusion of this data will requireembedded intelligent agents at each level. This willassure the proper tasking, processing, exploitation, anddissemination (TPED) of the information for each userrequest.

Intelligent, interactive displays will be required toprovide the Warfighter with the information in a formattailored to his needs and requirements to make rapidand effective decisions on the battlefield. Real timeaccess to local sensor arrays, coupled with synthesizedimagery from other databases (adapted from video-game technology and advanced visualizationtechniques), can also provide the Warfighter with aVirtual Presence in a tactical area from a remotelocation, thereby aiding him in mission planning andrehearsals prior to conducting a military mission.

We start with data from local sensor networks(imagery, weather, weapon platforms), collect andprocess the digitized data, fuse it with data from othersources (synthesized data, intelligence data,logistics/utilities data, terrain/cultural data, etc), anddisplay it to the Warfighter, upon demand, in the rightway at the right time. This will allow the Warfighter tomake faster and more effective battlefield decisions.

Thus, the Information Superiority achieved by therapid,, real time or near real time, exploitation of anintelligent, web-centric architecture will lead toDecision Dominance on the battlefield.

2 Enabling Technologies

Smart Sensor Web is a vision and a thrust in Scienceand Technology ; it is not a specific program. But thecapabilities of SSW will be realized through theapplication and integration of numerous enablingtechnologies currently being pursued by the defenseS&T community, the commercial sector, and inAcademia.

2.1 U.S. DoD activities

The DoD continues to make major investments inscience and technology efforts directly relevant to SSWthroughout the Services and Agencies.

The Services’ requirements-based science andtechnology investment strategy is strongly focused onnetwork-centric sensor and information technologies,with many being pursued as Joint efforts, such as theAdvanced Concept Technology Demonstrations(ACTDs), sponsored by the Deputy Under Secretary ofDefense for Advanced Systems and Concepts. Forexample, the Military Operations in Urban Terrain(MOUT) ACTD is evaluating numerous technologies

with potential to improve urban warfighting, includingsmall unit communication systems that provide locationdata for situation awareness.

The DUSD (S&T) is sponsoring a related initiative inCognitive Readiness that has strong relevance toSSW. Cognitive Readiness will explore technologies toimprove our understanding of human conditions andresponses in the battlefield environment, to includeinformation visualization, cognitive understanding,and advances in human behavior modeling.

Joint efforts are also being sponsored by theDUSD(S&T) in Automatic Target Recognition(ATR) Technology Assessment, with emphasis onmulti-spectral fusion for improved clutter rejectionand camouflage pentetration. This includes technologytransition opportunities for advanced sensor sytems,such as the Long-Range Advanced Scout SurveillanceSensor System (LRAS3). In this case, ATR technologywill aid the Scout in target search and track functions,with potential upgrade to target recognition capabilityvia multisensor fusion. Target information from sensorsystems such as LRAS3 can be linked to SSW via thenetwork architecture.

Figure 6 : Long Range Advanced Scout Sensor Systemwill incorporate ATR to aid in Search and Track

Service specific efforts such as the Army’s WarriorEnhanced Battlespace Sensor (WEBS) effortstrongly support the concept of web-based networkedsensors. The Navy’s and Marine Corps’ ExtendedLittoral Battlespace (ELB) ACTD has validated theimportance of providing information and more efficientsensor-to-shooter connectivity to small, dispersedMarine units. At the same time, the Navy’sCooperative Engagement Capability (CEC) effortdemonstrates the value of efficient information flowand target tracking and hand-off in its Network CentricWarfare concept. Finally, the Air Force’s JointBattlespace Infosphere (JBI) is a scaled-up version ofSSW concept that provides integrated, current,consistent, globally accessible information through theuse of information centric JBI client applications andexploitation of relevant commercial technologies.

Page 5: Smart Sensor Web: Web-Based Exploitation of Sensor Fusion ...isif.org/fusion/proceedings/fusion00CD/fusion2000/... · By linking local arrays of sensors with battlefield information

The Defense Advanced Research Projects Agency(DARPA) continues to seek radical innovation andmake significant progress in advanced sensortechnology, sensor networking, image/data fusion,target tracking, dynamic databases, and visualizationtechnologies. Recent DARPA efforts are now beingintegrated into SSW, including the Small UnitOperations Situation Awareness System (SUO SAS)that provides mobile networking communication andsituation awareness capability, and the VideoSurveillance and Monitoring (VSAM) system whichprovides video-based ATR and target tracking andhand-off capabilities, utilizing a local array of sensors.For technical approaches to intelligent agent-basedinformation visualization, SSW is also assessingDARPA’s Command Post of the Future (CPoF) thatdeals with intelligent display and visualizationtechnologies for command-level decision-making.

The Defense Modeling and Simulation Office supportsthe development of the tools (e.g., High LevelArchitecture/Runtime Infrastructure), models anddatabases (e.g., multi-spectral environmentalrepresentations) that together provide a syntheticenvironment for emulating live demonstrations andexperiments of SSW. This provides a cost-effectivemeans for assessing various implementations of theSSW concept. These same models and databases canalso be used to support simulation-based decisionsupport tools for optimizing sensor placement, andmission planning and rehearsal.

The US National Laboratories and Federally-FundedResearch and Development Centers (FFRDCs), closelylinked with the academic community, are also playing amajor role in the pursuit of Smart Sensor Web.Significant research is continuing in advanced sensortechnology and micro, electro-mechanical systems(MEMS) and micro, electro-optical- mechanicalsystems (MEOMS). Lincoln Laboratory, at theMassachusetts Institute of Technology (MIT), isplaying a major role in the overall integration oftechnologies for SSW, as well as pursuing leading-edgeresearch in numerous supporting technologies,including : advanced sensor/processing technology,multispectral fusion, hypserspectral imagingtechnology, and information fusion.

2.1 Commercial technologies

Numerous technology advances in the non-DoDcommercial sector and the rapid advances of the uses ofthe Internet are being leveraged in the pursuit of theSSW initiative.

The communications network of SSW will exploitadvances in mobile, wireless networks supporting cellphones, interactive pagers, and laptops with PCMCIAcommunication cards. The miniaturization of powerfulprocessors coupled with novel user interfacetechnologies (e.g., voice, touch screen, gesturerecognition, heads-up displays, etc.) supports the

development of personal data systems that fit in thepalm of a hand or on the back of a wrist, as well ashands-free computers. These devices are significantfor linking the individual combatant to the power of theSSW.

Commercial wristwatches now provide GlobalPositioning System (GPS) information, weatherinformation, and even remote sensing (e.g., sensorsthat can determine the temperature of a remote object).A remote connection through the Internet providesreal-time control of cameras (pan, tilt, and zoom), animportant step toward the real time, Virtual Presencethat SSW will provide the Warfighter.

Commercial security systems, burglar alarm systems,multi-mode sensors for law enforcement activities andthermal imagers for firefighters all provide acommercial base for large networked arrays of cheap,throwaway sensors. Even high resolution satellite-based imagery is now affordable and available throughthe Internet for general public use.

2.2 Multidisciplinary University ResearchInitiatives

The Multidisciplinary University Research Initiative(MURI) is a multi-agency DoD-sponsored programthat supports multidisciplinary university researchteams whose efforts intersect more than one traditionalscience and engineering discipline. Multidisciplinaryteam efforts can accelerate research progress in areasparticularly suited to this approach by cross-fertilization of ideas and expertise, hasten the transitionof basic research findings to practical applications, andhelp to train students in science and/or engineering inareas of importance to DoD. By supporting these teamefforts, MURI complements other DoD programs thatsupport university research principally through single-investigator awards.

For example, of the numerous MURI efforts, two areparticularly relevant to SSW:

• Data Fusion in Large Arrays of Micro-sensors(SensorWeb) -- The objective of this initiative is todevelop a quantitative basis for information

Figure 7 : Leveraging commercial Technologies

Page 6: Smart Sensor Web: Web-Based Exploitation of Sensor Fusion ...isif.org/fusion/proceedings/fusion00CD/fusion2000/... · By linking local arrays of sensors with battlefield information

processing in large arrays of distributed micro-sensors under strong limitations on the capabilitiesof each sensor and severe constraints oncommunication.

• Adaptive Mobile, Wireless Networks for HighlyDynamic Environments -- Develop new adaptivewireless network technologies applicable to thedistributed, dynamic, mobile, multi-hop militaryenvironment based on a rich exchange ofinformation between the various system layers.

The DoD also sponsors significant research atuniversities and private research laboratoriesthroughout the US through Small Business InnovativeResearch programs. In gnereal, the Basic Researchareas include : Nanoscience, Biomimetrics, Mobile,wireless communications, Smart structures, Intelligentsystems, and Compact power sources. Each of theseareas has potential value to the realization of SmartSensor Web.

3 Technology Challenges

Because SSW is a multi-disciplinary thrust, there arenumerous technology challenges covering a broad areaof disciplines, including : communications, bandwidth,signal processing, robust, self-organizing networks,power sources and power management, and sensordeployment. Further discussion in this paper highlightsthose technology challenges dealing with image fusionexploitation and visualization.

3.1 MultiSensor Fusion

Smart Sensor Web will be able to link and exploit bothimaging and non-imaging sensors. Imaging sensorswill include : visible/Near Infrared cameras (EO),thermal imagers (IR) in the mid-wave (3-5 microns)and long-wave (8-12 microns), Laser Radar (1-2microns), and Synthetic Aperture Radar (SAR).Multisensor imaging also includes multispectralimaging (processing simultaneous images from a singlesensor in multiple spectral bands) and hyperspectralimaging (processing simultaneous images from a singlesensor in a larger number of narrow spectral bands).Non-imaging sensors include : acoustics, seismic,magnetic, ground moving target indicators (GMTI),and other cueing sensors, as well as numerous micro-weather sensors.

Fusion of multisensor information will providesignificant benefit for robust ATR capabilities forsearch, recognition, and tracking of targets, even in thepresence of camouflage and decoys.

3.2 4-D Image Fusion

Three-dimensional image fusion requires accurateimage registration and geolocation from each sensor

acquiring information. In order to fuse imagery from amultisensor array, fusion algorithms will need preciselocations and viewing angles as well detailedperformance parameters (magnification, range, etc.).Image processing can then provide the necessary imagetransforms for image registration and fusion ofmultisensor imagery, particularly of fixed assets.

Dynamic battlefield conditions require a four-dimensional fusion space. Moving targets, changingenvironments, mobile sensors and tactical battlefieldevents require rapid updates and image fusion in timeas well as space. Major technical efforts are continuingin this area and multisensor fusion capabilities areanticipated for SSW in the near term.

3.3 Information Fusion and Visualization –the ultimate pay-off to the Warfighter

The real pay-off of SSW for the Warfighter will likelybe in the Information Fusion and Visualization area.Here, the challenge is to fuse the massive amount ofsensor data available into the right information at theright time. This will involve significant advancementsin intelligent agents, integration of live and virtualimagery and information, multi-level resolution andfidelity, and interactive displays that present thenecessary battlefiled information without inducingcognitive overload.

Three-dimensional, immersive displays are needed toprovide the Warfighter with the capability to beanywhere on or above the battlefield at anytime, in realtime, giving him Virtual Presence on demand.

4 Achieving Smart Sensor Web…a Testbed Approach

4.1 Testbeds and Sub-Webs

To translate the vision of the SSW into reality, OSD issponsoring a Testbed to demonstrate state-of-the-arthardware and software technologies, from on-going

Figure 8 : Multispectral Color Fused Imagery, Monterey, CA

Page 7: Smart Sensor Web: Web-Based Exploitation of Sensor Fusion ...isif.org/fusion/proceedings/fusion00CD/fusion2000/... · By linking local arrays of sensors with battlefield information

DoD efforts and from the commercial sector; and to useexperiments to assess technical and operational utilityof these technologies. The other aspect of the Testbedis to determine technical needs (power sources,bandwidth, etc.) and operational requirements(information needs, presentation capabilities, sensoremployment, etc.).

For the first 2-3 years of the effort, OSD will utilize theMilitary Operations in Urban Terrain (MOUT) site atFort Benning, Georgia, for testbed integration anddemonstration. This site was selected particularlybecause local situation awareness is most difficult andmost critical in built-up, urban areas. The MOUT site isfully instrumented for collecting experimental data canleverage the site personnel knowledge gained frommany Joint and Service exercises and technologydemonstrations over the past several years.

To identify the major SSW components and provide aframework for this multi-disciplinary effort, “Sub-Webs” were established. Initially, there were five Sub-Webs : Image, Weather, Weapons, Simulation, andInformation Integration. Physio-Med Sub-Web wasrecently added and additional Sub-Webs are beingconsidered, as additional capabilities (e.g., chemicaland biological sensing) are included in the experimentsand demonstrations.

The SSW Testbed activitiy is a joint, DoD-wide, effort,involving all the Services, including : the Air ForceResearch Laboratories (AFRL), the Army ResearchLaboratory (ARL), the Army Night Vision Electronicsand Sensors Directorate (NVESD), the Office of NavalResearch (ONR), and other Service Labs and ResearchCenters throughout the U.S.

The elements of the current SSW Sub-Webs include:

• Image Web – visual and infrared imagery,coupled with ATR; acoustic, magnetic,seismic, infrared, and laser beam sensors,coupled with solution models for targetidentification and alerts; personal sensors onindividual Warfighters for observing around

corners or through windows; and a host ofsensor platforms, including ground, tree, andbuilding mounts, as well as robotic groundand aerial vehicles.

• Weather Web – temperature, humidity,pressure, wind, visibility, and other weathersensors coupled to micro and macro weathermodels to provide current (nowcasts) andpredictive (forecasts) local weatherinformation.

• Weapons Web – initial focus on integratingthe data from the laser radar sensor and on-board ATR of the Air Force’s Low CostAutonomous Attack System (LOCAAS) intothe SSW database, thus utilizing valuableintelligent sensor data that would normally belost when the LOCAAS self-destructs duringits attack phase.

• Simulation Web – advanced models,simulations, and associated databases (e.g.,Paint the Night database of multi-spectralenvironmental data) to fulfill two roles withinthe SSW effort: (1) as a c omponent of SSW,visualize information and provide virtualpresence for mission planning and rehearsal,and (2) as a component of the Testbed,support the simulation-based experimentationand assessment of the SSW concept and itstechnology.

• Information Integration Web – intelligentagents, knowledge bases, and smart networksto integrate and present information obtainedfrom SSW sensors, models, and databases,and appropriate external databases (e.g.,maps, blueprints, higher echelon sensors,location data, etc.) in an interactive mode(information push and pull) to the Warfighter.

• Physio-Med Web – temperature, bloodpressure, heart rate, hydration, and otherphysical and medical sensors and associatedmodels that aid in identifying physical(overheating, needs hydration, etc.) andmedical (heat exhaustion, wounded, etc.)problems. Will evaluate configurations for

Live

Virtual

Figure 10: Virtual Presence via Live/Virtual Image Fusion

Figure 9 : MOUT Site, Ft. Benning, Georgia

Page 8: Smart Sensor Web: Web-Based Exploitation of Sensor Fusion ...isif.org/fusion/proceedings/fusion00CD/fusion2000/... · By linking local arrays of sensors with battlefield information

performing triage, medical diagnostic aid,and use for Personnel Recovery.

In addition to the Sub-Webs, there are other DoDagencies who will participate in the SSW Testbedexperiments. This will include the Special OperationsCommand (SOCOM), utilizing the Testbed to evaluatetechnologies for web-centric sensor systems.

4.2 Technology Transition Opportunities

Experiments and successful Advanced TechnologyDemonstrations (ATDs) from the SSW Testbed canbe transitioned to on-going development andacquisition programs within the Services and can beintroduced as Product Improvements to fieldedequipment, where appropriate.

Maturing technologies that offer important advancedcapabilities in Situational Awareness for the Warfightercan also be integrated and evaluated for military utilityvia Advanced Concept Technology Demonstrations(ACTDs), and rapidly transitioned to the field.

SSW is being pursued as a multi-year S&T Testbedeffort, with the intent of transitioning to an ACTD(s)for military utility evaluations by joint warfighter unitsin actual field conditions. Successful outcome of theseevaluations will result in early fielding and furthertransition opportunities to acqusition programs.

International collaboration with NATO allies is in itsearly stages and is also being pursued. This willprovide further opportunities for accelerateddevelopment and expanded applications for fielding.

5 Smart Sensor Web ends with OperationalUtility for the Warfighter

Smart Sensor Web is a vision of revolutionarycapabilities for the Warfighter. For the first time,battlefield commanders - company commanders,

platoon leaders, squad leaders and fire teams - willhave access and connectivity to essential battlefieldinformation in real time, on demand. The recenttechnological advances in low-cost microsensors,microelectronics, signal/image processing, informationtechnology in both the military and commercial sectorsare allowing this vision to be realized. Smart SensorWeb will be achieved incrementally through theintegration and demonstration of these enablingtechnologies via the SSW Testbeds. Rapid transitionto the field will be accomplished through military userevaluations via Advanced Concept TechnologyDemonstrations (ACTDs) and acquisition programs.

Just as the Internet began as a small network to link aspecific technical community at DARPA and grew toits massive, ubiquitous, state, revolutionizing oursociety, so too may be the evolution of Smart SensorWeb and its contribution to the survivability andmilitary effectiveness of the Warfighter in the NewMillennium.

Acknowledgements

The author would like to thank LtCOL BruceGwilliam, Special Projects Officer – Smart SensorWeb, ODUSD(S&T), Dr. Jasper Lupo, Director,Sensor Systems, DUSD(S&T)/SS, and Mr. AlSciarretta, CNS Technologies, Inc., Springfield,Virginia, for their significant help and consultation inthe preparation of this paper.

“Seeding” the Battlefield with a networkof distributed sensors

The Sower, Artist, Lee Hodges Ventura, CA

SituationalAwareness

IMAGE

WEAPONS

WEATHER

Clear

Fog

Clouds&

Rain

SIMULATION Enhanced

Figure 11 : Situational Awareness via Information Fusion