lots of bits in imagery not enough pipe to push imagery to the field
DESCRIPTION
Using Data Compression to Increase the Bandwith of Existing Tactical Control System Malcolm Vant Deputy Director General at DREO Chairman of NATO IST. Lots of bits in imagery Not enough pipe to push imagery to the field Visualisation quality is tied to compression. Problems. - PowerPoint PPT PresentationTRANSCRIPT
Using Data Compression to Increase the Bandwith of Existing Tactical Control System
Malcolm VantDeputy Director General at DREO
Chairman of NATO IST
Lots of bits in imagery
Not enough pipe to push imagery to the field
Visualisation quality is tied to compression
Problems
Information Management
Click to add sub-title
Margaret Varga
Sensors and Electronic Sectors
Telephone: +1684 895712Facsimile: +1684 894952Email: [email protected]
Defence Evaluation & Research AgencyMalvernWorcestershireWR14 3PSUK
Background
Huge volumes of diverse data to be analysed
Existing systems are outpaced
Manual examination is still the main approach
Need for fast, cost effective and reliable semi-automatic or automatic approaches
To retrieve, screen, evaluate, correlate, disseminate, archive and present information at the right time, place and format
Information Deluge
An Overview of an Information Management Process
Raw Data
Target DataPre-processedData
TransformedData
Patterns
KnowledgeData Selection
Pre-processing
Transformation
Data Mining
Interpretation /Evaluation
Analysis
Intelligence
Visualisation
Data Collection
InformationData
Information Management Process(MoD)
Data Collection - Reports (digital/hardcopy), formatted, application oriented
Data Selection - Information Retrieval/Extraction/Routing
Pre-Processing - Elimination/reduction of outliers
Transformation - Conversion into appropriate format for processing
Pattern Finding - Data mining
Interpretation/Evaluation of Patterns - Knowledge discovery
Analysis of Knowledge - Intelligence
Summarisation
Information visualisation
Information Management Principles
Relevance
Effectiveness and Efficiency
Responsiveness
Accessibility
Custody
Preservation
Accountability
Open Source Textual Information Exploitation(Military Relevance)
Timely access to relevant, accurate, precise, comprehensible and credible information (e.g. for decision making)
Ability to track what information is available (covert and overt)
Ability to define rapidly changing topics of interest from diverse users
Zoom or expand over levels of specialised knowledge (i.e. summary or detailed analysis)
Limitations of Existing Information Retrieval Approaches
Commercial Off The Shelf (COTS) products are only effective in a few domain specific applications
COTS based on user-specified ad hoc keywords, phrases or topics:– subject scope can become too general and unfamiliar– costly (time & money), depends on predefined search definition– irrelevant reports are retrieved and relevant reports may remain hidden
Full reports are presented
Duplicate information from same and different sources is presented
Retrieved reports are ranked by existence and frequency of keywords- not necessary equivalent to relevancy
Open Source Textual Information Exploitation Research Programmes
VisualisationPlumber, TextScape,Delos
Delos - Textual (and speech) information sourcesDelos - Textual (and speech) information sources
DERA-OKAPI Demonstrator
RedundancyElimination
Accurate KeyGeneration &
Topic spotting
RelevanceRanking
Summarisation
Indexation, Profiling,Archival & intelligence Analysis
DERA-OKAPI Information Retrieval Demonstrator
DERA-OKAPI Information Retrieval Demonstrator
DERA-OKAPI Information Retrieval Demonstrator
User Interest Oriented Text Summarisation
Accurate ‘keys’ are used as salient summary indicators of
the news feeds
Extracted sentences with the occurrence of the accurate keys
Extract the leading paragraph of the news item as a ‘rough’
summary of the reporting event
Extract the sentence before and after the ‘key’ sentence -
reduce dangling problems
Paragraphs with more than 3 ‘keys’ or more than 70%
relevant sentences
Text Summarization
Keywords/phrases
First paragraph
Sentences with ‘keys’
Paragraphs with more than 70% of sentences deemed relevant or > 3 keys
News Items
1 - 3%
5 - 10 %
15 - 20%
25 - 30 %
30 - 35%
100%
Amount of textAmount of text
Sentences before and after the sentence with ‘keys’
Salient Keys
1. I-FOR2. CONVINCED3. DAYTON4. CIVILIAN5. CRIMES6. MISSION7. SERIOUS8. INVESTIGATIONS9. SITUATION10. LIFE11. SUCCESS12. Nato13. WAR14. POLICE15. STATEMENTS16. PROCESS17. PROBLEMS18. keeping19. peace20 FORCE
A.1.1 Original News
Record 0 weight 1000 Length: 2/~1 pages
<Subject:> BOSNIA/SMITH <L-ONLY> BY NICK SIMEONE /WASHINGTON<ORGANISATION:> Defence Research AgencyDATE=/4/2/96TYPE=CORRESPONDENT REPORTNUMBER=2-195223TITLE=BOSNIA/SMITH (L-ONLY)BYLINE=NICK SIMEONEDATELINE=WASHINGTONCONTENT=VOICED AT:
INTRO: THE COMMANDER OF NATO FORCES IN BOSNIA, ADMIRAL LEIGHTON SMITH, SAYS IT IS TOOEARLY FOR MILITARY COMMANDERS TO RECOMMEND WHETHER U-S PEACE-KEEPING TROOPSSHOULD REMAIN IN BOSNIA BEYOND THEIR SCHEDULED DEPARTURE IN DECEMBER. THAT’S ADECISION, HE SAYS , THAT WILL HAVE TO BE MADE LATER THIS YEAR BUT NEARLY FIVE MONTHSINTO THE PEACE-KEEPING OPERATION, ADMIRAL SMITH IS RAISING CONCERNS ABOUT CIVILIANASPECTS OF THE BOSNIA PEACE AGREEMENT, INCLUDING THE ABILITY OF LOCAL POLICE TOACTUALLY ENFORCE LAW. WE HAVE THIS REPORT FROM V-O-A’S NICK SIMEONE.
TEXT: WITH WAR CRIMES INVESTIGATORS STARTING TO LOOK INTO MASS GRAVES, ADMIRALSMITH SAYS ENORMOUSLY DIFFICULT TASKS STILL LIE AHEAD AS THE WORLD WATCHES TO SEEWHETHER BOSNIA’S FORMER WARRING PARTIES WILL BE ABLE TO FULFILL TERMS OF THE PEACEACCORD THEY SIGNED TO END THE WAR. // OPT // HE MADE THE STATEMENTS TUESDAY AT APENTAGON BRIEFING. //
LATELY, SOME POLITICAL LEADERS IN EUROPE HAVE BEEN PRIVATELY SUGGESTING THE PEACEPROCESS IN THE FORMER YUGLOSLAVIA IS TOO DELICATE AND THE COUNTRY IS TOO LIKELY TOSLIP BACK INTO WAR FOR NATO FORCES - PARTICULARLY U-S TROOPS ----- TO CONSIDER PULLINGOUT AT THE END OF THE YEAR. PRESIDENT CLINTON HAS REPEATEDLY SAID THAT U-SPARTICIPATION IN THE PEAC-KEEPING MISSION IN BOSNIA WIIL ONLY BE FOR ONE YEAR. NATOCOMMANDER SMITH SAYS THE MILITARY, AT THIS STAGE AT LEAST, IS NOT MAKING PREDICTIONSEITHER WAY.
// SMITH ACT //
THERE ARE SOME PEOPLE IN EUROPE WHO HAVE EXPRESSED ANOPINION BOTH FOR AND AGAINST I-FOR REMAINING AFTERDECEMBER. I WILL JUST TELL YOU THAT THE SMART MAN WOULDGOO LOOK AT THE SITUATION, JUDGE IT AT THE TIME, MAKE ADECISION AND THEN FROM THAT POINT ON BE GUIDEDACCORDINGLY.
A.1.1 Leading Paragraph Summarisation
INTRO: THE COMMANDER OF NATO FORCES IN BOSNIA, ADMIRALLEIGHTON SMITH, SAYS IT IS TOO EARLY FOR MILITARY COMMANDERS TORECOMMEND WHETHER U-S PEACE-KEEPING TROOPS SHOULD REMAIN INBOSNIA BEYOND THEIR SCHEDULED DEPARTURE IN DECEMBER. THAT’S ADECISION, HE SAYS , THAT WILL HAVE TO BE MADE LATER THIS YEAR BUTNEARLY FIVE MONTHS INTO THE PEACE-KEEPING OPERATION, ADMIRALSMITH IS RAISING CONCERNS ABOUT CIVILIAN ASPECTS OF THE BOSNIAPEACE AGREEMENT, INCLUDING THE ABILITY OF LOCAL POLICE TOACTUALLY ENFORCE LAW. WE HAVE THIS REPORT FROM V-O-A’S NICKSIMEONE.
INTRO: THE COMMANDER OF NATO FORCES IN BOSNIA, ADMIRALLEIGHTON SMITH, SAYS IT IS TOO EARLY FOR MILITARY COMMANDERS TORECOMMEND WHETHER U-S PEACE-KEEPING TROOPS SHOULD REMAIN INBOSNIA BEYOND THEIR SCHEDULED DEPARTURE IN DECEMBER. THAT’S ADECISION, HE SAYS , THAT WILL HAVE TO BE MADE LATER THIS YEAR BUTNEARLY FIVE MONTHS INTO THE PEACE-KEEPING OPERATION, ADMIRALSMITH IS RAISING CONCERNS ABOUT CIVILIAN ASPECTS OF THE BOSNIAPEACE AGREEMENT, INCLUDING THE ABILITY OF LOCAL POLICE TOACTUALLY ENFORCE LAW. WE HAVE THIS REPORT FROM V-O-A’S NICKSIMEONE
LATELY, SOME POLITICAL LEADERS IN EUROPE HAVE BEEN PRIVATELYSUGGESTING THE PEACE PROCESS IN THE FORMER YUGLOSLAVIA IS TOODELICATE AND THE COUNTRY IS TOO LIKELY TO SLIP BACK INTO WAR FORNATO FORCES - PARTICULARLY U-S TROOPS ----- TO CONSIDER PULLINGOUT AT THE END OF THE YEAR. PRESIDENT CLINTON HAS REPEATEDLYSAID THAT U-S PARTICIPATION IN THE PEAC-KEEPING MISSION IN BOSNIAWIIL ONLY BE FOR ONE YEAR. NATO COMMANDER SMITH SAYS THEMILITARY, AT THIS STAGE AT LEAST, IS NOT MAKING PREDICTIONS EITHERWAY.
BUT IN EITHER CASE, ADMIRAL SMITH SAID THAT THOUGHT IS BEINGGIVEN TO SCALING BACK THE NATO FORCE, WHICH NOW STANDS ATABOUT 60-THOUSAND TROOPS, AS THE OPERATION HEADS INTO THESUMMER MONTHS. SO FAR NATO COMMANDERS SEE NO EVIDENCE THATTHE WARRING PARTIES ARE PREPARING FOR WHAT IN YEARS PAST HASBEEN THE START OF A WARM WEATHER, SPRING OFFENSIVE.
HE SAYS A KEY TO THE SUCCESS OF THE BOSNIA PEACE PLAN WILL BETHE BOSNIAN ELECTIONS, WHICH, ACCORDING TO THE PEACE PLAN, ARETO BE HELD IN SEPTEMBER, BY THEN NATO COMMANDERS, AS WELL ASTHE SPONSERS OF THE DAYTON PEACE ACCORD, WILL LIKELY HAVE AGOOD IDEA WHETHER BOSNIA’S POST-WAR ATMOSPHERE IS FREE OFTHREATS AND IF, IN ADMIRAL SMITH’S WORDS, POLITICIANS CAN GIVESPEECHES WITHOUT BEING SHOT. SUCH INDICATIONS COULD INFLUENCETHE DECISION ON WHETHER COME DECEMBER, U-S TROOPS STAY ORLEAVE.(SIGNED)
Salient key-based summary
A.1.1 Simple Summarisation Based on User’s Ad-hoc Keywords(Nato/peace/keeping)
INTRO: THE COMMANDER OF NATO FORCES IN BOSNIA, ADMIRALLEIGHTON SMITH, SAYS IT IS TOO EARLY FOR MILITARY COMMANDERS TORECOMMEND WHETHER U-S PEACE-KEEPING TROOPS SHOULD REMAIN INBOSNIA BEYOND THEIR SCHEDULED DEPARTURE IN DECEMBER.
BUT NEARLY FIVE MONTHS INTO THE PEACE-KEEPING OPERATION,ADMIRAL SMITH IS RAISING CONCERNS ABOUT CIVILIAN ASPECTS OF THEBOSNIA PEACE AGREEMENT, INCLUDING THE ABILITY OF LOCAL POLICETOACTUALLY ENFORCE LAW.
TEXT: WITH WAR CRIMES INVESTIGATORS STARTING TO LOOK INTO MASSGRAVES, ADMIRAL SMITH SAYS ENORMOUSLY DIFFICULT TASKS STILLLIE AHEAD AS THE WORLD WATCHES TO SEE WHETHER BOSNIA’S FORMERWARRING PARTIES WILL BE ABLE TO FULFILL TERMS OF THE PEACEACCORD THEY SIGNED TO END THE WAR.
LATELY, SOME POLITICAL LEADERS IN EUROPE HAVE BEEN PRIVATELYSUGGESTING THE PEACE PROCESS IN THE FORMER YUGLOSLAVIA IS TOODELICATE AND THE COUNTRY IS TOOL LIKELY TO SLIP BACK INTO WARFOR NATO FORCES - PARTICULARLY U-S TROOPS - TO CONSIDER PULLINGOUT AT THE END OF THE YEAR. PRESIDENT CLINTON HAS REPEATEDLYSAID THAT U-S PARTICIPATION IN THE PEACE-KEEPING MISSION IN BOSNIAWILL ONLY BE FOR ONE YEAR.
NATO COMMANDER SMITH SAYS THE MILITARY, AT THIS STAGE AT LEAST,IS NOT MAKING PREDICTIONS EITHER WAY.
BUT IN EITHER CASE, ADMIRAL SMITH SAID THAT THOUGHT IS BEINGGIVEN TO SCALING BACK THE NATO FORCE, WHICH NOW STANDS ATABOUT 60-THOUSAND TROOPS, AS THE OPERATION HEADS INTO THESUMMER MONTHS.
SO FAR NATO COMMANDERS SEE NO EVIDENCE THAT THE WARRINGPARTIES ARE PREPARING FOR WHAT IN YEARS PAST HAS BEEN THE STARTOF A WARM WEATHER, SPRING OFFENSIVE.
WHAT WE WANT TO BE ABLE TO DO IS TO KEEP THE MILITARY APART ANDIF WE CAN DO THAT WITHOUT A LARGE FORCE OR WITHOUT A FORCE,THAT REALLY THE GOAL.
Information Visualisation
Represent both the structure & content of the retrieved information
Readily comprehensive manners
Facilitate high level tasks
– Identification of similar/duplicate information– Identification of topic trends– Identification of anomalies
Provides access to different levels of summary
DERA TextScape
DERA TextScape
DERA TextScape
DERA TextScape
DERA TextScape
DERA TextScape
Evaluation Issues
Retrieval effectiveness: Dynamic data sources User’s oriented/task oriented Key word saliency
Summarisation Summaries first Followed by full report
Visualisation Effective in showing the
– Identification of similar/duplicate information– Identification of topic trends– Identification of anomalies
Inter-relationship? Temporal ?
Event Stream Analysis
Analysis of large datasets generated during intensive man/machine interaction
The analysis of the large log file generated by these interaction is assisted by DERA’s Plumber
Plumber - a visual programming tool which enables a new visual environment for data processing
Part of a commercial package of DP+MS
Plumber
Main canvas
Palette ofprocessing
widgets
Libraries ofpre-written
widgets
Built-invisualisers such
as bar charts
Access to externalprocessing tools
such as Excel andMatlab
Store resultsto disk
Java version