An introduction to McIDAS the Man-computer Interactive Data Access System

An introduction to McIDAS the Man-computer Interactive Data Access System(as part of the 22st Annual CIMSS Summer Workshop on Atmospheric and Earth Sciences 23 June 2014)10:30 AM 12:30 PM - McIDAS/satellite overview (Gary); McIDAS-V training (Jay and Bob). - McIDAS hands-on development of mini-projects (w/additional mentors Rick, Will, Barry, and Mat) 01:15 PM 02:15 PM - review McIDAS mini-projects, concluding with student presentations of such.

11. Thanks to Becky Schaffer and McIDAS folks for McIDAS slides taken from the training ppts from the Jun 2011 Beijing trip.2. File updated, with 2012 schedule, MUG users, and current GOES DPI example. File name: McIDAS-ovrvw-SmrWrkShp-v20120625.ppt.Final (post CSW) file updated, with (updated) sats-o-day-gsw-20120627.ppt slide to: McIDAS-ovrvw-SmrWrkShp-v20120627.ppt.File updated, with 2013 schedule, sats-o-day slide, and current GOES Sounder examples (20pan and EUS DPI). File name: McIDAS-ovrvw-SmrWrkShp-v20130624f.pptx.

Who We Are SSEC (Space Science and Engineering Center) is part of the Graduate School of the University of Wisconsin-Madison (UW).

SSEC hosts CIMSS (Cooperative Institute for Meteorological Satellite Studies), including a NOAA/NESDIS (National Oceanic and Atmospheric Administration /National Environmental Satellite, Data, and Information Service) research branch ASPB (Advanced Satellite Products Branch).

What Is McIDAS-V?McIDAS Man computer Interactive Data Access System

Powerful data analysis and 3-D visualization toolMcIDAS-V is the fifth generation of McIDASV stands for 5 (the Roman Numeral V)The first generation of McIDAS began in 1972

Storm cloud temperatures, showing overshooting tops in 2-D from above

Satellite composite image overlaid with GFS relative humidity contour cross-sectionSame overshooting tops, rotated to view from the side

Viewing data, developing algorithms, validating resultsIntegration of geophysical dataEase of reprojectionRemote and local data accessIncludes a bridge to McIDAS-X, allowing X users to continue using legacy code, but to visualize in McIDAS-V

McIDAS-X VisAD + IDV + HYDRA = McIDAS-VWhat Is McIDAS-V?McIDAS Users Group (MUG) members June 20123TIER Group Seattle, WAAgencia Estatal de Meteorologia (AEMET) Madrid, SpainAntarctic Meteorological Research Center Madison, WIAustralian Bureau of Meteorology Melbourne, AustraliaAviation Weather Center - Kansas City, MO [NOAA]The Boeing Company Seattle, WAComision Nacional de Actividades Espaciales (CONAE) Buenos Aires, ArgentinaComprehensive Large Array-Data Stewardship System (CLASS) Asheville, NC [NOAA]Energia Logistics Ltd. Long Beach, CAEnvironment Canada Downsview, OntarioEnvironmental Satellite Processing Center Suitland, MD [NOAA]European Organization for the Exploitation of Meteorological Satellites (EUMETSAT) Darmstadt, GermanyHarris Corporation Melbourne , FLHoneywell Aerospace Electronic Systems Redmond, WAHong Kong Observatory Kowloon, Hong KongImpactWeather, Inc. Houston, TXITOCHU Techno-Solutions Corporation Tokyo, JapanJohnson Space Center Spaceflight Meteorology Group Houston, TXKwajalein Atoll - Atmospheric Technology Services Company Kwajalein AtollMasdar Institute of Science and Technology Abu Dhabi, UAEMexico National Water Commission Mexico D.F., MexicoNASA Langley Research Center Hampton, VANASA Marshall Space Flight Center Huntsville, ALNational Hurricane Center Miami, FL [NOAA]National Transportation Safety Board Washington, DCNational Weather Service Pacific Region Headquarters Honolulu, HI [NOAA]National Weather Service Western Region Headquarters Salt Lake City, UT [NOAA]Northrop Grumman Information Systems McLean, VARange Weather Operations, Cape Canaveral Air Station Patrick Air Force Base, FLRegional and Mesoscale Meteorology, Cooperative Institute for Research in the Atmosphere Fort Collins, CO [NOAA]Storm Prediction Center Norman, OK [NOAA]Telvent DTN Burnsville, MNUnidata Program Center Boulder, COUniversity of Wisconsin - Madison, Space Science & Engineering Center Madison, WIVisionTech, Inc. Ibaraki, JapanWeather Central, LP Madison, WIWeather Decision Technologies, Inc. Norman, OKWeathernews America, Inc. Norman, OKWTVT Fox 13 Weather Tampa, FL

On 6 December 1966, the Applications Technology Satellite (ATS-1) was launched. We have had the benefit of the geostationary perspective for over 45 years!ATS-1's spin scan camera (UWs Suomi and Parent 1968) provided full disk visible images of the earth and its cloud cover every 20 minutes. The spin scan camera on ATS-1 occurred because of extraordinary efforts by Verner Suomi and Homer Newell, when the satellite was already well into its fabrication.Verner E. Suomi and Robert J. Parent

ATS-3 1967-Nov-18 15:03Z

Professor Suomi and McIDAS

(Man computer Interactive Data Access System)1972 McIDAS2014 McIDAS-VIncluding VIS-AD and HYDRAanWater vapor tracked winds from Meteosat during FGGE (the First Global Atmospheric Research Program (GARP) Global Experiment)

(15 Nov 1979)1968WINDCO generates wind vectors from ATS images1972First Generation of McIDAS Runs on Harris /5 with 96 KB of programmable memory and 2-5 MB hard drives1976GOES ingest system added to McIDAS1979Wichita Falls, TX tornado disaster: Congress directs operational McIDAS system to be installed at the NOAA National Severe Storms Forecast Center1983Peoples Republic of China funds port of McIDAS software to IBM mainframe1985McIDAS runs on PCs under DOS operating system1987McIDAS runs on PCs under OS/2 operating systemand McIDAS Users Group is formed1991McIDAS runs on UNIX workstations1994Satellite and conventional data is served from UNIX workstations, beginning the use of ADDE (Abstract Data Distribution Environment)1997McIDAS Users Group sunsets support for mainframe McIDASHistory of McIDAS19701980199020001111History of McIDAS-V2006 - Investigations of a new approach to data analysis and visualization2007 Collaboration with Unidata to advance VisAD and IDV as the basis of McIDAS-V2008 McIDAS-V becomes an alpha 2009 January beta 12010 January beta 52010 September V1.02012 May V1.2 (scripting capability demonstrated and promoted)2014 June latest version is V1.4Eastern Hemisphere:Meteosat-8,7INSAT-3A,3B,3DKalpanaElectro-LFY-2D,ECOMS-1MTSAT-1RGOES-14- Imager- Sounder105 W(Some) Meteorological Satellites and Instruments (2014)GOES-13- Imager- Sounder75 WGOES-12- Imager- Sounder60 WGOES-15- Imager- Sounder135 WTerra- MODIS10:30Aqua - MODIS AIRS13:30Meteosat-10 MSG-2- SEVIRI0 EMTSAT-2(Himawari-7)Imager145 ENOAA-18 - AVHRR/3- HIRS/4 - AMSU - MHS15:00NOAA-19 - AVHRR/3- HIRS/4 - AMSU - MHS13:30~ 36,000 km -- 24 hrs~ 850 km -- 100 minPMAMDMSPF-17 - SSM/IS05:30DMSPF-18 - SSM/IS08:00Suomi NPP - VIIRS CrIS ATMS13:30(http://www.wmo.int/pages/prog/sat/satellitestatus.php)Metop-A - AVHRR/3 HIRS/4 IASI - AMSU- MHS09:30 Metop-B - AVHRR/3 HIRS/4 IASI - AMSU- MHS09:30 GEOLEO[ sats-o-day-gsw-20140620.pptx ]X13(1) File name/update is: sats-o-day-gsw-20120627.ppt.(2) Launch dates: - 1999-12-18 --- Terra - 1998-03-06 --- Meteosat7 - 2001-07-23 --- GOES-12 - 2002-05-04 --- Aqua - 2002-06-24 --- NOAA-17 - 2005-05-20 --- NOAA-18 - 2005-12-22 --- Meteosat-9 - 2006-05-24 --- GOES-13 - 2006-02-18 --- MTSAT-2 - 2006-10-19 --- Metop-A - 2006-11-04 ---DMSP F-17 - 2009-02-06 --- NOAA-19 - 2009-10-18 --- DMSP F-18 - 2010-03-04 --- GOES-15 - 2011-10-28 --- Suomi NPP(3) Update as: sats-o-day-gsw-20130623.pptx.

Convection case study:19 June 2007MODIS data - define a transect to display radiance measurements14Can drag transect across image to view the data interactively. This would be a good tool to use with a product. Validation tool.

CIMSS Cloud Top Pressure15Here a window channel image is displayed with a derived cloud top pressure on top. The data probe is used to compare the product with the source data.15

Setvak:http://www.convection-wg.org/sandwich.phpBrunner et al.: http://cimss.ssec.wisc.edu/snaap/enhanced-vThunderstorm features: over-shooting top and enhanced-V (thermal couplet)16Trying to reproduce a product that is traditional done using a photoshop merge. The sandwich product was developed by Martin Setvakhttp://www.convection-wg.org/sandwich.php to highlight regions of convection.

Here is the sandwich product as produced in McV. This is an enhanced-V pattern Jason Brunner, KirsBedka (http://cimss.ssec.wisc.edu/snaap/enhanced-v/)

16

AIRS Level 1B window channel image (grey-scale) and moveable 2-D slice of ECMWF-AIRS Single FOV water-vapor retrieval (color-scale). Slice values are re-sampled on the fly from the 3-D difference field and auto-updated as the slice is dragged in space - demonstrating interactive direct manipulation, data integration, and python driven data computation.Advanced Display Capability1717

High spectral data from AIRS (onboard the Aqua satellite) is used to generate the UW Cloud Amount Vertical Profile (CAVP), shown as a 3-D cloud in McIDAS-V .18A 3-D cloud product the UW CAVP (Cloud Amount Vertical Profile) - orbit overlayed on topography. CAVP is derived from AIRS data. See work by E. Garms et al.18Bringing observations of clouds together:MODIS (passive) and CALIPSO (active)

1919

AMV derived wind color scaled by wind speed ; GFS gridded wind field in magentaUnder development: interrogation of vertical structure of surrounding reference winds model analysis and/or in-situ obs at location of flagged AMV derived wind.20Steve Wanzong (Atmospheric Motion Vector). Everything not magenta is a satellite derived wind.20

Infrared electromagnetic spectrum of the atmosphereSpectral response functions for the 18 infrared GOES Sounder bands, separated into long, mid, and shortwave sections.Incoming solar radiation peaks in the visible, near 0.5 um (corresponding to a radiative temperature of 5780K for the Suns surface); outgoing terrestrial radiation peaks in the infrared, near 11 um (corresponding to an average earth radiative temperature of 255K). [ from Weins Law] Various gaseous constituents effect the amount of absorption at the different wavelengths. All spectral bands of the GOES Sounder

( GOES-13 (East) at 15:46 UTC on 23 June 2013 )Looking at real-time GOES Sounder data

Look at all spectral bandsLook at a Sounder product

cimss.ssec.wisc.edu/goes/rt/exp-work.php[This workshop PowerPoint is available at -- http://cimss.ssec.wisc.edu/~garyw/sum-wrkshp/2014/.]Atmospheric soundings from geostationary orbitBelow is total precipitable water (TPW) derived from the Sounder on GOES-13 at 12 UTC on 23Jun 2013.As a cold front drops across the N Hi Plains bringing drier air, the E and rest of the central US stay under very moist air (previously advected from the Gulf of Mexico region) as a warm front moves up across the Great Lakes. Smooth GOES retrievals (profiles) are similar to near-by radiosonde profiles.[ http://cimss.ssec.wisc.edu/goes/realtime/eus/begin-eus.html ]

Sift and Winnow a Wisconsin idea

Extra slidesWho We Are SSECScientists - examining the Earth from satellites to gain insight into weather and climate

Modelers - developing diagnostic and forecast models to explain the weather and climate

Engineers - developing new observing tools for spacecraft, aircraft, and ground-based platforms

Data Center real-time satellite imagery from 16 satellites and managing online archive of 1 Petabyte (back to 1978)

Software developers - creating tools to visualize and manipulate data for use by researchers and operational meteorologists

What Is McIDAS-V?Uses an extensible framework - Built a framework on top for adapting new sources of data (format and type; local or remote), for updating user interface components, and for creating novel displays and analysis techniques

Developed in the Java programming language object oriented; write once - run anywhere (very portable)

Persistence mechanism (bundles) for saving and sharing interesting displays/analysis with other McIDAS-V users

Python based user defined computation for scripting

Java-based, open-source, and freely available

IBMMainframeHistory of McIDASMainframe CommunicationNOAAPORTSignal

ClientsServersHistory of McIDASADDE Client-Server Communication(Abstract Data Distribution Environment)Fundamental CIMSS research: striving to make quality real-time GOES Sounder radiance observations into practical useful information for weather forecasting

Atmospheric continuity and evolution are clearly evident in multi-spectral animation.Where will clouds be? Comparison between observed imagery (bottom) and forecast imagery (top) builds confidence in how well the CRAS model is assimilating retrieved GOES Sounder cloud and moisture information.

Where will forecast (GFS) moisture need to be modified, monitoring trends, to provide a better forecast for convection (as across Texas)? Differences between retrieved GOES Sounder TPW and the GFS forecast values are plotted over the GOES TPW Derived Product Imagery (DPI). [1800 UT 2 Apr 2004]Atmospheric soundings from geostationary orbit

Below is total precipitable water (TPW) derived from the Sounder on GOES-13 at 12 UTC on 21 Jun 2012.As a cold front moves across GrtLks and Cen Plns, dry air fills in across N Plns as moist air (previously advected from the Gulf of Mexico region) remains over most of rest of E US.Smooth GOES retrievals (profiles) are similar to near-by radiosonde profiles.[ http://cimss.ssec.wisc.edu/goes/realtime/eus/begin-eus.html ]

The setting for 23 June 2013

500Z/300 winsfc TD/850 winDavenport (IA) radiosonde from 12 UT 23-Jun-2013(with sfc parcel of 86F (69F TD) at 17 UT)