nsidc work summary · nsidc work summary october 2009 – october 2010 podag 29 9/20/10 10:54 am...

20
NSIDC Work Summary October 2009 – October 2010 PODAG 29 9/20/10 10:54 AM updated Podag members please review the following summary of work performed in the past year. These reports are intended to provide you with the context and general outline of the specific project and to list the major accomplishments. The authors of each section were asked to provide responses to the following 3 points: 1. Summary paragraph describing the work 2. Summary (bullets or text) of work accomplished since October 2009 3. Any issues or questions for PoDAG consideration After you have read these summaries please send me any questions that you would like us to address at the PoDAG meeting. We will modify the meeting schedule to accommodate the most important questions or issues you all raise. Major Data Streams MODIS Doug Fowler, Product Team Lead NSIDC archives and distributes Level-2 and Level-3 snow cover and sea ice data products from the Moderate Resolution Imaging Spectroradiometer (MODIS) sensor on NASA's Earth Observing System (EOS) Aqua and Terra satellites. These Level-2 and Level-3 products deal with snow cover and sea ice. MODIS (or Moderate Resolution Imaging Spectroradiometer) is a key instrument aboard the Terra (EOS AM) and Aqua (EOS PM) satellites. Terra's orbit around the Earth is timed so that it passes from north to south across the equator in the morning, while Aqua passes south to north over the equator in the afternoon. Terra MODIS and Aqua MODIS are viewing the entire Earth's surface every 1 to 2 days, acquiring data in 36 spectral bands, or groups of wavelengths. http://nsidc.org/data/modis/index.html Activities Collection 5 archive and distribution continues for data from both the Aqua and Terra satellites o Distribution continues to increase over time

Upload: others

Post on 24-Mar-2020

8 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: NSIDC Work Summary · NSIDC Work Summary October 2009 – October 2010 PODAG 29 9/20/10 10:54 AM updated Podag members please review the following summary of work performed in the

NSIDC Work Summary October 2009 – October 2010

PODAG 29 9/20/10 10:54 AM updated

Podag members please review the following summary of work performed in the past year. These reports are intended to provide you with the context and general outline of the specific project and to list the major accomplishments. The authors of each section were asked to provide responses to the following 3 points:

1. Summary paragraph describing the work 2. Summary (bullets or text) of work accomplished since October 2009 3. Any issues or questions for PoDAG consideration

After you have read these summaries please send me any questions that you would like us to address at the PoDAG meeting. We will modify the meeting schedule to accommodate the most important questions or issues you all raise.

Major Data Streams

MODIS Doug Fowler, Product Team Lead NSIDC archives and distributes Level-2 and Level-3 snow cover and sea ice data products from the Moderate Resolution Imaging Spectroradiometer (MODIS) sensor on NASA's Earth Observing System (EOS) Aqua and Terra satellites. These Level-2 and Level-3 products deal with snow cover and sea ice. MODIS (or Moderate Resolution Imaging Spectroradiometer) is a key instrument aboard the Terra (EOS AM) and Aqua (EOS PM) satellites. Terra's orbit around the Earth is timed so that it passes from north to south across the equator in the morning, while Aqua passes south to north over the equator in the afternoon. Terra MODIS and Aqua MODIS are viewing the entire Earth's surface every 1 to 2 days, acquiring data in 36 spectral bands, or groups of wavelengths. http://nsidc.org/data/modis/index.html

Activities

• Collection 5 archive and distribution continues for data from both the Aqua and

Terra satellites o Distribution continues to increase over time

Page 2: NSIDC Work Summary · NSIDC Work Summary October 2009 – October 2010 PODAG 29 9/20/10 10:54 AM updated Podag members please review the following summary of work performed in the

o User Services handles user inquiries which are up two to three times over what they were preceding this past 12 months

• We are continuing to update Google Earth with the MODIS/Terra Snow Cover Monthly L3 Global 0.05Deg CMG (MOD10CM) product

• Preparations for Collection 6 are under way o Load testing should be taking place in September or early October o Reprocessing rates are expected to be 20 to 30x of normal data rates,

thus the need for load testing

MODIS Distribution increases from 9/07 – 9/10

GLAS Doug Fowler, Product Team Lead NSIDC archives and distributes Level-1 and Level-2 data products from the Geoscience Laser Altimeter System (GLAS) instrument aboard the NASA Ice, Cloud, and land Elevation (ICESat) satellite. These products cover ice sheet elevations, sea ice thickness, atmospheric measurements, and vegetation cover. The level 0 and ancillary products are also archived at the data center. The ICESat/GLAS instrument has three lasers, each of which has a 1064 nm laser channel for surface altimetry and dense cloud heights, and a 532 nm lidar channel for the vertical distribution of clouds and aerosols. The three lasers have been operated one at a time, sequentially throughout the mission. To extend mission life, the current operational mode has included 33-day to 56-day campaigns, several times per year. ICESat was successfully decommissioned from operations Saturday August 14, 2010, at approximately 17:37:00 Greenwich Meridian Time (GMT). ICESat was brought down in

0  

500000  

1000000  

1500000  

2000000  

2500000  

3000000  

3500000  

AQUA   TERRA  

2007  

2008  

2009  

2010  

Page 3: NSIDC Work Summary · NSIDC Work Summary October 2009 – October 2010 PODAG 29 9/20/10 10:54 AM updated Podag members please review the following summary of work performed in the

the Barents Sea on 30 August 2010, after operating for more than seven years. http://nsidc.org/data/icesat/index.html

• All campaigns have been reprocessed using Release 31 software o This included data for four (L3K L2D, L2E, and L2F) never- before-

released campaigns o Previous versions of data, 28 & 29, have been deleted from the archive o ECS Distribution is increasing from year to year o GLAS rSCF Subsetter distribution, also increasing

• Work has begun on long-term archive requirements • User Services supporting numerous users questions, typically about data and

data tools • Attending teleconferences on changes that will incorporated into Release 33, the

final release. • Finalizing work with the GSFC P.I. and local scientists on the Sea Ice Thickness

& Freeboard data set and documentation. Expecting it to be released before PoDAG

GLAS Distribution increases from 9/07 – 9/10

0  

100000  

200000  

300000  

400000  

500000  

600000  

Total  

2007  

2008  

2009  

2010  

Page 4: NSIDC Work Summary · NSIDC Work Summary October 2009 – October 2010 PODAG 29 9/20/10 10:54 AM updated Podag members please review the following summary of work performed in the

AMSRE Amanda Leon, Product Team Lead The Advanced Microwave Scanning Radiometer - Earth Observing System (AMSR-E) is a mission instrument launched aboard NASA's Aqua Satellite on 4 May 2002. It is a multichannel passive microwave radiometer that is capable of measuring geophysical variables in the global water cycle, such as snow, sea ice, sea surface temperature, precipitation and soil moisture, providing finer spatial resolution than previously possible with spaceborne microwave radiometers. NSIDC DAAC archives, distributes, and supports all AMSR-E standard products, including Level-1A, -2, and -3 data. In the past year, AMSR-E algorithm refinements have been implemented by the Science Team and reprocessing campaigns have been completed by the Science Investigator-led Processing System (SIPS) leading to improved products.

• L3 Sea Ice: V12 corrects an error in the method for applying the multi-year sea ice mask; V13 corrects an error in the sea ice concentration difference field calculation

• L3 Ocean: V04 incorporates measures to further reduce Radio Frequency Interference (RFI), and the products now include an RFI angle grid as a parameter.

NSIDC developed and released a Google Earth animation showing the past 30/60/90 days of AMSR-E L3 global daily soil moisture (http://nsidc.org/data/virtual_globes/index.html). NSIDC also received and published three AMSR-E related data sets:

Brown, Ross. Canadian Meteorological Centre (CMC) Daily Snow Depth Analysis Data. (http://nsidc.org/data/nsidc-0447.html) Kimball, John. Daily Global Land Surface Parameters Derived from AMSR-E. (http://nsidc.org/data/nsidc-0451.html) Tedesco, Marco. Co-Registered AMSR-E, QuikSCAT, and WMO Data. http://nsidc.org/data/nsidc-0450.html

Page 5: NSIDC Work Summary · NSIDC Work Summary October 2009 – October 2010 PODAG 29 9/20/10 10:54 AM updated Podag members please review the following summary of work performed in the

IceBridge Marilyn Kaminski, Project Lead NSIDC manages data for products derived from NASA's Operation IceBridge aircraft missions, and implements tools and services extending the uses of IceBridge data products. The Operation IceBridge mission, initiated in 2009, collects airborne remote sensing measurements to bridge the gap between NASA's Ice, Cloud and Land Elevation Satellite (ICESat-I) mission and the upcoming ICESat-II mission. IceBridge mission observations and measurements include coastal Greenland, coastal Antarctica, the Antarctic Peninsula, interior Antarctica, the southeast Alaskan glaciers, and Antarctic and Arctic sea ice. The NASA IceBridge mission combines multiple instruments to map ice surface topography, bedrock topography beneath the ice sheets, grounding line position, ice and snow thickness, and sea ice distribution and freeboard. Data from laser altimeters and radar sounders are paired with gravitometer, magnetometer, mapping camera, and other data to provide dynamic, high-value, repeat measurements of rapidly-changing portions of land and sea ice. There are currently 17 instruments included in the IceBridge, ICECAP, and UAF missions, all of which are included in NSIDC's "IceBridge" project. There is one additional instrument flying in '10. There may be different providers or different instruments in future years, depending on the results of the ROSES call. The 18 instruments are managed by 9 different teams with 10 PIs. We currently have completed or are working on ingest of 44 products. 12 of them are published and available to users. In addition to those products, we know there will be L0 products that correspond to these higher-level products (approx 10 more products). We also know there are a significant amount of ancillary data products that will be coming, but have no idea yet as to what/how many that will include. There are also other related data sets that need to be archived, including historical data sets (such as the early ATM data) and flight reports. This list continues to grow. Accomplishments for past year: -­‐ Wrote a proposal for IceBridge data management that was accepted by ESDIS and

HQ -­‐ Sub-award made in early 2010, and work began in February 2010 -­‐ Put together a team from existing staff and new hires -­‐ Ingested, archived, and made available to users 12 data sets from IceBridge ’09

campaigns in Greenland and Antarctica, in advance of schedule requested by NASA HQ to support a public announcement of data availability

Page 6: NSIDC Work Summary · NSIDC Work Summary October 2009 – October 2010 PODAG 29 9/20/10 10:54 AM updated Podag members please review the following summary of work performed in the

-­‐ Engineered streamlined processes for ongoing ingest cycles; and began engineering robust processes for ingest into ECS for data preservation and enhanced data access services.

-­‐ Established Science Working Group for ongoing interchange on technical developments; conducted first meeting in March 2010

-­‐ Participated in planning meetings for Greenland and Antarctica ’10 missions

Issues/questions: -­‐ What exchange of information would the PoDAG membership like to have from the

IceBridge SWG? Molotch and Stearns are members of both teams. Does PoDAG want updates from them, from NSIDC, or neither?

Passive Microwave Data Sets, including Passive Microwave Workflow Improvements Donna Scott, Product Team Lead, Walt Meier PM Scientist NSIDC's Passive Microwave (PM) data sets are one of the longest environmental records from satellites with consistent data sources (SMMR and SSM/I) spanning late 1978 to present. The PM snow and ice product suite includes snow and sea ice products in both polar stereographic and EASE grids and encompasses near-real-time products as well as long-term climate records with considerable quality control measures to assure accuracy and consistency throughout the record. Summary of PM accomplishments for FY 2010 – PoDAG Fall 2010

• NISE team: Reprocessed NISE to V4, providing consistency with V2 • NISE team: Presentation at the MicroRad conference on the NISE F17

calibration procedures and subsequent submission of a manuscript to TGARS • Consolidation of the NISE, EASE-Grid, and Polar Stereographic teams into one

Passive Microwave Team o Consolidation of meetings to one regularly scheduled monthly meeting,

decrease in core team members o Implementation of ticketing system to enhance productivity and

understanding of priorities of the team o Presentation of a passive microwave 101 to educate the team (and

NSIDC staff) on concepts of passive microwave, and the production workflow (quite successful)

• FTP and archive analysis and clean up o Easier maintenance for OPS o Less confusion for users to find data

• Upgraded EASE-Grid TBs from F13 V4 to F13 V6 produced and published for distribution

Page 7: NSIDC Work Summary · NSIDC Work Summary October 2009 – October 2010 PODAG 29 9/20/10 10:54 AM updated Podag members please review the following summary of work performed in the

• All F13 polar stereographic TBs for 2008-2009 produced and published for distribution (includes bad data)

• Bootstrap production code upgraded to produce a preliminary version of the GSFC Bootstrap sea ice product ((This still waiting on last minute changes from Jeff, and validation from Walt, hoping to be done by PoDAG))

o This will eliminate NSIDC’s Bootstrap product, alleviating confusion to users about our multiple sea ice products

• Consideration and validation for changing EASE-Grid from Backus Gilbert to Inverse Distance Squared, necessary for F17 change ((This still waiting on Walt’s validation, hoping done by PoDAG))

• Automation of daily NISE data to Earth Observatory’s NASA’s Earth Observations (NEO) site

• Ingest of enhanced SSMI, AMSRE, Qscat data from D. Long (J. Stroeve proposal)

o Creation of new data set for the enhanced SSM/I-AMSRE, including data set documentation (not yet public, data validation in process)

• Update of polar stereographic TB production code to work with F17 data format • F17 RSS data ingest began Sept 2010, includes 4 years of data (may not be

complete by podag) • Development of new data display tool, NSIDC DataViewer, providing users with

or without IDL, better display, animation, and image creation o Easier maintenance for programmers o Major documentation updates for addition of NSIDC DataViewer and

removal of other IDL tools o Removal of several stand-alone IDL display tools from distribution

• Complete assessment of all polar stereographic related tools o Updates to tools to work with newer versions of Windows operating

system o Comments added to all codes specifying applicable data sets o Updates to data set documentation where needed o Less confusion for users trying to apply our tools to the passive

microwave data sets o Addition of tools to subversion – easier maintenance for programmers

• Defined requirements and approach for overhauling passive microwave data production (work to begin 1st or 2nd quarter of FY2011)

• Submission of two poster presentations for Fall AGU 2010 related to passive microwave team

o Transition from the DMSP SSM/I to SSMIS sensors for NSIDC near-real-time snow and ice climate records – Peter Gibbons, et al.

o Revealing passive microwave data production at NSIDC – Donna Scott, et al.

Issues: The passive microwave team faced obstacles in 2010.

Page 8: NSIDC Work Summary · NSIDC Work Summary October 2009 – October 2010 PODAG 29 9/20/10 10:54 AM updated Podag members please review the following summary of work performed in the

• We fell further behind in providing timely updates to our SSMIS brightness temperature and sea ice data sets. We were not provided with the input data from RSS until September 2010.

o All TBs and sea ice updates were impacted by data source not providing the data when initially stated.

• We faced numerous resource issues, as team members moved to other roles with little/no transition period. Representation from USO, Ops, and programming staff were key support factors. The EASE-Grid updates and preliminary bootstrap creation were the primary data sets impacted by NSIDC resources.

o The team focused on housekeeping tasks, including tools development & clean up, documentation updates, production prep, and FTP/archive cleanup.

o

PoDAG feedback: • Polar Pathfinder Daily 25km EASE-Grid Sea Ice Motion Vectors

(http://nsidc.org/data/nsidc-0116.html) – Chuck Fowler is interested in NSIDC taking over production of this data set. We already distribute it, and receive irregular updates. He will provide his production code, but it may need a lot of work to fit in our system. This is a popular data set and is often several years out of date, prompting many user requests for updates. We would like to work on pulling the production code into our system and streamlining the data for a more regular update. Ideally we would prefer doing this in the Spring of 2011 to ensure communication Chuck before he retires next summer (likely).

• Chuck & Jim Maslanik are also producing an ice age product. NSIDC does not currently distribute this product. The Arctic Sea Ice News & Analysis site often uses this data to enhance the information about current sea ice conditions. Walt is very interested in acquiring this product, adding the production of this product to our system, and making it available for public distribution.

• Based on conversation with NSIDC DAAC management, we need to consider a more regular update to several EASE-Grid related products:

o Global Monthly EASE-Grid Snow Water Equivalent Climatology (http://nsidc.org/data/nsidc-0271.html)

Issue new version with improved regression between SMMR & SSM/I TBs and an atmospheric correction over Tibetan Plateau. Change to a more NRT approach instead of batch processing. Consider NetCDF format.

o Northern Hemisphere EASE-Grid Weekly Snow Cover and Sea Ice Extent V3 (http://nsidc.org/data/nsidc-0046.html)

Change to a more NRT model with automated updates, instead of batch approach.

o Global EASE-Grid 8-day Blended SSM/I and MODIS Snow Cover (http://nsidc.org/data/nsidc-0321.html)

Page 9: NSIDC Work Summary · NSIDC Work Summary October 2009 – October 2010 PODAG 29 9/20/10 10:54 AM updated Podag members please review the following summary of work performed in the

MeaSures Acquisitions Various Poduct Leads, consult Ron Weaver if you have questions The DAAC’s role is to provide counsel on data management issues and house certain products upon NASA HQ approval, possibly with external funding. NSIDC is a participant in several current ReaSON CAN, MEaSURES, and ACCESS projects: Projects with active or potential data transfers to NSIDC DAAC GLIMS: We are archiving and managing the GLIMS data sets. Polar Scatterometry: This project has developed data sets relevant to the cryosphere community. NSIDC scientist Dr. J. Stroeve is a co-investigator on this proposal to the University of Utah and her role and funding specifically calls for archival of data in the NSIDC DAAC. The Cryospheric Change Analysis Web Services (CCAWS, ACCESS, Stroeve PI) project will directly address a Greenland specific science need by allowing the comparison of data and simplifying data access and delivery through the use of modular services designed to support a map server portal to the data. Existing components, especially the existing Searchlight infrastructure, are being reused where possible. Enhanced Resolution SSM/I and AMSR-E Data: As part of the REASoN CAN grant with D. Long at BYU, NSIDC agreed to make available the enhanced resolution passive microwave data developed under this CAN. This includes enhanced resolution brightness temperatures from both SSM/I and AMSR-E. Greenland Ice Mapping Project: Measuring rapid changes in ice flow. (MEaSURES: I. Joughin PI) Dr. T. Scambos at NSIDC is a co-PI on this award. Work at NSIDC under Scambos’ guidance will foster data sets to be archived here. (Will be reviewed at PoDAG meeting.) Ice Velocity Mapping of the Great Ice Sheets, Antarctica. (MEaSURES: E. Rignot, PI) Dr. T. Scambos is a co-PI on this award and will be coordinating NSIDC DAAC efforts. (Will be reviewed at PoDAG meeting.) Earth System Data Record for Land Surface Freeze-Thaw State: Quantifying Terrestrial Water Mobility Constraints to Global Ecosystem Processes. (MEaSURES: J. Kimmel, PI). One data set from this project has already been archived at NSIDC. (Will be reviewed at PoDAG meeting.)

Page 10: NSIDC Work Summary · NSIDC Work Summary October 2009 – October 2010 PODAG 29 9/20/10 10:54 AM updated Podag members please review the following summary of work performed in the

Northern Hemisphere Snow and Ice Climate Data Record, (MEaSURES, D. Robinson, PI). We expect further interactions with Dr. Robinson’s team; however we believe he will manage the data sets produced under this project for some years to come. NSIDC already incorporates output from his snow cover products in our EASE grid Northern Hemisphere snowcover extent data. (Will be reviewed at PoDAG meeting.)

Major Projects

Searchlight and Polaris Donna Scott, Product Owner, Brendan Billingsley Scrum Master1 NSIDC has built a robust, extensible infrastructure to provide our users with the ability to preview and analyze data sets on the Web. The NSIDC Searchlight engine powers the new Beta Advanced Data Search Interface and the Services for Analysis of the Greenland Environment (SAGE) interface. In November the Beta interface will become operational and it’s name will be Polaris. We are changing the name to remove confusion between Searchlight which provides services and infrastructure and Polaris which provides a GUI and data interface to users. The main objective for building the NSIDC Searchlight engine-Polaris interface was to reduce the distance between our users and our data. See: http://newice:8000/data/searchlight/about.html The NSIDC Searchlight engine supports the following functionality: • Tools and services to support inventory-level, simple-to-use, on-line graphical

interface to NSIDC's cryospheric data. (Polaris, Beta Advanced Data Search interface.)

• Get the users just the subsetted/reprojected/reformatted data they need, in the format they want, as quickly as possible.

• Allow users to display and interact with the data. • Provide new data delivery services to our users that include: • On-the-fly download of archived files. • On-the-fly reformatting to

o geotiff (gridded data) o netCDF (gridded and point data)

1 Searchlight is being developed using the Agile Project Management approach. Product Owner and Scrum Master have specific definitions in the Agile approach. Product Owner roughly translates to Project Leader, and Scrum Master to software development manager.

Page 11: NSIDC Work Summary · NSIDC Work Summary October 2009 – October 2010 PODAG 29 9/20/10 10:54 AM updated Podag members please review the following summary of work performed in the

o ASCII csv (point data). • On-the-fly reprojection to a limited set of canned projections:

o Cylindrical Equidistant (latitude/longitude) o Lambert Northern Azimuthal o Lambert Southern Azimuthal o Cylindrical Equal-area o Polar Stereo Graphic North o Polar Stereo Graphic South o Easily extensible to other projections.

• On-the-fly resolution resampling. • Packaging that includes links to data set documentation. • Packaging that includes data provenance for any reformatting /reprojecting

/resampling operations that are done. • User-controlled sorting and filtering of search results. • Inventory-level graphical Results Explorer functionality providing browse

image movies and selected metadata exploration. • Bookmark-able links into search results that can be saved, shared, or e-

mailed. • Free text search. Polaris serves as a navigation point for NSIDC data sets using Searchlight infrastructure adding subsetting, reformatting, and reprojecting capabilities. Release Milestones • Successful demonstrations at NASA One Booth and NSIDC display booth at

Fall AGU, San Francisco, Dec 2010 o we reached a large audience (can USO get us a number?) at AGU

with live demos of the interface o we presented 2 posters on the work we had done

Collins, J., M. J. Brodzik, B. Billingsley, Lineage management and user support considerations for on-demand data

Billinsgley, B. Software reuse example and challenges at NSIDC

• Beta 0.1 Release, January 2010 o we released the first Beta version of the system, delivering Searchlight

services for 14 of our most popular data sets • 4 minor Beta releases between February and August • Current efforts are targeted for our Production 0.1 Release, December 2010 Searchlight Infrastructure • Major work on Enterprise Data Base

o to replace current operational MDDB collection-level holdings, and support new application development, including:

new writer's interface to populate and maintain holdings dynamic, on-line catalog pages

Page 12: NSIDC Work Summary · NSIDC Work Summary October 2009 – October 2010 PODAG 29 9/20/10 10:54 AM updated Podag members please review the following summary of work performed in the

dynamic, on-line forms pages on our web site internal ops and USO applications

o to simplify maintenance by reducing the number of data bases we require for operations

o to expand DB holdings to include accurate, complete file-level information

o to expose holdings to users within 24 hours of acquisition o to establish processes and procedures for data base integrity and the

ability to regenerate the DB contents directly from our holdings o to improve the position of NSIDC to respond to the DB needs of

current and potential funding, e.g. IceBridge, Libre, OPERA o data base work was performed by our talented developers stepping up

in the absence of a staff DBA, more recent work has been accomplished by two new DBA we hired in May

• Supported SAGE project goals • Supported OpenSearch and Data Casting project goals • Continuing work to support IceBridge

o data base enhancements to handle airborne data types o prototyping connection to ECS data holdings directly via web service

APIs • Grid technology for efficient, scalable job processing • Job persistence so jobs are never lost and output data can be regenerated

after it is removed • System makes use of continuous integration procedures, which incorporate

integration and system testing into daily work process • Refinements to make deployment process as automated as possible The Production System will provide: • faster performance and improved user experience, by separating data

processing system from web server hardware • faster performance and improved data reliability, by replacing MDDB with

new, more robust Enterprise Data Base • improved system reliabiltity, by supporting persistent user jobs (that can be

recovered in case of system restart) • improved user experience through interface bug fixes Polaris • production level user interface utilizing the Searchlight interface • collection-level search and access for 14 of our most popular gridded data

sets, including: o gridded final and preliminary sea ice concentrations o gridded Snow Water Equivalent data o gridded brightness temperatures (SMMR, SSM/I, AMSR-E) o gridded AVHRR data o GLAS Greenland DEM o ice sheet melt data

Page 13: NSIDC Work Summary · NSIDC Work Summary October 2009 – October 2010 PODAG 29 9/20/10 10:54 AM updated Podag members please review the following summary of work performed in the

o SNODAS (NOHRSC snow data assimilation) data • on-the-fly browsing, subsetting, reprojecting, reformatting, downloading • modern, OpenLayers-based map interface • output formats include binary, netCDF (to appeal to modelling community)

and geotiff (in response to frequent user requests) • text-based search

Recent unsolicited comment from a Searchlight user: “I would also like to mention how brilliant I think the Beta Advanced Data Search tool is. I use a few of the NSIDC data sets, and it can save scientists such as myself considerable time converting between different grids and formats. I have also found the interface very user friendly and I like the way I can preview images on the data in the Results Explorer, and subset the data as required. Thank you all and I look forward to seeing more of your data sets available in this way. “

Searchlight Management • Continued refinement of agile processes on the Searchlight team, including:

o periodic reviews of progress with stakeholders o team retrospectives to review and modify development processes o change to more formal approach with a scrum master and a product

owner o enhanced standing scrum to better understand goals and organize

workload • Established team response procedures and priority triage; defined and

implemented roles for USO, ops, writers, developers to support the Beta system

• Regular exposure of system to users, to incorporate user feedback early and often.

SAGE (NASA ACCESS funded project) Dave Gallaher Project Lead (Julienne Stroeve PI) The overall aim of this NASA ACCESS-funded project is to create multi-faceted web services for access, browse, online analysis and delivery of data sets important for understanding the processes that control the mass balance of Greenland’s ice sheet. The project addresses a Greenland specific science need by allowing the comparison of data and simplifying data access and delivery through the use of modular services designed to support a map server portal to the data. Visualization and Analysis Processes allow the user to explore the available data through the use of overlays, time series, and plots; and with user-specified aggregation, averaging, or sampling intervals.

Page 14: NSIDC Work Summary · NSIDC Work Summary October 2009 – October 2010 PODAG 29 9/20/10 10:54 AM updated Podag members please review the following summary of work performed in the

SAGE Accomplishments FY2010 The majority of the project has been completed. Over 1000 project related tickets have been generated on this Agile development of Services for the Analysis of the Greenland Environment (SAGE). We are currently completing several GUI enhancements related to the production of plots and analysis as a part of the SAGE system. The interactive analysis component of the SAGE application is nearly completed. User can select from 10 datasets of data and select a large number of plots and analysis based on the parameters of interest (Figure 1). The data sets currently available through SAGE include microwave Brightness Temperatures from SSM/I and AMSR-E, AVHRR Polar Pathfinder products, DEM elevation, QuikSCAT and surface melt from SSM/I. OpenSearch data discovery services are completed. PoDAG Issues We would appreciate any comments from PoDAG on the GUI and the services. The analysis function developed by SAGE will eventually be incorporated into the Searchlight driven Polaris GUI.

Green Data Center Dave Gallaher Project Lead, (Mark Serreze PI) NSIDC has won an NSF ARRA grant for $525,000 to build a “green” data center. The project includes; cooling with external air and indirect evaporative cooling; consolidation of facilities; server virtualization and hot/cold aisle; a solar/battery array for power; internet connectivity to 1 Gb/s. While our research provides evidence of climate change, there is an irony in the large carbon footprint of NSIDC’s computing center. By creating a 'green data center', NSIDC will demonstrate leadership to CU and the nation. This project will reduce the computer center’s carbon footprint and reduce the power need for cooling by more than 90% (Figure 1). The design work was funded by the DAAC along with project staff time. The NSF grant covers only infrastructure and construction cost.

Page 15: NSIDC Work Summary · NSIDC Work Summary October 2009 – October 2010 PODAG 29 9/20/10 10:54 AM updated Podag members please review the following summary of work performed in the

Figure 1. Consolidated IT facility, with upgraded cooling computer room air conditioning (CRAC) units, uninterruptible power supply (UPS), and connectivity. Green Data Center Accomplishments FY2010

• Grant submitted • Negotiations with NFS completed • Final design completed • Project execution plan completed • Project awarded June 2010 • University issues resolved • Bid documents to be issued in September

Green Data Center Issues for PoDAG PoDAG may want to stay informed on the status of this fairly high profile project. We expect the project to be completed during FY2011.

Tech Infusion Projects Ron Weaver, Overall Project Coordinator In May of 2010 ESDIS released an informal request for technology infusion proposals to the earth science data centers (ESDC). The intent was to provide some modest incremental funding to the ESDCs for joint work on technologies to enhance our data systems in a collaborative manner. Most of the work is to be complete by the end of 2010 (12/31/2010).

Page 16: NSIDC Work Summary · NSIDC Work Summary October 2009 – October 2010 PODAG 29 9/20/10 10:54 AM updated Podag members please review the following summary of work performed in the

NSIDC was successful on three proposals as described below. Below is a high level bullet list of what we are doing.

• Install OPENDAP interface to Searchlight • Install Data Casting software • Open up Searchlight subsetting through an API • Consult with ASF on NetCDF, HDF5 and provide some of our holdings in

formats more compatible with ASF SAR data Simple Subset Wizard Proposal – GES DISC lead, Lynnes

1. Callable API to Searchlight subsetter on a specified and limited set of data 2. SSW requests, we generate and stage a subset of a data set (time, space only) 3. Data set(s) are not yet chosen, but the “Get Data” and Open Search protocols

being developed under Searchlight and SAGE will provide the API to the SSW. 4. Work is on schedule

Consolidated Level 2 Swath Search and Subsetting Web Services – JPL PO.DAAC lead, Thompson Enhance PO.DAAC’s NAIAD Level 2 swath search and subsetting system. Add RESTful search and subset web services

1. Identify data sets appropriate for inclusion into NAIAD and provide documentation

a. MODIS L2A Snow product is first candidate 2. Install and configure OPENDAP server software

a. Server is installed, we are working on network connection issues between the ECS FTP site and the OpenDAP server – really this is a firewall rules issue

3. Configure data casting software to create a. A feed of historical data b. Real time feed of update granules

Making Synthetic Aperature RADAR Data Accessible to Terrestrial Ecologists – ASF Lead, Cunningham Our proposal with ASF was combined with one ASF submitted with ORNL.

1. Provide consultation services with ASF on NetCDF and HDF5 data formats 2. Install OPENDAP interface to Searchlight and stand up several data sets

complementary to SAR data 3. Present combined results to UWGs for testing and evaluation

In addition to the above funded proposals, NSIDC and LP.DAAC are considering how we can staff a proposal that was not funded. That work is to support MODIS reprojection tool adaptation to web services environment (MRTweb) development

Page 17: NSIDC Work Summary · NSIDC Work Summary October 2009 – October 2010 PODAG 29 9/20/10 10:54 AM updated Podag members please review the following summary of work performed in the

with LP.DAAC and ASF. LP.DAAC will modify the MRTWEB tool to allow us to acquire MODIS products and incorporate them into SAGE and Searchlight. ASF will provide us with geotif versions of Radarsat-1 data, which has been released to the IPY community. All data are just for Greenland.

1960’s Sea-ice Extents from Early NIMBUS Satellites David Gallaher, Project Lead There is a wealth of early Earth-observing satellite data that were never fully explored due to the limitations of historical programs and processing systems. The data was generally of high quality but the output from the early computers was marginal by today’s standards. With the recent recovery of some of these data by GSFC, there is now an opportunity to use this 1960s vintage data to create sea ice extent, climate data records (CDRs) for this era. The objective of this project is to generate monthly (or better) sea- ice extent CDRs for the time intervals covered by NIMBUS I, II and III and TIROS 9 &10. Prior Nimbus analysis by J. Sissala (1972) determined the Antarctic sea Ice extent for 2 discrete months in 1966, 1969 and 1970 from Nimbus AVCS and HRIR data. Their data indicates that the 1960s may have had higher sea ice variability then the existing 30-year record. This new project will reexamine this earlier work and will develop a multi-step process to define and build the new CDRs. The initial step is the reprocessing and correcting of all the available HRIR data. The available MRIR data will be examined to help in the processing. AVCS visible band video images and TIROS visible images (film) will be acquired and scanned to provide additional channels for the analysis. Next techniques must be developed to determine the monthly average sea-ice extent from these data. We expect clouds will be the most difficult challenge in the process. With data records provided on multiple orbits on a daily basis, our preliminary research confirms that we can separate clouds from ice by animating the data in Google Earth. Finally, the standards for these 1960s CDRs must be published along with the resultant data.

Page 18: NSIDC Work Summary · NSIDC Work Summary October 2009 – October 2010 PODAG 29 9/20/10 10:54 AM updated Podag members please review the following summary of work performed in the

Nimbus II September 23rd, 1966 Nimbus Accomplishments FY2010

• Initial concept tested under CIRES IRP grant • Dataset processed into Google earth as proof of concept • Potential solution for harmonic distortion determined • Proposal/request submitted • Funding approved • Contract for software services completed • Source of Nimbus AVCS data found • Project Kickoff September • Staff hiring started • Process and procedures determined

NIMBUS Project Issues for PoDAG The long-term hope of this project (this is beyond the scope of the current project) is to create monthly sea-ice extents from 1962-1979. This will not be possible for all months and will be difficult in the arctic before 1964. PoDAG could offer suggestions on how to proceed and if they are aware of any old NIMUS tapes.

Page 19: NSIDC Work Summary · NSIDC Work Summary October 2009 – October 2010 PODAG 29 9/20/10 10:54 AM updated Podag members please review the following summary of work performed in the

ISO Metadata Tiger Team Siri Jodha Khalsa, Team Leader, Ron Weaver PI NASA ESDIS requested NSIDC to lead a tiger team to investigate how ESDIS data systems, primarily ECHO could adapt to support ISO metadata standards. A project team has been formed under Dr. Siri Johda Khalsa’s leadership to conduct this investigation over the coming 4-5 months. The "Metadata Evolution for NASA Data Systems" (MENDS) project, is analyzing the ISO 19115 metadata standard and its applicability to the NASA Earth Science Data Information System. The set of metadata standards in the ISO 191xx family will be considered, as needed.

Other Projects

Metrics and ACSI survey Reports on these topics will be coming in a few days.

Sensing our Planet Jane Beiltler, Project Lead Sensing Our Planet, known informally as the "DAAC Annual," is a yearly print publication designed to highlight research uses of Earth observing data from NASA’s Science Mission Directorate, for the purpose of demonstrating the utility of the EOSDIS data centers to science. The twelve articles produced each year are also published online on the NASA Earth System Science Data and Services Web site. This DAAC-sponsored publication now encompasses other EOSDIS data centers such as CDDIS, MODAPS LAADS and Ocean Biology Processing Group. Each of the twelve data centers will be represented in at least one feature article, with some represented in more than one article, as permitted by available resources and story leads.

Page 20: NSIDC Work Summary · NSIDC Work Summary October 2009 – October 2010 PODAG 29 9/20/10 10:54 AM updated Podag members please review the following summary of work performed in the

Title page to current edition under review

Table of Contents for the current SOP version, in press