independent technology assessment within the esip testbed
TRANSCRIPT
![Page 1: Independent Technology Assessment within the ESIP Testbed](https://reader035.vdocument.in/reader035/viewer/2022062523/587a8e581a28ab58288b662f/html5/thumbnails/1.jpg)
and 170+ member organizations
ESIP is supported by
www.esipfed.org @ESIPfed | #ESIPfed
INDEPENDENT TECHNOLOGY ASSESSMENT WITHIN THE ESIP TESTBED
Annie Burgess1, Erin Robinson1 and John Graybeal21ESIP, 2Stanford University
18 December 2016 | AGU Fall Meeting
![Page 2: Independent Technology Assessment within the ESIP Testbed](https://reader035.vdocument.in/reader035/viewer/2022062523/587a8e581a28ab58288b662f/html5/thumbnails/2.jpg)
EVALUATION NEEDEarth sciences technologies benefits from diverse
perspectives and end-user input.
![Page 3: Independent Technology Assessment within the ESIP Testbed](https://reader035.vdocument.in/reader035/viewer/2022062523/587a8e581a28ab58288b662f/html5/thumbnails/3.jpg)
• Workspace for projects to mature and gain visibility.
• Funding mechanism for project
development.
• Framework for technology evaluation.
ESIP TESTBED
![Page 4: Independent Technology Assessment within the ESIP Testbed](https://reader035.vdocument.in/reader035/viewer/2022062523/587a8e581a28ab58288b662f/html5/thumbnails/4.jpg)
BACKGROUND• NASA’s Advanced Information Systems Technology (AIST)
Program supports information technology development.
• A Technology Readiness Level (TRL) is used to assess project maturity.
• TRL assessment is internal only.
• Research to operations transitions have suffered from surprises in technology adoption.
![Page 5: Independent Technology Assessment within the ESIP Testbed](https://reader035.vdocument.in/reader035/viewer/2022062523/587a8e581a28ab58288b662f/html5/thumbnails/5.jpg)
BACKGROUND• NASA’s Advanced Information Systems Technology (AIST)
Program supports information technology development.
• A Technology Readiness Level (TRL) is used to assess project maturity.
• TRL assessment is internal only.
• Research to operations transitions have suffered from surprises in technology adoption.
![Page 6: Independent Technology Assessment within the ESIP Testbed](https://reader035.vdocument.in/reader035/viewer/2022062523/587a8e581a28ab58288b662f/html5/thumbnails/6.jpg)
ESIP/AIST COLLABORATIONESIP was tasked with:
• Providing an independent assessment of AIST project TRL.
• Identifying
opportunities/roadblocks for project infusion within the broader Earth science community.
![Page 7: Independent Technology Assessment within the ESIP Testbed](https://reader035.vdocument.in/reader035/viewer/2022062523/587a8e581a28ab58288b662f/html5/thumbnails/7.jpg)
EVALUATION GOALS• Achieve consistency,
traceability and defensibility of evaluation results.
• Be recognized as comprehensive and fair.
• Provide a valuable experience for PIs and project evaluators.
![Page 8: Independent Technology Assessment within the ESIP Testbed](https://reader035.vdocument.in/reader035/viewer/2022062523/587a8e581a28ab58288b662f/html5/thumbnails/8.jpg)
TIMELINELead Time Aug 1 Aug 15 Sep 1 Sep 15 Oct 1 Oct 15 Nov 1 Nov 15 Dec 1
Project Selection
PI Communications
Evaluator Selection
Intro. Telcons + Planning
Testing Period
Evaluators Final Report Prep.
ESIP Final Report Prep.
Reports Submitted to AIST
AISTESIP
Evaluators
![Page 9: Independent Technology Assessment within the ESIP Testbed](https://reader035.vdocument.in/reader035/viewer/2022062523/587a8e581a28ab58288b662f/html5/thumbnails/9.jpg)
WORKFLOW
ESIP Edits and Submits Reports to AIST
Evaluators Submit Final Report Content
Evaluators Fill Out Evaluation Structure
Evaluators Carry Out TestingEvaluators/AIST PIs Create Test Plan
ESIP Selects Evaluators AIST Selects Projects
8 Evaluators: - 1 Private Sector - 1 Non-Profit - 2 Federal Government - 4 Academic
3 Projects- ESIP solicits suggestions from PI.- ESIP reaches out to community.
![Page 10: Independent Technology Assessment within the ESIP Testbed](https://reader035.vdocument.in/reader035/viewer/2022062523/587a8e581a28ab58288b662f/html5/thumbnails/10.jpg)
WORKFLOW
ESIP Edits and Submits Reports to AIST
Evaluators Submit Final Report Content
Evaluators Fill Out Evaluation Structure
Evaluators Carry Out TestingEvaluators/AIST PIs Create Test Plan
ESIP Selects Evaluators AIST Selects Projects
Communicate with:- Telecons- Slack- Wiki- In-Person Meet-ups- Email
1-2 Telecons
![Page 11: Independent Technology Assessment within the ESIP Testbed](https://reader035.vdocument.in/reader035/viewer/2022062523/587a8e581a28ab58288b662f/html5/thumbnails/11.jpg)
WORKFLOW
ESIP Edits and Submits Reports to AIST
Evaluators Submit Final Report Content
Evaluators Fill Out Evaluation Structure
Evaluators Carry Out TestingEvaluators/AIST PIs Create Test Plan
ESIP Selects Evaluators AIST Selects Projects
Checklist for software development best practices
![Page 12: Independent Technology Assessment within the ESIP Testbed](https://reader035.vdocument.in/reader035/viewer/2022062523/587a8e581a28ab58288b662f/html5/thumbnails/12.jpg)
CHECKLIST FOR SOFTWARE DEVELOPMENT BEST PRACTICESSupportability
PortabilityTestabilityAccessibilityCommunityGovernanceLicensingCopyrightInstallabilityBuildabilityLearnabilityDocumentationUnderstandabilityFriendliness
200+ questions about software development
![Page 13: Independent Technology Assessment within the ESIP Testbed](https://reader035.vdocument.in/reader035/viewer/2022062523/587a8e581a28ab58288b662f/html5/thumbnails/13.jpg)
SupportabilityPortabilityTestabilityAccessibilityCommunityGovernanceLicensingCopyrightInstallabilityBuildabilityLearnabilityDocumentationUnderstandabilityFriendliness
200+ questions about software development
CHECKLIST FOR SOFTWARE DEVELOPMENT BEST PRACTICES
![Page 14: Independent Technology Assessment within the ESIP Testbed](https://reader035.vdocument.in/reader035/viewer/2022062523/587a8e581a28ab58288b662f/html5/thumbnails/14.jpg)
WORKFLOW
ESIP Edits and Submits Reports to AIST
Evaluators Submit Final Report Content
Evaluators Fill Out Evaluation Structure
Evaluators Carry Out TestingEvaluators/AIST PIs Create Test Plan
ESIP Selects Evaluators AIST Selects Projects
Content was KEY!- AIST Reviews Reports- PIs receive Final Reports
![Page 15: Independent Technology Assessment within the ESIP Testbed](https://reader035.vdocument.in/reader035/viewer/2022062523/587a8e581a28ab58288b662f/html5/thumbnails/15.jpg)
FEEDBACK+ Multiple communication choices. + Compensation.+ Selecting evaluators from potential infusion sites.
- Short evaluation period lead time.- Software best-practices checklist overly
complex.
![Page 16: Independent Technology Assessment within the ESIP Testbed](https://reader035.vdocument.in/reader035/viewer/2022062523/587a8e581a28ab58288b662f/html5/thumbnails/16.jpg)
LESSONS LEARNED
• Start evaluation with a technical exchange meeting.– Motivation– Challenges (Technology access)– Infusion Potential
• Diversify evaluation teams with domain and technical expertise.
• Each evaluation will be different, coordination and communication are KEY!
![Page 17: Independent Technology Assessment within the ESIP Testbed](https://reader035.vdocument.in/reader035/viewer/2022062523/587a8e581a28ab58288b662f/html5/thumbnails/17.jpg)
OUTLOOK
Provide the Earth sciences community with a novel, needed evaluation framework to improve technology development and infusion potential.
![Page 18: Independent Technology Assessment within the ESIP Testbed](https://reader035.vdocument.in/reader035/viewer/2022062523/587a8e581a28ab58288b662f/html5/thumbnails/18.jpg)
PARTICIPANTSEvaluators PIs
![Page 19: Independent Technology Assessment within the ESIP Testbed](https://reader035.vdocument.in/reader035/viewer/2022062523/587a8e581a28ab58288b662f/html5/thumbnails/19.jpg)
THANK YOU!Contact:
and 170+ member organizations
ESIP is supported by
www.esipfed.org @ESIPfed | #ESIPfed