judges score card overview mark turner nasa ames research center
TRANSCRIPT
Judges Score Card Overview
Mark TurnerNASA Ames Research Center
Scorecard 2
Ground Tournaments
• Purpose is to provide interim monetary awards and feedback to Teams
• Four Ground Tournaments (GT4 is required for EM-1 launch) six months apart
• Evaluation, analysis and simulations will be performed by ARC’s Mission Design Center, WFF’s Mission Planning Lab and Subject Matter Experts
January 7, 2015
3
GT Concept of Operations
January 7, 2015
Teams Prepare Data
40% - Probability of Mission Success 60% - Compliance with SLS Interface Requirements & Specific Challenge Rules
Judges
Simulations and Review byARC-MDC WFF-MPL SMEs
Review by SLS PIM SLS S&MA SLS Safety Panel
Judges
Centennial Program Office
Scores & Evaluations
Judges Final Scoring & Ranking
Awards
Questions for clarification Safety Hazard Report action items
Scores (and feedback) to Teams
ARC- MDC: Ames Mission Design CenterWFF-MPL: Wallops Mission Planning Laboratory SMEs: Subject Matter Experts
Scorecard 4
Purpose of the Score Card
• Provide a method to objectively judge each team to a set of clearly defined metrics
• Provide to each team a clear picture of what data they will need to provide for each Ground Tournament (GT)
• Provide to Teams a clear picture of how they will be evaluated for each GT
January 7, 2015
Scorecard 5
Score Card Development
January 7, 2015
• Subject Matter Experts from ARC, WFF and GSFC developed and reviewed the Score Card over several months.
• ARC-MDC and WFF-MPL developed detailed inputs necessary from Teams to run simulations.
• Two judges reviewed the Score Card.
Scorecard 6
Scorecard Evaluation Criteria
• Probability of Mission Success – 40% of total score– Likelihood of Achieving Communications Goals– Likelihood of Achieving Orbit or Distance Goals – Likelihood of Meeting Longevity Goals– Systems Design Maturity Relative to GT
• Compliance with SLS Interface Requirements & Specific Challenge Rules - 60% of total score– Specific Challenge Rules– SLS SPDS IDRD requirements (beginning with GT2)– Safety Panel Hazard Report
• GT1 identify potential safety hazards• GT2 & 3 Identify how you plan to address these hazards• GT4 Provide data to prove hazards have been mitigated
January 7, 2015
Scorecard 7
Inputs from Teams
• Each category identifies REQUIRED inputs and RECOMMENDED inputs (as noted in the tabs on the Score Card)
• REQUIRED inputs identify data critical for judges to evaluate each Team– Required data is the minimum acceptable data required for judging the
competition– By minimizing the amount of required inputs, each team can allocate its
resources as needed to develop a successful mission design• RECOMMENDED inputs identify data NASA would typically develop for in-
house missions– Given that teams are operating with limited resources, each team can decide
which recommended data they will provide– Recommended data provides additional insight to the judges as to how
robust and complete the proposed mission design is
January 7, 2015
Scorecard 8
Scorecard Evaluation Criteria
• Probability of Mission Success – 40% of total score– Likelihood of Achieving Communications Goals– Likelihood of Achieving Orbit or Distance Goals – Likelihood of Meeting Longevity Goals– Systems Design Maturity Relative to GT
• Compliance with SLS Interface Requirements & Specific Challenge Rules - 60% of total score– Specific Challenge Rules– SLS Secondary Payload Deployment System (SPDS) IDRD requirements
(beginning with GT2)– Safety Panel Hazard Report
• GT1 identify potential safety hazards• GT2 & 3 Identify how you plan to address these hazards• GT4 Provide data to prove hazards have been mitigated
January 7, 2015
Scorecard 9
Expectations
• Teams will reach higher fidelity of designs as the series of GTs progress. Less fidelity is expected at GT1 and high fidelity is expected at GT4.
• Judges’ grading will become more stringent as the series of GTs progress.
January 7, 2015
Scorecard 10
Evaluation Criteria for GTs
• GT 1 – Evaluate the mission architecture and concepts to meet Challenge Goals and
evaluate plans to meet Rules & Interface Requirements.• GT 2
– Evaluate plans and designs to meet Challenge Goals, Rules & Interface Requirements.
– Determine if the integrated design is appropriately mature to continue with final designs and fabrication.
• GT 3– Evaluate the readiness to begin system Assembly, Integration & Testing and
progress toward meeting Challenge Goals, Rules & Interface Reqts. • GT 4
– Evaluate the readiness for launch, likelihood of meeting Challenge Goals and ensure final compliance to Rules & Interface Requirements.
– Three Flight selections shall be made at GT4 with one additional team as backup
January 7, 2015
Scorecard 11
Schedule
January 7, 2015
O N D J F M A M J J A S O N D J F M A M J J A S O N D J F M A M J J A S O N D J F M A M J J A S O N D
Vehicle Related EventsSPUG AvailableCube Quest Summit (7th & 8th)Payload ~PDR (maturity check)IDRD AvailablePayload ICD CompletePhase 0/I Safety ReviewPayload ~CDR (maturity check)Phase II Safety ReviewVerif. Testing CompletePhase III Safety ReviewKSC Safety ReviewCOFRIntegrated Payload Turn OverPayload Battery ChargingLaunch, EM-1
Centennial Challenge EventsChallenge Released to Public XGT #1 - Rank Top 5 20GT #2 - Rank Top 5 30GT #3 - Rank Top 5 30GT #4 - Select Flight Group 20
Notes: All safety packages must be delivered 14 days prior to the scheduled review.Depending on the number of payloads, some milestones may be shifted to reduce technical loads on review panels.
FY2018 FY2019
Centennial Challenge EM-1 CubeSat Milestone Schedule
Payload MilestonesCY2014 CY2015 CY2016 CY2017 CY2018
FY2015 FY2016 FY2017
Point of no return
SPUG: SLS Secondary Payload User’s GuideIDRD: Interface Definition Requirement Document (IDRD)
Scorecard 12
Scorecard Examples
January 7, 2015