Joint Mission Environment Test Capability (JMETC)

Program Overview

Mr. Ryan NormanDeputy Program Manager, JMETC

Test Resource Management Center (TRMC)ryan.t.norman.civ@mail.mil

Distribution Statement A: Approved for public release. Distribution is unlimited.

JMETC Mission

2

JMETC provides the robust distributed infrastructure (network, enterprise resources,

integration software, tools, reuse repository) andtechnical expertise to integrate Live, Virtual, and Constructive (LVC) systems for test and evaluation

in Joint Systems-of-Systems and Cyber environments.

Standard Processes and Repositories

Configuration Management

Information AssuranceSecurity

VV&A

Distributed Data Collection

MOP Framework

Test Support NetworkCross Domain Solutions

System IntegrationProtocol

Translators

Live Tactical Systemsin Labs and on Ranges

Virtual & Constructive Red & Blue System 

Simulations

Tactical Network Emulation

Realistic Tactical Scenarios

HLADIS

OPERATIONALLY RELEVANT

COST EFFECTIVE

SOS ENVIRONMENT 

AT‐SCALE

System Adaptors

Common ToolsPre‐Event:Planning Event:

Monitoring, Management, and Control

Post‐Event:Data Reduction, Analysis

and Visualization

TENA Middleware

Cloud ServicesVirtualizationVirtualization Massively Parallel ProcessingMassively Parallel Processing Big Data StorageBig Data Storage

Big Data AnalyticsBig Data Analytics

GatewaysCommon 

Development Environment

Live-Virtual-Constructive Test & Training Environment

3

JMETC Benefits Acquisition Programs, Testers, & Evaluators

• Enables early verification that systems work in Joint and Cyber contested environments

• Test whether systems work well together• Test whether systems are resilient to cyber threats• Identify issues early when they are less costly to fix

• Provides access to high-demand, low availability systems• Supplements number of live Systems Under Test (SUTs), threats, or “supporting cast” to

create a realistic environment• Feasible alternative to Live testing in early DT and risk reduction for OT

• Provides access to cyber ranges• Ability to conduct unconstrained but nondestructive cyber activities in representative

environments

• Provides a collaborative engineering environment• Gives SMEs an opportunity for collaboration without leaving home station

• Supports all aspects of testing across the acquisition lifecycle• Interoperability, cybersecurity, rapid fielding, DT, OT, etc.

Reduce Acquisition Cost, Schedule, and Risk 4

JMETC and TENA

JMETC provides secure, robust distributed testing infrastructures tailored to meet user requirements

• Enables early risk reduction - finding problems early in acquisition when less costly to fix• Enables continuous testing across the acquisition life cycle• Reduces acquisition cost, schedule and risk by not requiring all test assets be brought to one location• Reduces test integration and setup time by providing a DoD-wide interoperability architecture for T&E

JMETC Networks Support __Distributed T&E every day

• JMETC SECRET Network (JSN) provides persistent connectivity to 78 government / industry range & labs

• JMETC MILS Network (JMN) supports multiple classifications concurrently up to and including TS//SCI/SAP/SAR

• Standing accreditations allows for minimal external coordination and approval

• Dedicated SMEs available to support distributed T&E planning, execution and post-test activities

Cybersecurity T&ESupport

• Regional Service Delivery Point (RSDP) cloud services add capacity and capability for DoD Cyber Ranges

• Enterprise resources rapidly generate robust virtualized environments for realistic test and training activities

• Cost effective and adaptable design that supplements capacity at existing DoD Cyber Ranges

• Dedicated SMEs available to support cybersecurity T&E planning, execution and post-test activities

• Supports National Cyber Range

Test and Training Enabling Architecture (TENA)

• Open and evolving architecture that enables interoperability between inter-and intra-range assets

• Provides a core set of 23 tools that address common event requirements

• Auto-code generated for 34 operating system / compiler combinations

• Web Repository facilitates sharing and reuse across T&E community

• Dedicated system integration SMEs available to support T&E planning, execution, and post-test activities

5

JMETC Distributed Test Architecture

6

SystemsUnderTest

Joint Operational Scenarios

IntegratedTest

Resources

JMETC Infrastructure

on DREN

* TENA:  Test and Training Enabling Architecture

VirtualPrototype

Hardwarein theLoop

InstalledSystemsTest

Facility

Range EnvironmentGenerator

ThreatSystems

TENACommon

Middleware

TENACommon

Middleware

TENACommon

Middleware

TENACommon

Middleware

TENACommon

Middleware

TENACommon

Middleware

Reuse Repository

TENAStandard Interface Definitions

TENAStandard Interface Definitions

TENAStandard Interface Definitions

TENAStandard Interface Definitions

TENAStandard Interface Definitions

TENAStandard Interface Definitions

Distributed Test Support Tools

JMETC depends on TENA to support distributed testing

6

T&E Infrastructure Interoperability

Driver Challenge

Goal/EndstateSolution• A highly flexible infrastructure for operationally realistic T&E.

Existing Components:• Integration Architecture: Test and Training Enabling

Architecture (TENA)• Mature, continuously improved software architecture (15+ years)• Architecture enables integration of Red/Blue capabilities

• Interface Standardization: TENA Object Models• Interface definitions for integrating both Red and Blue systems

• Common Tools: JMETC Tools and TENA Utilities• Proven tools suite to rapidly integrate and operate LVC-DE

• Corporate MILS Network: Joint Mission Environment Test Capability (JMETC)

• Mature, continuously improved network infrastructure (9+ years) • Used to support 250+ distributed test events since 2007

• DoD test range infrastructure was not originally designed to be interoperable. Impacts:

• Data loss between systems degrades test quality• Government maintains redundant system interfaces• Changes break functionality between related systems

• Producing complex, immersive environments for T&E is expensive and time consuming. Impacts:

• Systems do not get stressed against realistic conditions

• T&E infrastructure won’t be able to meet demands of agile acquisition

• Persistent network connectivity between government and industry test locations

• Interoperability between inter- and intra-range Test and Training assets

• Testing that can span kinetic and cyber test infrastructure• Elimination of proprietary interfaces on test infrastructure• Pre-Integrated common tools that reduce resources

needed for test planning, execution, and analysis• Pre-Integrated scenarios that can be pieced together into

complex and challenging test environments via automation• Sharing and reuse of common capabilities across existing

and new investments

Goal: Make complex, realistic T&E environments efficient and routine7

TENA at a Glance

• What does TENA enable?• Interoperability between inter- and intra-range assets• Elimination of proprietary interfaces to range instrumentation• Efficient incremental upgrades to test and training capabilities• Integration of Live, Virtual, and Constructive assets (locally or distributed)• Sharing and reuse of common capabilities across existing and new

investments• What is included in the TENA architecture?

• Customizable “data contracts” that standardize repeatable information exchange

• Interoperability-enabling, auto-code generated software libraries• A core set of tools that address common test and training requirements• Collaboration mechanisms that facilitate sharing and reuse

• TENA has a plan for continued evolution and funding to execute this plan

TENA is the DoD GOTS range integration architecture

8

JMETC has a hybrid network architecture

• The JMETC Secret Network (JSN), based on the Secret Defense Research and Engineering Network (SDREN), is the T&E enterprise network solution for Secret testing

• The JMETC Multiple Independent Levels of Security (MILS) Network (JMN) is the T&E enterprise network solution for all classifications and cyber testing

JMETC Network Architecture

9

JMETC SECRET Network (JSN)

• Focus is on persistent connectivity• Standing Agreements

• All sites have valid Authority to Operate (ATO) and Authority to Connect (ATC)

• Daily full mesh, end-to-end network characterization ensure optimized performance • On demand usage with little to no coordination necessary

• MOAs in place to authorize connections between all sites

• Persistency enables user to…• Test capabilities early and often• Execute unscheduled/unplanned testing whenever needed• Focus on the test rather than the network

• Operates at SECRET Collateral • Leverages SECRET Defense Research & Engineering Network (SDREN) for

connectivity• Functional and growing since 2007

10

Customer time and dollars not spent on infrastructure by leveraging JMETC

JMETC SECRET Network (JSN) Site Map

Ft Hood (2): CTSF, TTEC

WPAFB: SIMAF

As of 25 Apr 2018

Leverages the SECRET Defense Research and Engineering Network (SDREN) for connectivity

Operates at SECRET classification, available 24/7 Continuous monitoring, troubleshooting, and optimization of the

end-to-end network infrastructure Capable of supporting numerous simultaneous test events

Functional JSN Locations: 49 (access to 82 labs/facilities)Planned JSN Locations: 6 Connection Points to Other Networks: 5

Pax River:(12) E2C, E2D, MCL, ATRACETEF, SAIL, MFS, P-8,E-2C SIL, UASIL, EP-3, Triton Data Analysis JMETC SYSCONEast Interconnect Rtr.

Aberdeen: (2)ATC-STE, ARL

Langley TDLITCJS J6 DDC4 C4AD

Wallops Island SCSC

Dahlgren:(3) IWSL, DOC, ICSTFRapid-SIL/C2

Dam Neck CDSA

Newport News NGC VASCIC

McLean MITRE NSEL

Redstone (12): SMDCRTC: DTCC, DISTL, AvSTILAMRDEC SSDDAMRDEC SED: Patriot, THAAD, FAAD, GSIL, JLENS, MUSE, C-RAM

Crane (2):NSWC RDT&E, NTN Lab

Moorestown (2): CSEDS, CPTS

ArmyAir ForceNavyMarines

JointIndustry

GTRI

Ft Huachuca: (3) JITC, JTDL, JTRS

Metro Park JMETC / TENA SDA Lab

Dugway Proving Ground

Melbourne NGC JSTARS

Keyport NUWC

Rome NY AFRL

Newport NUWC

Tinker AFB AWACS

Greenville Rivet JointWSMR IRCC

Raytheon Tucson

LMCO Global Vision Network

DISTRIBUTION A. Approved for public release: distribution unlimited.

Pittsfield GD LCS

Arlington IDT

Manassas LM ASW

Ft. Sill, SMART Lab

Camp Pendleton:MCTSSA

China Lake (4):F/A-18, IBAR,

TSPIL, AV-8BLand Range, WSCI

Edwards (2):

Corona: NSWC

Point Loma : (2)SSC-PAC ITCL SSC-PAC CTB LMMTFTEC (Keyport Det.)

Point Mugu (4):ITEC, EW CyCon,AEA, Sea RangeMAGTF EW

W. Interconnect Rtr.Rancho Bernardo NGC Triton

412th EWG IFASTRidley

Port Hueneme: NSWC

MHPCCHawaii

Pearl Harbor FIST2FAC (Keyport Det.)

Hanscom AFB HCIC

Eglin AFB (4):AOC, DTF, KHILSGWEF / COOP

Panama City: NSWC

11

Sampling of Assets Available on JSN

12

China Lake:F/A-18, IBAR

Eglin AFB: 46th Test Squadron

Redstone

Pax River: Hawkeye, Manned Flight simulator.Dahlgren: Rapid SILWallops Island : Ship Self-Defense SystemMcLean MITRE: National Security Experimentation Laboratory

WSMR IRCC

WPAFB:SIMAF

Ft Huachuca: JITC, JTRS, C4ISR

SPAWAR Systems Center Pacific

Camp Pendleton

Edwards

Ft Hood:TTEC, CTSF

JSN Event Support Services

• Pre-Test / Test Integration Emphasis• Test Development/Design - help users leverage JSN capabilities and services to meet

with infrastructure solutions• Network Engineering - designs, configures, establishes, and baselines connectivity

solutions for test customers• Cybersecurity Engineering - support site/user accreditation efforts• User Support - ensures JMETC sites have the knowledge, skills, abilities, and site-

specific examples to address test resource interoperability issues

• Test Execution Emphasis• JMETC SYSCON - verifies infrastructure readiness and proactively troubleshoots

problems as they are discovered• Event Support - provides direct support to customer test activities on an as-needed

basis

• Post Test Emphasis• Capture Lessons Learned and Infrastructure Gaps/Limitations• Data dissemination and distributed analysis

13

14

JSN Connectivity Services

• JSN Systems Control (SYSCON)• JMETC Personnel available to test, monitor, and troubleshoot network connectivity• Web-Based Help Desk and Phone Support• Assistance with site Ports, Protocols & Services management• Assistance with site device configuration• 9x5 and after-hours support as necessary

• Inter-Site Collaboration• VoIP Call Manager• Chat Server (XMPP)• Secure File Transfer Protocol (SFTP) Server• Adobe Connect

• Information Assurance Compliance• Linux and Windows Patches (YUM and WSUS)• Anti-virus (McAfee, Symantec, TrendMicro)• Scan/STIG tools (SRR, Gold Disk, Retina, etc.)

Continue to expand services offered based on community requirements

Major JSN Events Supported(August 2016 – July 2017)

Customer Event Execution Dates Onsite Support

Navy MQ-4C Triton Ongoing -Air Force Small Diameter Bombs II (SDB) Live Fly Testing Sep-14 – Aug-16 -

Air Force Air Force System Interoperability Test (AFSIT) Feb-17Apr-17 -

Joint Joint Interoperability Test Command (JITC) Joint Interoperability Tests (JIT)

Oct-16Jan-17Mar-17Jul-17

Yes

Navy Distributed Integration & Interoperability Assessment Capability (DIIAC) V&V Sep-16Jan-17Jul-17

Yes

Navy Interoperability Development and Certification Testing (IDCT)Aug-16Nov-16Apr-17

Yes

Joint Air Ground Integrated Layer Exploration (AGILE) Fire IX Aug-16 Yes

Joint Joint Distributed IRCM Ground-test System (JDIGS) Apr-17Jul-17 -

Joint Navy Integrated Fires Nov-16 Yes

Joint F-35 Joint Strike Fighter Record & Playback Aug-16 YesNavy Alpha Omega Live Virtual Constructive (LVC) Event Sep-16 -Joint Undersea Communication Jan-17 Yes

Navy Digital Integrated Defense System (DIADS) Demo Feb-17Jul-17 -

Navy Amethyst Mar-17 Yes

Joint Joint Integrated Air and Missile Defense Office (JIAMDO) Joint Distributed Engineering Plant (JDEP) Black Dart

Mar-17May-17 Yes

Navy Bloodstone Jun-17 YesNavy Apollo Captive Carry (CC) for Integrated Test Events (ITE) Jul-17 -

15

JSN Event Examples

Battlefield Airborne Communication Node (BACN) Joint Urgent Operational Need

• Integration of BACN payload onto multiple platforms for solution to urgent in-theater need :

• Combat requirement for beyond line-of-sight comm

• Relay, bridge, and range extension for ground forces and supporting aircraft

• Distributed Testing included Live-fly, DT, and Operational Utility Evaluation

IMPACT• Efficient integration of DT and OT• Testing successfully completed without

need for live assets to be co-located• Distributed Testing saved “~$1.2M” (OTA)• Urgent capability fielded-quickly

16

Joint Interoperability Tests (JITs)

ImpactJMETC provides a venue for TDL interoperability testing with its persistent infrastructure capabilities. JMETC enables readily available tactical systems and infrastructure at little or no cost to JITC and the Program Offices, as well as rapid scalability for extended participation of sites and systems as needed. The JITs benefit from the JMETC persistent test infrastructure, resulting in significant reduction in test time and allowing system-specific follow-on testing, when necessary, with little or no notice or setup. In FY17, JITC has already tested 16 systems-under-test (SUTs) and are scheduled for one more test event in the Summer of 2017. Four JITs are already scheduled for FY18.

Program OverviewThe Joint Interoperability Test Command (JITC) Joint Tactical Data Link (TDL) Branch is responsible for ensuring systems that implement TDLs (Link-11/11B/16), Joint Range Extension Application Protocol (JREAP), Variable Message Format (VMF), and United States Message Text Formatting (USMTF) are interoperable and comply with the applicable Joint standards. The JITC accomplishes this mission by conducting tests, called JITs. JMETC provides the network infrastructure, event support tools, and event planning and execution support for multiple Joint Interoperability Tests for JITC each year. JITs have been executing 4-5 times per year on the JMETC persistent infrastructure since FY10.

JMETC / TENA ToolsJMETC provides JITs with a persistent, robust infrastructure: the JMETC Secret Network (JSN), integration software -- the Test and Training Enabling Architecture (TENA), tools, reuse repository, and the technical expertise to integrate live, virtual, and constructive (LVC) systems for test and evaluation in Joint systems-of-systems and cyber environments. TENA is used to exchange simulation data between sites and monitor the applications and network. The JSN Systems Control (SYSCON) services included network troubleshooting, Ports and Protocol (PPS) for firewall setup, Voice over Internet Protocol (VoIP), and Secure File Transfer Protocol (SFTP) Server.

17

Air Force System Interoperability Test (AFSIT)

ImpactThe AFSIT Event leverages the persistent JMETC Secret Network (JSN) infrastructure to integrate facilities, labs, and JSN Systems Control (SYSCON) services; to include network troubleshooting and Voice over Internet Protocol (VoIP). In addition, AFSIT uses the Test and Training Enabling Architecture (TENA) for the distribution of Simulation data.

SUTs are evaluated performing mission threads in a simulated environment that includes real AF command and control (C2) and weapons platforms. Due to such rigorous testing, superior TDL platforms are delivered to the Warfighter, operating per MIL-STD specifications.

Program OverviewThe Air Force System Interoperability Test is a U.S. Air Force series of tests that provide system-level interoperability testing to validate compliance to tactical data link (TDL) standards, and to verify compatibility and interoperability with participating systems. The emphasis of each AFSIT is to use actual hardware-in-the-loop (HWIL) communications assets to confirm interoperability of hardware / software modifications and / or upgrades.

Approach• The focus of AFSIT is to “immerse” systems under test (SUTs)

into a simulated battle theater to validate compliance to TDL standards, as well as verifying compatibility and interoperability with participating systems

• Research, evaluate, and coordinate test objectives with A5/JI (USAF participating Test Unit Coordinator)

• Perform formal, highly structured, rigorous MIL-STD conformance testing

• Make continuous improvements to testing methodologies• Maintain Test Center to current and emerging standards• Provide detailed, concise reports to the bit level on each / every

AF tactical data link platform• Assist all AF platforms through Joint certification testing,

provide connectivity, defend system performance parameters at the Joint Analysis Review Panel (JARP)

18

JMETC Multiple Independent Levels of Security (MILS) Network (JMN)• T&E enterprise network solution for all classifications and cyber

testing• Supports National Cyber Range (NCR)

JMETC JMN

19

Why Do We Need Cyber Ranges?

• To assess advanced cyberspace technologies or exercise tactics, techniques, and procedures (TTPs) that require isolated environments of complex networked systems (e.g., movement on the Internet)

• To conduct activities that cannot occur on operational networks due to potential catastrophic consequences (e.g., releasing self-propagating malware)

• To rapidly and realistically represent cyber contestedenvironments at different levels of security, fidelity, and/or scale (e.g., Blue [friendly] force, Red [adversary] force, and Gray [neutral]networks)

• For precise control of the event environment that allows for rapid reconstitution to a baseline checkpoint, reconfiguration, and repeat of complex use cases (e.g., rapidly running variation of conditions to quickly evaluate hundreds of scenarios)

20

Requirement Drivers for Additional Cyber Range Capabilities

• Need increased virtualization capacity to meet expected demand but must be… • cost effective to scale efficiently

• remotely accessible to support distributed activities

• interoperable with other “cyber range” capabilities

• able to support multiple security classifications (including coalition)

• able to support unconstrained activities

These Virtual Ranges Can Also Be Utilized To Meet More Traditional Requirements

21

JMETC MILS Network (JMN)

• Focus is on providing secure distributed testbeds to support unconstrained cyber activities and users access to enterprise resources at multiple classifications

• Employs Multiple Independent Levels of Security (MILS) architecture• Allows for segregation of data

streams by protocol, system, event, COI, etc.

• Capable of supporting multiple simultaneous events at multiple classifications concurrently

• Ability to create isolated “sandboxes”

• Accredited by Defense Intelligence Agency (DIA) to operate from Unclassified up to TS//SCI

• Included NSA Red Team assessment

22

JMN Connectivity Services

• JMN NOSC• Manage, optimize and troubleshoot network connectivity• Help Desk• Provide pre-event checkouts as requested• Local infrastructure assistance as requested• 10x5 with after hours support as necessary

• Inter-Site Collaboration• VoIP• Chat Server• Secure File Transfer Protocol (SFTP) Server

Continue to expand tools & services offered based on user requirements

23

• Provide enterprise resources to rapidly generate virtualized representative cyber environments

• Comprised of computational and storage resources to host 1000s of high fidelity virtual representations

• Large, integrated Red-Blue-Gray environments• Platform specific high-fidelity representations• Tailored, independent student classrooms

• Automated provisioning to minimize deployment time

• Each is capable of supporting numerous events and varying classifications concurrently

• Serves as a platform for tools and services

• Geographically dispersed to minimize latency and maximize usability

• Designed to be cost-effective and adaptable• Also supports more conventional types of testing

Regional Service Delivery Points(RSDPs)

24

RSDP CONOPS

• Accessibility by users• Sites / users can utilize any RSDP (assuming latency is not an issue)• Sites / users can access multiple events, at multiple classifications on multiple

RSDPs concurrently

• Extensibility to address extremely large scale, high fidelity requirements• Multiple RSDPs can be used in conjunction to support a single event • A RSDP can be used in conjunction with other Cyber capabilities (e.g., NCR) as part

of a larger virtual environment

• Reusable RSDP Resources• All RSDP resources are sanitized after each event

• Addresses compute, storage and management assets • Allows for “non-destructive” cyber testing

25

Technical Support Across the Event Lifecycle

(1 of 2)

• Pre-event Phase• User/Event Support

• Help users understand what services and capabilities are available to meet with their infrastructure requirements

• Identify new site requirements• Develop event agreements and support coordination efforts

• Virtual Environment Support• Help users develop and define environment requirements• Develop new automation workflows as necessary• Optimize the automation to minimize environment build time • Configure customized traffic generation and instrumentation

• Network Operations Security Center (NOSC)• Bring on new sites as needed• Ensure end-to-end network is optimized • Work with local site personnel to troubleshoot issues• Provide collaboration tools for use in event planning

26

• Event Execution Phase• User/Event Support

• Provide remote and onsite support to user activities as needed• Virtual Environment Support

• Rapidly deploy virtualized environments• Sanitize resources and redeploy environments as needed between runs

• NOSC• Build out event specific network linkages• Work with local site personnel to troubleshoot issues• Provide collaboration tools for use in event execution

• Post Test Phase• User/Event Support

• Assist with data analysis and dissemination• Capture lessons learned and identify infrastructure gaps

• Virtual Environment Support• Sanitize RSDP computational and storage resources for reuse

• NOSC• Tear down of event specific network linkages

Technical Support Across the Event Lifecycle

(2 of 2)

27

• JMN site map, event examples, and more detailed information available at the FOUO level

Unclassified Restrictions

28

• JSN / JMN infrastructure continues to grow to meet user requirements

• JMETC provided tools, services, and support are all institutionally funded capabilities• Sites typically pay for underlying DREN (and SDREN for JSN) but JMETC

covers all other equipment and O&M costs• RSDP operations are funded by JMETC and centrally managed by the JMN

NOSC• JSN and JMN engineering and event support services are no cost to the user

• JSN, JMN and RSDP are proven capabilities• Fully accredited to operate at multiple classifications • Have supported a variety of test and training activities

• JMETC capabilities are driven by user requirements• Deployment of JSN/JMN nodes and RSDPs are based on user need• JMETC provided tools and services are based on user input

• TENA is an Architecture for Ranges, Facilities, and Simulations to Interoperate, to be Reused, & to be Composed into greater capabilities

29

Summary