F-35 Joint Strike Fighter (JSF)

Executive Summary• All three F-35 variants had entered flight test by June 2010.

For the first time, all three integrated test forces at Fort Worth,Texas; Patuxent River Naval Air Station (NAS), Maryland;and Edwards AFB, California, conducted flight test operationswith seven Systems Design and Development (SDD) testaircraft. The cumulative data for test sorties and pointsindicate progress slightly ahead of that planned. The testteams exceeded the goal of 39.4 total sorties for calendar year2010 by early December 2010. However, progress in testingthe Short Take-Off and Vertical Landing (STOVL) aircraft wasless than planned.

• Immaturity of STOVL design and unexpected componentdeficiencies limited successful accomplishment of testpoints in areas critical to short take-off and vertical landingcapability. Development of mission systems softwarecontinued to experience delays that affected flight testprogress.

• Program leadership began re-planning SDD flight testingat the end of FY 10, in conjunction with a restructuring ofmission systems soil are development plans. These effortsfollowed the recommendations of the Program ExecutiveOffices (PEO) Technical Baseline Review (TBR) Of theprogram, which was a technical. "bottoms-up," independentreview of the air vehicle platform, sustainment, missionsystems software. and test. Finalization of the test scheduleand integration into a master program schedule continued intoearly FY II.

• Service plans for initial training and operational capability. andacquisition plans for full-rate production need to be adjusted toa realistic timeline consistent with certification through testingof the incremental capability aircraft will actually provide, aswell as later completion of SDD. Although the integrated testforces and development teams made significant progress, theresults of flight testing and the TBR indicate more time andresources will be needed to complete SDD than incorporatedin the June 2010 program baseline.

Actual versus Planned Test Flights and Points

ALL VARIANTSALL TESTING

SToVL ONLYFLIGHT SCIENCES

CToL oNLYFLIGHT SCIENCES

CV ONLYFLIGHT SCIENCES

MISSIONSYSTEMS

Flights Points Flights Polnts Flights Points Flights Points Flights Points

ACTUAL 282 2,948 130 1,467 111 963 14 344 28 174FY10

PLANNED 256 2,496 173 1,678 43 485 9 77 32 256

Cumulative ACTUAL 427 4,614 216 2,649 155 1,294 24 466 32 205

(as of 1 Dec 10) PLANNED 390 4,404 254 3,010 69 825 21 201 46 368

System• The F-35 Joint Strike Fighter (JSF) program is a joint,

multi-national, single-seat, single-engine family of strikeaircraft consisting of three variants:- F-35A Conventional Take-Off and Landing (CTOL)- F-35B Short Take-Off and Vertical Landing (STOVL)- F-35C Aircraft Carrier Variant (CV)

• It is designed to survive in an advanced threat (year 2012 andbeyond) environment using numerous advanced capabilities.

It is also designed to have improved lethality compared tolegacy multi-role aircraft.

• Using an Active Electronically Scanned Array (AESA) radarand other sensors, the F-35 is intended to employprecision-guided bombs such as the Joint Direct AttackMunition and Joint Standoff Weapon, AIM-120C radar-guidedair-to-air missiles, and AIM-9 infrared-guided air-to-air missiles.

F-35 JSF 13

The program incrementally provides mission capability:Block 1 (initial . ), Block 2 (advanced), Block 3 (full).

• The F-35 is under development by a partnership of countries:the United States. Great Britain, Italy, the Netherlands, Turkey.Canada, Australia, Denmark, and Norway,

Mission• A force equipped with F-35 units should permit the Combatant

Commander to attack targets day or night, in all weather, inhighly defended areas of joint operations.

• Targets include fixed and mobile land targets, enemy surfaceunits at sea, and air threats, including advanced cruise missiles.

Major Contract&Lockheed Martin, Aeronautics Division. Advanced DevelopmentPrograms Fort Worth. Texas

Activity

Activity Affecting Test Strategy, Planning, and Resourcing

Joint Estimate Team 11• The second independent joint Estimate Team (JET)

review concluded last year that the SDD flight test planlacked sufficient resources and incorporated unrealisticassumptions for flight test productivity relative to historicalexperience. At the time of the JET II review, the programhad accomplished approximately 25 flight test hours ononly two STOVL SDD test aircraft; no aircraft had ferriedto the Hight test centers. .

• In early FY 10, the program began the process ofincorporating the review's key recommendations: addingtest aircraft to the SDI) test fleets from productionlots, adding down-time for aircraft maintenance andmodifications, reducing the assumed productivity of certainflight test aircraft, increasing anti extending engineering andtest operations staffs to support concurrent developmentand test, and adding an additional software integrationand test lab. The program was also directed to implementrecommendations of the first Independent ManufacturingReview Team, to include reducing production in the FutureYears Defense Plan by 122 aircraft, thereby reducingconcurrency of development and production.

• These reviews and actions, along with a review of costand risk in development of the propulsion system, fed tothe acknowledgement of a breach of the Nunn-McCurdy"critical" cost thresholds for the JSF program.

Nunn-McCurdy Certification• An Integrated Test Review occurred in April to support.

the Nunn-McCurdy certification. Representatives fromthe Edwards and Patuxent River flight test centers. .ISFOperational Test Team, and the Services conducted thereview and identified numerous issues affecting theexecutability of the flight test schedule.

• The Nunn-McCurdy program certification occurred inJune. At the time of the certification of the new programbudget baseline, the flight test program had accomplishedapproximately J90 flight test hours and ferried five totalaircraft to the test centers, including two CTOL flightsciences aircraft, with an overall average number of 3.2months on-site at the flight test centers. Low fly rates

on STOVL flight sciences aircraft and unanticipateddeficiencies in the design had begun to emerge in flighttest. Analysis during the review indicated STOVL flightsciences was becoming the critical path to complete SDI)flight test. The program acknowledged later ferry dates forremaining SDD test aircraft. The estimate of SDD flighttest completion was extended to July 2015.

Technical Baseline Review (MR)• The new FED commissioned a TBR of the program in June

to determine the technical adequacy or program plans andresources. The TBR benefited from more flight test resultsthan previous reviews because the three Integrated TestForce sites had accumulated over 440 flight test hours andthe overall average in months on-site for SDD aircraft atthe flight test centers Was 72 months, However, during themonths since the last program review, more problems withSTOVL design and mission systems software arose.The TBR recommended further changes to the parametersused to plan and model flight test schedules, as well asnumerous changes in staffing and other resources neededto complete SDI) and enter lOT&E. Specific changesto the schedule recommended by the TBR include lowerflight rates for test aircraft that are tailored to each variant(lower than prior independent reviews), additional re-flyand regression sorties that are tailored to the type of testing,and more flight test sorties. The TBR also determined moretime was needed for completion of all remaining softwareincrements. The result is a completion of developmentalflight test in late 2016, with STOVL fligh t sciencescompleting later than the other two variants.

F-35 Flight Test

STOVL Flight Sciences, Flight Test with BF-1 BF-2, andBF-3 Test Aircraft• BF-3 ferried to Patuxent River NAS, Maryland, in February

2010; it is the last of three B-model flight sciences aircraft.• Maintenance, test operations, and engineering, staffs

increased significantly (approximately 25 percent) in FY10at Patuxent River, NAS. The program intends to reach fullstrength in 2011, pending hiring of qualified contractorpersonnel.

14 F -35 JSF

• The government-contractor test team attempted test pointsin up-and-away flight envelope expansion. STOVL-modeflight, handling qualities, propulsion testing and readinessfor the first ship integration test period (planned forlate 2011).

• In FY 10. STOVE Flight Sciences aircraft flew 130 of 173planned sorties; the test team completed 1,467 of l,678planned test points. However, the test team accomplishedonly 10 of 42 planned vertical landings between March andNovember 2010; these are key to the shore-based build-upto testing on L-class amphibious ships at sea, In the firsttwo months of FYI l, STOVE flight sciences aircraft flew54 sorties. 5 more than planned; the test team accomplished356 of 506 planned test points, From mid-Augustuntil early November, the test team flew CTOL-modeconfigurations due to limitations of the vertical-liftcapability of the STOVL system. STOVE-mode flight testoperations began again in BF-1 in November 2010,

• In July, the program made changes to supply chainmanagement to provide timely spares and implementedsurge scheduling and 7-day/week maintenance operations,These actions contributed to an increase in flights per monthof approximately 25 percent.

• Discoveries during STOVL Flight Sciences testing thisfiscal year include transonic wing roll-off, greater thanexpected sideslip during medium angle-of-attack testing,higher and unanticipated structural loads on STOVL doors,and poor reliability and maintainability of key components.

CTOL Hight Sciences, night Test with AF-1 and A F-2 TestAircraft• AF-1 and AF-2 ferried to Edwards AFB. California, in May.

as planned.• Maintenance, test operations, and engineering staffs

increased significantly (approximately 50 percent) in FY10.The program intends to reach full strength in 2011, pendinghiring of qualified contractor personnel.

• In FY 10, the test team made progress in envelopeexpansion, handling qualities, and propulsion test points.CTOL Flight Sciences aircraft flew 111 sorties, 6$ morethan planned. The test team completed 963 test points,exceeding the 485 planned flight test points for the fiscalyear. In the first two months of FY11, CTOL flight sciencesaircraft flew 44 sorties, 18 more than planned; the test teamaccomplished 331 4340 planned test points.

• The program anticipates the remaining CTOL FlightSciences aircraft, AF-4, will ferry to Edwards, AFB,California. by January 2011, approximately two monthslater than planned.

• Discoveries during CTOL flight sciences flight test in thisfiscal year include transonic wing roll-off, greater thanexpected sideslip during medium angle-of-attack testing,and problems with reliability and maintainability of keycomponents.

CV Flight Sciences, Flight Test with CF-1 Test AircraftCF- I flew for the first time in June 2010. The aircraftferried to Patuxent River NAS, Maryland, in earlyNovember 2010, one month later than planned.

• While at Fort Worth. Texas, CF- 1 flew airworthinessand initial-service-release propulsion system test flights,accomplishing 14 flight test sorties, five more than planned.As a result, CF-1. flew 344 test points, significantly morethan the 77 planned for the fiscal year. In the first twomonths of FY II. aircraft CF-1 flew 10 of 12 plannedsorties; the test team accomplished 4 of 14 planned testpoints.

• The Integrated Test Force at Patuxent River NAS,Maryland, built up maintenance and engineering supportpersonnel in anticipation of the arrival of CF-1, which theprogram delivered to the test center in November 2010.

• The program anticipates the remaining CV flightsciences test aircraft, CF-2 and CF-5. will ferry toPatuxent River NAS. Maryland, in February 2011 andlate 2013. respectively. Aircraft CF-2 would then arriveapproximately two months later than planned.

Mission Systems, 13F-4 and A F-3 Right Tests and SoftwareDevelopment Progress• Block 0.5 Infrastructure

- The program released Block 0.5 software for flighttest in March 2010, five months later than planned.The software had completed mission systems labintegration activity and integration flights on theCooperative Avionics Test Bed (CATB). Block 0.5is the infrastructure increment, which containscommunications, navigation, and limited radarfunctionality.Aircraft BF-4, loaded with Block 0.5, accomplished firstflight in April 2010, five months later than planned, andthen ferried to Patuxent River NAS, Maryland, in June,two months later than planned, and began Block 0.5flight test.Test teams attempted approximately 70 percent of theplanned Block 0.5 flight test points on BF-4. Softwareproblems occurring before and during flight test werenot resolved in the Block 0,5 configuration. Programleadership deemed Block 0.5 unsuitable for initialtraining and adjusted the software development plan toimplement fixes for the Block 0.5 problems in the initialrelease of Block flle integrated test force is re-flyingselected Block 0.5 flight test points in the Block 1configuration.

• Block I, Initial Training Capability- The program delivered aircraft AF-3 in a Block 1

configuration to Edwards AFB, California, in December2010, approximately five months later than planned.

- The program intends the Block I design (which includesmulti-sensor fusion capability) to support the initial

F-35 JSF 15

training, syllabus for the initial cadres at the trainingcenter. The development team conducted integrationactivity with an initial version of Block 1, including fixesto Block 0.5 problems. in the mission systems labs andon the CATB,The program planned to release the first Block Jincrement to flight test aircraft in August 2010, but. F-35flight testing did not begin until November 2010. Bythe end of November. the test team flew 4 of 14 plannedsorties and accomplished 31 of 112 test points.

• Block 2 and Block 3 Software Development ProgressThe Block 2 detailed flight test planning process beganin September 2010.In August, the program began re-planning the softwaredevelopment schedule for completing and certifyingBlock 1, Block 2, and Block 3 increments of SDDcapabi 1 ity,

• Ferry of Remaining SDD Mission Systems Flight TestAircraft- The program anticipates ferry of BF-5 in late March

2011 and CF-3 in May 2011: these deliveries to the testcenters arc approximately four and five months later thanplanned, respectively.

Modeling and SimulationVerification Simulation (VSIM)

• The program commenced planning of validation effortsfor F-35 modeling, development of the virtual battlespaceenvironment, and integration of the two into one simulationintended for developmental test and evaluation.

• The program identified funding shortfalls for theVerification Simulation (VSIM) to meet OT&.E needs,primarily in the battlespace environment, and provided datafor an independent cost assessment leading to inclusion ofVSIM costs in the program baseline. The Services havebeen directed to fully fund VSIM for OT&E.

• The PEO completed a VSIM Sufficiency Review todetermine the means to provide the required OT&E VSIMcapability.

Oilier Models and Corporate Labs• The program continues to plan to accredit a total of 32

models and virtual laboratories for use as test venues(including VSIM) in developmental testing. The programplanned to accredit 11 models by the end of FY 10;however, the program office accredited only three venuesby September 2010.

• Due to software development delays and shifts in capabilityto later software blocks, the program decided severalmodels are not needed to support testing of Block 1 mission

systemic.Static Structural and Durability Testing• The test teams completed STOVL and CTOL static structural

testing ahead of schedule, which is an important input toenvelope expansion through flight test. The CV static testarticle completed initial drop tests for carrier suitability.

• CTOL and STOVL durability testing began in FY10. Resultsfor a loading, equivalent to one aircraft lifetime (8,000 hours)were expected in mid-FY11 for the STOVL aircraft and earlyFY 12 for the CTOL aircraft, However, a major fatigue crackwas found in the STOVE test article at approximately 1,500flight hours. Failure of the bulkhead in flight would havesafety of flight consequences. The program stopped fatiguetesting on both the STOVL and CTOL test articles andbegan root cause analysis in November 2010. The STOVLbulkhead is constructed of aluminum alloy. The CTOL andCV bulkheads have a similar but not identical design and aremade of titanium, The difference in bulkhead material is dueto actions taken several years ago to reduce the weight of theSTOVL aircraft

Propulsion System Testing• F135. The program delivered the first initial-service-release

Fl35 engines to SDD CV and STOVL test aircraft. By theend of November 2010, CF-1 had flown 36 flight hours withthis engine; however, BF-5 had not yet flown. The programbegan implementing plans to modify test aircraft to rectifythe afterburner "screech" problem, a problem that preventsthe engine from sustaining full thrust. These modificationsare necessary for the test aircraft to complete envelopeexpansion at the planned tempo.

• 1=136. Engine testing accomplished approximately 430 of739 planned ground test hours by the end of the fiscal year.The program is examining ways to accelerate testing in orderto meet the !planned start of flight test with the Fl 36 in late2011 for CTOL, and late 2012 for STOVL.

Operational Test and Evaluation• In June, the JSF Operational Test. Team (JOTT) began

OT-2E, the fifth operational assessment of progress towardsdeveloping an operationally effective and suitable Block 3mission capability in all three variants, The JOTT plans tocomplete this assessment in late 2011.

• At the request of the JSF Program Executive Officer (PEG).the JOTT is also developing plans to assess the initialtraining capability intended for use with the first fleet pilotsand maintenance crews in 2011,

• The JOT. T reviewed and re-validated the November 2008requirements documentation for the VSIM for OT&E.DOTE approved the re-validated requirements.

• The JOTT began the Readiness-to-Test evaluation processin FY 10, which uses an assessment template to determineactions necessary for the weapons system to be ready tosuccessfully enter and complete the planned OT&E periods.This process identifies potential gaps between verification ofcontract specification compliance and delivery of the missioncapability necessary to meet the operational requirements.

• The JOTT significantly increased its work force and theServices identified pilots and maintenance crews forexecution of early operational testing and assessments.

16 F-35 JSF

Air System-Ship Integration and Ship Suitability Testing• Coordination continued between the JSF program office,

Naval Sea Systems Command, and Naval Air SystemsCommand offices responsible for planning and implementingactions to integrate the JSF aircraft and support systemson naval ships. The teams focused efforts on readiness forinitial ship trial periods that the program now plans in late2011 (one year later than previously planned), as well as onplanning the other actions needed to achieve initial operatingcapabilities of the B-model on L-class amphibious ships andthe C-model on large-deck carriers.

• The coordination teams are working significant issues inthese areas: identification of personnel hazard zones aroundB-model aircraft, interoperability of the Autonomic LogisticsInformation System with Service and joint systems, carrierjet blast deflector modifications needed for CV aircraftoperations, aircraft-ship connectivity for alignment of inertialnavigation systems. secure facilities for handling specialaccess material, and spectrum limitations.

• The first ship trial period for the B-model STOVL aircrafthas slipped from March 2011 to no earlier than late 2011 dueto the slow flight test progress in accomplishing the shore-based build-up test points. The first C-model trial period ona large deck carrier is planned for early 2013.

Live Fire Test and Evaluation• LFT&E conducted On-Board Inert Gus Generations System

(OBIGGS) tests during FY09-FY10.• The Weapons Survivability Lab at China Lake took delivery

of the Full-Up System-Level (FUSL) F-35 aircraft. Theaircraft is being prepared for ballistic testing. The test teamwill begin this testing in IQFY11.

Assessment

Test Schedule Re-Planning and Implementation of Changes• The year-long process of analyses during FY 10 (,1E111

implementation. Nunn-McCurdy certification. and TBR)served to develop a more realistic estimate of SDDcompletion for Block 3 in all variants and identify stepsto reduce risk in execution of the verification test andevaluation strategy. Although the sample size of experiencewith the CV is still small, the STOVL design emerged as thehighest risk of all variants and the most difficult to progressthrough flight test. This is due in part to the difficulty inmaking progress in vertical lift operations compared to thatplanned. The analyses also revealed that the F-35 missionsystems software development and test is tending towardsfamiliar historical patterns of extended development,discovery in flight test, and deferrals to later increments. Themodifications recommended by the TBR (lower fly rates,more regression and re-fly margin, more nights, and otherresource additions) that result in completion of SDD flighttest for Block 3 in all three variants later than previouslyestimated are realistic and credible. Completion of STOVLflight sciences in this timeframe is dependent on whetheror not the necessary changes to STOVL design can be

implemented and tested. It will also depend on whetherthese changes result in fewer aircraft operating limitationsand greater aircraft availability for test. The program willpotentially need as much as a year longer than the othertwo variants to complete this variant's Right sciences andship integration testing. The expectations approaching10 to 12 flight sciences sorties/month/aircraft in previousschedules are not achievable in the flight test program untilchanges are made to all variants that improve reliabilityand maintainability in flight test operations, Additionally,the process must begin to reduce the aircraft operatinglimitations, which inhibit flight test progress particularly invertical lift STOVL testing.

• Mission Systems flight test still contains significantuncertainty, which will affect any estimate of a Block 3completion date. This is primarily due to the delaysincurred in development thus far and the filet that only theBlock 0.5 flight test plan has actually been completed andapproved. A test plan for Block 1 is currently in reviewby test center authorities, and the Block 2 test plan is in aninitial draft state. Additionally, technical issues in the helmetmounted display and sensor fusion. along with uncertaintiespertaining to new capabilities with which the program haslimited experience on the F-35 aircraft (multi-functionadvanced data link, distributed aperture system, infrared/

electro-optical fused sensor tracks) are risks that affectthe ability to accurately predict the conclusion of missionsystems flight test. Completion by early 2016 is possibleprovided further delays in delivery of Block 2 and Block 3software are not incurred, and the program can overcome thehelmet mounted display problem before Block 2 flight testmust begin, Mission systems labs and CATB are importantto software integration and test; use of these assets hasenabled the resolution of many problems before flight test.However. F-35 flight test must include integration sorties todemonstrate software performance before performing flighttest points for verification of capability, F-35 flight test forthe purposes of software and sensor integration has not been,but needs to be, an explicit part of the flight test plan suchthat integration precedes verification events.

• The TBR also revealed a number of changes neededto directly support the Edwards and Patuxent RiverIntegrated Test Force flight test centers to assure thehighest possible rate of execution. Recommendations foradditional maintenance and test operations work forces,improving spare parts supply chain management, increasingengineering support for test data analysis, standardizingnetwork connectivity at all sites, and improving priority ofthe program on test ranges are credible, important effortsthat need follow-up and require sustained emphasis for theduration of SOD flight test.

Verification Simulation for Operational Test and Evaluation• Open-air testing is constrained by range limitations that

are incapable of providing realistic testing of many keycapabilities provided by Block 3 aircraft. Consequently, a

F-35 ,JSF 17

robust. operationally realistic VSIM is critical to performingIOT&E of JSF, as required by the Test and EvaluationMaster Plan (TEMP).

• The program office and contractor team have begun work onthe simulation for Block 2 capability needed for the OT-2Foperational utility evaluation, and are beginning to focus onthe process and data requirements to validate installed F-35

performance in the simulation. This critical work needs to becarefully resourced and coordinated, and should be subject toindependent review.

• The JSF VSIM developed for IOT&E will have significantutility for development and testing of upgrades to aircraftcapabilities beyond Block 3 occurring well after IOT&Eis complete. The JSF Program Office Sufficiency Reviewdetermined a path for completing the simulation for Block 3IOT&E within the baseline budget adjustment made in theNunn-McCurdy certified program. Challenges remain inidentifying and collecting the needed validation data forF-35 installed performance and completing the battlespaceenvtronment.

TrainingThe Integrated Training Center made significant progressin preparation for receiving aircraft, support systems, andpersonnel. The development of the syllabi and trainingdevices proceeded essentially on the pace planned inFY I0. However. the adequacy of the training system forthe Integrated Training Center requires reassessment. Usershave expressed concerns about the adequacy of coursecontent and its allocation between training venues, suchas the self-paced computer-based lessons, electronicallymediated instructor lectures. desktop Pilot Training Aid.training events conducted in the cockpit simulators. and on/in-aircraft training.

• The slower than planned pace of mission systems softwaredevelopment and significant aircraft operating limitationsaffect readiness to begin formal training, which is not likelyto occur in mid-2011 as planned. The JOTT operationalassessment of the intended training system and its plannedproducts requested by the PEO will provide an independentidentification of issues, and progress towards resolution. Theeffects of immature aircraft and support systems, along with.

user concerns about adequacy of training venues for intendeduses, will be key aspects of this assessment.

Live Fire Test and Evaluation• The OBIGGS system fails to inert the fuel tank tillage spaces

throughout the combat flight envelopes evaluated.

Recommendations• Status of Previous Recommendations. The program and

Services are satisfactorily addressing four of eight previousrecommendations. The remaining four recommendationsconcerning adequate flight test resourcing, coordinatingexpected level of low-rate initial production capability withusers including the JOTT, accreditation of models used as testvenues, and restoring the means to minimize fueldraulics leaksand coolant shutoff valves are outstanding,

• FY 10 Recommendations. The program should;I. Assure the re-planned detailed mission systems

development schedule and detailed flight test schedule arerealistic.

I, Annually evaluate flight test progress against plannedperformance, assess resources, and recommend adjustmentof Service early fielding goals. Remain prepared to dealwith continued discovery in flight test as more complextesting begins.

3. Determine the impact of resolution of known criticaltechnical issues, including Helmet Mounted Display,STOVL mechanization, handling characteristics, andafterburner "screech" on plans for flight test and fieldingcapability.

4. Assure that there is explicit use of F-35 flight test forsoftware integration before verification,

5. Finalize plans to verify and validate the mission data lo adproducts through dedicated flight test.

6. Complete VSIM development for OT&E in accordancewith the operational testing requirements document andTEMP.

7, Re-design the OBIGGS system to ensure that the fuel tanktillage volume oxygen concentrations are maintained belowlevels that sustain fire and/or explosion throughout thecombat flight envelopes.

18 F-3 5 BF

'my wh..iA•. 1 D

usedLogC2

Intelzt'''ojt..7-'11Z1, --

NET-CENTRIC 1WEB-BASED

WEB-ENABLED ;BROWSER

Global Combat Support System — o n (GCSS-J)

Executive Summary• The Joint Interoperability Test Command (JITC) conducted

a risk assessment for each version of the software andrecommended a level of test for DOME approval. Do &Eapproved the plan for a full operational test for Global CombatSupport System Joint (GCSS-J) version 7.J.0. Because oflower risks for versions 7.1.1 and 7.1.2, DOT&E approved theplan for operational assessments based on JITC's observationof the developmental tests conducted by the program manager.

• The latest version, 7.1.2, is operationally effective andoperationally suitable, The application is survivable againstcyber attacks- The primary host server site. however. did notmeet the required level of cyber attack detection measures.

System• The GCSS-J is a web portal that enables users at combatant

commands and joint task forces to access joint logisticsapplications.

• The system supports planning, execution, and control forengineering, health services, logistics services. supply,distribution, and maintenance operations- It is comprisedof strategic servers (located in Montgomery, Alabama, andPearl Harbor, Hawaii). a commercial otT-the-shelf-basedinfrastructure, and Public Key Infrastructure-

• GCSS-J supports the situational awareness of the militaryoperators by providing applications for the following. : searchquery, and reports capability: Watchboard (allowing rapidcomparison of planned actions with actual events): electronicbankbook (organizing tiles and web pages into categories):knowledge management: business intelligence: mappingcapability: joint engineer planning: and execution capability.

• GCSS-J Increment 7 follows an agile acquisition strategythat supports multiple releases of the updated software incoordination with the user, program manager. and testers-In 2010. the Defense Information Systems Agency (DISA)released three versions of GC SS-J: 7.1.0, 7-l.1. and 7.l.2.

Mission• Joint commanders use GCSS-J to move and sustain joint

forces throughout, the entire spectrum of military operations.

iy Visibility

l• Decision Support Tools-C,aiiiiii.ii:w FLiiiiiiim

' ,■ 'A' :gi Atli:NI A 1,1? ,IA; i.i.,,o, c ■1/4.,,,,, , ,...4

I NET-CENTRICPORTAL / SERVER ]

NET-CENTRICWEB-BASED

APPLICATIONS(normalized data)

- Command and ControlDFAS • Defense Finance and Accounting ServiceDIMHRS - Defense Integrated Mititary Human Resources System

DLA-AV - Defense Logistics Agency Asset VistbilityDLA-lDE Defense Loglstics Agency - Integrated Data EnvlronmentDMDC • Defense Manpower Data CenterECSS -Expeditionary Combat Support:GCSS-Army - Gtobal Combat Support System-ArmyGCSS-MC - Globat Combat Support System-Marine CorpsGCSS -Navy Global Combat Support System-NavyGTN -.Global Transportation NetworkIDE-AV - Integrated Data Environment - Asset VisibilitySALE - Singte Army Logisttcs EnterpriseSIPRNET - Secret Ihternet Protocol Routed NetworkSOCOM - Speclal Operations CommandSOF-TAV - Speclat Operattons Forces - Total Asset VislbilityTMIP Theater Medical Information Program

• Combatant Command and Joint Task Force commanders andlogistics stalk use the GCSS-J to gain end-to-end visibilityof combat support capability up through the strategic level,facilitating information flow across and between combatsupport and command and control functions.

Major ContractorNorthrop Grumman Mission Systems Herndon. Virginia

Activity• JITC conducted a risk assessment for each version of the

software and recommended a level of test for DOT&Eapproval. DOT&E approved the plan for a lull operational testfor version 7,1.0. Because of lower risks for versions 7.1.1and 7,J,2. DOT&E approved the plan for operational

assessments based on JITC's observation of the developmentaltests conducted by the program manager.

• JITC conducted an operational test of GCSS-J version 7.1.0from October 20 through November 3, 2009, in accordancewith the DOT&E-approved test plan.

GCSS-J 19

JITC assessed GCSS-J version 7.1.1 Secret InternetProtocol Router Network (SIPRNet) based on participationin the developmental test conducted from February 22 toApril 22, 2010. JITC conducted a separate operationaltest at the primary hosting site in Montgomery, Alabama,March 8 - 19, 2010, to assess operational survivability againstcyber attacks,JITC assessed (GCSS-J version 7.1.2 using the results fromthe developmental tests conducted by the program manager inaccordance with the risk assessment recommendations. TheSIPRNet GCSS-J version 7,1.2 developmental test was fromJuly 26 through August 3, 2010, and Unclassified but SensitiveInternet protocol Router Network (NIPRNet) version 7,1.2developmental test was September 3 - 7. 2010.

Assessment• GCSS-J version 7.l,0 was operationally effective. It

incorporates an improved Query Tool. The Joint LogisticsManagement functional areas incorporated tools providingimproved user visibility into the status of ammunitioninventories and allowed easier interface with query toolsand the Watchboard. Maintenance, Supply and Services,Movement, Personnel Management, and Health Servicesfunctional areas were operationally effective. However, theJoint Engineering Planning and Execution System had criticalproblems. making that function not operationally effective.

• GCSS-J version 7. 1.0 was operationally suitable, withgood training and good system reliability and availability.

The performance of the help desk showed improvement.Evaluation of IA was limited. IA controls associated withprotect. detect, react, and restore at the application level weresatisfactory, but IA controls at the server level could not beassessed, DOTE agreed to de fer a vulnerability assessmentof the DISA host server suite to the 7.1.1 operational test.

• (GCSS-J version 7.1.1 corrected errors discovered inoperational testing of GCSS-J version 7.1.0, and the systemwas assessed to be operationally effective and suitable. Aseparate vulnerability assessment of the DISA host server suiterevealed that GCSS-J version 7.1.1 did not add significantvulnerability against cyber attacks, but that the primary hostsite in Montgomery, Alabama, could not detect the cyberattacks to the required level,

• Both the SIPRNet and NIPRNet version 7.1.2 are operationallyeffective and operationally suitable. Like version 7.1.1,version 7,J.2 does not cause an unacceptable increase invulnerability of the DISA network. I However, the primary hostsite must make additional improvements toward meeting therequired level of cyber attack detection measures.

Recommendations• Status of Previous Recommendations. DISA has taken

appropriate action on the previous recommendations.• F1'10 Recommendation.

1. DISA should improve the security posture of the serverhosting sites and perform penetration tests on an annualbasis.

20 GCSS-J

Joint Chemical Agent De ector

Executive Summary• In April 2009. the Joint Project Manager chose Smiths

Detection's Lightweight Chemical Detector 3.3 to replace theM4 to improve chemical warfare agent detection sensitivity,reduce false alarm rate, increase battery life (to 25 hOursfrom 12), and reduce acquisition and lifecycle costs,

• The program office conducted M4E I developmental testingfrom May to October 2010-

• Integrated developmental and operational test and evaluationevents included Chemical Warfare Agent Detection andIdentification, Toxic Industrial Chemical Detection andIdentification, Chemical Warfare Agent Detector Clear Down,and Chemical, Biological, and RadiologicaJ ContaminationSurvivability.

• The Army 'Pest and Evaluation Command led a multi-serviceoperational test at Dugway Proving Ground, Utah from July toAugust 2010-

System• JCAD is a hand-held device that automatically detects,

identifies, and alerts warfighters to the presence of nerve andblister vapors, as well as one blood chemical agent vapor andone toxic industrial chemical vapor.

• JCAD is a non-developmental item modified from acommercially available device- It operates as a stand-alonedetector. It is carried by personnel and placed onto variousplatforms, including ground vehicles, at fixed-site installations,and at collective protection shelters. It supplements orreplaces existing fielded chemical agent vapor detectors.

• The total Acquisition Objective for JCAD, M4 and M4E1, is109,705 units. The JCAD will be issued to:

Army squadsMarine platoonsAir Force base reconnaissance, and ground-servicepersonnelNavy shore installations, and riverine or land-based units

Mission• Units use JCAD to provide hazard level indication of chemical

warfare agent and toxic industrial chemical vapors. This alerts

personnel to take personal protection measures. includingmasking and unit force protection measures (contaminationavoidance and increase in mission-level protective posture).JCAD is used for the following purposes:

Personal chemical vapor detectorMonitor in and around a vehicle or shelter's interior andexterior, or aircraft interiorFixed installation Monitor or array of monitors to provideremote alarming

Major ContractorSmiths Detection - Edgewood. Maryland

Activity• In April 2009, the Joint. Project Manager chose Smiths

Detection's Lightweight Chemical Detector 3.3 to replace theM4 to improve chemical warfare agent detection sensitivity,reduce false alarm rate, increase battery life (to 25 hoursfrom 12), and reduce acquisition and lifecycle costs. The JointProject Manager intends to procure 49,705 of the enhanced

detector, designated the M4E1- The Joint Project Managerplans to begin production of the M4EI in March 2011-

• DOT&E approved the JCAD Test and Evaluation Master Plan(TEMP) on July 22. 2010. It provides for side-by-side testingin both developmental and operational testing of the M4and M4E1.

JCAD 21

• The program office. conducted WE 1 developmental testingfrom May to October 20J0.

• The integrated test and evaluation program includes thefollowing developmental/operational test events:- Chemical Warfare Agent Detection and Identification

Toxic Industrial Chemical Detection and Adentification- Chemical Warfare Agent Detector Clear Down- Chemical. Biological, and Radiological Contamination

Survivability.• The Army Test and Evaluation Command led a multi-Service

operational test at Dugway Proving Ground. Utah from July toAugust 2010.

• All testing was conducted in accordance with theDOT&E-approved TEMP.

Assessment• DOT&E is uu evaluating the test data and plans to

publish an evaluation report to support the February 2011production decision.

Recommendations• Status of Previous Recommendations- There was no 1 Y09

report for this program. In accordance with DOT&E's FY08recommendation, the Joint Program Manager plans to conductsurveillance and inspection of the fielded JCADs beginning inOctober 20J1. Fielded systems found to be out of compliancewith the initial set-up parameters will be returned to Smiths •Detection for repair.

• FY 10 Recommendations- None.

JCAD

Joint Tactical Radio System (JTRS)Ground Mobile Radio (GMR)

Executive Summary• The Joint Tactical Radio System (JTRS) Ground Mobile

Radio (GMR) System Integration Test (SIT) was delayed fromMay 17 to June 29, 2010- due to problems with late hardwaredeliveries and immature software.

• The program completed the .ITRS GMR SIT, which wasmodified into a customer test and a government-assesseddevelopmental test, from June 29 to September 14. 2010. Dueto JTRS GMR performance deficiencies revealed during. theSIT, the Army delayed the planned December 2010 JTRSGMR Limited User Test (LW') until June 2011 to allow timefor reliability and performance improvements.

• The Early Infantry Brigade Combat Team (E-IBCT")Increment 1 LUT in September 2010 assessed the JTRS GMRas a component oldie Network Integration Kit (NIK).

• The E-IBCT LUT 10 demonstrated the JTRS GMR'scapability (as a component of the NlK) to transfer imagesfrom E-1BCT sensors to other NIKs within the brigade NIK/GMR network, The E-1BCT LUT 1() highlighted GMRdeficiencies including poor support for secure voice networks,limited transmission range, difficulty connecting to sensorfields and concerns with reliability.

System• JTRS is a family of software-programmable and

hardware-configurable digital radios intended to provideincreased int eroperability. flexibility, and adaptability tosupport numerous tactical communications requirements-

• JTRS GM R components include portable control displaydevices, universal transceivers, network/information securityinterface units, and power amplifiers, which combine to createradio sets for installation in Army and Marine Corps groundvehicles.

NIU --Network Infomation/Security UnitPCDD - Portabte Controt Dtsplay DevtceUT - Universat IranscetverVUPA - VHF (Very High Frequency)/UHF (Ultra High

Frequency) Power AmplifierWBPA - Wideband Power Amplifler

MissionCommanders from the Army and the Marine Corps intend to useJTRS GMR to:• Communicate and create networks to exchange voice, video,

and data during all aspects of military operations.• Interface with other JTRS product line radios and legacy radio

systems in joint and coalition operations,

Major ContractorThe Boeing Company, Integrated Defense Systems HuntingtonBeach. California

Activity• The JTRS GMR has an approved Milestone B Test and

Evaluation Master Plan with requirements based upon JTRSOperational Requirements Document 3.2.l. The Army is inthe process of staffing an updated JTRS GMR CapabilitiesProduction Document.

• The Army delayed the start of the GMR SIT developmentaltest from May 17 to June 29, 2010, due to problems with latehardware deliveries and immature software, The SIT usedEngineering Development Model (EDM) GMRs hosting aselection of terrestrial and satellite JTRS waveforms, including

the Wideband Networking Waveform (WNW) v4.0. The SITwas split into two pans:- SIT Part J, June 29 - August 19, 2010, was a customer test

using 35 GMR nodes to conduct continued test-fix-testactivities of the WNW and to provide an instrumentednetwork for the Director, Defense Research andEngineering to collect data to assess GMR's technologyreadiness levels.

- SIT Part 1 September 1 - 14, 2010, was executed as agovernment developmental test of JTRS GMR and its

JTRS GMR

Increment 1 waveforms conducted by Amy Test andEvaluation Command (ATEC)-

• The JTRS GMR program participated in the Vice Chief ofStaff of the Army (VCSA) Brigade Combat Team IntegrationNetwork exercise in July 2010. This event demonstratedexchange of information among the JTRS GMR, theWarfighter Information Network — Tactical, and the JTRSHandheld, Manpack. and Small Form Fit radios-

• ATEC assessed J5 JTRS GMRs each serving as a component°fa NIK, during the E-1BCT Increment 1 LUT 10 inSeptember 2010. The JTRS GMR within the N K employs thefollowing waveforms:

Frequency hopping, secure Single Channel Ground andAirborne Radio System (SINCGARS)

- Soldier Radio Waveform (SRW) v .0cWNW v4-0

Assessment• The FY 10 JTRS GM R schedule delays were due to late

hardware deliveries and a technically immature operatingenvironment and waveform software.

• In 2009, the JTRS GMR supported the 30-Node WidebandNetworking Waveform (WNW) Field Test and the E-1BCTLUT 09 (as a component of the NIK) with pre-EDM (Witsand an earlier version of the WNW. These CiMRs operatingwith the WNW had difficulty establishing a stable network.poor throughput. unsatisfactory message completion rates, andpoor operational reliability.

• The September 2010 JTRS GMR SIT initial results indicteethat the JTRS GMR and WNW performed better withinportions of the network (subnets), but when tested across29 GMR WNW nodes. the radio continued to demonstratedeficiencies noted in 2009. including:- Difficulty establishing the network

Low data throughput as data is transmitted throughout thenetworkLow message completion rates that decrease ascommunications traffic travels across the larger network

- Low message completion rates that decrease as GMRnodes physically moved within the test range

• The September 2010 E-1BCT LUT 10 demonstrated the NM(with ,JTRS GMR as a component) as capable of transmittingBSOs and images from E-1BCT sensors to others NIKswithin the brigade NIK/GMR network. The JTRS GMR'sperformance (as a component of the NIK) highlighted thefollowing deficiencies:

Undependable SINCGARS frequency-hopping securewaveform (used for voice command and control)Limited transmission •tinge of 7-10 kilometers whichrequired the use of two NIK/GMR relay nodesDifficulty connecting to E-1BCT sensors using the SRW

- CGMR and waveform difficulties which contributed to thepoor reliability of the NIK.

• Due to deficiencies in -ITRS GMR performance revealedduring the SIT, the Army delayed the planned December 2010LUT (in support of the program's Milestone C) until June 2011to allow time for reliability and performance improvements-

• The .JTRS GMR program faces potential restructure underNunn McCurdy that may impact the program's schedule andsupporting test events.

Recommendations• Status of Previous Recommendations. The program addressed

two of the four FY09 recommendations- The previousrecommendations regarding the correction of deficienciesnoted in the 30-Node WNW Field Test and the E-1BCT LUT09, as well as the synchronization of activities to create anintegrated approach between JTRS QMR, JTRS NetworkEnterprise Domain, and the E-1BCT programs remain valid.

• FY 10 Recommendation.I. The JTRS GMR program should execute an independent

government. developmental test to verify correction of allreliability and performance deficiencies revealed duringprevious testing.

24 JTRS GMR

Joint Tactical Radio System (JTRS) Handheld, Manpack,and Small Form Fit (HMS)

Executive Summary• In April 2009, the Army completed the Rifleman Radio

Limited User Test (LUT). DOT&E assessed the RiflemanRadio's LUT performance as supportive of missionpreparation, movement, and reconnaissance, but the radio didnot demonstrate usefulness during squad combat engagementsand exhibited deficiencies in operational reliability.transmission range. battery life, and concept of operations.

• The Joint Tactical Radio System (JTRS) Handheld, Manpack.and Small Form Fit (HMS) Overarching Integrated ProductTeam (OJPT) postponed the program's Milestone C to allowthe program time to resolve program shortfalls and prepare astrategy to address poor reliability and performance problemsdemonstrated during the Rifleman Radio LUT.

• The Army tasked the JTRS HMS program to participate in theVice Chief of Staff of the Army (VCSA) July 2010 BrigadeCombat Team Integrated Network exercise and delayedthe JTRS HMS Rifleman Radio Verification of Correctionof Deficiencies (VCD) test planned for May 2010 untilJanuary 2011- The 2009 Rifleman Radio LUT and 2011 VCDtest will support a Milestone C decision.

System• JTRS is a family of software-programmable and

hardware-configurable digital radios intended to provideincreased interoperability. flexibility, and adaptability tosupport numerous tactical communications requirements.

• The JTRS HMS program provides Handheld and two-channelManpack Radios supporting Army, Marine Corps, Navy, andAir Force operations- The program develops Small FormFit (SFF) radio configurations that include the stand-aloneArmy Rifleman Radio and embedded SFF variants that servein Army host platforms such as the Early Infantry BrigadeCombat Team (E-1BCT) Unattended Ground Sensors (UGS)and Unmanned Aircraft System (Class I ), and Neu Warrior-

• The program strategy has two phases of HMS production.Phase 1 is Rifleman Radios with National Security Agency(NSA) 'Fyne 2 encryption of unclassified information. Phase 2is Manpack Radios with NSA Type J encryption of classifiedinformation.

MissionCommanders from the Army, Marine Corps. Navy, and Air Forceintend to:• Use JTRS Handheld, Manpack, and Rifleman Radios to

communicate and create networks to exchange voice, video,and data using legacy waveforms or the Soldier RadioWaveform during all aspects of military operations-

• Jntegrate JTRS SFF variants into host platforms to providenetworked communications capabilities for users engagedin land combat operations to support voice, video, and dataacross the air, land, and sea battlespace,

Major ContractorGeneral Dynamics, C4 Systems — Scottsdale, Arizona

Activity• In April 2009, the Army conducted the Rifleman Radio LUT

at Fort Bliss, Texas, to support the program's Milestone Cdecision scheduled for November 2009. The LUT assessedthe operational effectiveness, suitability, and survivability

of the Rifleman Radio under numerous mission scenariosexecuted by an infantry platoon within the Army EvaluationTask Force-

.112S HMS 15

• On October 20, 2009, the JTRS HMS owl- postponed theprogram's Milestone C. The OIPT took this action to allowthe program time to resolve unit cost issues and a commercialGPS waiver, and to prepare a strategy to correct reliabilityand performance deficiencies highlighted during the RiflemanRadio LUT

• The JTRS HMS program scheduled a Rifleman Radio VCDtest for May 2010 at the Maneuver Battle Lab, Fort Benning,Georgia. The Army postponed the VCD test to January 2011when the program was tasked to support the VCSA BrigadeCombat Team Integrated Network exercise in July 2010.

• The JTRS HMS program provided SEP A radios to the BrigadeCombat Team Modernization program to support the 4QFY 10E-1BCT LUT 10. These radios functioned as components ofthe E-1BCT UGS.

• The Army Training and Doctrine Command refined twoRifleman Radio requirements as a result of the JTRS HMSOIPT-

The Rifleman Radio range requirement changed froma point-to-point range of 2 kilometers to uninterruptedcommunication for an infantry squad dispersed in a circleof 2 kilometers diameter for urban terrain and 1 kilometerfor dense vegetation.Battery life was reduced from 24 hours to eight hours dueto lechnology limitations.

• The JTRS HMS program initiated a complete redesign of theRifleman Radio hardware to address the deficiencies identifiedduring the April 2009 LUT The Production-RepresentativeRadio (PRR) version of the Rifleman Radio incorporatesdesign improvements in size, weight, and battery life, as wellas increased radio frequency power out. PRR Rifleman Radiosparticipated in the July 2010 VCSA Brigade Combat TeamIntegrated Network exercise. The JTRS HMS program intendsto use PRR Rifleman Radios exclusively in the January 2011VCD test that supports the Army's planned July 2011 RiflemanRadio Milestone C-

• The JTRS HMS program office has on aggressive schedulefor conducting the Manpack Radio WI in 3QFY II and theRifleman Radio IOT&E in 1QFY12-

• The Army is developing a JTRS HMS Manpack RadioAcquisition Strategy Report, Capabilities ProductionDocument, and Test and Evaluation Master Plan (TEMP).

Assessment• DOT&E assessed the performance of the Rifleman Radio

as useful during mission preparation, movement, andreconnaissance activities- During combat engagements. theradio demonstrated poor performance and the squad haddifficulty with employment of the radio.

• During the 2009 Rifleman Radio LUT:- Operational reliability was less than one-half of the radio's

Milestone C requirement 01'840 hoursTransmission range fell short of the radio's requirement of2,000 meters, demonstrating connectivity to 1,000 metersBatteries had a short lifespan and generated excessive heatConcept of operations for employing the radio provedvague and at times hindered operations

• The JTRS HMS program did not assess the developmentof Position Location Information, Information Assurance,Electronic Warfare, and Nuclear. Biological, and Chemicaloperations during the Rifleman Radio LUT. These areas willbe assessed in future developmental testing and the RiflemanRadio IOT&E-

• The J IRS HMS program continues preparation for itsrescheduled January 2011 Rifleman Radio VCD test.The FIRS HMS Manpack does not have an approvedCapabilities Production Document or TEMP.

Recommendations• Status of Previous Recommendations- The JTRS HMS

program is addressing all previous recommendations-• FY 10 Recommendations. The JTRS HMS program should:

1. Develop a strategy to address poor reliability, poorperformance, and the immature intro-platoon concept ofoperations demonstrated during the Rifleman Radio LUT-

These improvements are critical for success during thescheduled 1QFY12 IOT&E-

2. Complete necessary Manpack radio documentation tosupport future operational test,

3. Assure that adequate developmental testing is performedprior to future operational tests.

26 JTRS HMS

Joint Warning and Reporting Network (JWARN)

Executive Summary• Joint Warning and Reporting Network ()WARN) Joint Mission

Application Software (JMAS) is operationally effective as anautomated tool to increase situational awareness regardingpotential chemical, biological, radiological, and nuclear(CBRN) attacks and support force protection decisions.

• JMAS is operationally suitable when hosted on GlobalCommand and Control System (GCCS)-Army, GCCS-Joint,and Command and Control Personal Computer (C2PC)/JointTactical Common Workstation (JTCW) command and controlsystems.

System• JMAS provides a single CBRN warning, reporting, and

analysis tool for battalion/squadron-level units and above tosupport joint operations.

• The program office designed JMA S to improve the speed andaccuracy: of the NATO CBRN basic warning and reportingprocess through automation.

MissionJMAS operators in command cells support CBRN battlefieldmanagement and operational planning by predicting chemical,biological, and nuclear hazard areas based on sensor and observerreports, identifying affected units and operating areas, andtransmitting warning reports.

Major ContractorNorthrop Grumman Mission Systems — Winter Park, Florida

Activity• The Service Operational Test Agencies (OTAs) conducted a

post-operational test demonstration in October 2009 and areliability assessment conference in 2010 to assess the effectof software changes on system reliability.

• DOT&E provided an Operational Evaluation of the JWARNJMAS to support the JMAS Full Deployment Decision onGCCS-Army, GCCS-Joint. and C2PC/JTCW command andcontrol systems.

Assessment• A unit equipped with JMAS is able to provide CBRN warning

reports in time to institute force protection actions beforeencountering CBRN hazards for units operating 10 or morekilometers from area in wh ich the CBRN hazard is initiallyidentified-

•JMAS not effective for warning at the company level due toits limited capability for warning units operating between 2.5and 10 kilometers from the hazard release, depending uponmeteorological conditions and distance-

• JWARN is interoperable with GCCS-Joint, GCCS-Army, andC2 P TC W command and control systems.

• Engineering changes resolved JMAS software reliabilityshortfalls identified during the FYO8 Multi-Service OT&E.

Recommendations• Status of Previous Recommendations: The program manager

addressed all of the previous recommendations-• FY 10 Recommendations. None.

JWA RN

2$

Mine Resistant Ambush Protected (MRAP)Family of Vehicles

Executive Summary• DOT&E transmitted its Assessment of the Mine Resistant

Ambush Protected (MRAP) Family of Vehicles to Congressand the Secretary of Defense in March 2010.

• In FY 10, the (MRAP program continued a capabilities insertionprogram in FY 10 to acquire, test, and assess enhancedcapabilities and solutions to be integrated across the MRAPFamily of Vehicles. The capability insertions are undergoingdevelopmental, live fire, and operational testing to assess theircontribution to MRAP operational effectiveness-

• The Army Test and Evaluation Command (ATEC)completed the operational testing of the Force ProtectionIndustries, Inc. (FPI) Cougar Independent Suspension System(ISS) and Navistar Dash MRAP variants in December 2009 atYuma Proving Ground, Arizona.

•DOT&E assessed the FPI Cougar (Category I (CAT I) andCategory II (CAT 11) as possessing the off-road mobilityneeded to transport units over Afghanistan terrain, The FPICougars are operationally suitable.

• The Navistar Dash, as tested, is not operationally effective foruse in Operation Enduring Freedom. These vehicles couldnot negotiate cross-country terrain. The program plans toincorporate an JSS to improve off-road mobility. The vehicleswere not operationally suitable due to poor reliability.

System• MRAP vehicles area family of vehicles designed to provide

increased crew protection and vehicle survivability againstcurrent battlefield threats, such as IEDs, mines, and smallarms. DoD initiated the MRAP program in response to anurgent operational need to meet multi-Service ground vehiclerequirements- MRAP vehicles provide improved vehicleand crew survivability over the High Mobility Multi-purposeWheeled Vehicle (HMM WV) and are employed by unitsin current combat operations in the execution of missionspreviously execut ed w ith t h e Hmmwv,

• This report covers two categories of MRAP vehicles andthe MRAP-Ambulance variant- The MRAP CAT I vehicleis designed to transport six persons and the MRAP CAT IIvehicle is designed to transport 10 persons. The MRAPAmbulance variant vehicle is designed to transport up to threelitter casualties and from three to six ambulatory casualties.MRAP vehicles incorporate current Service command andcontrol systems and counter-1ED systems. MRAP vehiclescontain gun mounts with gunner protection kits capable ofmounting a variety of weapons systems such as the M240Bmedium machine gun, the M2 .50 caliber heavy machinegun, and the Mk 19 grenade launcher- The program hasdevelopmental efforts underway to integrate improved

FPI Cougar Category J

FPJ Cougar Category JI

Navistar Dash Category I

BAE RO-32L Category JI

GDLS RG -31 A2 Category I BAE TVS Caiman Category I

NAVISTAR AmbuJance Cat. J BAE AmbuJance Category AI

protection against Rocket Propelled Grenades (RPGs) onexisting MRAP vehicles-

• Five vendors have been awarded ongoing productioncontracts for MRAP CAT land CAT 11 vehicles: FPI, GeneralDynamics Land Systems Canada (GDLS-C), NAVISTARDefense, BAE-Tactical Vehicle Systems (BAE-TVS), andBAE Systems (BAE)- Six CAT 1 and CAT II variants havebeen purchased:- FPI Cougar CAT I- FPI Cougar CAT II- NAVISTAR Defense axxPro CAT 1 vehicle and

Ambulance variant

MRAP Family of Vehicles

BAE RG-33L CAT Ii and Ambulance variant- GDLS-C RG-31 A2 CAT 1- RAE TVS Caiman CAT I

• Units equipped with the MRAP CAT 1 vehicles will conductsmall unit combat operations such as mounted patrols andreconnaissance. Many of these Operations are conducted inurban areas. Units equipped with MRAP CAT II vehiclesconduct ground logistics operations including convoy security,troop and cargo transportation, and medical evacuation- TheMRAP Ambulance variant supports the conduct of medicaltreatment and evacuation.

• M RAP vehicles support multi-Service missions and are fieldedto units based upon priorities established by the operationalcommander.

Major Contractors• Force Protection Industries (FPI), Inc, — Ladson, South

Carolina• General Dynamics Land Systems Canada — Ontario, Canada• NAVISTAR Defense Warrenville, Illinois• BAE-TVS — Rockville, Maryland• RAE Systems — Santa Clara, California

Activity• In FY I0. the MRAP program continued a capabilities insertion

program to acquire, test, and assess enhanced capabilitiesand solutions to be integrated across the MRAP Family ofVehicles. The major capability insertions are the following:ISS: Command. Control. and Communication Suite: CommonRemote Weapon Station: and Gunner Protective Kit OverheadProtective Cover.

• The Joint Program Office is managing the capability insertionprogram through Engineering Change Proposals. Thecapability insertions are undergoing developmental, live fire,and operational testing to assess their contribution to MRAPoperational effectiveness.

• ATEC completed the operational testing of the FPI Cougar ISSand Navistar Dash MRAP variants in December 2009 at YumaProving Ground, Arizona.

• In March 2011, the program will execute a Limited UserTest (LUT) at Yuma Proving, Ground, Arizona, to examinea unit's ability to execute missions with the MRAP Familyof Vehicles modified with an ISS and the operationaleffectiveness of the FPI Cougar CAT II ambulance variant-

• Following publication of the DOT&E MRAP report inMarch 2010, the Secretary of Defense directed additional livetire testing on two of the MRAP variants. ATEC completedthese tests in accordance with a DOT&E-approved test plan-

• ATEC completed Live Fire testing orate FPI Cougar Al andA2 ISS upgrades in 3QFY10- DOT&E plans to issue a singlevulnerability report on these vehicles in early

Assessment• The FPI Cougar CAT I and CAT II demonstrated the off-road

mobility needed to transport units over Afghanistan - like terrainduring the MRAP All Terrain Vehicle IOT&E.

• The FPI Cougars with ISS enable units to be less predictable intheir movement, continue operations under minor protection,and conduct a greater variety of mounted maneuver toapproach and secure an objective than possible with currentMRAPs.

• The FPI Cougars are operationally suitable-• The Navistar Dash, as tested, is not operationally effective

For use in ;Operation Enduring Freedom. These vehiclescould not negotiate cross-country terrain. The programplans to incorporate an ISS to improve off-road mobility-The MRAP program intends to complete operational testingof an ISS-equipped. Dash in FY11. The vehicles were notoperationally suitable due to poor reliability, The NavistarDash demonstrated 121 Mean Miles between OperationalMission Failure (MMBOMF) versus its operationalrequirement of 600 MMBOMF.

Recommendations• Status of Previous Recommendations. The MRAP program

continues to address all previous recommendations.• FY 10 Recommendation-

I, The program should improve the Navistar Dash reliabilityand off-road mobility capability by integrating an ISS andcompleting operational testing of an ISS-equipped Dash-

30 MRAP Family of Vehicles

•Special Operations

Forces MATVBase MATV

Mine Resistant Ambush Protected (MRAP)All Terrain Vehicle (M-ATV)

Executive Summary• The Army Test and Evaluation Command (ATEC) completed

the IOT&E for the Mine Resistant Ambush Protected (MRAP)All Terrain Vehicle (M-ATV) at Yuma Proving Ground,Arizona, in December 2009,

• ATEC completed the majority of the Live Fire testingdocumented in the M-ATV Operational -fest Agency Test Planby March 2010.

• DOT&E provided the M-ATV Live Fire and OperationalTest and Evaluation Report to Congress and the Secretaryof Defense in June 2010- DOT&F, assessed the M-ATV asoperationally effective. operationally suitable, and survivablefor armored tactical mobility and transport to units in supportof Operation Enduring Freedom missions.

• The Special Operations Force (SOF) M-ATV variant IOT&EGs planned for November 2010.

Sytem• The M-ATV is the smallest vehicle of the MRAP family of

vehicles. The M-ATV is designed to have the current MRAPlevel of protection and mobility similar to the High MobilityMulti-purpose Wheeled Vehicle (H.MM WV). The vehicle willsupport combat and stability operations: in highly restrictedrural, mountainous- and urban terrain with off-road movementconducted greater than 50 percent of the time-

• The M-ATV is designed to improve vehicle and crewsurvivability over the up-armored HMMWV The M-ATV hasthe capability to add protection against attacks by ExplosivelyFormed Penetrators (Fn .'s) and Rocket-Propelled Grenades(RPGs) to support mounted patrols, reconnaissance, security,and convoy protection.

• The M-ATV is designed to transport five persons andincorporates current Service command and control andcounter-IED systems. The M-ATV includes gun mountswith gunner protection kits capable of mounting a variety ofweapons systems such as the M240B medium machine gun,

the M2 -50 caliber heavy machine gun, and the Mk 19 grenadelauncher.

• Oshkosh Defense has been awarded a production deliveryorder for 10,000 M-ATV vehicles-

• The U.S. Special Operations Command (USSOCOM) requiredmodifications to the base M-ATV vehicle to support SOFmissions- These vehicles are referred to as the SOF M-ATVvariants- The modifications included an additional fifth seatposition, protection for the cargo area, rear area access, andsome other improvements for human factors-

Mission• Units equipped with the M-ATV vehicle conduct mounted

patrols, convoy patrols, convoy, proteetion, reconnaissance,and communications, as well as command and controlmissions to support combat and stability operations in highlyrestricted rural. mountainous, and urban terrain. The MATVis reconfigurable to meet mission requirements-

• M-ATV vehicles support multi-Service missions and arefielded to units based upon priorities established by theoperational commander-

Major ContractorOshkosh Defense Oshkosh, Wisconsin

Activity• ATEC completed the IOT&E for the M-ATV at Yuma Proving

Ground, Arizona. in December 2009.• In March 2010. the Marine Corps Operational Test and

Evaluation Activity completed M-ATV high-altitude testing atCamp Navajo, Arizona-

• The MRAP program has procured 421 SOF M-ATV variantsfor Special Operations Command.

• Live Fire testing of the SOF M-ATV variant commencedin 3QFY 10-

• USSOCOM began the SOF M-ATVloTkvar-ant inNovember 2010 at Yuma Proving Ground, Arizona.

M -ATV 3

Assessment• The M-ATV is operationally effective, operationally suitable,

and survivable for armored tactical mobility and transport tounits in support of Operation Enduring Freedom missions.

• The M-ATV successfully demonstrated off-road mobilitycomparable to the up-armored HMMWV with FragmentationKit 5 in operational testing.

• The off-road mobility and maneuver capability of the M-ATVenables units to be less predictable in their movement.continue operations under armor protection, and conduct agreater variety of mounted maneuvers than possible withcurrent MRAPs-

• The M-ATV has very limited room for the crew to egress thevehicle due to the location of mission command, control, andcommunication equipment on the center console.

• Based on results of the M-ATV high-altitude testing. vehiclemobility in son soils can be a risk on unimproved trails and

roads through mountainous terrain, especially when operatingat the edge of the road, or should the trails give way- As aresult, M-.ATVs can roll over under these conditions-

• The SOF M-ATV variant Endurance Testing is ongoingat Yuma Proving Grounds, Arizona. The M-ATV hasaccumulated 586 operational miles of the planned 3,000 miles.

• Results from the SOF M-ATV variant automotive testingindicate the vehicJe has comparable performance to theBaseline M-ATV.

Recommendations• Status or Previous Recommendations- The MRAP program

continues to address all previous recommendations.• FY 10 Recommendations. None-

M - ATV

Multi-Functional Information Distribution System (MIDS)(includes Low Volume Terminal (LVT) and Joint Tactical Radio System (JTRS))

Executive Summary• The Multifunctional Information Distribution System Joint

Tactical Radio System (MIDS-JTRS) core terminal completeddevelopment and entered into IOT&E of the MI DS-JTRSintegrated into the FA- I SPE aircraft during July 2010-IOT&E is scheduled to complete in November 2010.

• Open issues at the transition from developmental test toindependent operational test included Tactical Air Navigation(TACAN) range accuracy and excessive Built-In Test (BIT)False Alarms- Commander Operational Test Force (CUTF)will examine these issues more closely, as well as theapproved Critical Operational Issues, during the operationaltest-

• During the MIDS-JTRS IOT&E, the program successfullycompleted a multi-channel demonstration in whichMIDS-JTRS simultaneously exercised Link 10. TACAN,and Single Channel Ground and Airborne RadioSystem (SINCGARS) functionalities,

• Not all MIDS-JTRS core terminal capabilities, such asLink 16 enhanced data throughput and instantiation (AJTT6Software Communications Architecture waveforms, will beoperationally tested due to limited current aircraftrequirements and funding availability.

System• Multifunctional Information Distribution System - Low

Volume Terminal (MIDS-LVT) is a communications andnavigation terminal in full-rate production. When integratedinto a host platform. MIDS-LVT provides Link 16 digitaldata link. Link 16 digital voice communications, and TACANcapabilities. Since production started, the MIDS-LVT hasevolved with hardware, firmware. and software updates toresolve performance and stability deficiencies and to providenew Link 16 capabilities-

• MIDS-JTRS is a pre-planned product improvement of theMIDS-LVT system. When integrated into a host platform,MIDS-JTRS provides MIDS-LVT capabilities, plus threeadditional programmable channels capable of hosting JTRSSoftware Communications Architecture-compliant waveformsin the 2 to 2,001) megahertz radio frequency bandwidth. Inaddition, MIDS-JTRS will provide the capability for enhancedthroughput and Link 16 frequency re-mapping-

• The system under test includes the MIDS terminals and thehost platform interfaces such as controls. displays. antennas.high power amplifiers. and any radio frequency notch filters.

• TACAN has an air-to-air mode and an air-to-ground modeand is a Primary means of air navigation by military aircraft.

Link 16 data link is a joint and allied secure anti-jam highspeed data link that uses standard messages to exchangeinformation among flight or battle-group host platforms orbetween combat platforms and command and control systems.Link 16 digital voice provides host platforms as secure anti-jamvoice line-of-sight communications capability.

Mission• U.S. Services and many allied nations will deploy MIDS-LVT

and MIDS-JTRS-equipped aircraft. ships. and ground unitsin order to provide military commanders with the abilityto communicate with their forces by voice, video, and dataduring all aspects of military operations- MIDS-JTRSnetworking capability and multiple waveforms (including newwaveforms such as the Joint Airborne Networking,— TacticalEdge (JAN-TE)) are intended to allow collaboration despitegeographical and organizational boundaries-

• MIDS-JTRS-equipped units should be able to exchangeinformation including air and surface tracks, identification,host platform fuel, weapons, mission status, engagementorders, and engagement results.

Major Contractors• United States:

- ViaSat -- Carlsbad, California- Data Link Solutions - Wayne. New Jersey, and Cedar

Rapids, Iowa• Europe:

- EuroMIDS Paris, France

MIDS 33

ActivityMl DS-LVT (B-1 B Integration)• Detachment 5, Air Force Operational Test and Evaluation

Center (AFOTEC) conducted an Operational Assessmentof the integration of the MIDS-LVT into the B-I B Bomberaircraft at Edwards AFB, California, from February toJune 2010- The Common Link Information Processor(CLIP) facilitated the integration of MIDS-LVT into theB-1B aircraft- The Operational Assessment included threeoperational test flight sorties by the B-I B aircraft.

MIDS-JTRS• The Naval Air Warfare Center completed the ground

and flight Developmental Test and Evaluation of theMIDS-JTRS as integratcd on the FIA-18E/F operating fromthe Naval Air Station (NAS) Patuxent River, Maryland. andNAS China Lake, California.

• The Naval Air Warfare Center, with participation by the AirTest and Evaluation Squadron Nine, conducted integratedaircraft carrier suitability flight testing of the F/A- 1 8E/Fwith the integrated MIDS-JTRS core terminal.

• cuff, started the F/A-18E/F MIDS-JTRS IOT&E inJuly 2010 at NAS China Lake, California, and participatedin a large joint force training exercise conducted atNellis AFB- Nevada to collect Link 16 interoperability testdata.

• All testing was conducted in accordance with aDOT&E-approved Test and Evaluation Master Plan(TEMP) and operational test plans.

AssessmentM (B-1B Integration)• Results from the B-1B MIDS-LVT and CLJP integration

Operational Assessment indicate that the transfer ofMIDS-LVT Link 16 data via CLIP to the B-1 B hostcomputer was problematic- Specifically, inaccurateinformation transferred for some of the Link 16message types, and CLIP could not completely andaccurately process information from near simultaneousbeyond-line-of-sight and line-of-sight transmissions ofLink 16 messages. CLIP software was riot yet mature,as indicated by the quantity of unresolved softwarediscrepancies and a constant (not yet decreasing) rate ofdiscovery of software discrepancies-

MIDS-JTRS• The aircraft carrier suitability integrated test results indicate

compatible operation of the F/A-18E/F M IDS-JTRS duringapproach, take-off, and landing on the aircraft carrier.

• The MIDS-JTRS lOT&E data collection and analysiseffort is ongoing. There are emerging concerns related toMI DS-JTRS TACAN air-to-air range errors- There areemerging concerns regarding the ability to achieve thereliability requirement of 220 hours for the MIDS-JTRSterminal, as evidenced by ten MIDS-JTRS terminal(hardware) failures during IOT&E,

• Data show the BIT Use alarm rate associated with theintegration of the MIDS-JTRS into the F/A-18E/F isexcessive, potentially affecting maintainability ratings forthe system.

• The operational test squadron, Air Test and EvaluationSquadron Nine, based at NAS China Lake. California,identified aircraft integration discrepancies during theNi MIDS-JTRS installation and pre-operational flight checksconducted prior to the start of the IOT&E. Aircraftmaintenance personnel resolved the issues by conductingsignificant levels of minor maintenance to correctaircraft electronic systems discrepancies or developingworkarounds to permit IOT&E to begin as scheduled.

Recommendations• The Air Force and Navy made satisfactory progress on the

previous recommendations primarily related to the fieldedMiDS-LVE

• FY I0 Recommendations-I. The Air Force B-1 B and CLIP program offices should

review the findings from the CLIP (B-I B/MIDS-LVT)integration Developmental Test and Operational Assessmentand correct all major deficiencies prior to entry into theB-1 B CLIP integration OT&E,

2. The MIDS-JTRS and FA-IS program offices shouldcontinue to characterize the TACAN discrepancies anddevelop a solution for test prior to fielding.

3. The Navy should modify the MIDS-JTRS F 1 A-18installation checklist for fielding to ensure appropriatemaintenance personnel adequately prepare all antenna andcabling connections for installation of the terminal.The MIDS program office should focus industry efforts onachieving improved terminal manu facturing processes inorder to elevate the overall reliability of the MIDS-JTRSsystem,

5, The Navy. [AIDS program office and industry shoulddevelop a solution to the excessive BIT false alarm rate forthe MIDS-JTRS system as integrated into the F/A-I8 FIR

4-

34 MIDS

Oecete spe:elffeInfafrestism for

people

MetadataViscPve!Y

.44e.a.n

Aeries, to netweer.develop to rest d

semantics

CantEnt., • DiScOrry. ..

••••• • •••fr'.',Aarne

impravid Cement •awareness

Service

Mediation

&lune OW

unintinIP•tidwere Aiamau

Network Centric Enterprise Services (NCES)

Executive Summary• The Operational Test Team conducted a series of FOT&E II

events on a subset of Increment 1 Net-Centric EnterpriseServices (NCES) from November 2009 through July 2010.The FOT&E 11 was conducted in accordance with theDOT&E-approved test plan. However, testing of someservices (Enterprise Service Management (ESM) andMachine-to-Machine Messaging) relied heavily on pilotprograms and demonstrations providing less confidence in theresults-

• The Milestone Decision Authority authorized full deploymentfor Service Discovery on February 12, 2010, and made afielding decision for Enterprise File Delivery, EnterpriseSearch, ESM. Machine-to-Machine Messaging, and PeopleDiscovery on September 20. 2010.

• Penetration testing during the FOT&E, revealed InformationAssurance (IA) shortfalls at the Defense EnterpriseComputing Centers (DECCs) housing NCES capabilities.The Defense Information Systems Agency (DISA) addressedthese shortfalls and conducted a Verification of Correctionsassessment from May to June 2010. Further incident responserecommendations for the DECCs are being incorporated inplans, processes, and procedures to be released in late FY I0.

• Machine-to-Machine Messaging is rated as operationallyeffective and operationally suitable with limitalions. Withthe suspension of key pilot programs and demonstrations, nonew programs have committed to use Machine-to-MachineMessaging. Users report a preference for another DISAservice, the Joint User Messaging Service.

• DOME is unable to determine the operational effectivenessof Service Discovery and ESM based on FOT&Es I and II.

Service Discovery requires additional testing incorporatingan expanded data base of registered services and a largeruser base- The DoD Chief Information Officer (CIO)memorandum "Net-Centric Enterprise Services (NCES)Service Discovery Governance" (August 3, 2010) outlinesa plan of action to improve the governance process andencourage the use of Service Discovery.ESM testing also had a limited set of users who only useda portion ofdie capabilities provided. More guidancefrom DoD CIO and Under Secretary of Defense forAcquisition, Technology, and Logistics (USD (AT&L))to the DoD Services, Agencies, and Commands is neededto help software service integrators select and configureappropriate ESM capabilities,

System• NCES is a suite of individual capabilities that support

automated information exchange across the DoD on bothclassified and unclassified networks- These capabilitiesinclude collaboration, discovery, and mediation and user

access through the Defense Knowledge Online pertal- Eachservice is unique and has its own IOT&E and acquisitionfielding decision.

• NCES collaboration tools enable users to hold meetings andexchange information by text, audio, and video-

• The discovery capabilities (content, people, services,metadata, publish/subscribe) allow data producers to postinformation, alert others to the presence of new information,and evaluate the relevance Of the data to their current roles andactivities.

• NCES includes security and management capabilities thatintegrate with and rely upon:

capabilities providing data on performance,operational status, and usage of web services that enhancenetwork situational awareness t6 the Global JnfrastructureServices Management Center

- 1A/computer network defense• The software for all the NCES services is comprised of

commercial off-the-shell- and government off-the-shelfproducts- The concept is to provide commercially availableproducts managed under a contract that specifies maintenance,support, and performance levels :for each Service, commonlyknown as Service-Level Agreements (SLAB).

• The warfighting, intelligence, and business communities(i.e-, finance, medical, and logistics) will access NCEScapabilities either directly or through a portal that controlsaccess by the use of Public Key Infrastructure profiles.

• NCES services are available to all operational and tacticalusers who connect to a Defense Information System Networkpoint-of-presence-

• NCES is a collection of services from which users can selectthose that best fit their needs. Users can be syStem or softwaredevelopers, system or network administrators, communitiesof interest, programs of record, or personnel executingwarfighting, business, and intelligence missions.

NCES 35

MissionJoint Force Commandersillw... use selected NCES servicesto enable shared understanding. interface with otherdecision-makers, orient forces, assess the situation, and/orsynchronize operations-

Major ContractorGovernment Integrator -- DISA

Activity• The Joint Interoperability Test Command (J ITC) led a

multi-Service operational test team that conducted a series ofFOT&E II events on a subset of NCES Increment I servicesfrom November 2009 through July 2010. The FOT&E AI wasconducted in accordance with the DOT&E-approved Test andEvaluation Master Plan and operational test plan- However,testing of some services (ESM and Machine-to-MachineMessaging) relied heavily on pilot programs anddemonstrations providing less confidence in the results-Services tested include the following:- Enterprise File Delivery- Enterprise Search- ESM

Machine-to-Machine MessagingPeople DiscoveryService Discovery

• The Milestone Decision Authority authorized full deploymentfor Service Discovery on February 12, 2010, and made afielding decision for Enterprise File Delivery, EnterpriseSearch, ESM- Machine-to-Machine Messaging, and PeopleDiscovery on September 20, 2010-

• The test team conducted an IA assessment for a subset of theNCES services hosted at the DECCs. These assessmentsincluded IA documentation reviews, interviews with programoffice and hosting site personnel, penetration testing, andspecial events to assess restoration, failover, and incidentresponse.

• The test team plans to conduct a continuous evaluation of theESM service to assess how it performs as the number of usersincrease.

Assessment• The FOT&E II events were adequate to assess the operational

effectiveness and suitability of a subset of NCES- Immatureprocedures and processes and the absence of sufficientnumbers of end users limited the ability of DOT&E to assessESM and Service Discovery as operationally effective.During the FOT&E II events, testers encountered significantlimitations. An extremely limited user base for many servicesprecluded an assessment of scalability to the levels envisionedin the Capabilities Production Document for the DoDenterprise. In addition, inconsistent quality of data providedby the various Managed Service Providers precluded anassessment of suitability-

The following is a synopsis of the results for each serviceevaluated during the FOT&E events:

The Enterprise File Delivery Service successfullysynchronizes content to Multiple consumers on theunclassified network and is operationally effective- It isconsidered operationally suitable with limitations. TheSLA and documentation need improvement-Enterprise Search is assessed as operationally effectivewith limitations. Improvements were made in timeliness,relevance of results, methods for exposing information,and the number of content sources since the IOT&Ein 2008. However, users did not find that the NCESEnterprise Search provided advantages over existing searchtools- Enterprise Search is also assessed as operationallysuitable with limitations- User documentation and supportto content owners need improvement-DOT&E was unable to determine the operationaleffectiveness of ESM due to the low level of use (onlyone of three monitoring agents was used during the test)and because a majority of services being monitored werenot being used in an operational context. Users reportESM provides a needed capability to optimize softwareperformance and provide them situational awarenessfor distributed software services- More guidance fromDoD CIO and USD (AT&L) is needed to enable softwareservice integrators to select and configure appropriate ESMcapabilities. Also further testing should be conductedin which ESM provides situational awareness and

management of operational services. ESM is consideredto be operationally suitable due to its ease of use and goodhelp desk support.Machine-to-Machine Messaging is rated as operationallyeffective and operationally suitable with limitations. Usersreported the service performs the required functions, butthe users had problems troubleshooting faults, configuringsecurity devices, and monitoring service health and status,and were not notified of upgrades or outages- With thesuspension of key pilot programs and demonstrations, nonew programs have committed to use Machine-to-MachineMessaging. Users report a preference for another DISAservice, the Joint User Messaging Service. Since theDOT&E report, Machine-to-Machine Messaging has beenintegrated with a new Program of Record and the ProgramOffice is currently working with several potential users.

36 NCES

- Upgrades to People Discovery demonstrated markedimprovement and it is now considered operationallyeffective with limitations and operationally suitable.There is improved consistency of results and access tomore authoritative personnel sources- The service meetsavailability and performance requirements and is easy touse. Users would like to see more contact informationlisted for each person, especially for People Discoverysearch results on the classified network. There is a need forthe various authoritative personnel databases to maintaina consistent amount and type of personnel information/attributes on individuals to ensure accurate PeopleDiscovery search results (e.g. phone numbers, e-mailaddresses, work organization).

- An upgrade to Service Discovery resolved findings fromthe FOT&E 1, including, user interface and documentationshortcomings. The DoD CIO signed a memorandum(NCES Service Discovery (Governance) on August 3, 20J0,with a Plan of Action to improve the governance processesand encourage the use of Service Discovery and thereuse of registered services. However. there is still alack of sufficient governance processes for practical

implementation of Service Discovery and Enterpriseadoption to assess the mission benefits of service re-use-

• FOT&E IJ survivability assessments identified significantIA deficiencies at the sites hosting NCES capabilities, As aresult of the FOT&E II assessment, DISA made adjustmentsto hosting sites and security practices- JITC conducted aVerification of Corrections assessment of these AA deficienciesfrom May to June 2010, Further incident responserecommendations are being incorporated in plans, processes,and procedures to be released by DISA in late FY10.

Recommendations• Status of Previous Recommendations. The DoD C10, DISA,

and JITC satisfactorily addressed two of the three FY09recommendations- The recommendation concerning periodicindependent assessments to evaluate scalability of services toEnterprise levels remains-

• FY 10 Recommendation-1. DISA and the NCES Program Management Office should

Conduct assessments evaluating adoption across the DoDand the utility of the Machine-to-Machine Messaging andPeople Discovery-

NCES 37

38

Mr.! ECA Loot r_At.ro r

+17-0.11,1+c

Netwnr1+ Backbone+ andRotated It rtrastrucluro

GIGdtptt••

SorvIcirs

lia Ian AITim

o

, • .

nook..

PNI

PKI.Vor I Oh

ekrec nrark:nr.

Public Key Infrastructure (PKI) Increments and 2

Executive Summary• DoD Public Key Infrastructure (PKI) Increments I and 2

provide authenticated identity management via apassword-protected Common Access Card (CAC) and SecureInternet Protocol Routing Network (SIPRNet) token toenable DoD members, coalition partners, and others to accessrestricted websites, enroll in online services, and encrypt anddigitally sign e-mail.

• JITC conducted separate FOT&Es for DoD PKI Increment I,Spirals 3 and 4 in April 2010 and August 20J0. respectively.The capabilities provided in the spirals were operationally ,

effective, suitable, and survivable for deployment. However,Spiral 3 capabilities provided inaccurate reporting ofcertificate revocations that most be corrected.

• The Operational Assessment f i r Increment 2, Spiral 1 showedRegistration Authorities (RAs) were able to efficiently issueSIPRNet tokens, and end users were able to use those tokensto facilitate missions through digital signing, encryption. andweb-server authentication. However- reliability of the tokenswas unacceptable when approximately ten percent of thosedistributed during the Operational Assessment were defective-

System• DoD PKI is a critical enabling technology for Information

Assurance (IA). It supports the secure flow of informationacross the Global Information Grid (DIG) (Non-SecureInternet Protocol Routing Network (NIPRNet) and SIPRNet)-as well as secure local storage of information.DoD PKI provides for the generation, production, distribution,control, revocation, recovery, and tracking of Public Keycertificates and their corresponding private keys. DoD PK Iworks with commercial off-the-shelf (COTS) and governmentoff-the-shelf (GOTS) applications to provide IA ande-business capabilities.

• Using authoritative data, DoD PKI creates a credential thatcombines identity information with cryptographic informationthat is non-forgeable and non-changeable. In this way, DoDPK 1 provides a representation of physical identity in anelectronic form.

• DoD PKI Certification Authorities (CAS) for the NIPRNetand SIPRNet software certifications reside in the DefenseInformation Systems Agency (DISA) Defense EnterpriseComputing Centers (DECC) in Chambersburg, Pennsylvania,and Oklahoma City. Oklahoma. PK 1 CAs for issuanceof the SIPRNet hardware tokens reside in the DECC inChambersburg, Pennsylvania.

DoD PKI is comprised of COTS hardware, COTSsoftware, and other applications software developed by theNational Security Agency (NSA)-Certificates are imprinted on the DoD CAC token forNIPRNet personnel identification using data taken

n En

Elornent 4:3r.rx.

ARA - Auto-Key Recovery Agent

LRA - Locof Registration AuthorityCA Certification Authority

ORC Operatronal Researclt Consultants. Inc

CAC - Continent Access Card

PK -Peek KeyDEERS DOOrISII) EfigreA

RA Regisnation AulhorityRepertine Syklare

RAPIDS - Real-Tone Automated PersohnelDMDC - Dblettae Manpower Dina Center

Inentificalion System

DEM r Data Recoyory ?Tanager

SIPRNer - SECRET Internet Protocol RattlerECA Enterer ine. Certification Authority

Network

GIG - Global information Grid

VO Verify:rag arreralklenctficatern

from the Defense Enrollment Eligibility ReportingSystem (DEERS)- The Secret DEERS provides thepersonnel data for certificates imprinted on a separateSIPRNet token-

• DISA and NSA are jointly developing DoD PKI in multipleincrements- Increment J is broken into five spirals, fourof which have been operationally tested and deployed onNIPRNet. Increment 2 is being developed and deployed inthree spirals on the SIPRNet,

Mission •• DoD PKI enables net-centric operations by allowing military

operators, communities of interest, and other authorizedusers to securely access, process, store, transport, anduse information, applications, and networks regardless oftechnology. organization, or location-

• Commanders at all levels will use DoD PKT to provideauthenticated identity management via password-protectedCAC or SIPRNet token to enable DoD members, coalitionpartners, and others to access restricted websites, enrollin online services, and encrypt and digitally sign e-mail-Commanders will use specific PKI services to:

Enable and promote it common ubiquitous secure webservices environment.Enable the integrity Of data/forms/orders moving withinthe GIG (both NIPRNet and SIPRNet), via use of digitalsignatures-

PKI 39

Enable management of identities operating in groups orcertain roles within GIG systems.Ensure the integrity and confidentiality of what is operatingon a network by providing assured PK I-based credentialsfor any device on that network

Major ContractorBAIT Systems (ncorporated - Arlington, Virginia

ActivityIncrement 1 Spirals 3 and 4 (FOT&E)• JITC conducted separate FOT&Es for DoD PKI

Increment I, Spirals 3 and 4 in April 21)10 and August2010, respectively- Integrated developmental andoperational testing was accomplished according toDOT& [-approved test plans and procedures in the JITCPKI laboratory at Fort Huachuca, Arizona.- Spiral 3 testing evaluated software applications

encompassing three new or improved capabilities: LocalRegistration Agent version 4 (LRA v4), Web Based BulkRevocation (WBBR) server, and the Certificate HistoryRepository Information Service (CHR-IS).

- Spiral 4 testing evaluated system upgrades to the RobustCertificate Validation Service (RCVS), which includesmigration to a new operating system and architectureenhancements.

Increment 2, Spiral 1 (Operational Assessment)• For Increment 2, Spiral 1,JITC conducted an Operational

Assessment in June and July 2010 in accordance with theDO T& E-approved test plan and procedures- Typical usersfrom a variety of operational environments participated inthe test event.

• The PK 1 Program Management Office (IMO)experimented with varying network conditions in Februaryand September 2010 to better define problems with PKI usein tactical and austere environments.

AssessmentIncrement 1, Spirals 3 and 4 (FOT&E)• The testing conducted by JITC was adequate to assess

the operational effectiveness and suitability of the DoDPKI Increment 1, Spirals 3 and 4 enhancements- Thecapabilities provided in the spirals were operationallyeffective, suitable, and survivable for deployment.

• Eleven deficiency reports (DRs) were opened during Spiral3 testing, with no critical items identified, The two highestpriority DRs involved certificate revocations, Inaccuratereporting of certificate revocations resulted in the RAconcluding all certificates were revoked when that was notthe case. These DRs must be resolved prior to deployment,or written guidelines must be given to users warning themof this potential reporting error-

• There are no outstanding issues with the Spiral 4capabilities-

Increment 2, Spiral 1 (Operational Assessment)- The testing conducted by JITC was adequate to assess

the capabilities and limitations of DoD PKI Increment 2,Spiral l.

• RAs were able to efficiently issue SIPRNet tokens and endusers were able to use those tokens to facilitate missionsthrough digital signing, encryption, and web-serverauthentication.

• Nearly ten percent of all tokens were found to be defective,Some tokens failed prior to or during the issuance process.A sizable fraction failed after issuance, having an adverseimpact on the users. Problems with the software thatformats tokens, inaccurate system documentation, and lackof a back-up system which prevents RAs from performingtheir duties when the system is down for maintenance alsoadversely affected operational suitability.

• Token problems MINI be resolved prior to starting the10T& E.

RecommendationsStatus of Previous Recommendations, The PKI PMOsatisfactorily addressed one of two recommendations fromthe FY08 annual report for Increment I, Spirals 1 and 2. Therecommendation concerning correction of a physical securityvulnerability at Letterkenny Army Depot remains.

• FY10 Recommendations,Increment 1, Spirals 3 and 4I- The PKI PMO should provide data regarding the expected

system load for Increment 1 at full deployment so artadequate capacity assessment can be done by DOT&E tosupport the full deployment decision following IncrementSpiral 5 testing in FYI I.

Increment 2, Spiral I2. The PKI PMO should correct unresolved Increment 2,

Spiral 1 SIPRNet token deficiencies identified during theOperational Assessment and confirm through testing thatthe fixes are operationally viable before purchasing moretokens in support of the Increment 2, Spiral 1 IOT&E.

3. The PM° should provide a written Continuity ofOperations Plan for Increment 2 and ensure the alternateSIPRNet sight is operational: conduct IA testing duringthe IOT&E in accordance with DOT&E guidance to assessprotect, detect, react, and restore capabilities: and develop aLife-Cycle Sustainment Plan-

40 PK1

Suite of Integrated Radio Frequency Countermeasures(SIRFC) AN/ALQ 211

Executive SummaryArmy Speeial Operations Command• After a lengthy engineering investigation the Suite of

Integrated Radio Frequency Countermeasures (SIRFC)Program Office directed a complete redesign of the radiofrequency (RF) switch that was the primary source of poorsystem reliability. The Army was scheduled to receive thenewly designed switch in late 20J0 and should completeadditional qualification testing in early 20

• Because of continued delays in resolving the RF switchreliability problem, DOT&E published the SIRFC IOT&Ereport to Congress in October 20J0. In previous reporting,DOT&E stated that SIRFC was operationally effectivebased on preliminary analysis of operational test results.However, upon completion of a comprehensive analysisof all test data in context with SIR ['Cs ability to supportoperational mission accomplishment- DOT&E determinedthat the initial effectiveness assessment was incorrect- Thefinal assessment (characterized below) and supportinganalysis is included in the October 2010 IOT&E report.

• DOT&E assessed that SIRF(' was not operationallyeffective and remains not operationally suitable- SIRFCdoes not provide sufficient survivability even with currentaircrew tactics to allow penetration in to the weaponengagement zone of many current radar-guided threatsystems. Pending successful qualification and flighttesting of the new RF switch, the SIRFC system should beoperationally suitable.

• SIRFC provides more capability than the legacy RFcountermeasures systems on the MH-d7 and MI 1-60aircraft: nonetheless, it is not effective against its intendedthreat environment.The SIRFC radar warning sub-system, which can operateseparately from the RF countermeasures portion, providesaircrew with excellent situational awareness, rapidlydetecting, identifying, and providing accurate relativebearing to threat radar systems-

Air Force Special Operations Command and Navy• DOT&E released the CV-22 OT&E Report in

January 2010. assessing SIRFC integration on that aircraftas not effective and not suitable.

System• SIRFC is an advanced radio frequency self-protection system

designed for installation on aircraft-• Major SIRFC subsystems are:

- Advanced threat Radar Warning Receivers(Numbers 1, 2, 3. 6, and 9 in picture)

1 LRU (Line Replaceable Unit) Receive Processor

6 - Electrical Assembly2 Receive Antenna

7 - LRU High Power Receive Transmit

3 Amplifier

8 • Transmit Antenna4 LRU Advanced Countermeasure

9 - Cockpit Display

5 Radlo Frequency Switch Assembly

- Advanced threat radar jammer/Electronic Countermeasures(Numbers 4, 5, 7, and 8 in picture)

SIRFC is integrated onto Army Special Operations Command(ASOC) MH-47 and MH-60 helicopters and Air Force SpecialOperations Command (AFSOC) CV-22 tilt rotor aircraft. TheAFSOC CV-22 aircraft is supported by the Navy V-22 JointProgram Office (PMA-275)-

• The SIRFC system integration is 90 percent common betweenthe Service platforms, though the Army MH-47 and MI-60aircraft have a higher power transmitter installed, Earlyintegration challenges on the A FSOC CV-22 aircraft dictatedthe installation or a lower power transmitter. Future CV-22block upgrades are scheduled to incorporate a higher powertransmitter-

MissionSpecial Operations Forces will use SIRFC to enhance thesurvivability of aircraft on missions that penetrate hostile areas-SIRFC-equipped units should be able to provide self-protectionagainst threat radar-guided weapons systems by:• Improving aircrew Situational Awareness and threat warning• Employment of active electronic jamming countermeasures• Expending countermeasures (i.e-, chatf)

Major ContractorITT Electronics Systems — Clifton, New Jersey

SIRFC 41

ActivityArmy Special Operations Command▪ The SIRFC Program Office engineering investigation

discovered deficiencies in the current RF switch design.Asa result, the Technology Applications Program Office(TAP()) directed a complete switch redesign and isplanning additional qualification testing to be completed inearly FY11.

• As an interim solution, the SIRFC Program Office. incoordination with ASOC, reduced the power to the forwardtransmitter via a software change to minimize the chanceof a switch failure- Testing of the reduced power wascompleted at Eglin AFB, Florida, in July 2009. Althoughlimited in scope. the flight tests experienced no RF switchfailures and indicated no change in system effectivenessagainst the very limited number of threats that SIRFC waseffective against during IOT& E.

• DOT&E published the SIRFC IOT&E report to Congressin October 2010 based on the IOT&E and post-IOT&Etesting.

Air Force Special Operations Command and Navy• DOT&E released the CV-22 OT&E Report in January 2010

assessing the SIRFC' integration as not effective and notsuitable.

Assessment• In previous reporting. DOT&E stated that SIRFC was

operationally effective based on preliminary analysis p

operational test results. However, upon completion of acomprehensive analysis or all test data in context with SIRFC'sability to support operational missGon accomplishment.DOT&E determined that the initial effectiveness assessmentwas incorrect. The final assessment (Characterized below) andsupporting analysis is included in the October 2010 IOT&Ereport.

• Despite the common SIRFC hardware among all the platforms,some unique aircraft system integration challenges haveresulted in a disparity in performance with each Serviceaircraft.

• Although the Services conducted SIRFC development andtesting under two separate Test and Evaluation Master Plans,inter-program communication and coordination allowed theCV-22 program to benefit from the ASOC SIRFC program-Army Special Operations Command• SIRFC integration on ASOC helicopters is not operationally

effective and remains not operationally suitable- The

program's newly redesigned RF switch could resolve thesuitability problems. pending successful qualification andflight testing-

• SIRFC provides more capability than existing RFcountermeasures systems on the M11-47 and M1-1-60aircraft. which include two legacy radar warning receivers(APR-39 and APR-44) and two legacy RF countermeasuressystems (A LQ-136 and A LQ-J62).

• Nonetheless. SIRFC' does not provide sufficientsurvivability even with current aircrew tactics to allowpenetration into the weapon engagement zone of manycurrent radar-guided threat systems. I d.1 does- however,reduce the ability of some threat radars to track the aircraftand it reduces (but not eliminates) the ability of someradar-guided threat systems to shoot the aircraft. SIRFChas poor to marginal performance against a number oflikely threats, and does not reduce the number of shotstaken by an air defense system during an entire engagementsufficiently to provide the high survivability a sfow-movinghelicopter, operating covertly without support, requires.

• The SIRFC radar warning sub-system. which can operateseparately from the RF countermeasures portion, providesexcellent situational awareness. rapidly detecting.identifying. and providing accurate relative bearing to threatradar systems.

Air Force Speeial Operations Command and Navy• As part of DOT&E's assessment of the CV-22 OT&E,

SIRFC was assessed to be not operationally effective andnot operationally suitable- Effectiveness performance wassimilar to that on the ASOC helicopters. but the suitabilityissues were un ique to the CV-22 platform.

Recommendations• Status of Previous Recommendations. The Services are

satisfactorily addressing the two FY09 recommendationsto conduct additional SIRFC flight testing on the RF switchredesign and to conduct CV-22 flight testing to verifycorrection of situational awareness problems in IOT&E;however, the recommendations have yet to be completed andtherefore remain valid.

• FY 1(1 Recommendations. None.

42 SIRFC

Teleport

Executive Summary• A November 2009 Multi-Service Operational Test and

Evaluation (MOT&E) demonstrated that the TeleportGeneration 2. Phase 2 (G2P2) system is effective and suitablewith some limitations. DoD Teleport G2P2 added militaryKa-band terminals to access Wideband Global Satellites(WGS), the LinkStar Internet Protocol (IP) modem, andupgraded the software versions of the LinkWay and iDirect IPmodems to use existing Teleport capabilities-

• Follow-on integrated testing in April 201)) demonstrated thatthe program manager corrected a power control problem withthe iDirect IP Version-7 modem observed during the MOT& E.

• The Milestone Decision Authority granted the DefenseInformation Systems Agency (DISA) the authority to enterTeleport Generation 3, Phase I (G3P 1 ) into the Production andDeployment Phase. Teleport (BPI will integrate AdvancedExtremely High Frequency (A EHF) Extended Data Rate(X DR) capability into the existing Teleport system ExtremelyHigh Frequency (EHF) architecture.

System• DoD Teleport sites are globally distributed satellite

Communications (SATCOM) facilities- The system has sixcore Teleport facilities located in Virginia, Germany, Italy,Japan, Hawaii, and California. Teleport sites consist of foursegments:

The radio frequency segment consists of SATCOM earthterminals that operate in X-, C-. Ku-, Ka-. Ultra HighFrequency (UHF), and EHF bands- The terminals provideradio frequency links between the Teleport site and thedeployed user SATCOM terminal via commercial ormilitary satellites.The Teleport base-band segment includes encryption,switching, multiplexing, and routing functions forconnecting data streams or packetized data to the DefenseInformation Systems Network (DISN)-

- The network services segment provides connectivity to theDISN long-haul networks and data conversion functionsnecessary to meet the user's requirements-

- The management control segment provides centralizedmonitoring and control of Teleport base-band hardware,

SATCOM earth terminal hardware, transmission security,and test equipment.

Mission• Combatant Commanders, Services, and deployed operational

forces use Teleport systems in all phases or conflict to gainworJdwide military and commercial SATCOM services-

• "Teleport provides deployed forces access to standard fixedgateways from anywhere in the world for all six DISNservices:- Secret Internet Protocol Router Network (SIPRNet)- Non-secure Internet Protocol Router Network (NIPRNet)

Defense Red Switch Network (DRSN)Defense Switched Network (DSN)Video Teleconferences (VTC)Joint Worldwide Intelligence Communications System(JWICS)

Major ContractorGovernment Integrator DISA

Activity• The Joint Interoperability Test Command (JITC) conducted

the G2P2 MOT&E November 2009 at the Hawaii DoDTeleport site and the Pacific Theater Network OperationsCenter (TNC) at Wheeler Army Air Field, Hawaii- Elevenoperational units along with a special LinkStar unit operatedfrom deployed locations in California, Oklahoma, Arizona,Idaho, Hawaii, and Korea.

• The MOT&E focused on Teleport's capability to providedeployed users Ka-band access, DISN services using threedifferent current force IP modem variants, and the systemcontrol and management capability.

• JITC and the program office conducted a follow-on test atthe Wahiawa Teleport in April 2010 to verify correction of a

Teleport 43

power control problem associated with the iDirect Version-7 IPmodems.

• Teleport G3P1. which consists of adding Navy Multi-bandTerminals (NMT) to communicate over AFHF satellites, hasachieved Milestone C-

Assessment• The MOT&E was adequate to evaluate the operational

effectiveness and suitability of the Teleport G2P2 system.The Teleport G2P2 system is effective and suitable withlimitations. The newly installed Ka-band access providesconnectivity over the WGS system. The newly installedLinkStar IP modem and upgraded Linkway and iDirectVersion-7 IP modems provide connectivity over commercialKu-band and military X- and Ka-band satellites.

• The two major deficiencies in effectiveness observed duringMOT&E were an iDirect Version-7 IP modem power controlproblem that prevented consistent IP access over WGS and theinability of users to complete DSN secure calls on a consistentbasis.

• The major deficiencies in suitability were lack ofdocumentation and training preparing Teleport Operators

to troubleshoot system-level problems, and lack of detailedConcepts of Operations that adequately describe the roles andresponsibilities and operational configurations of IP-basedSATCOM networks.

• Follow-on testing in April 2010 confirmed that the programmanager corrected the power control problem observed duringMOT&E with the iDirect Version-7 IP access over WGS.

Recommendations• Status of Previous Recommendations- DISA has satisfactorily

addressed all previous recommendations.• FY10 Recommendations. DISA should:

I. Develop a 'technical or procedural solution for DSN securecalk approved by the user community and JITC shouldvalidate the corrective action,

2. Improve system documentation and maintenance training,particularly with respect to system-level troubleshootingprocedures-Develop documentation that adequately describes theroles and responsibilities and operational configurations of1P-based SATCOM networks.

44 Teleport