DOING MORE WITH LESS:

IMPROVED CONFIDENCE LEVELS VIA REDESIGNED

AMMUNITION QUALIFICATION TESTING

A method to improve small arms ammunition qualification for use in existing ADF small arms weaponry using statistical test techniques applied within the US Defense Department

Mr Roy Henry / Dr Keith Joiner CSC

SCOPE

• Why This• Background• Current Test - DMSP• Problem• History – 6000 rounds• Proposal • Old vs New – Round / Test Design• Summary• Recommendations

WHY THIS• ADF desire to IIS new ammunition• I wanted to understand the test regime• Tests appeared unnecessarily repetitive • Opportunity to study it – and apply sound

principles to redesign the test• Dr Keith Joiner’s Advanced Test and

Evaluation Course

BACKGROUND• Policy: ADF DMSP-1QMS (ENG) 12-8-031 Safety and suitability for Australian

service (S3) assessment for small arms weapon systems

• Predominately based off two standards:• NATO STANAG 4608 Ammunition below 12.7mm Cal Design Safety Requirements

and Safety and Suitably for Service• NATO AC / 225 D/14 Evaluation Procedures for Future NATO Small Arms Weapons

Systems, 6 Jul 01

• Endstate• “Obtain Sufficient OQE to support Technical Certification of system (weapon and

ammunition)”• The system is therefore safe to conduct manned firing, user trials or introduce new

ammunition or weapons into service.

CURRENT TEST: DMSP

• Outlines the four different tests:• Manned firing clearance• User Trial• Provisional Design Authority / TC with Conc)• Full Design Authority / Tech Cert)

• Ambiguous and contradictory in parts

• 6000 round “endurance” requirement

*Taken from DSMP, Table 2

CURRENT TEST: ROUND COUNT• One Ammunition Type

• Minimum 18 260 rounds required (3 weapons)

• With Suppressor – newSteyr

• 36000 rounds

• 1200 magazines• ~5h to load (2 rounds /

second)

• 12000 rounds perweapon – incredible useage20 240

1320

6000

3960

18000

0

2000

4000

6000

8000

10000

12000

14000

16000

18000

20000

Technical Manned Firing User Trial Manned Firing Advice(One Temperature)

S3 for Provisional DesignAcceptance

S3 for full Design Acceptance

NUM

BER

OF

ROUN

DS

Amount of rounds required IAWDMSP for each stage

One weapon Three Weapons

PROBLEM

• Test Parameters:• External Ballistics• Endurance• Temperature assessment• Function and Casualty

• 6000 rounds fired over four discrete temperature ranges

• -46 OC• -6 OC• +21 OC• +71 OC

• Test Outputs:• Muzzle Velocity• Accuracy (Group)• MPI shift• Ejection Pattern• Weapon / Ammunition failures

1050

1050

1050

1800

1050

0

1000

2000

3000

4000

5000

6000

7000Breakdown of the 6000 Rounds

Temp .-46 oC Temp .-6 oC Temp 21 oC Temp 21 oC Temp 71 oC

HISTORY - 6000 ROUNDSQuirk of history which has been applied to ammunition qualification testing

Production Testing, Lot acceptance, Serviceability RegimesTests are Weapon Focused

1987 M4A1 Mil SpecMIL-C-70599

1995 Army(AUST) 6443 Australian Steyr F88

2007 US TOP 3-2-045Small Arms – Hand and Shoulder Weapons

2001 NATO D/14

Production Lot Acceptance5 in every 500 weapons

Production Lot Acceptance1 in every 7500QA program

“determine the functioning and endurance performance of the

weapon behaviour and serviceability of all parts”

“determine the functional life of the weapon and its component parts. …data helps establish

logistical requirements for parts stockage and replacement schedules…”

ANALOGY

Fuel is the ‘ammunition’ for the car (Weapon)

Car needs to drive long distance (6000km) without falling apart

Quality of car being tested, not fuel

You don’t drive long distance as a test each time you change fuel providers…

PROPOSAL : AMMUNITION S3 REIMAGINED

DOE inspired model of the ammunition qualification system• Assumptions:

• OEM has qualified round • Round has undergone

certification/compliance activities

• In production / use with other users

Y4 – Ejection pattern reduced in scope as deemed qualitative measure

PROPOSAL: IMPROVING STATISTICAL CONFIDENCE

• Used Projectile Velocity as the basis• Velocity is directly related to group size, and therefore MPI shift. Slower velocities mean the

projectile drops quicker (and vice versa)• Standard Deviation of 6m/s was used as representative of mass produced factory

ammunition• One sample equates to five rounds

PROPOSAL: DOE INSPIRED RESULT

OLD VS NEWROUND COUNT COMPARISON

• 92% reduction in rounds / testing

• Reduced• Labour• Laboratory time• Funding• Waste

• Savings can be reinvested in other areas20 240

3960

18000

20 220 1140 13800

2000

4000

6000

8000

10000

12000

14000

16000

18000

20000

Technical Manned Firing User Trial Manned FiringAdvice (OneTemperature)

S3 for Provisional DesignAcceptance

S3 for full DesignAcceptance

Current vs Proposed Ammunition Usage

Current DMSP Proposed DOE Model

OLD VERSES NEWTEST COMPARISON

• Green = similar• Orange = modified• Red = removed

• More streamlined

• 18260 vs 1400 Rounds

SUMMARY• Proposed Test regime demonstrates that test data quality is not

compromised and a more informed approach can be taken for full technical certification

• The research found the 6000-round per weapon requirement probably derived from weapon qualification standards rather than ammunition qualification standards.

• The research has proposes a first-principled and statistically rigorous rationale for such S3 testing based on what the S3 output responses should be and the use of U.S. DoD design-of-experiment techniques.

• The historical context combined with the analysis attempts to show that endurance testing should be by exception only.

RECOMMENDATIONS• ADF has an opportunity to revisit the current small arms ammunition S3 testing

procedure (DMSP, 2015) to:

• remove the identified ambiguities and inconsistencies• improve its applicability,• remove the default endurance requirement• specify statistically guided testing within required confidence limits to provide a more

informed outcome so as to achieve a significant reduction in test time and resources• seek US ATEC DOE discussion to confirm our approach

DOING MORE WITH LESS:

IMPROVED CONFIDENCE LEVELS VIA REDESIGNED

AMMUNITION QUALIFICATION TESTINGMr Roy Henry / Dr Keith Joiner CSC

BackgroundCurrent Test - DMSPProblemHistory – 6000 rounds

Proposal Old vs New Test

SummaryRecommendations