a quantitative acquisition process …csse.usc.edu/new/wp-content/uploads/2017/01/a-quantitative...a...
Post on 24-Apr-2018
214 Views
Preview:
TRANSCRIPT
AQUANTITATIVEACQUISITIONPROCESSMODELINGAPPROACHTOWARDEXPEDITING
SYSTEMSENGINEERING
YvetteRodriguez06April2017
USCCenterforSystemsandSoftwareEngineering2017AnnualResearchReview
ResearchMotivation:BetterBuyingPower(BBP)3.0
• BBP3.0InitiativetoEliminateUnproductiveProcessesandBureaucracybymeansofstreamliningprocessesandreducingcycletime.
• BBP3.0Initiative:EliminateUnproductiveProcessesandBureaucracy
• Unnecessaryandlow-valueaddedprocessesanddocumentrequirementsareasignificantdragonacquisitionproductivityandmustbeaggressivelyidentifiedandeliminated.
“BetterBuyingPower(BBP)istheimplementationofbestpracticestostrengthentheDefenseDepartment'sbuyingpower,improveindustryproductivity,andprovideanaffordable,value-
addedmilitarycapabilitytotheWarfighter.”
ResearchFocus:Pre-MilestoneBData
• DepartmentofDefense(DoD)decision-makingduringearly(pre-MilestoneB)systemsengineeringprocesseshavelastingimpacts,bothpositiveandnegative,throughoutthelifecycle.
• SourcesofdataonDODsystemsengineeringtimelinesanddecisionprocessesprovidedquantitativeimprovementinsights.
DoD ReviewCycle
Letter
Figure 5: DOD Levels Reviewing Information Requirements
Page 15 GAO-15-192 Acquisition Reform
Many different functional organizations within each level review the information before the document is approved. The number of organizations conducting reviews varies depending on the information included in each document. A few documents that include a wide breadth of information can be reviewed by many offices at each level. For example, Air Force acquisition strategies, that on average took over 12 months to complete for the programs we surveyed, can be reviewed by
USAirForceReviewProcessExample
• 30LevelsofOfficeoftheSecretaryofDefense(OSD)Reviews
• 22ServiceLevelsofAcquisitionExecutive(SAE)Reviews
• 7LevelsofProgramExecutiveOffice(PEO)Reviews
ProgramExecutiveOffice(PEO)LevelReviews
1. ProgramExecutiveOffice2. FinanceFunctionalStaff3. DeputyProgramExecutiveOfficer4. EngineeringFunctionalStaff5. ContractingFunctionalStaff6. ProgramExecutiveOfficerExecutionGroup7. LogisticsFunctionalStaff
ServiceAcquisitionExecutive(SAE)LevelReviews
1. AssistantSecretaryoftheAirForceforAcquisition(ServiceAcquisitionExecutive
2. AssistantSecretaryoftheAirForceInstallations&Environment
3. AirForceLogistics,Installations,&MissionSupport
4. AirForceOperations,Plans,&Requirements
5. AirForceIntelligence,Surveillance,&Reconnaissance
6. AirForceFinancialManagement&Comptroller
7. AirForceTest&Evaluation8. AssistantSecretaryoftheAirForceSmall
BusinessPrograms9. AssistantSecretaryoftheAirForceChief
InformationOfficer10. AssistantSecretaryoftheAirForceTest&
Evaluation(PolicyandPrograms)11. AirForceOperations,Plans&
Requirements(OperationalCapabilityRequirements)
12. AirForceLogistics,Installations&MissionSupport(Logistics)
13. AssistantSecretaryoftheAirForceInstallations&Environment(Logistics)
14. AirForceIntelligence,Surveillance,&Reconnaissance(Strategy,Plans,Doctrine&ForceDevelopment)
15. AssistantSecretaryoftheAirForceChiefInformationOfficer(Policy&Resources)
16. AssistantSecretaryoftheAirForceDeputyGeneralCounselforAcquisition
17. AirForceFinancialManagementandComptrollerDeputyAssistantSecretary(CostandEconomics)
18. AirForceFinancialManagementandComptrollerDeputyAssistantSecretary(Budget)
19. AssistantSecretaryoftheAirForceDirectorateofScience,Technology&Engineering
20. AssistantSecretaryoftheAirForceDirectorateManagementPolicy&ProgramIntegration
21. AssistantSecretaryoftheAirForceDirectorateofContracting
22. AirForceAcquisitionCapabilityDirectorate
OfficeoftheSecretaryofDefense(OSD)LevelReviews
16. Director,SystemsEngineering17. Director,CostAssessmentandProgram
Evaluation18. Director,DevelopmentalTest&Evaluation19. Director,AcquisitionResources&Analysis20. DeputyAssistantSecretaryofDefense,
Manufacturing&IndustrialBasePolicy21. PrincipalDeputyUnderSecretaryofDefense
(Acquisition,Technology,&Logistics)22. Director,InternationalCooperation23. AssistantSecretaryofDefense(Acquisition)24. Director,PerformanceAssessmentandRoot
CauseAnalysis25. AssistantSecretaryofDefense(Logistics&
MaterialReadiness)26. AssistantSecretaryofDefense(Legislative
Affairs)27. DeputyUnderSecretaryofDefense(Installations
andEnvironment)28. Director,DefenseProcurementandAcquisition
Policy29. DeputyGeneralCounsel(Acquisition&Logistics)30. AssistantSecretaryofDefense(Operational
EnergyPlansandPrograms)
1. DefenseAcquisitionExecutive2. AssistantSecretaryofDefense(Research&
Engineering)3. ViceChairmanoftheJointChiefsofStaff4. DeputyAssistantSecretaryofDefense,
Strategic&TacticalSystems5. UnderSecretaryofDefense(Policy)6. DeputyAssistantSecretaryofDefense,
Space&Intelligence7. UnderSecretaryofDefense(Comptroller)8. DeputyAssistantSecretaryofDefense,
Communication,Command,andControlCyber
9. UnderSecretaryofDefense(Personnel&Readiness)
10. Director,NationalGeospatial-IntelligenceAgency
11. UnderSecretaryofDefense(Intelligence)12. DeputyDirector,CostAssessment13. ChiefInformationOfficer14. Director,DefensePricing15. Director,OperationalTest&Evaluation
Hypothesis&ResearchQuestions• Hypothesis:Thereexistsabaselinesetofcriticalsuccess
factordatavariablesthatidentifyearlyDODAcquisitionprogramslikelytoexperiencedelays.
• ResearchQuestions:– RQ1:Whatearlyacquisitionpredictivedatavariablesactas
criticalsuccessfactorsindistinguishingpreviousExpeditedversusDelayedearly-SEacquisitionprocesses?
– RQ2:Howcanthepeople,product,andprocessorganizationalpracticesidentifiedintheESEFframeworkbetterpromoteexpeditedsystemsengineeringthroughouttheearly-SEDoDacquisitionprocess?
– RQ3:WhatvalueisaddedtotheExpeditedSystemsEngineeringFrameworkthroughtheanalysisofquantitativereporteddata?
BoxPlotOutliers
EstimatedDelay
PhaseA(ESISTOASP)#ofDays
PhaseB(ASPtoASD)#ofDays
PhaseC(ASDtoRFP)#ofDays
PhaseD(RFPtoCA)#ofDays
TotalDays(StarttoCA)
0
200
400
600
800
1000
1200
1400
1600
Boxplots
Mean Minimum/Maximum Outliers(1)
0
5
10
15
20
25
DollarV
alue
Level
Boxplot(DollarValueLevel)
ScatterPlotOutliers
EstimatedDelay
PhaseA(ESISTOASP)#ofDays
PhaseB(ASPtoASD)#ofDays
PhaseC(ASDtoRFP)#ofDays
PhaseD(RFPtoCA)#ofDays
TotalDays(StarttoCA)
0
200
400
600
800
1000
1200
1400
1600
Scattergrams
Mean Median
0
5
10
15
20
25
DollarV
alue
Level
Scattergram(DollarValueLevel)
RegressionResults
ΣDContract Pr ocess = 55.4+ 0.8*DEstimatedDelay−8.1DVContract + 0.6DPhaseA+1.4DPhaseB+ 0.4DPhaseC +1.2DPhaseD
D = Number of DaysDV = Dollar Value Level
-500
0
500
1000
1500
2000
-500 0 500 1000 1500 2000
Days(C
ontractP
rocess)
Pred(Days(ContractProcess) )
Pred(Days(ContractProcess))/Days(ContractProcess)
Given the R2, 91% of the variability of the dependent variable DContractProcess is explained by the 6 explanatory variables.
Conclusions• QuantitativeResults:Theresultssupportthehypothesisidentifying
thatthereexistsabaselinesetofcriticalsuccessfactorsdatavariabletoprovideevidence-baseddecision-makinginexpeditingsystemsengineering.
• StudyResults:TheresultsaddressesfindingsspecifictotheESEFprocessbyidentifyingprocessphasetrends.
• DoD AcquisitionDataCollectionPractices:Anextensivesearchwasconductedtofindanappropriatesetofdatatoconductasignificantquantitativestudyacrossmultipleprogramsanddatalimitationscontinuetomakequantitativeanalysisparticularlychallenging.
• EarlySystemsEngineeringPractices:Thepracticesexploredspecificallyfocusedonthecontractawardprocesswithobservationsbasedonexpertopinion.
LevelsofKnowledgeDistribution
Letter
Figure 5: DOD Levels Reviewing Information Requirements
Page 15 GAO-15-192 Acquisition Reform
Many different functional organizations within each level review the information before the document is approved. The number of organizations conducting reviews varies depending on the information included in each document. A few documents that include a wide breadth of information can be reviewed by many offices at each level. For example, Air Force acquisition strategies, that on average took over 12 months to complete for the programs we surveyed, can be reviewed by
FinalRecommendations• Anearlierunderstandingofthewidersystemicviewofthe
missionobjectivecanprovideawiderandmoreeffectiverangeoftruealternatives(trade-space)inearlysystemsengineeringprocesses.
• Proposedsolutionsapprovedatlowerlevelscanhavebelatedrejectionsanddelayslaterintheprocess,thereforeearlycommunicationbetweenhigherauthoritativelevelsandtheprogramofficeisrecommended.
• Identificationofthespecificdangersinexpeditingpredecessorphasesandconcurrentlyaccomplishingtasksinmultiplephasestowillprovideanimprovedunderstandingofrisksandopportunities.
top related