a guide to effective benchmarking of applications development
DESCRIPTION
TRANSCRIPT
04/10/2023
Webinar Sponsored by Computer Aid, Inc.
Slide: 1
A Guide to Effective Benchmarking of Applications Development
October 30, 2012 4:00 PM - 5:30 PM
04/10/2023
Webinar Sponsored by Computer Aid, Inc.
Slide: 2
Pam MorrisCEOTotal [email protected]
Hosted by:
Michael MilutisDirector of MarketingComputer Aid, Inc. (CAI)[email protected]
04/10/2023
Webinar Sponsored by Computer Aid, Inc.
Slide: 3
About Total Metrics
• “Supporting organizations in their quest to optimize the cost effectiveness and efficiency of their software delivery”
• Internationally based - Certified IFPUG and COSMIC Function Point and Metrics Consultants and Trainers – Europe, UK, USA, Australia
• Provide metrics related tools, procedures, consulting and training - Thought leader in functional size measurement and developers of :
• SCOPE Project Sizing Software™ – Function point Counting, Portfolio Metrics
Management, Industry Benchmarking Analytics.
• FP Outline™ Wizard for Estimating Application and Project Functional Size
• For organizations serious about measuring Functional Size
• Multi-lingual Korean, Chinese, Japanese, Portuguese, Spanish , German, Dutch, Italian, French, English etc.
• NEW Release FREEdownloads see WWW.totalmetrics.com
04/10/2023
Webinar Sponsored by Computer Aid, Inc.
Slide: 4
CAI Achieves IT Operational Excellence
04/10/2023
Webinar Sponsored by Computer Aid, Inc.
Slide: 5
PDU Credits Available for this Webinar
• The PMI has accredited this webinar with PDUs
• You will be eligible to receive 1.0 PDU credits
• Your PDU email will be sent to you within 24 hours
04/10/2023
Webinar Sponsored by Computer Aid, Inc.
Slide: 6
Online Webinar Recordings
NOW AVAILABLE
• Anytime Access• Hundreds of Topics
Visit: www.ITMPI.org/library
04/10/2023
Webinar Sponsored by Computer Aid, Inc.
Slide: 7
Enjoy the benefits of ITMPI Membership
JOIN TODAY!
• UNLIMITED Free Webinar Recordings• UNLIMITED Free PDU Credits• Hundreds of Topics
Visit: www.ITMPI.org/subscribe
04/10/2023
Webinar Sponsored by Computer Aid, Inc.
Slide: 8
Overview
• What is a Benchmark?
• Why Benchmark?
• What are the Risks?
• Ensuring Success - Terms of Reference
04/10/2023
Webinar Sponsored by Computer Aid, Inc.
Slide: 9
Benchmarking Definitions
“Benchmarking is the process of continuously comparing and measuring an organisation with business leaders anywhere in the world to gain information which will help the organisation take action to improve its performance.”
American Productivity and Quality Centre Benchmarking Management Guide
Cambridge, Mass., 1993, Productivity Press
“Activity of comparing objects of interest to each other or against a benchmark to evaluate characteristic(s)”
ISO /IEC FCD 29155-1, Software and Systems Engineering – IT Project Performance Benchmarking Framework – Part 1: Concepts & Definitions
“Benchmarking is the process of determining a metric baseline for an organisational or functional unit for the purpose of comparison”
Total Metrics – Glossary of Terms
04/10/2023
Webinar Sponsored by Computer Aid, Inc.
Slide: 10
Types of Benchmarks
• Internal Benchmark
– Benchmark against a group of commonly usable and authorised benchmarks based upon data sourced from within the target organisation.
– Procedure of comparing and measuring one’s own performance within an organisation
• External Benchmark
– Benchmark against a group of commonly usable benchmarks based upon data sourced from external organisations and issued by external repository providers.
– Procedure of comparing and measuring an organisation with other organisations
04/10/2023
Webinar Sponsored by Computer Aid, Inc.
Slide: 11
Why Benchmark?
Answer: To be able to make informed decisions!
– Are we more or less effective than our competitors? (Should we offshore?)
– Are we more or less effective than industry? (Should we upgrade our skills? Technology? Or should we outsource?)
– Is our IT consuming more money than it should ? (Should we change suppliers?)
– Are we improving at the rate required by our contract? (Do we need to prepare for likely penalties?)
– Are we achieving ROI on investments in tools and technology? (Do we need to review our IT investment strategy?)
– Are all organization units performing at the same level? (Which units do we keep or cut?)
04/10/2023
Webinar Sponsored by Computer Aid, Inc.
Slide: 12
What are the Risks?
Answer: Incorrect strategic decisions based on invalid numbers!
» Critical to get the Benchmarking Right!
04/10/2023
Webinar Sponsored by Computer Aid, Inc.
Slide: 13
How do we to ensure Valid Benchmark Results?
Answer: Establish rigorous ‘Terms of Reference’ and get stakeholder agreement on the Benchmark’s:
• Strategic Intent• Type• Performance Metrics• Standards for Measures• Scope• Frequency• Peers for Comparison• Report Requirements• Dispute Resolution Process
04/10/2023
Webinar Sponsored by Computer Aid, Inc.
Slide: 14
Strategic Intent
Common Understanding: Why are we Benchmarking?
• Strategy is determined by:
– What results are required
– When - within what time frame
– How results will be used
04/10/2023
Webinar Sponsored by Computer Aid, Inc.
Slide: 15
Common purposes for Benchmarks - External
• Client initiated Benchmarks
– Establish current IT efficiency and effectiveness compared to Industry norms – should we look at outsourcing?
– Evaluate supplier performance against Industry standards – should we change suppliers?
– Set competitive range for performance baseline and targets – how can we monitor that we are getting value for money from our suppliers?
• Supplier initiated Benchmarks
– Establish competitive position for productivity, pricing and service levels – how can we ensure that we are in a competitive bidding position?
04/10/2023
Webinar Sponsored by Computer Aid, Inc.
Slide: 16
Common purposes for Benchmarks - Internal
• In-house initiated Benchmarks
– Which operational units are the most efficient? - Should we review where work is allocated ?
– Which operational have “Best Practice” processes? - How can we implement practices in all units?
– Did we achieve the planned benefits of implementing new tools, techniques or technologies? - Can we justify the expenditure?
– How consistent is our development productivity rate? - Can we be confident in using it to estimate planned projects?
– Is our M&S support productivity improving? - Are we doing more with less staff than Industry norms?
04/10/2023
Webinar Sponsored by Computer Aid, Inc.
Slide: 17
Example : M&S Support efficiency improvement
04/10/2023
Webinar Sponsored by Computer Aid, Inc.
Slide: 18
Comparisons – Internal? External? Both?
– Internal • To demonstrate improvement trends over time for the
organisations internal processes, or – External
• To compare internal results with external independent organisational units, or Industry as a whole. Eg. ISBSG
04/10/2023
Webinar Sponsored by Computer Aid, Inc.
Slide: 19
What will we measure?
• Cost-effectiveness of the process – are we getting ‘value’ for money invested?
• Efficiency of the process – how ‘productively’ is our software being developed or maintained?
• Speed of Delivery – how ‘quickly’ can we deliver software product or ‘solve a problem’?
• Quality of the product – how ‘good’ is the software product or service we deliver?
• Quality of the process – how much time and money was wasted in ‘rework’?
• Customer Satisfaction – how well does the delivery of our software products and related services meet and or exceed their customer’s expectations?
04/10/2023
Webinar Sponsored by Computer Aid, Inc.
Slide: 20
Comparable Measures?
• Establish Rules for Comparison:
• Units of measurement – Day? Hrs?
• Measurement scope – which processes , whose effort, which FPs?
• Data accuracy
• Validation requirements
04/10/2023
Webinar Sponsored by Computer Aid, Inc.
Slide: 21
Industry Data Collection Standards
• Glossary of Terms
• Data Collection Standards
• Development and Enhancement (D&E)
• Maintenance and Support (M&S)
• ISO/IEC 29155. Systems and software engineering - Information technology project performance benchmarking framework
04/10/2023
Webinar Sponsored by Computer Aid, Inc.
Slide: 22
Industry Benchmarking Standards
IT project performance benchmarking framework overview
International standard (ISO/IEC 29155-1. Systems and software engineering -- Information technology project performance benchmarking framework -- Part 1: Concepts and definitions. )
ISO/IEC 29155-1 :2011
04/10/2023
Webinar Sponsored by Computer Aid, Inc.
Slide: 23
Scope - What is ‘in’ and what is ‘out’
• Comparing “apples to apples”
• Decide on the inclusion and exclusion criteria for projects and applications into the Benchmarking Set
• Delivery Options
• Types of Requirements
• Resourcing Profiles
• Technology Profiles
• Size Profiles
04/10/2023
Webinar Sponsored by Computer Aid, Inc.
Slide: 24
Scope - What is ‘in’ and what is ‘out’
• Aim for homogeneous data sets for comparison
• User Base
• Functional Domains
• Project Classifications
04/10/2023
Webinar Sponsored by Computer Aid, Inc.
Slide: 25
How often should we compare?
Collect data and compare at a rate that is useful and meaningful!
• Demonstrating Trends
• Allocating Projects to Benchmarking Periods
• Usefulness of the Result
• Statistical Validity of the Result
04/10/2023
Webinar Sponsored by Computer Aid, Inc.
Slide: 26
Comparable External data
• Organisational Type
• Different User Priorities
• Quality of the External Dataset
• Filtering of Submission Data in an External Dataset
04/10/2023
Webinar Sponsored by Computer Aid, Inc.
Slide: 27
Benchmarking Report
• Structure and content
• Level of granularity
• Presentation technology
• Confidentiality and Audience
• Review Process and Acceptance criteria
• Feedback process
04/10/2023
Webinar Sponsored by Computer Aid, Inc.
Slide: 28
Dispute Resolution Process
• Who is responsible for:– resolution costs– liability payments– consequential loss
• Who will arbitrate?
04/10/2023
Webinar Sponsored by Computer Aid, Inc.
Slide: 29
Summary
• Benchmarking results provide important input into tactical and strategic decisions
• Benchmarking can be done relatively cost effectively using Industry data
• BUT ….Careful upfront planning is the most important step!
– Get the planning right and
– The rest is easy (…..well easier!!)
04/10/2023
Webinar Sponsored by Computer Aid, Inc.
Slide: 30
Other Considerations
• Submitting to the ISBSG Data Repository
– Maintenance and Support
– Development and Enhancement
• Benchmarking Internally
• Benchmarking against ISBSG
04/10/2023
Webinar Sponsored by Computer Aid, Inc.
Slide: 31
Thank You!
Total MetricsLevel 1, 191 Riversdale RoadHawthornVictoria 3122 Australia
Phone +61 (3) 9882 7611
[email protected] TO GET THE FULL PAPER ON WHICH THIS PRESENTATION WAS BASED.
04/10/2023
Webinar Sponsored by Computer Aid, Inc.
Slide: 32
Questions?
04/10/2023
Webinar Sponsored by Computer Aid, Inc.
Slide: 33
CAI Sponsors Proudly Sponsors
The IT Metrics & Productivity Institute• IT and Software Knowledge Center: WWW.ITMPI.ORG
• Weekly Webinars Hosted by Industry Leaders: WWW.ITMPI.ORG / WEBINARS
• Access Webinar Recordings Anytime at WWW.ITMPI.ORG / LIBRARY
• Enjoy the Benefits of ITMPI Membership at WWW.ITMPI.ORG / SUBSCRIBE
• Follow Us on TWITTER at WWW.TWITTER.COM / ITMPI
• Follow Us on FACEBOOK at WWW.ITMPI.ORG/ FACEBOOK
• Join Our Network on LINKED IN at WWW.ITMPI.ORG/ LINKEDIN
• Find Out About Our CONFERENCES at WWW.ITMPI.ORG/ EVENTS
04/10/2023
Webinar Sponsored by Computer Aid, Inc.
Slide: 34
Subscribe to the AITS Newsletter
• Daily articles in your mailbox on improving the management of IT. Subscribe at www.AITS.org
• Listen to our “Thought Radio” podcasts on iTunes or at www.AITS.org
04/10/2023
Webinar Sponsored by Computer Aid, Inc.
Slide: 35
Pam MorrisCEOTotal [email protected]
Hosted by:
Michael MilutisDirector of MarketingComputer Aid, Inc. (CAI)[email protected]