distributed multi-source development with bill … · distributed multi-source development with...
TRANSCRIPT
Distributed Multi-Source Development with
Open Source:
How New Tools, Processes and Free Code are Redefining Software Development
LinuxCon 2010
Bill McQuaideEVP Products & Strategy
Black Duck Software, Inc.
Copyright © 2010 Black Duck Software, Inc. All Rights Reserved. 2
Agenda
Introduction
Market Trends
Development Challenges
Agility and Innovation via Multi-Source Development
Best Practices
Customer Use Cases
Copyright © 2010 Black Duck Software, Inc. All Rights Reserved. 3
Market Dynamics – “Multi-Source”Software development has changed
Collaborative developmentComponentization & Search for re-useAgile methods
OSS is compellingGartner: 85% of enterprises use OSS, 45% use is mission-criticalAccenture: Top benefits are quality, reliability, bug fixing, cost
Distributed, “Multi-Source” development using Agile methods represent new pragmatism
Market Need – “Managing Abundance”< 40% of customers have any OSS PoliciesNeed: address challenges of Multi-Source development:
- Compliance/Management – IP, security, export- Management/Automation – policy, process, multi-source
451 Group Survey on OSS Use (December 2009)
• 87% of companies say OSS meets or exceeds cost savings expectations
• 39% of OSS users ranked Flexibility as the primary benefit
Copyright © 2010 Black Duck Software, Inc. All Rights Reserved. 4
Source: Accenture, August 2010
Open Source is Changing the Way Business Operates its IT
Copyright © 2010 Black Duck Software, Inc. All Rights Reserved. 5
Abundance of Open Source
SugarCRM
MyS
QL
zlibPentaho
BIRT
Xerceslog4j
Asterix
ACEGI
Hibernate
OpenSIP
Alfresco
OpenSSL
SpringOpenNMS
HipHop
Var
nish
Android
SphinxFileZilla
Nagios
Subversion
JBOSS
Tomcat
OpenVista
Mumble
OpenHeX
FreeNA
S
BaculaWordpress
Ganglia
Virtual Dub
Jython
TweetC
raft
OpenEMR
FreeMedPatientOS
Hudson
Ant
Bioclipse
ANTLR
Free
BSD
Webkit
Archiva
Copyright © 2010 Black Duck Software, Inc. All Rights Reserved. 6
Today’s Development Org & Process
Cambridge, MA Los Gatos, CA
Bristol, UK Bangalore, India
Outsourced Code Development
OSS Community
Commercial ISV CodeOffshore
Code Development
Distributed Agile “Multi-Source”
Copyright © 2010 Black Duck Software, Inc. All Rights Reserved. 7
Challenges: Multi-Source Development at-Scale
Management & Control
Find & leverage the right software from many internal, external sources
Get a handle on code base after years of ad hoc ‘ism
Encourage standardization of components & versions
Compliance & Security
Comply with company’s or organization’s policy
Manage licensing and associated obligations
Complying with regulations
Formal control of open source software lags adoption:
~60% of companies surveyed do not have formal polices or guidelines for OSS
Far fewer using tools for automation / management
Source: 451 Group, December 2009
Copyright © 2010 Black Duck Software, Inc. All Rights Reserved. 8
…and Open Source is Playing a Big Role
Please estimate what percent of your code is?
.00
10.00
20.00
30.00
40.00
50.00
60.00
Open source Internally written 3rd party proprietary Other (pleaseexplain)
From recent study of commercial software projects:– Sampled hundreds of commercial projects
Millions of files, representing hundreds of GB of code
– 22% of typical application/project is open sourceAvg project size: ~ 700MB of codeDozens to hundreds of OSS components
From development projects in progress:
OSS is a significant portion of code in DevelopmentSource: Survey of Users from WWW.Koders.com(January-March 8, 2010)
Copyright © 2010 Black Duck Software, Inc. All Rights Reserved. 9
What We’re Hearing from Customers
Concerns about OSS use evaporating– Budget realities trump lingering concerns about OSS security,
quality
CIO’s not “wasting a good crisis”…– Agile: broader OSS use easing evolution to Agile methods
Reduction in design cycle time of 10% to 75%– Goals for Re-Use/Standardization of code of up to 80%
Now recognizing new challenges – managing & controlling use of OSS at-scale in multi-source environments– Need for defined OSS Policy– Making good choices – Search, Selection, Validation– Managing complexity & “abundance”
ad hoc use of hundreds of OSS components has led to governance, tracking, support challenges
– “Provenance” & “software supply chain” visibilityWhat’s in the code being received
Copyright © 2010 Black Duck Software, Inc. All Rights Reserved. 10
Benefits of Multi-Source Development when Best Practices are Followed
Innovate more, code less
Accelerate Time to Market
Open source software to avoid reinventing
the wheel,
Faster delivery of functionality
Increase Innovation & Product Capability
Readily available code to fill out
feature list
Focus resources on features, innovation
Control Development Costs
Re-use to lower development and
licensing costs
Improve development and group
productivity
Copyright © 2010 Black Duck Software, Inc. All Rights Reserved. 11
Pro-Active, Managed Use of Open Source
Cost of defects– Minimal when issues are
detected early in lifecycle– Grows 100-1,000X late in
the lifecycle or after release
For OSS, invest in process and automation:– Make better OSS code
choices up front (via search/selection)
– Validate the in-bound code before pushing it into the development process
– Validate all code prior to release
Follow QA best practicesCapers Jones, Applied software measurement: assuring productivity and quality, 1999.
Copyright © 2010 Black Duck Software, Inc. All Rights Reserved. 12
Multi-Source Development Best Practices
Manage as a cross-functional business process
Published OSS use policy
OSS review board, and process owner
Supply chain techniques
Code reduction/re-use initiatives
Defined approval processes & workflow
Agile methodology
Automated validation at acquisition and in development
Automated monitoring & tracking
Obligation verification
Policy Process Technology
Copyright © 2010 Black Duck Software, Inc. All Rights Reserved. 13
Case Study: “Design in” Compliance
Strategic acquisition of OSS
Developer
Catalog
KnowledgeBase
Development
Security
IT
Legal
Management
Quality
Approval Board
SourceForgeRubyForgeEclipse.orgApache.org
etc…
Open Source
Approval FlowAlertsAlerts
Subversion Build Test/GASoftware Bill
ofMaterials:
Validation Engines
Component, Licensing
•Open source•Home grown•Commercial•Outsourced
Continuous Multi-Source Development
Copyright © 2010 Black Duck Software, Inc. All Rights Reserved. 14
SPDXTM - Standard for Exchanging License Information
What:A data exchange standard to share OSS license and component information (metadata) for software packages and related content with the aim of facilitating license and other policy compliance
Why:Enable easy exchange of license information between companies reducing burden on both suppliers and consumersAvoids due diligence redundancy where the same source code package is analyzed multiple times by different receivers
Who:Participation from over 16 organizations including software, systems and tool vendors
Sponsors:Linux Foundation & FOSSBazaar (governance best practices group under Linux Foundation)
Software Package Data Exchange™ (SPDX™)
Copyright © 2010 Black Duck Software, Inc. All Rights Reserved. 15
OSS 3rd Party
LegacyCode
OSS
Case Study: Multi-Source Management in a Software Supply Chain
OSS 3rd Party
LegacyCode
Typical Smartphonehas over 300 components
SoftwareAsset
Corporate-Owned IP
Proprietary/Licensed IP
XML
Security
Networking
Graphics
Database
Web Services
Customer
Development/Integration
Out Source/Offshore
Your Company
Development/Integration
InternalCode
OSSOSS OSS
Copyright © 2010 Black Duck Software, Inc. All Rights Reserved. 16
SAP – Complex Software Supply Chain Automated via Black Duck Multi-Source Management
Open Source ProjectOpen Source Project
SAP CustomerSAP Customer
SAP product
Open source
component
Embedded third-party softwareContribute
Integrate
SAP product
Embedded third-party software
Redistribute
Complementary third-party software
Custom development
SAP PartnerSAP Partner
Partner component Distribute
Integrate
2001-06: use of OSS in SAP products was seen as a risk and managed as an exception
2007: approvals delegated to product units; OSS
contributions were limitedto Eclipse Foundation
2008-09: shift to OSS as productivity enabler – OSS
contribution approvals now delegated to the product units
New OSS usage
New Contributions
Copyright © 2010 Black Duck Software, Inc. All Rights Reserved. 17
SummarySoftware development is changing rapidly– New processes, new tools, new standards, multi-source
Benefits of distributed agile development are significant– Reduced operating costs ($M’s)– Reduced cycle times (up to 75%)– Increased innovation and leverage
Benefits of multi-source development are significant– Reduce operating costs– Speed innovation, supports Agile methods, increases flexibility
New processes and tools enable Dev teams to realize the full potential
17
Thank You!