ibm software group ® aspect oriented software development from promise to reality dr. daniel sabbah...
TRANSCRIPT
IBM Software Group
®
Aspect Oriented Software Development
from Promiseto Reality
Dr. Daniel SabbahVice President, IBM Software Group Strategy and Technology
IBM Software Group
Outline
Industry trend toward on demand Business IBM’s Customers and their Changing Requirements
IBM’s Own Transformation
Aspect Oriented Software Development
Driving AOSD Technology within IBM
Future Activities around AOSD
Challenges
Conclusion & Questions
IBM Software Group
Traditional The Internet On Demand
StructuredCalculations
Data ProcessingTransactions
Open StandardsConnectivity
FlexibilitySimplicity
Modular Componentseasily defined and manipulated
Dynamic definition and operations
Deepening Integration of IT with BusinessEmerging On Demand Computing Model
IBM Software Group
An (inter)enterprise whose business
processes integrated end-to-end
across the company and with key partners,
suppliers and customers—can
respond with speed
to any customer demand, market opportunity
or external threat.
On Demand Business… is Not Just About Getting Better…
IBM Software Group
Bridging the gap between business transformation and IT.
Partner Relationship
Mgmt.
Product Lifecycle Management
Category Management/
Merchandising
SCM / Retail Operations
Procurement
Bu
siness
Pro
cesses
DatabasesWorld Wide
WebWeb
Server
CustomerConnections
Transactions
SupplierNetworks
InternalSystems
IT S
op
histicatio
n
The Need to Become More Horizontal
IBM Software Group
Process Layer (modeling, execution, management)
Dealer Integration, Warranty Claims
Management
Collaborative Engineering
Direct and Indirect
Procurement
Sequencing, Order
Management
Inventory Control, Sourcing, Supply Chain visibility
Application Integration LayerConnectors/Collaboration/Flows
SystemsEnvironment
SystemsEnvironment
ApplicationEnvironment
ApplicationEnvironment
Human Interactionand Collaboration
Transactions & Messaging
Information Management
Systems Management
& Security
Software Development
Middleware PlatformMiddleware Platform
Business efficiencyBPMBAM
Business Process Integration EvolutionBusiness Process Integration Evolution
AutonomicIT EfficiencyResource optimization
Distributed OS EvolutionDistributed OS Evolution
Linux Unix Windows OS/400 z/OS
Servers Storage
Evolution towards Middleware
IBM Software Group
• Executing strategy to repurpose WAS as the kernel for IBM’s On Demand offerings
• Leveraging a very large investment in WAS • WAS technology has been reused and/or bundled by
129 product offerings across IBM Software Group:•WebSphere
•Commerce•WSAD•WSSD•…
•Tivoli•Policy Director•Privacy Manager•…
• Lotus• Portal• Domino• Workplace Messaging• Learning Management System
• Pervasive• Voice Systems
• Data Management• UDB Universal Developers Edition
• …• Growing reuse of Application Server technology• GRID computing• On Demand• Additional SWG and Server Group products
• Leveraging another very large investment in UDB • UDB technology has been reused and/or bundled by
90+ product offerings across IBM Software Group:•WebSphere
• WAS• Commerce• WSAD, WSADIE• WSSD• MQ Series Extended Security Edition• MQ Workflow• Business Integration Connect, Message Broker, Server• Everyplace Access
•Tivoli• Policy Director• Configuration Manager• …
• Lotus• Portal• Domino• …
Reuse Across the WebSphere Platform: WebSphere Application Server (WAS) and UDB
• Current approach gives reuse value, but is:• too expensive because components are too large, monolithic;• and results in high service and support expenses
• A more cost-effective solution is needed
IBM Software Group
Reuse Across the WebSphere Platform: Portal Server
Key Pressures on Development/Test:
• Marketplace demands for achieving aggressive release cycles
• Currency with versions of O/S, Application Servers, Web Servers, Databases, Directory Servers, Browsers, Security, etc.
• Breadth of O/S platforms: Windows, AIX, Solaris, zLinux, Linux/Intel, iSeries, zOS and OS390, etc.
• Version currency of WAS, Domino, HTTP Server, DB2, Oracle, Informix, SQL, iPlanet, Active Directory, etc.
• Integration with Tivoli Access Manager, Site Analyzer, SiteMinder, Lotus Sametime, Lotus QuickPlace
• Supporting multiple, concurrent service streams for Portal Server V4.1.x, V4.2.x, V5.0.x
• Increased customer support involvement for service, and Q&A
Portal Server(V4 and V5)
WSSD WAS EE SametimeQuickPlace
DominoSite Analyzer
IBM DirectoryServer
TAM and CM(V4 only)
WEA WVAA IBM Content
MgrCommerce Lotus
Workplace
UDB
TranscodingPublisher
Machine Translation
WPCP
3rd party:LDAP Directories,
Security Subsystems,Databases
IBM Software Group
Case Study of a Cross-Platform Offering: Portal Server Re-use of Platform products enabled faster delivery of function
But encountered some inconsistencies and lack of interoperability of the parts Development / technical hurdles
• Smooth installation procedure very difficult due to complex pre-reqs• Complex interactions between Portal, Application Server, Tivoli Access Manager, Lotus / Domino and
IBM Directory all part of the integrated offering• No full binary compatibility for runtime APIs• Product testing vs. Platform solution testing
Platform hurdles
• Plan prioritization of requirements in other products’ plans (Application Server, Tivoli, Lotus, etc.)• End of Support mismatches between all dependencies• Support measurements: number of PMRs rises in base products due to additional usage as a
bundled component
Sales and Marketing hurdles
• Lag between availability dates of Portal and base products causes sales conflict• Need to cross-sell to address total solution needs, which compounds our current complexities
IBM Software Group
Changes the Software Industry must make to facilitate customer transformation to on demand: Products must become more flexible, more modular, more
adaptive, higher quality, and simpler for the customer
The way the software industry creates, extends and services software must undergo a similar transformation.
AOSD enables both of these transformations Introducing on demand characteristics into IBM’s software
• E.g., quality, reuse, componentization, reengineering Accelerating IBM as an on demand company.
• E.g., organization matches architecture
The rest of the talk will describe our initial steps.
We know intuitively that AOSD can help
IBM Software Group
AOSD is the next step in the series of advances inSoftware Engineering modularity
Structured Programming
Modular Programming
Data Abstraction
Object Oriented Programming
Aspect Oriented Software Development
Compiled Programming Languages
IBM Software Group
Gartner Hype Cycle for Application Development, 2003
“Hype Cycle for Application Development”, J. Duggan, M. Blechar, M. Driver, J.Feiman, M. Light, T. Lanowitz, J. Sinur, D.Vecchio, D. Stang, J. Fenn, A. Linden, Y.Natis, W. Andrews. 30 May 2003© Copyright Gartner Research 2003. Used by permission.
IBM Software Group
It took a while to get here
Visibility
Maturity
Insight Incubation
As of May 2003
Trigger Peak Trough Slope Plateau
Aspect-OrientedSoftware Development
There have been manyyears of incubation andinnovation
IBM Software Group
AOSD within IBMIBM Somers, NY, SWG StrategyTJ Watson Research, NY. Core Technology Research
Hursley SW Laboratory, UK. Core Tech. Dev.
Raleigh, NC. WebSphere Development Software Group System House. Serviceability Team
Tech Focus Group, Toronto lab China Research Lab, Business AO
Approximately 75 peopleFocused on AOSD across IBM
AOSD is a core IBM Software Architectural Activity
IBM Software Group
IBM Roadmap
Core Technology
… 2003 2004 2005 2006 …
Driving Organizational Acceptance/Adoption
Reengineering Software
Enhancing the quality of software
Reengineering Software Engineering
NOW
New Services Offerings
IBM Software Group
Investing in core technologies AspectJ™
• AO technology for the Java™ language
• AspectJ 1.1 recently awarded a Software Development Magazine Jolt Productivity Award
AJDT
• Development environment for AspectJ
CME
• Cross-artefact, cross life-cycle capability
To provide the underpinning capability for internal, and external exploitation
All are Eclipse based, Open Source projects
Core Technology Investment
IBM Software Group
Core Technology
Application within IBM, and with the community
Improvements
PerformanceWeaving flexibility
Load Time weavingScalability
RobustnessRemoval of adoption barriers
…
JDT RestructureCross-artefact capability
… more
PerformanceWeaving flexibility
Load Time weavingScalability
RobustnessRemoval of adoption barriers
…
JDT RestructureCross-artefact capability
… more
Core Technology Investment
IBM Software Group
Delivering new Qualities of Service (QoS) into a WebSphere product
Adding problem diagnostic support into large software stack (a WebSphere component – WSIF (web services invocation framework) 110 source files 17,210 lines of code
FFDC = First Failure Data Capture. Captures contextual information within exception catch blocks Used for post-failure diagnosis FFDC is a cross cutting concern
A key proof-point for broader application across IBM Software products
Enhancing the quality and serviceability of software
IBM Software Group
First-failure data capture in a WebSphere component.
Each bar is a .java file. A red bar indicates an explicit catch block and FFDC site.
Only a subset of the110 source files shown
Modularizing Cross Cutting Concerns
Enhancing the quality and serviceability of softwareDelivering new Qualities of Service (QoS) into a WebSphere product
IBM Software Group
The AOSD solution.
The FFDC policy and logic have been modularized into an aspect.
The new FFDC aspect applies to 241 exception catch blocks.
Modularizing Cross Cutting Concerns
Enhancing the quality and serviceability of softwareDelivering new Qualities of Service (QoS) into a WebSphere product
IBM Software Group
Re-engineering of existing QoS solutions to improve on manual/automated techniques
Intuition, and now, experience, tells us that logging and FFDC can be more reliably applied with AO technology
But – perhaps it is worth re-engineering existing solutions?
Using a custom Eclipse plugin, we did some analysis SM (admin) Component of WebSphere
• Total of 355 errors found
− missing trace points
− 33% FFDC entries are incorrect
Runtime component of WebSphere
• missing trace points
• 17% FFDC entries are incorrect
None of these errors would have occurred with an AO approach
Enhancing the quality and serviceability of software
IBM Software Group
org.apache.wsif
WebSphere RAS
WebSphere FFDC
WebSphere PMI
+
WSIF for Open Source Community
Composition WSIF for WebSphere
Exploring Re-Use and Product Line Implications: The WSIF Story
Enhancing the quality and serviceability of software
IBM Software Group
Exploring Re-Use and Product Line Implications: The WSIF Story
Enhancing the quality and serviceability of software
241 FFDC join points1060 RAS (Log4j) join points 46 PMI join points
IBM Software Group
Delivering Technologies to enhance problem diagnosis and serviceability
HyperProbe: Java instrumentation technology based on Hyper/J
Effective at enhancing IBM J2EE field service situations, e.g.,
• HTTP Session state data ballooning in production systems
• JDBC connection deadlock due to mismanagement in application code
• Misuse of Finalize() to free system resources
Next Steps Developing robust/reusable diagnostics around AspectJ
Developing new AOSD-based Serviceability technologies coupled with FFDC to enhance service processing.
• Targeting multi-component interactions and problem isolation
Enhancing the quality and serviceability of software
IBM Software Group
Delivering higher quality code through capturing and enforcing architectural standards and best practices: API Scanner Application
Product Deployment Tools (version N)
Scanner Report210 deprecation warnings
5 obsoletion errors3 contraventions
AspectsPackage/Class Map
Rules & FiltersMessages
Conversion scripts
Scanner rulesdatabase
(Deprecated, Deleted,Illegal Interfaces)
API ContraventionScanner
DeployedApplicationUnit (EAR)
Deployer /System Administrator
Application Developer
WebSpherePlatform-supplied
Product-supplied
legend
Contravention Data-in human-readable form-for development tools
Enhancing the quality and serviceability of software
IBM Software Group
Results Scanner being used with products under development, e.g.,
• WebSphere Portal• WebSphere Commerce• Tivoli products
Goal: identify use of private APIs by products that run on top of the WebSphere Application Server
Result: 50,000+ violations found
Application was very easily constructed
Positive contribution to goals of clean separation for: componentization
Delivering higher quality code through capturing and enforcing architectural standards and best practices: API Scanner Application
Enhancing the quality and serviceability of software
IBM Software Group
Business Investment – IBM Software Group System House
Focused on platform integration challenges – Cross Cutting Concerns: Capability: integration & interoperability
Usability: enhancing ease of learning & understanding
Reliability: addressing high customer Problem rates
Installability: better out-of-box experience
Maintainability: improving problem determination
Documentation: improving documentation
Service: support and maintenance challenges – consistency and effectiveness
Across all IBM Software brands – Rational/Tivoli/DB2/Lotus and WebSphere
AOSD delivers tools/technology that facilitates Business goals
Driving Organizational Change
IBM Software Group
Technology Enablement and buy-in
Enabled use of AspectJ in a newly-unified WebSphere product build environment
All IBM Middleware Developers will be able to create, integrate and deploy AO-based solutions.
Products will be able to leverage AOSD in their construction WebSphere Application Server
WebSphere Portal Server
Lotus Workplace
WebSphere Business Integration
WebSphere Messaging
Developers, and Customers, will be able to use AO technology with IBM Middleware, e.g.,
Create AspectJ aspects that add value to J2EE artefacts created by IBM Rational tools
Deploy these composed J2EE artefacts (e.g., Servlets, EJBs)
Driving Organizational Change
IBM Software Group
RunQuery
AnalyseQueryReport
AlterEJB Supportdefinition?
Refactor acomponent
Componentization Investigation: Refactoring the WebSphere Container
Concern Modelling
Visualization
Concern-based queries At one point, Query
capability reported > 1000 links to resolve
Refactoring using OO and AspectJ
Reengineering software – IBM’s and IBM’s customers
IBM Software Group
Scale of the exercise 15000 java source files. Around 1500 packages.
90 components, largest components around 250kloc.
Substantial entanglement complexity
The tools stood up to the test compiled > 20,000 files with AspectJ
build time ok
queries ran fast
Was an early use of CME query capabilities Success!!
Reengineering software – IBM’s and IBM’s customersComponentization Investigation: Refactoring the WebSphere Container
IBM Software Group
Componentization – Realizing the Shared Capabilities of IBM’s Software Portfolio
LotusWebSphere DB2Tivoli
Re-factor to SWG Product Offerings
Componentization
LotusTivoli WebSphere DB2
New or Enhanced
Capabilities
New or Enhanced
Capabilities
New or EnhancedCapabilities
New or Enhanced
Capabilities
New or Enhanced
Capabilities
SharedComponents
Product Offerings
Product SpecificInvestment
SharedCapabilities
InitialBase
Product
Reengineering software – IBM’s and IBM’s customers
IBM Software Group
Programming-level aspects (such as AspectJ) are now becoming more common, and are easier to develop and use
Benefits of AOSD can be applied to entire solutions: A solution may have elements in multiple programming
languages (such as Java or C), or in XML (such as BPEL)
Considering entire solutions is becoming technologically easier (SMD, EAR etc.)
Applying aspects can be ‘taken up a level’ from direct coding
Re-engineering Software EngineeringSolution Level Aspects
IBM Software Group
“Create Policy” Business Process
“Credit History” Service
WS
DL
WS
DL
DB2
VB.NET
ServletWeb
Browser
WS
DL
Insurance Broker Solution External Service Providers
.bpel
.javaweb.xml
.javaejb-jar.xml
.ddl
“Price Quote” ServiceEJB
.vb
Re-engineering Software EngineeringSolution Level Aspects
Examples of Solution Level Monitoring and Measurement Aspects:
“Generate a business event every time a customer requests a price quote over $500”
“Measure how long it takes to update customer details in the database”
Consider this Insurance Broker application dependent on Insurance company Web Services. Many distinct artefacts, e.g., Web Service can be called from BPEL and EJB.
CME will provide the underpinning cross-artefact capability
IBM Software Group
CME – Concern Manipulation Environment
Broad View of Software Engineering There's more to software engineering than just writing code
• There are more artefact types than just code (descriptors, UML designs, …) There are many phases in the engineering lifecycle
• Cross-cutting concerns also apply at these phases, and• Integration needs are just as profound at design time, or deployment time, as
development
CME Is an open source IDE, integrated with Eclipse
• Novel capabilities, tools to aid in full-lifecycle engineering
Enables consistent manipulation across multiple artefact types in an integrating platform:
• With control over everything that goes into a product - e.g. end user docs, build scripts, deployment descriptors, WSDL files, …
• Supporting multiple AOSD approaches (e.g., AspectJ, Hyper/J)• With components, frameworks for tool builders• With integration points on many levels
A unifying (meta-)model based on concerns
• Integrating different AO approaches
Re-engineering Software Engineering
IBM Software Group
IBM Roadmap
Core Technology
… 2003 2004 2005 2006 …
Driving Organizational Acceptance/Adoption
Reengineering Software
Enhancing the quality of software
Reengineering Software Engineering
FUTURE
New Services Offerings
IBM Software Group
Continuous improvement in core technologies Focus on extending cross artefact capability
• Through CME
Drive use up the software stack
• With Solution Aspects; with integration into Rational Tools
Broader technology exploitation across and within the products
For critical Qualities of Service
To enable componentization needed for customer (and IBM) flexibility
Future
IBM Software Group
Future
Bringing the next level of AO value and capability to customers requires:
first class support in design and development tools
• E.g., Rational Development tools
first class support in the core runtime servers
• E.g., WebSphere Application Server, Portal Server, BI Server, etc.
first class representation in the programming model.
• E.g., Rational XDE Developer
And – bringing value to the level of Business Modelling
IBM Software Group
Challenges for the AOSD Research Community
Scalability through ‘complexity reduction’ Commercial software is large and complex
• Experience with Container refactoring, and with legacy re-engineering provide some experience and challenges in tool scaling
• But future (and legacy) applications may well be even larger
Cross Artefact Querying and Composition Essential for robust, full-solution integration
CME is an important start
Organizational Flexibility “Organizational aspects” (e.g., Problem Determination, or
Serviceability, organizations) are assisted with AOSD technology
• It is a transformational technology What is the right organizational structure?
Who owns cross-cutting code?
IBM Software Group
Challenges for the AOSD Research Community
Standards: Do we need them? For commercial adoption at the end-user level – Yes.
Standards will be important to allow customers, ISV’s to have flexibility and to preserve investment
Complexity versus Simplification. Does AOSD really help reduce complexity?
• Need work toward gaining understanding of this question
• But clearly WE think it DOES
AOSD introduces its own learning curve
• Consequences for industrial adoption?
• Who will be the practitioners?
IBM Software Group
Our Conclusions
AOSD’s time has come. The Software Industry needs it, and IBM is using it now.
Our customers stand to benefit significantly.
IBM is taking AOSD very seriously From a technical and business perspective
AOSD has development impact today across all major IBM brands –
• Tivoli, WebSphere, DB2, Lotus, Rational Takeup in IBM is growing – no longer a “push”; there is now a
lot of pull from across IBM’s development teams
Future impact will become more visible in IBM’s runtimes and in development tools
IBM Software Group
Trademarks
AspectJ is a trademark of Palo Alto Research Center Incorporated
Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both.
Microsoft, Windows, Windows NT, BizTalk, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both.
Gartner is a registered trademark of Gartner, inc., or its Affiliates
Solaris is a trademark of Sun Microsystems, Inc. in the United States, other countries, or both.
UNIX is a registered trademark of The Open Group in the United States and other countries.
Intel, Pentium, Pentium Pro, Pentium II, Pentium III are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the US and other countries.
HP-UX is a registered trademark of Hewlett Packard Company.
Linux is a registered trademark of William R. Della Croce, Jr. (last listed previous owner was Linus Torvalds)
"SAP is the trademark of SAP AG in Germany and in several other countries.
AIX, AS/400, Blue Gene, BlueDrekar, Lotus, Tivoli, Rational, XDE, Z/OS, DB2, Deep Blue, Deskstar, Discoverylink, IBM, Microdrive, OS/390, Scrollpoint, ServeRAID, Thinkpad, TransNote, Travelstar, Ultrastar, Websphere, Workpad, are all trademarks and registered trademarks of International Business Machines Corporation in the United States and/or other countries.
IBM Software Group
Thank You!
Questions?