eclipsecon 2010 api design and evolution (tutorial)
DESCRIPTION
In this tutorial, we explain how to design and evolve APIs without breaking client code: API Design Patterns and Idioms, Guidelines for writing Specifications, and Process for good API Design.TRANSCRIPT
March 24, 2010© Copyright 2007, 2010 IBM Corp. and Wind River; made available under the EPL v1.0
API Design and Evolution
Boris Bokowski, IBM RationalMartin Oberhuber, Wind RiverMichael Scharf, Wind River
Based on presentation material co-authored with John Arthorne and Jim des Rivieres, IBM Canada
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 2
Authors
Dr. Boris Bokowski, IBM Rational, Ottawa CanadaFull time committer on the Eclipse Platform and e4, Technical lead of the Eclipse Platform UI team, Member of the Eclipse Architecture Council, Committer Representative on the Eclipse Board of Directors.
Dipl.-Ing. Martin Oberhuber, Wind River, Salzburg AustriaSenior Member of Technical Staff at Wind River, Certified ScrumMaster, Chair of the Eclipse Architecture Council, Eclipse PMC Member, Target Management Project Lead, Eclipse Platform Core and e4 Committer.
Dipl.-Phys. Michael Scharf, Wind River, Heidelberg GermanyPrincipal Technologist and Architectural Overseer at Wind River, Member of the Eclipse Architecture Council, Model Enthusiast, Committer and Officially Approved Eclipse Troublemaker.
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 3
A Simple Example
Requirement:As a Tutorial Presenter, I want to know how many committers vs non-committers are in my talk, such that I can present material that matters to the audience.
API, 1st cut:
public interface Course {public int countCommitters();public int countNonCommitters();
}
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 4
Specification Example
Adding Specifications:
What's broken with this?
/** * Represents a course at a conference. */public interface Course {
/** * Returns the number of committers who attended the course. * @return the number of committers who attended the course. */public int countCommitters();/** * Returns the number of non-committers who attended the course. * @return the number of non-committers who attended the course. */public int countNonCommitters();
}
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 5
What's broken with this?
...committers who attended the course is not what I want!- I want a live count (current snapshot)
I'm also interested in the total number of attendees!- Ok that was not in the requirement, but I found it when testing the API
I want the total number to be exact- Approximate percentage is OK for attendee role, but percentages
should sum up as expected- Another new requirement
Actually, I may be interested in more roles in the future- E.g. contributor, commercial vs private- Yet another requirement
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 6
What can we learn from this?
Specification: API Documentation is important (“live count” vs “attendees”) to make crystal clear what the API does.
Process: Testing the API against an actual client is important from the earliest stages, to verify that it does what was expected.
- Client and implementer learn from each other when discussing API
Good API needs time and multiple iterations against real clients.
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 7
Requirements, revisited
As a Tutorial Presenter, I want to know after the talk how many people attended, such that I can boast how successful it was.
- What about people who came and left again?- What about knowing whether they liked the talk?
➔ Eliminate waste: Avoid a new API iteration by first discussing requirements!
➔ Remember, API design is a process involving (at the very least) 2 parties.
- In the ideal case, for making API rock solid, there's 3 distinct clients- Yet working with 1 primary client closely saves time, use others for
validation of concepts.
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 8
Requirements, revisited (2)
As a Tutorial Presenter, I want to know after the talk how many distinct people heard the talk, and how many left a “+1” card when leaving, such that I get feedback about audience satisfaction.
As a Tutorial Presenter, I want to get live approximate statistics about committers vs non-committers in my talk, such that I can present material that matters to the audience. I may be interested in other attendee roles in the future, such as contributors, private or commercial attendees.
Contest: There will be a prize for the best API for these reqs- Use the patterns you'll learn in this course, and your ingenuity- E-Mail Solutions until today 7pm to [email protected] Will discuss solutions and give out the prize at the Unconference,
7:15pm in this room
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 9
Contest
Launch Eclipse SDK or any package (JDT preferred) File > Import > General : Existing Projects into Workspace Archive: org.eclipsecon.apitutorial.zip Hand off: File > Export > General : Archive File
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 10
Goals of this Course
Pick up some usable advice for designing good API- Understand why good API is important- Understand the key aspects of good API- Best practices, patterns and idioms for process, code and spec
We'll talk mostly about style – this is not a course on SW Architecture
Know how to safely evolve existing API- Learn what's new in terms of tooling and policy around API
Have fun hearing some API Stories Get some good references to read on or read up on this talk
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 11
Agenda
Introduction Why API? API Design
- What is Good API?- Design – Guidelines, Patterns and Idioms- Specification – Language, Do's and Dont's
API Evolution- Binary, Source and Contract Compatibility- API Deprecation
Tools for Crafting API- API Process- API Tooling
References, Q&A
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 12
Why APIs?
APIs are interfaces with specified and supported behavior.- Specification: Guideline for implementations, judge what's a bug- Supported: Bugs will be fixed
Key for successful componentization and scaling up- Develop, test and evolve components individually- API: Provide a stable Platform to build on- SPI: Allow multiple implementations, e.g. improve performance
Good APIs capture and bind clients- Clients invest into API (by code, and by learning the API)
Good APIs minimize waste- Encourage code re-use instead of re-writing- Reduce code and complexity- Bad APIs lead to bugs, maintenance and documentation needs
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 13
Different styles of API evolution
open-endedset of Clients
clients known
lock-step development(“screw the client”)
not in lock-step(“maintain compatibility”)
Internal productdevelopment
Clients will adapt tochanges, cost is known
Platform development
Clients will continue to workwithout being recompiled
Product developmentwithout compatibility
promises
Clients must adapt tochanges, cost is unknown
Internal productfamily development
Clients should continue towork when recompiled
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 14
API Design
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 15
Characteristics of good API
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 16
Characteristics of good API
?
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 17
Characteristics of Good API
Easy to Learn Leads to Readable CodeHard to MisuseCompleteStable
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
Make your API Easy to Learn
Implies simplicity- But not too simple
Minimal surprise (consistency)- Meets the expectations of a user- Depend on the knowledge of the users- Mimic the environment
Avoids boilerplate code
Client Centric
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
Make your API Client Code Easy to Read
Implies the right level of abstraction Think in terms of clientsNaming matters!
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
Easy to Read vs. Easy to Write (Learn)
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
Easy to Read vs. Easy to Write (Learn)
new Label(parent, SWT.SEPARATOR | SWT.HORIZONTAL);
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
Easy to Read vs. Easy to Write (Learn)
new Label(parent, SWT.SEPARATOR | SWT.HORIZONTAL);
Easy to Read Hard to Write
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
Easy to Read vs. Easy to Write (Learn)
new Label(parent, SWT.SEPARATOR | SWT.HORIZONTAL);
new Label(parent, true, true);
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
Easy to Read vs. Easy to Write (Learn)
new Label(parent, SWT.SEPARATOR | SWT.HORIZONTAL);
new Label(parent, true, true);
Hard to Read Easy to Write (code completion)
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
Easy to Read vs. Easy to Write (Learn)
new Label(parent, SWT.SEPARATOR | SWT.HORIZONTAL);
new Label(parent, true, true);
Label label=new Label(parent);label.setSeparator(true);label.setHorizontal(true);
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
Easy to Read vs. Easy to Write (Learn)
new Label(parent, SWT.SEPARATOR | SWT.HORIZONTAL);
new Label(parent, true, true);
Label label=new Label(parent);label.setSeparator(true);label.setHorizontal(true);
Easy to Read (but verbose) Easy to Write (but verbose)
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
Make your API Hard to Misuse
Use java (JDT) to guide the user Avoid hidden contracts
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
The “ultimate” API
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 29
The “ultimate” API
class U { static Object do(Object o) {}}
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 30
The “ultimate” API
class U { static Object do(Object... o) {}}
A little Java 5 Convenience
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 31
Hidden Contracts
class U { static Object do(Object... o) {}}
Object w=U.do("createWindow");U.do(w,"setSize",500,500);U.do(w,"show");
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 32
Hidden Contracts
class U { static Object do(Object... o) {}}
API is more than the signature
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 33
Is this Problematic?
void setup(Map map);
Collection elements();
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 34
Is this Problematic?
void setup(Map map);
Collection elements();
Easy to MisuseHard to WriteNeeds Lots of API Documentation
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 35
Is this Better?
void setup(Map<String,Object> map);
Collection<IElement> elements();
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 36
Is this Better?
void setup(Map<String,Object> map);
Collection<IElement> elements();
It still needs lots of API documentation
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
Make your API Stable
Easy to extend and evolve Do not over-specify Do not break clients
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
Make your API Complete
Lets the users do everything they want Fits the requirements Do mot miss important concepts
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 39
Recipes for Good API
Heuristics Deviation should be with a reason
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
API is a Cover Story
Hide implementation Hide information Hide internal data formats Think about concepts rather than code
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
Names Matter
Good names drive development- Easy to Learn, Easy to Read
If it's hard to name, it's probably broken- take a step back
Avoid abbreviations- Hard to remember where abbreviation is used
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
One API should do One Thing and do it Well
Less is better than too much
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
As Small as Possible (but not Smaller)
Minimal to satisfy requirements - can always add later but not remove
Avoid the need for boilerplate code in the client
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
Consider non-functional aspects
Performance Threading State Ownership Cleanup
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
Mimic the base Platform
Naming Patterns and Language
- makes it easy to learn new API
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 46
Java API Guidelines
Class Guidelines Method Guidelines Exception Guidelines
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 47
Class Design Guidelines
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
Try to keep Classes Immutable
Immutable objects are simple, thread-safe and reusable If there must be state, keep it small and known, and document it
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
Prohibit Subclassing – or clearly document it
Subclasses are sensitive to implementation change in the base
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
Subclass only “IsA” relations
A Set is a Collection Properties is not a HashMap
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
Avoid interfaces that clients may implement
Use abstract classes instead to allow evolving
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
Separate API from SPI (Service Provider Interface)
Evolve API and SPI separately SPI may have less strict rules Example: org.eclipse.core.filesystem
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 53
Method Design Guidelines
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
Prefer factory method over public constructor
Allows object caching and sharing Life-cycle control
Dependency Injection is even better!
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
Make simple things simple, make hard things possible
Don't force the client write boilerplate code Don't violate the principle of least astonishment Don't clients into exception processing
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
Use consistent naming and ordering of parameters
Platform: IProgressMonitor always comes last Avoid parameter lists longer than 4
- impossible to remember
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
Use proper parameter and return types
Input types as specific as possible - as generic as makes sense
Avoid String if something better exists - static type checking
Avoid Collections
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 58
Exception Design Guidelines
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
Exceptions are for the exceptional
Don't use exceptions for control flow Don't use return values for exceptional behavior
- Java is not C
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
Use Checked Exceptions only if client must be alerted
Don't force the client handle situations it doesn't care about
Allow exceptions fall through to proper handler
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
Fail fast
Report errors soon – compile time is best Include failure cause in exceptions
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 62
Summary: API Design
Make your API easy to read, write and evolve Be client centric API is a Cover Story Names Matter! Repeat until happy Keep API Small: If in doubt, leave it out Make simple things simple, make hard things possible Consider non-functional aspects: No Spec - No API. Emulate the Platform
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 63
Eclipse Design Idioms
Use int option flags to allow for future extension- Same effort but more flexible than boolean- Common Pattern used in the Platform, e.g.IProject.open(IResource.NONE, IProgressMonitor mon)
Separate concerns: Do one thing at a time- Adapter pattern to mix-and-match functionality- IAddonInterface addon =
(IAddonInterface)Platform.getAdapterManager().getAdapter( baseObject, IaddonInterface.class)
e4: Use Dependency Injection
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 64
API problems that make life difficult for clients
Spec too vague, no single consistent Story Completeness (e.g., add but no remove) Names (avoid overly long names for elements that are used a lot) All interfaces, but no constructor Check assumptions about defaults Internationalization: specify if strings are visible to the end-user Lack of considering non-functional requirements
- thread safety- progress reporting- being able to cancel long-running operations- nesting, composeability
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 65
API Specifications
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 66
API specs
There's many things an API User needs to know,which are not expressed in the Java signatures...
Specification Design Goals by RoleA. Tell client what they need to know to use itB. Tell an implementor how to implement it properlyC. Tell tester about key behaviors to testD. Determines blame in event of failure
➔ Document religiously, but only as far as the cover story is concerned (don't let implementation leak)
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 67
Question: What do we spec for methods?
/** * * ??? * * * */public Object pop();
/* © Copyright 2007 IBM Corp. All rights reserved. This source code is made available under the terms of the Eclipse Public License, v1.0. */
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 68
Question: What do we spec for methods?
/** * Removes and returns the object on the top of this stack. * The stack must not be empty. * * @return the object popped off the stack; may * be <code>null</code> */public Object pop();
Method specs include- Preconditions- Results
/* © Copyright 2007 IBM Corp. All rights reserved. This source code is made available under the terms of the Eclipse Public License, v1.0. */
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 69
Lessons learned
API is not just public methods Document religiously, and carefully
No specs. No API.
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 70
Appropriate level of specification detail
Is the specification too specific or detailed, making it difficult to evolve or implement?
- Gain flexibility by hiding implementation.- API is a cover story.
Is the spec too vague, making it difficult for clients to know the correct usage?
- Always assume clients are unfamiliar with your code.- Tell a compelling and seamless story.- If incomplete, clients will look up your implementation :(
Is the API designed to be implemented or extended by clients?- Be aware of different kinds of clients.- Keep Client API separate from Service Provider API (SPI).
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 71
Specification Checklist
What do YOU want to read in an API Spec?
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 72
Over-specification makes Evolving hard
Exposing unnecessary implementation details Factory methods: specifying that it "returns a new instance"
rather than "returns an instance" - prevents future caching/pooling
Specifying the whole truth, rather than just the truth- Specifying all failure cases (instead of "reasons for failure
include...")- Specifying that an enum or similar pool of types is a closed set -
prevents adding entries later (eg: resource types)- Specifying precise data structures of return types (HashSet, rather
than just Set or Collection)
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 73
Over-specification makes Implementation hard
Making promises that are hard to keep:
Over-specifying precision of results (returns the number of nanoseconds since the file was changed)
Returning a data structure that promises to remain up to date indefinitely (leaking live objects)
Promises about the order of operations - prevents future concurrency
Real-time promises
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 74
API Contract language
The language used in an API contract is very important
Changing a single word can completely alter the meaning of an API
It is important for APIs to use consistent terminology so clients learn what to expect
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 75
API Contract language
RFC on specification language: http://www.ietf.org/rfc/rfc2119.txt
Must, must not, required, shall: it is a programmer error for callers not to honor these conditions. If you don’t follow them, you’ll get a runtime exception (or worse)
Should, should not, recommended: Implications of not following these conditions need to be specified, and clients need to understand the trade-offs from not following them
May, can: A condition or behavior that is completely optional
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 76
API Contract language
Some Eclipse project conventions: Not intended: indicates that you won’t be prohibited from doing
something, but you do so at your own risk and without promise of compatibility. Example: “This class is not intended to be subclassed”
Fail, failure: A condition where a method will throw a checked exception
Long-running: A method that can take a long time, and should never be called in the UI thread
Internal use only: An API that exists for a special caller. If you’re not that special caller, don’t touch it
Emulate the Platform language since your clients will know it.Use API Tooling tags.
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 77
Summary: API Specification review checklist
Purpose: Summarized in 1 introductory sentence Contract: Pre-conditions, Post-conditions Data: Argument and result types, ownership, special cases Failure: Any cases worth specifying? Don't overdo! Non-functional aspects: as relevant for Cover Story
- Side effects- Concurrency- Event ordering- Callbacks
Is the story complete, compelling, and seamless? Does your spec help your clients without limiting your freedom?
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 78
Summary: API Design
Know your Clients and Iterate over Requirements with them Make simple things simple, make hard things possible Keep API Small: If in doubt, leave it out. Names Matter! Repeat until happy. API is a Cover Story Specs are important! No Spec - No API. Emulate the Platform
Know where to look for details: http://wiki.eclipse.org/API_Central
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 79
Evolving API
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 80
Evolving APIs
Different situations when evolving an API
Compatibility
Techniques for evolving APIs
Techniques for writing APIs that are evolvable
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 81
But remember
Try to get it right the first time. Failing that, do not break clients.
APIs should be stable.
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 82
Compatibility
Contract – Are existing contracts still tenable?
Binary – Do existing binaries still run?
Source – Does existing source code still compile?
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 83
Contract compatibility
Before:After:
/** * Returns the current display. * @return the display; never null */public Display getDisplay();
/** * Returns the current display, if any. * @return the display, or null if none */public Display getDisplay();
• Not contract compatible for callers of getDisplay• Contract compatible for getDisplay implementors
/* © Copyright 2007 IBM Corp. All rights reserved. This source code is made available under the terms of the Eclipse Public License, v1.0. */
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 84
Contract compatibility
Weaken method preconditions – expect less of callers- Compatible for callers; breaks implementors
Strengthen method postconditions – promise more to callers- Compatible for callers; breaks implementors
Strenghten method preconditions – expect more of callers- Breaks callers; compatible for implementors
Weaken method postconditions – promise less to callers- Breaks callers; compatible for implementors
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 85
Compatibility quiz
Is the code snippet a binary compatible change? Is it source compatible?
For clients? For implementers?
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 86
Compatibility quiz #1
Before:After:
public class Test {public void foo() {
System.out.print(“Yes”); }}
public class Test {public void foo() {
System.out.print(“Oui”); }}
• Binary compatible• Method bodies do not affect binary compatibility
/* © Copyright 2007 IBM Corp. All rights reserved. This source code is made available under the terms of the Eclipse Public License, v1.0. */
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 87
Compatibility quiz #2
Before:After:
public class Test {public void foo() {}public void bar() {}
}
public class Test {public void foo() {}
}
x• Not binary compatible• Not source compatible• Deleting methods is a breaking change
/* © Copyright 2007 IBM Corp. All rights reserved. This source code is made available under the terms of the Eclipse Public License, v1.0. */
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 88
Compatibility quiz #3
Before:After:
public class Test {public void foo() {}
}
public class Test {public void foo(int flags) {} x
• Not binary compatible• Parameters are part of the method signature
/* © Copyright 2007 IBM Corp. All rights reserved. This source code is made available under the terms of the Eclipse Public License, v1.0. */
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 89
Compatibility quiz #4
Before:After:
public class Test {public void foo(String s) {}
}
public class Test {public void foo(Object o) {}
} x• Not binary compatible• Source compatible
/* © Copyright 2007 IBM Corp. All rights reserved. This source code is made available under the terms of the Eclipse Public License, v1.0. */
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 90
Compatibility quiz #5
Before:After:
public class Test {public void foo(Object o) {}
}
public class Test {public void foo(Object o) {}public void foo(String s) {}
}
• Binary compatible• When source references are recompiled they may be bound to the new method• Will cause errors in source references with a null argument, such as test.foo(null)
/* © Copyright 2007 IBM Corp. All rights reserved. This source code is made available under the terms of the Eclipse Public License, v1.0. */
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 91
Before:After:
public class Super {public void foo(String s) {}
}
public class Sub extends Super {public void foo(String s) {}
}
public class Super {public void foo(String s) {}
}
public class Sub extends Super {}
• Binary compatible• A different method will be called at runtime when method “void foo(String) is invoked on an object of type “Sub”
Compatibility quiz #6
/* © Copyright 2007 IBM Corp. All rights reserved. This source code is made available under the terms of the Eclipse Public License, v1.0. */
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 92
Compatibility quiz #7
Before:After:
public class Test {public static final int x = 5;
}
public class Test {public static final int x = 6;
} x• Not binary compatible• Constant values that can be computed by the compiler are in-lined at compile time. Referring code that is not recompiled will still have the value “5” in-lined in their code
/* © Copyright 2007 IBM Corp. All rights reserved. This source code is made available under the terms of the Eclipse Public License, v1.0. */
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 93
Before:After:
public class Test {public static final
String s = “foo”.toString();}
public class Test {public static final
String s = “bar”. toString();
• Binary compatible• Constant value cannot be computed at compile-time
Compatibility quiz #8
/* © Copyright 2007 IBM Corp. All rights reserved. This source code is made available under the terms of the Eclipse Public License, v1.0. */
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 94
Compatibility quiz #9
Before:After:
package org.eclipse.internal.p1;public class Super {
protected void foo(String s) {}}
package org.eclipse.p1;public class Sub extends Super {}
package org.eclipse.internal.p1;public class Super {}
package org.eclipse.p1;public class Sub extends Super {}
x
• Not binary compatible• Protected members accessible from an API type are API• Is this valid if class Sub is final or says clients must not subclass?
/* © Copyright 2007 IBM Corp. All rights reserved. This source code is made available under the terms of the Eclipse Public License, v1.0. */
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 95
Compatibility quiz #10
Before:After:
public class E extends Exception {}
public class Test {protected void foo() throws E {}
}
public class E extends Exception {}
public class Test {protected void foo() {}
}
• Binary compatible• There is no distinction between checked and unchecked exceptions at runtime• Not source compatible because catch blocks in referring methods may become unreachable
/* © Copyright 2007 IBM Corp. All rights reserved. This source code is made available under the terms of the Eclipse Public License, v1.0. */
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 96
Compatibility quiz #11
Before:After:
public class A {public void foo(String s) {}
}public class C extends A {
public void foo(String s) {super.foo(s);
}}
public class A {public void foo(String s) {}
}public class B extends A {}public class C extends B {
public void foo(String s) {super.foo(s);
}}
• Binary compatible• The super-type structure can be changed as long as the available methods and fields don’t change
/* © Copyright 2007 IBM Corp. All rights reserved. This source code is made available under the terms of the Eclipse Public License, v1.0. */
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 97
Compatibility quiz #12
Before:After:
public class Test {}
public class Test {public Test(String s) {}
}x
• Not binary or source compatible• When a constructor is added, the default constructor is no longer generated by the compiler. References to the default constructor are now invalid• You should always specify at least one constructor for every API class to prevent the default constructor from coming into play (even if it is private)• A constructor generated by the compiler also won’t appear in javadoc
/* © Copyright 2007 IBM Corp. All rights reserved. This source code is made available under the terms of the Eclipse Public License, v1.0. */
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 98
Compatibility quiz #13
Before:After:
public class Test {public void foo() {}
}
public class Test {public boolean foo() {
return true;}
}
x
• Not binary compatible because the return type is part of the method signature• Source compatible only if the return type was previously void
/* © Copyright 2007 IBM Corp. All rights reserved. This source code is made available under the terms of the Eclipse Public License, v1.0. */
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 99
Lessons learned
It is very difficult to determine if a change is binary compatible Binary compatibility and source compatibility can be very different You can’t trust the compiler to flag non-binary compatible changes
Evolve once, check twice.
If in doubt, check “The Bible”: Evolving Java-based APIs, Jim des Rivieres et alavailable from http://wiki.eclipse.org/API_Central
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 100
Binary compatibility DONT's for API elements Rename a package, class, method, or field Delete a package, class, method, or field Decrease visibility (change public to non-public) Add or delete method parameters Change type of a method parameter Add or delete checked exceptions to a method Change return type of a method Change type of a field Change value of a compile-time constant field Change an instance method to/from a static method Change an instance field to/from a static field Change a class to/from an interface Make a class final (if clients may subclass) Make a class abstract (if clients may subclass)...
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 101
Binary compatibility DO's for API elements Add packages, classes, and interfaces
Change body of a method
Do anything you want with non-API elements
Add fields and type members to classes and interfaces
Add methods to classes (if clients cannot subclass)
Add methods to interfaces (if clients cannot implement)
Add non-abstract methods to classes (if clients may implement)
Reorder class and interface member declarations
Change value of a field (if not compile-time constant)
Move a method up to a superclass
Make a final class non-final
Make an abstract class non-abstract
Change name of method formal parameter...
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 102
Evolving Generified API
Breaks compatibilityAdd, delete, or change type bounds of type parameters
Binary compatibleRename type parameters
Breaks compatibilityRe-order type parameters
Breaks compatibilityDelete type parameter
Breaks compatibility(unless type was not generic)
Add type parameter
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 103
Techniques for evolving APIs
Create extension interfaces, use naming convention (ITurtleExtension, ITurtle2)
Wiegand’s device
Deprecate and try again
Proxy that implements old API by calling new API
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 104
Techniques for enabling API evolution
Use abstract classes instead of interfaces for non-trivial types if clients are allowed to implement/specialize
Separate service provider interfaces from client interfaces
Separate concerns for different service providers
Hook methods
Mechanisms for plugging in generic behavior (IAdaptable) or generic state, such as getProperty() and setProperty() methods
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 105
Non-Java API's
File Formats- Workspace Compatibility section in the Eclipse Project's Project Plan- Provide Migration code when changing, e.g. Preferences
Extension Points- Expected Parameters- Can be evolved like Java APIs: Version Numbering- No Tooling available yet
ID's- Even Extensions are API for consumers!- Example: ID's of Property Testers expected to be in the Platform- No Tooling, no good automatic documentation!
Data Protocols, Wire Formats, … Functionality of commandline args, config files, …
- Example: eclipse -configuration /tmp
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 106
API Deprecation
Reasons for deprecating API- A new story fixed issues (e.g. concurrency, performance)- Small API – Less to Learn, easy to understand- Planning a major implementation rewrite (think e4)
Reasons for removing API- At first, deprecation is just a recommendation
But nobody will act if there is no axe in the window May remain deprecated forever if just for ease-of-learning
- Schedule API Removal for serious issues- Removal breaks clients and must be clearly announced (Policies)- Announce removal date. Use API Tooling to find parties to discuss.
Eclipse Project Policy for Removal: 2 years lead timehttp://wiki.eclipse.org/API_Central
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 107
Who can remember all this?
API Design is hard...
… but there is rescue!
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 108
API Process
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 109
Before you begin
Have and agree on common API guidelines, e.g.:- Eclipse Naming conventions
http://wiki.eclipse.org/Naming_Conventions
- How to Use the Eclipse APIhttp://www.eclipse.org/articles/Article-API%20use/eclipse-api-usage-rules.html
Have someone in charge of the API early on
Have well-defined component boundaries and dependencies- There's no good API for bad Architecture- E.g., Core vs. UI
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 110
Work with a client
APIs exist to serve the needs of clients- Where are those clients? What do they need?
Important to work with actual clients when designing API- Designing APIs requires feedback from real clients who will use it- Otherwise risks crummy API that real clients cannot use
Find a primary client- Ideally: adjacent component, different team, same release schedule- E.g., JDT UI is primary client of JDT Core
Work closely with primary client- Listen to their problems with using API- Watch out for lots of utility classes in client code symptomatic of
mismatch between API and what client really needs- Work together to find solutions
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 111
API Design Process
Gather Requirements – but with healthy skepticism- Some “Requirements” are really implementation suggestions- API should be the Cover Story but allow different implementations!
Start the spec with 1 page- Iterate, rinse, return - take time to evolve
Code early against the API - Tests, Examples to see how it “feels” in real life- This is not throw-away code but may become regression tests or
examples as part of the spec later on- Even more important for SPI! - If there is just one implementation of a
service that's meant to be changeable, you'll surely mess up.
Have realistic expectations: expect to evolve
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 112
Development cycle, releasing an API
Important to distinguish between- On-going development- Released API
In each development cycle, opportunity to work with well-known clients on new API
Release cycles can be short or long Release cycles of API, implementation, and Client may be:
- The same (lock step development)- Different (more likely!)
What kind of compatibility promise do you make?
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 113
Across components, use API, not internals
Component provides API for arbitrary clients- API exists to tame inter-component coupling
Client components are expected to use API “to spec”- Not depend on behavior not covered in API spec- Not depend on internals- Foolish to make exceptions for close friends
Close friends don’t point out flaws Sets bad example for others
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 114
Eclipse API Process Guidelines
Provisional API Guidelines- Mark x-internal:=”true” in the Manifest until solidified- Allows for wider exposure to clients, even across releases- NEW: Allowed to have provisional API in its final namespace
No more forced breaking of clients when refactoring from “internal” package to “public” package namespace
X-internal is the premier markup of provisional API Must have a client
- Each extension point or API must have a client in Open Source- Ensure that example code and test code is there
Freeze by M6- With PMC Escalation process into M7
Reference: http://wiki.eclipse.org/API_Central
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 115
API Tooling
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 116
API Tooling for Developers
Has become indispensable for- Adding proper Javadoc Tags- Reporting mis-use according to tags- Detecting Binary Compatibility Breakage- Detecting Leakage of non-API Types into API- Updating Version Numbers correctly
Super easy setup- Window > Preferences > PDE > API Baselines : Register Baseline- Single directory with plugins to compare against- Problem Markers; use Ctrl+1 Quickfix to add problem filters
Documentation: http://www.eclipse.org/pde/pde-api-tools/
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 117
Version numbers
major.minor.service.qualifier From release to release, the version number changes as follows:
- When you break the API, increment the major number- When you change the API in an (binary) upwards compatible way,
increment the minor number- When you make other changes, increment the service number- The qualifier is changed for each build
Proposal: Less strict versioning for Service Provides (SPI)- Breaking changes in SPI are allowed with minor rev- A variant of the “I know my clients so I can break them” story- Better not make such API public, if I know you, use internals
Guidelines, see http://wiki.eclipse.org/API_Central
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 118
API Tooling for Build Engineers
.api_description file needed in binary bundles- Conveys information from API tags like @noimplement- No problem reporting without these in the binary baseline
Setup: Just add this to your build.properties:- generateAPIDescription=true
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 119
API Tooling for Reports
API (mis) Usage Reporting- Who uses my internals?- New with 3.6M5: Run > External Tools...
API Comparison- Generate an XML Snapshot of API- Compare two snapshots, generate XML / HTML Report- Useful for New&Noteworthy, Release Reviews
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 120
Summary: What you should take home
Why API – 4 quadrants Design Guidelines
- Make Simple things Simple, make complex things possible
No Specs – No API- Non-functional aspects: Performance, Threading, State, ...- Cover Story: Avoid Overspecification
API Evolution: binary – source – contract compatibility- API Tooling
Process for API: Requirements – Rinse – Repeat- No API without Clients
Where to look for more - References
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 121
References
API Central on Eclipse Wikihttp://wiki.eclipse.org/API_Central
- API Process: This Presentation, linked on API Central Eclipse API Process and Policies, all linked from API Central
- API Design: Joshua Bloch, Effective API Design (1 Hr Video)http://www.infoq.com/presentations/effective-api-design
- API Specifications: EclipseCon 2006 API Tutorial, see API Central Sun, Requirements for Writing Java API Specifications Sun, How to Write Doc Comments for the Javadoc Tool
- API Evolution Evolving Java-Based API's, see API Central EclipseCon 2007: Eclipse APIs and Java 5, see API Central
- API Tooling, Documentation and Wiki all linked from API Central
Joshua Bloch, Effective Java Programming Language Guide
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 122
And remember the Contest...
Create your API for counting Course attendees and Committer / User Statistics
E-Mail solutions to [email protected] Come to the Winner Selection at the Unconference
- Today 19:15 in this room (Lafayette)- We'll all learn a lot by looking at solutions- … and the winner can take their prize away
If you have any more questions about API...- Find us at the conference, or file a bug with the Architecture Council- http://wiki.eclipse.org/Architecture_Council
API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 123
Legal Notices
IBM and Rational are registered trademarks of International Business Corp. in the United States and other countries
Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both
Other company, product, or service names may be trademarks or service marks of others