© 2002 ibm corporation confidential | date | other information, if necessary © 2004 ibm...

© 2002 IBM CorporationConfidential | Date | Other Information, if necessary© 2004 IBM Corporation

Jim des RivièresIBM OTI Labs (Ottawa)[email protected]

Platform Quality APIs

2 © 2004 IBM Corporation

Outline of Talk

Platform quality APIs Long-haul APIs to support open-ended set of clients

Best Practices for API Design Design advice for good APIs

API Evolution Changing APIs while keeping existing clients happy

Best Practices for API Development How to work effectively on components with APIs


Topic – Platform quality APIs

Platforms are…

Useful Platform provides useful services to clients

Open Platform intended for open-ended set of diverse clients

Stable Platform provides stability across multiple releases

Growing Platform evolves from release to release to better meet client needs


Successful platforms

Examples of successful platforms Win32

Intel x86

IBM 360

J2SE

Mac OS

Successful platforms last a long time They don’t break existing clients – compatible evolution They keep existing binaries working – binary compatibility


Platform quality APIs

APIs just among friends (adjacent components in same release) Unsupported for arbitrary clients

Don’t need to be stable from release-to-release

You get chance to correct your mistakes

Stark contrast to platform APIs

Platform APIs must support open-ended set of clients and be stable Once API is released there’s no going back

Breaking changes are bad news

Either extend API in next release

Or provide improved API alongside old API


Platform quality APIs are unforgiving

One chance To tell clients how to use API – the “API specs”

To choose helpful class and method names

To decide between class or interface Once API is in service

Constrained to work within framework laid down

Repair and improve without breaking clients Unclear, inadequate, or incorrect API specs are bad news

Confusing to clients

Fixing API specs may make things worse


Good specs are key to platform quality APIs

Good APIs needs good specs Code is not enough (even source code)

API specs Describe intended behavior of API element

How it’s supposed to work How clients should behave to ensure they will not be broken by future

changes Client code written to spec

Every reason to believe it will work with any conforming implementation Implementation code written to spec

Every reason to believe it will support all conforming clients Best (&only) known recipe for long-term survival

Allows API to endure, evolve, and satisfy clients that build upon it


Topic – Best Practices for API Design

Design advice for good APIs


Clearly separate API from internals

APIs should be highly visible so that clients can find Publish API specs

Draw clear distinction between API and non-API Eclipse non-API packages “internal”; e.g., org.eclipse.ui.internal

API consists of public classes and interfaces in API packages

All interface members Public and protected members of classes

No “visible” connection from API to non-API API should be closed story

Implementation details should not leak into API


Separate Core from UI

“Core” component Provides basic infrastructure that can be used in headless manner

Usually provides API

May depend on other core components

Must not depend directly or indirectly on UI components “UI” component

Provides elements or frameworks for use in GUI

May have API (but often not)

May depend on other UI components

May depend on core components All inter-component dependencies are via APIs


Segregate client API from provider API

Component often provide APIs for different parties Client APIs

Used by bread-and-butter clients, like a UI Provider APIs (aka SPIs)

Used only for plugging in new pieces

Keep different kinds of APIs segregated Client API should be easy for clients

SPI should be flexible, powerful for service provider

Clients shouldn’t have to understand SPI too

Clients shouldn’t be able to meddle in SP affairs


Use Java classes and interfaces wisely

API interfaces that clients do not implement are useful Excellent way to insulate client from implementation internals

Allows there to be multiple implementations

Adding methods in future is non-breaking API interfaces that clients may implement are inflexible

Adding method breaks binary compatibility

Abstract classes work better for this API classes that clients do not subclass are useful

Use factory methods and hide constructors API classes that clients may subclass are useful, but difficult

API frameworks are notoriously hard to specify

Needs additional contracts for subclassers


Use Java modifiers to block unintended usage

If API class is not intended to be subclassed by clients Declare class final, or make constructors package-private

If API class is not intended to be instantiated by clients Declare constructors private

If API method is not intended to be overridden by client subclasses Declare method final

If API class is intended to be subclassed but not instantiated by clients

Declare class abstract

Declare constructors protected Use “soft” injunctions in specs if there’s no way to block

“This interface is not intended to be implemented by clients.”


Factory methods are better than constructors

Constructors are limiting as API All constructors effectively have same name

Cannot have two with same type parameters public Path(String platformNeutralString) public Path(String osSpecificString)

Constructors cannot return instance of subclass Factory methods are more flexible than constructors

Factory method have names

public static Path newPath(String platformSpecificString) public static Path newOSPath(String osSpecificString)

Factory method can return instance of subclass

Keep constructors hidden (private, package-private, or protected)


Methods are better than fields

API fields are very weak public String hostName;

No way to intercept field accesses either read or write

No lazy init on first read No possibility to report changes

Non-final fields vulnerable to overwrite

More flexible to expose get/set API methods public final String getHostName();

public final void setHostName(String hostName);

Use fields in API only for constants (public static final)


Program extra defensively at API boundary

Clients do the strangest things Usually accidentally, but sometimes intentionally

Check incoming arguments are to spec Don’t ignore - throw some form of RuntimeException

In Eclipse null argument is illegal unless explicitly allowed by spec Worry about argument capture

What are consequences if client later modifies object being captured?

Consider copying arrays and collections Worry about result disclosure

What are consequences if client later modifies object being return?

Consider copying arrays, returning unmodifiable collections Worry about threading, or reentrancy

Are you holding locks when you issue that callback?


Conclusion

No one can really tell you how to design good APIs Any more than they can really tell you how to write good programs

API design is just a form of high-stakes program design Talk to other programmers

Ask how they’d approach it Study successful APIs

APIs are generally available and well-documented

Recommended reading Effective Java Programming Language Guide, by Josh Bloch

http://java.sun.com/docs/books/effective/

How to Design a (module) APIhttp://openide.netbeans.org/nonav/tutorial/api-design.html


Topic – API Evolution

APIs need to evolve from release to release Changes to API could invalidate existing clients

Evolve API in compatible ways Preserve as much value as possible across API changes

Keep existing clients working

Two general considerations Contract compatibility – Honor existing API contracts

Binary compatibility – Keeping the JVM happy


Contract Compatibility

Before/** Returns the non-empty list of indices. */

public int[] getIndices();After

/** Returns the list of indices. The list may be empty. */

public int[] getIndices();Breaks some existing callers

int[] d = getIndices();

System.print(d[0]); // possible array index out of boundsHowever, existing implementers are fine

public int[] getIndices() {

…; return result; // result is non-empty

}


Evolving API Contracts

API contracts are expressed in API specs

API contracts promise the client certain things Clients can play multiple roles – e.g., caller, implementer

Different roles have different contracts

Changes to contracts should not invalidate existing clients


Binary Compatibility

Beforepublic void register(String key);

Afterpublic void register(Object key);

Existing calls re-compile as expectedregister(“foo”); // no compile error

But existing binaries no longer linkregister(“foo”); // link error


Binary Compatibility DON’Ts for API Elements

1. Rename a package, class, method, or field2. Delete a package, class, method, or field3. Decrease visibility (change public to non-public)4. Add or delete method parameters5. Change type of a method parameter6. Add or delete checked exceptions to a method7. Change return type of a method8. Change type of a field9. Change value of a compile-time constant field

10.Change an instance method to/from a static method11.Change an instance field to/from a static field12.Change a class to/from an interface13.Make a class final (if clients may subclass)14.Make a class abstract (if clients may subclass)…


Binary Compatibility DOs for API Elements

1. Add packages, classes, and interfaces2. Change body of a method3. Do anything you want with non-API elements4. Add fields and type members to classes and interfaces5. Add methods to classes (if clients cannot subclass)6. Add methods to interfaces (if clients cannot implement)7. Add non-abstract methods to classes (if clients may implement)8. Reorder class and interface member declarations9. Change value of a field (if not compile-time constant)

10.Move a method up to a superclass11.Make a final class non-final12.Make an abstract class non-abstract13.Change name of method formal parameter…


Binary Compatibility

Java VM has special rules for binary compatibility

API changes should be binary compatible Existing clients should continue to work without recompiling

N.B. Java compiler does not detect this kind of breakage

Ref: Evolving Java-based APIshttp://eclipse.org/eclipse/development/java-api-evolution.html


Adding Methods to API Interfaces

API interfaces used to hide implementation work well “/** … This interface is not intended to be implemented by clients */”

Add new methods to API interface

Add corresponding methods to implementing class

package org.eclipse.core.resource;/** … This interface is not intended to be implemented by clients */public interface IWorkspace { … }

package org.eclipse.core.internal.resource;class Workspace implements IWorkspace { … }

public boolean isTreeLocked(); // new

public boolean isTreeLocked() {…}


Avoid API Interfaces that Clients May Implement

API interfaces that clients may implement are problematic Adding method breaks binary compatibility

Use API class instead of API interface… When client may implement

When there is a chance new methods needed in future

N.B. converting interface to class breaks binary compatibility


Adding Methods via I*2 Extension Interfaces

If no choice, add new methods in extending API interface Avoids breaking existing clients that implement

package org.eclipse.ui;public interface IActionDelegate { … } // original interface

Usage

IActionDelegate d = new IActionDelegate2() {…};

if (d instanceof IActionDelegate2) { IActionDelegate2 d2 = (IActionDelegate2) d; d2.dispose(); // call new method}

public interface IActionDelegate2 extends IActionDelegate { void dispose(); // new }


How to Delete API

API deletion always breaks any existing clients

But replacing API with improved version is usually doable


Replacing API Methods

Add replacement API method Deprecate original method

Ensure original method continues to work

package org.eclipse.jdt.core.dom;public class Message { … /** … * */ public int getSourcePosition() { // rename getStartPosition() … }}

package org.eclipse.jdt.core.dom;public class Message { … /** … * @deprecated Use getStartPosition() instead */ public int getSourcePosition() { return getStartPosition(); // forward to new method }

public int getStartPosition() { … }}


API Evolution – Summary

Evolve API in compatible ways Honor existing API contracts

Observe technical rules for Java binary compatibility

Usually feasible to find way to improve API and keep existing clients working without recompiling

Design APIs with future evolution in mind


Topic – Best Practices for API Development

Good APIs don’t just appear overnight Significant design effort

Good APIs require design iteration Feedback loop involving clients and implementers

Improve API over time

Components build upon the APIs of other components Need collaborative working relationship between teams

Some ways to work effectively on components with APIs, based on Eclipse Project experience


Before you begin

Have and agree on common API guidelines Eclipse Naming conventions

http://dev.eclipse.org/naming.html

How to Use the Eclipse APIhttp://www.eclipse.org/articles/Article-API%20use/eclipse-api-usage-rules.html

Have someone in charge of the API early on

Have well-defined component boundaries and dependencies core vs. UI


Work with API clients

APIs exist to serve the needs of clients Where are those clients? What do they need?

Important to work with actual clients when designing API Designing APIs requires feedback from real clients who will use it

Risks crummy API that real clients cannot use Find a primary client

Ideally: adjacent component, different team, same release

E.g., JDT UI is primary client of JDT Core Work closely with primary client

Listen to their problems with using API

Watch out for lots of utility classes in client code symptomatic of mismatch between API and what client really needs

Work together to find solutions


APIs First

Basic “APIs First” workflow1. Work with primary client to decide what API you want/need (design

API)

2. Write API specs

3. Write API test suites

4. Implement API

(Expect numerous iterations within basic workflow)

Helps ensure APIs are Ready for clients to use (specs and impl. in place)

Of high quality and stable

Ready to evolve to meet next requirement


Don’t reveal API too early

“We shall ship no APIs before its time” (to paraphrase Orson Welles' old wine commercial)

Keep work in internal packages until new API is ready API specs in place

API test suite

Credible implementation When ready, release by moving to API package

Keeps everyone’s expectations realistic Clients

Component owner Avoids embarrassing recall of unfinished API just before shipping


Write comprehensive API unit tests

Unit tests for each API element Tests written “to spec”

Assumes and tests only what is covered in spec

Plays role of idealized client

Helps you to keep client view of the API Implementer’s safety net

Catches stupid mistakes before they can screw over clients

Helps working relationship with primary client Core components

Complete API coverage by automated tests UI components

Partial coverage by automated tests

May required additional manual tests for visual elements


Keep API specs consistent at all times

You write a story. Someone else changes the story. How do you know whether the story still reads well?

Errors and inconsistencies within specs are hard to detect/remove API spec is more like a book than like code

Maintaining consistency require re-reading

Start with initial set of consistent API specs Scrupulously maintain consistency as API spec grows/changes

Have a “Keeper of the Specs” Responsible long-term for maintaining specs and reviewing spec

changes

Requires thorough knowledge of API and how it got its spots


Keep API tests up to date at all times

Errors and inconsistencies within tests are found immediately But insufficient test coverage is hard to detect/remove

Start with a set of tests that completely cover API Scrupulously maintain tests as API spec grows/changes

Can be done as part and parcel of changing API Or by “Keeper of the Specs” if they keep the tests as well


Funnel all external dependencies thru API

Component provides API for arbitrary clients API exists to tame inter-component coupling

Client components are expected to use API “to spec” Not depend on behavior not covered in API spec

Not depend on internals

Foolish to make exceptions for close friends Close friends don’t point out flaws Sets bad example for others

Most common form of Eclipse component is single plug-in Internal packages have “.internal.” in name

Plug-ins should not reference internal packages of other plug-ins


Focus clients on API

Encourage clients to discuss/propose API changes in terms of existing API

Grounds discussion in reality

API owner ultimately decides how best to address issue API needs to make sense for all clients

Final decision may differ for what was proposed


Tread carefully in vicinity of APIs

Be especially careful when working on code in component with an API

Adding/deleting public method

internal class - no problem API class - changes API

Be careful with renaming/refactoring tools Make sure APIs not affected


Tag API elements with @since

Scrupulously record which release API element first appeared in In Javadoc with @since tag

@since 3.1 Elsewhere in words

During release cycle Distinguishes old API from new API added during current release cycle

Still flexibility to change (or delete) newly added API

After release has shipped Help clients targeting release >= N to avoid API added after N


Minimize disruptive API changes

Breaking API changes are very disruptive to clients Non-breaking API changes also cause work for clients

Work required to port to use new/improved APIs

During release cycle Schedule API work for early in cycle

Whenever possible, preserve API contracts

When not possible, coordinate with clients to arrange cut-over plan

After release has shipped Evolve APIs preserving API contract and binary compatibility


Replace, deprecate, and forward

Add replacement API in non-breaking way Deprecate old API but keep it working

@deprecated This method has been replaced by {@link #bar()}.

For new API added earlier in release cycle Negotiate a short fuse on deleting deprecated stuff

Make sure clean up happens well before end of release cycle

Schedule API review & polish before too late to fix Reviews often uncover inconsistencies

For API added in earlier release Deprecation warnings inform of better way

API contract and binary compatibility preserved


Outgoing API changes for a component

Component team works in HEAD stream

Other component teams compile and run against version as of latest (weekly) integration build

Keep HEAD compiling and working cleanly Do not release changes that would hose other team members

Re-run all unit tests before releasing

Outgoing API changes require coordination with downstream teams Post “preview version” of component a few days before I-build

Allows downstream components to coordinate their changes

Use nightly builds to reduce risk for a broken integration build


Incoming API changes affecting a component

For dependent components, use version as of latest (weekly) integration build

Team members upgrade to latest I-build each week Laggards slow the team down

Puts premium on everyone making each I-build better than last

Incoming API changes from upstream teams Receive “preview version” of component a few days before I-build

Allows component to prepare coordinated changes for next I-build


Staging large changes to existing APIs

E.g., JDT Core support for new J2SE 5 language features

Work done in stages, over several months Coordinated with primary client

Parcels of API changes with stub implementations throw new RuntimeException(“Not implemented yet”);

Allows clients to write and compile code (but not run/test)

Implementation of API parcels done later Clients can run/test code as soon as API is implemented


API-related Resources

How to Use the Eclipse API, by Jim des Riviereshttp://www.eclipse.org/articles/Article-API%20use/eclipse-api-usage-rules.html

Effective Java Programming Language Guide, by Josh Blochhttp://java.sun.com/docs/books/effective/

Requirements for Writing Java API Specificationshttp://java.sun.com/products/jdk/javadoc/writingapispecs/index.html

How to Write Doc Comments for the Javadoc Toolhttp://java.sun.com/products/jdk/javadoc/writingdoccomments/index.html

Evolving Java-based APIshttp://eclipse.org/eclipse/development/java-api-evolution.html

Contributing to Eclipse, by Erich Gamma and Kent Beckhttp://www.aw-bc.com/catalog/academic/product/0,4096,0321205758,00.html

Internal Tool (reports cross-plug-in references to internals)http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/jdt-core-home/tools/internal/index.html

How to Design a (module) APIhttp://openide.netbeans.org/nonav/tutorial/api-design.html

© 2002 ibm corporation confidential | date | other information, if necessary © 2004 ibm...

Documents

nonapi api

api method

internal api

separate api

api development

improved api

old api slide

good apis api evolution