adaptive object-model metadata evolver object-model metadata evolver atzmon hen-tov ... model...
TRANSCRIPT
1
Adaptive Object-Model Metadata Evolver
Atzmon Hen-Tov
Pontis
David H. Lorenz
Open University
Lena Nikolaev
Pontis
Lior Schachter
Open University
Rebecca Wirfs-Brock
Wirfs-Brock Associates
Joseph W. Yoder
The Refactory, Inc.
Introduction An Adaptive Object-Model (AOM) system represents user-defined domain classes (and
their relationships) as metadata [YBJ01; YJ02]. The AOM system is a model
represented by objects rather than classes. The object model is constructed at run time
by interpreting externally stored definitions (metadata). Users change the object model
(or the metadata) to reflect changes in the domain.
One way to support model evolution is to take the system off-line, evolve the metadata
and the domain objects, and perform a clean restart of the system after a careful release
and upgrade. However, when a system needs to keep running, or when updating the
entire model at the time a new version needs to come online is too time consuming, then
the evolution of the metadata and the domain objects can be included as a part of the
model loader.
Target Audience
The pattern presented in this paper focuses on updating the metadata as either the core
meta-architecture or the domain model of an AOM evolves. This pattern is particularly
relevant to developers of AOM systems. It may also be relevant to those who need to
evolve other types of meta-architectures. We assume the reader has some background
knowledge of AOM systems (mainly TYPE OBJECT [JW98], PROPERTIES [JW98], and
TYPE-SQUARE [YBJ01] or DYNAMIC OBJECT MODEL [RTJ05] based architectures).
2
AOM Metadata Evolver
Context
You are upgrading a 24x7 production system that is implemented using an AOM. You
find that implementing new functionality requires changes to the AOM architecture. For
example, a core update to the AOM architecture may be required. You want to make
these changes available to a running production system where there is already
considerable metadata and data in the system based upon a previous version of the
AOM architecture.
Your model relies on a TYPE OBJECT [JW98] to dynamically define new business
entities for the system. Entities have attributes, which are implemented with the
PROPERTIES [JW98] pattern. The TYPE OBJECT pattern is used a second time to define
the legal types of attributes, called AttributeTypes. Thus you are using TYPE
SQUARE [YBJ01], a combination of TYPE OBJECT [JW98] and PROPERTIES [JW98]
patterns. Additionally you might be using these patterns: ENTITYRELATIONSHIP
(ACCOUNTABILITY) [WYW09], RULEOBJECTS [Ars00], HISTORY OF OPERATIONS
[FCW08], SYSTEM MEMENTO [FCW08], and MIGRATION [FCW08].
Problem
How can you evolve at runtime the metadata and the data associated with a live and
running AOM as its core architecture changes?
Forces
Polymorphism: The rules for creating an entity may vary according to its type or
according to rules that apply to its data. A monolithic upgrade logic will not manage
model evolution well, i.e., a solution that doesn’t support polymorphism in the
upgrade logic will be hard to manage against a polymorphic model..
Customisability: There are many customers using the product, each customer may
modify its own AOM model. The new architecture needs to be able to understand
different versions of the AOM model. A solution that doesn’t let the customer team
resolve conflicts autonomously will require reimplementation of the customization.
Legacy Data: There may be large amounts of stored instances, excluding the
possibility for off-line data migration in a short maintenance window.
Solution
Use an AOM BUILDER [WYW09] to create entities (Figure 1). Add as the first step in
your AOM-Builder an UpgradeBuildStep to check whether the data associated
with an entity-type has been defined with the current version of the AOM. If not, invoke
the necessary upgrade scripts that will evolve the data to map to the current version of
the AOM. Each entity-type may have several upgrade scripts (Figure 2), some of which
are supplied with the new product version and some that are added manually by the user
(per customized installation of the AOM system).
3
Figure 1: Upgrade Builder Step
This process can include migrating metadata and data from many versions before so
there may be a chain of version upgrades that needs to be applied. For that to work,
every persistent object (Entity, EntityType) has a version field (Figure 2). A
system-wide parameter holds the currently installed version. When upgrading the
system a collection of ModelVersion entities and the associated upgrade scripts are
available to the UpgradeManager.
Figure 2: Upgrade Framework Class Diagram
4
The UpgradeManager is responsible for determining which scripts are necessary to
invoke. This is carried out by consulting the EntityType and the migration
converstions each Upgrader handles. This logic includes querying a flag
(overrideSuper) to determine whether or not the EntityType superclass’
Upgraders should be used.
An Upgrader edits the raw AOM data (e.g., XML) and transforms it to the new
version format. The raw data transformation is the responsibility of the
Upgrader.upgrade method (Figure 2). Updating the raw data values of AOM
objects in the BuildContext is carried out by the UpgradeManager (Figure 1).
The BuildContext is part of the AOMBuilder.
The upgrade framework provides an API (using the PROXY pattern) for manipulating
raw data format. This lets the script programmer concentrate on the transformation
from the old version to the new one, rather than dealing with the raw data
format [BMR+96].
Dynamics
The overall process requires a staging server (Figure 3) in which the new version is
installed and the AOM metadata from the production system is brought into the running
system. Conflicts between modification done by developers in the new version of the
product and modifications done at production by end users are reported in the Problems
view (see BREAK AND CORRECT [HNS+10]) and the user needs to resolve them. For
example, if an entity type has a new property with the same name of a property defined
in an AOM type, the user will need to change the dynamically defined property name.
The product team prepares placeholder update scripts for cases where they think a
customer specific upgrade script is needed (for example, when a new mandatory
property is defined and a simple default value will not suffice).
After all conflicts are resolved and all placeholder scripts are written, the customer team
can test the new version along with the entire set of upgrade scripts.
During production deployment, the upgrade framework is invoked as a first step in the
AOM-BUILDER and performs just-in-time instance upgrades as described above
(invoked as script hook points during the build process).
5
Figure 3. AOM Evolution Process
Example
For example, consider an AOM implementation of an online marketing system,
designed for running Telephony campaigns targeted at different subscriber segments
[HLPS09]. Figure 4 shows several classes in the AOM and several instantiated entities.
6
Figure 4. The AOM Model and Instances.
A generic product comprises several general-purpose classes, such as Benefit and
Event. Benefit describes specific campaigns. Event describes the trigger for
activating Benefits. Each Benefit is associated with an EventType to define
which type of Event can trigger grant of this Benefit type. BenefitType and
EventType are part of the AOM TYPE OBJECT pattern.
These generic classes can be specialized for a particular customer. In our example, the
customer defined two kinds of Benefits: FreeSms and
FreeAstrologicalPrediction; and three kinds of Events: SmsEvent,
TopupEvent, and VoiceCallEvent. FreeSms is activated by an SmsEvent and
grants the user with free units of Sms sends. The number of units is defined in the
Benefit class. FreeAstrologicalPrediction is activated by a
TopUpEvent and grants the user with free units of daily predictions. For simplicity we
omit details such as the conditioning for granting benefits, e.g., “send 200 text messages
7
this week and get 100 more sends for free.” In addition, the customer took advantage of
the AOM architecture and added a CAP instance variable in
FreeAstrologicalPrediction. This variable is used by the customer to set a
special cap on the number of free units that limits the overall cost of the benefit to the
service provider in keeping with the marketing budget for this campaign.
At runtime, instantiations of specific benefits represent specific products. For example,
100FreeSms sets the benefit to 100 units of free SMS's.
WeekFreeAstrologicalPrediction sets the benefit to 7 days of free
predictions with a maximum cap of 1904 days total for all users collectively to keep
within a $10K budget.
Upgrade Scenario
The next version of the AOM system needs to incorporate two improvements
(Figure 5). First, the cardinality of the association between BenefitType and
EventType needs to change from 1:1 to 1:M. In this new version each benefit may be
triggered by several event types. Second, to improve budget control, a new monetary
CAP is introduced in the generic framework (in the Benefit class), along with a
costPerUnit (in BenefitType). When granting benefits to end users, the system
checks that the CAP is not exceeded, otherwise it fails the grant. This is implemented
in the new version of the Benefit class.
Next we consider the conflicts that may occur during the upgrade and their resolution.
Generic upgrade scripts: The cardinality change is a change to the type of benefits, thus
affecting FreeSMS and FreeAstrologicalPrediction. This is resolved by
providing an upgrade script that converts an event type value to an event type list
containing a single event type value. The script is provided as part of the generic system
upgrade:
if (oldInstance.getTriggerType() != null) {
newInstance.getTriggerTypes().add(oldInstance.getTriggerType());
}
The generic product can contain a library of recurring refactoring scenarios and their
solutions (e.g., property rename).
Customer upgrade scripts: Not all upgrade scenarios can be resolved automatically. In
some cases specific business logic should be applied in order to resolve the conflict. For
example, the cost-per-unit modification cannot be resolved generically. The new
property is mandatory and therefore must be initialized. Therefore, the customer needs
to supply the cost on case-by-case basis. This is achieved by defining a placeholder
script that the customer team needs to implement. The addition of the CAP variable
clashes with pre-existing customizations performed to the model by the customer.
Specifically, the addition conflicts with the CAP introduced earlier by the customer. To
resolve this conflict, the customer needs to rename the original CAP to oldCAP and
provide an upgrade script that calculates the CAP value for existing model instances.
This can be accomplished by multiplying the oldCAP by the costPerUnit and the
number of units.
newInstance.setOldCAP(oldInstance.getCAP());
8
newInstance.setCAP( oldInstance.getOldCAP() *
oldInstancenewInstance.getType().getCostPerUnit() *
oldInstance.getUnits());
The FreeSMS is simpler: the customer just needs to define a default value.
Although the example presented here is of an application domain change, the pattern
applies as well to core AOM architecture changes, such as changing the serialization
format of AOM objects.
Figure 5. New Version of the AOM Model.
9
Alternative Solution
The core solution presented in this pattern is based upon lazily evolving the metadata as
the system evolves; primarily for modifying systems that need to be running 24x7.
However there is a very common alternative (or possible pattern) to the core solution
mentioned in this paper that can be used when you need to “Evolve your Metadata” and
you don’t have such execution constraints. This is to suspend your AOM system and
completely migrate all of your metadata before resuming the system.
This alternative is premised on the ability to completely shut down the running AOM
system. This allows for a script to completely evolve all of the metadata before
bringing your AOM system back online. This solution has the benefit of allowing the
metadata to always be consistent with the AOM and will also perform better as the
system will not need to invoke checks and hooks in order to possibly evolve the
metadata as it is loaded and instantiated. There are definitely tradeoffs to either solution
but the core solution mentioned in this paper requires interesting enhancements to the
core of the AOM architecture, while this alternative solution does not affect the
architecture of the AOM.
Resulting Context
High Availability: Changes to the AOM architecture can be made without having to
stop a running system and then evolve all the previous metadata and data at the
same time.
Amortized Cost: The overhead of converting data to the new AOM architecture
occurs incrementally.
Automatic Migration: Default migration can be provided for the changes to the core
AOM metadata and architecture using generic scripts. An AOM system may be
upgraded from a much earlier version to the latest version without going through the
intermediate versions
Reuse: Generic upgrades can be provided by the upgrade framework (e.g., for
rename property).
Usability: Users are supported in resolving version conflicts.
Higher Complexity: The AOM Builder code is more complex as hook methods and
evolution code will become part of the Builder.
Performance Hit: The AOM Builders can be slower at first as the upgrade to the
metadata is invoked. Running version checks every time an entity is created can
degrade performance.
Fragility: Cached AOM data may need to be flushed to force data evolution of
existing in-memory AOM instances.
Related Patterns
History Patterns [AND98] allows for different version of the rules to be available thus
going back in time but can add a lot of complexity to the system. Also, in some
systems you only want the current version of the AOM architecture and rules.
AOM BUILDER [WYW09] is an evolution of the BUILDER [GoF95] pattern.
10
AOM Builder is used to invoke the Upgrade Scripts [HNS+10].
Customer upgrade scripts are implemented using the PLACEHOLDER PATTERN [FJS99].
HOOKS [FPR01] are invoked in the AOM loader to enable customers to provide default
values for AOM attributes when migrating to a new version.
Known Uses
Pontis Ltd. (www.pontis.com) is a provider of Online Marketing solutions for
Communication Service Providers. Pontis’ Marketing Delivery Platform allows for on-
site customization and model evolution by non-programmers. The system is developed
using ModelTalk [HLPS09] based on AOM patterns. Pontis’ system is deployed in
over 20 customer sites including Tier I Telcos. A typical customer system handles tens
of millions of transactions a day exhibiting Telco-Grade performance and robustness.
There is a mutation technique developed for migrating objects in Smalltalk-80 [CW86]
that is a known use of the core of this pattern. These authors made an enhancement to
Smalltalk that changed how class redefinitions worked. The original Smalltak-80
system mutated all existing instances to use the new representation whenever a class
was redefined. To eliminate the overhead of this bulk object mutation, the new approach
used lazy mutation. Lazy mutation defers object mutation until it is actually used in a
computation. Lazy mutation is accomplished by replacing the method dictionary of the
old class definition with a dictionary which defines only the message doesNotUnderstand:.
In addition, the superclass of the old class definition is set to nil and a reference to the
new class definition is stored within the old definition. When a message is sent to such
an instance of the old class, a response to the message will not be found and the
doesNotUnderstand: method will be activated. This method contains the code to mutate the
instance into an instance of the new class. The difference between this approach and the
pattern described in this paper is that lazy mutation only applies a single evolution to an
object, not a chain of replacements.
An AOM system developed for the Illinois Department of Public Health used the
alternative solution mentioned above. There are also many other well-known uses of
the alternative solution done by some of the authors.
11
References
[AOM] Adaptive Object-Models. http://www.adaptiveobjectmodel.com
[And98] Anderson, F. A Collection of History Patterns. Proceedings of the 6th
Pattern Language of Programs Conference (PLoP 1998), Monticello,
Illinois, USA, 1998.
[Ars00] Arsanjani, A. Rule Object Pattern Language. Proceedings of the 8th
Pattern Language of Programs Conference (PLoP 2000). Technical
Report WUCS-00-29, Dept. of Computer Science, Washington
University Department of Computer Science. (2000).
[BMR+96] Buschman, F., Meunier, R., Rohnert, H., Sommerlad, P., and Stal, M.
Pattern Oriented Software Architecture, Volume 1: A System of
Patterns. Wiley & Sons. 1996.
[BR98] Bäumer, D. , D. Riehle. Product Trader. Pattern Languages of
Program Design 3. Edited by Robert Martin, Dirk Riehle, and Frank
Buschmann. Addison-Wesley, 1998.
[CW86] Caudill, P., Wirfs-Brock A. "A Third Generation Smalltalk-80
Implementation.", p. 119-130, OOPSLA '86 Conference Proceedings,
Portland Oregon, September 29-October 2, 1986.
[FCW08] Ferreira, H. S., Correia, F. F., and Welicki, L. 2008. Patterns for data
and metadata evolution in adaptive object-models. Proceedings of
the 15th Conference on Pattern Languages of Programs (Nashville,
Tennessee, October 18 - 20, 2008). PLoP '08, vol. 477. ACM, New
York, NY, 1-9.
[Fow97] Fowler, M. Analysis Patterns: Reusable Object Models. Addison-
Wesley. 1997.
[Fow02] Fowler, M. Patterns of Enterprise Application Architecture.
Addison-Wesley. 2002.
[FPR01] Fontura, M., Pree, W., Rump, B. The UML Profile for Framework
Architectures. Addison-Wesley. 2001.
[FY98] Foote B, J. Yoder. Metadata and Active Object Models. Proceedings
of Plop98. Technical Report #wucs-98-25, Dept. of Computer
Science, Washington University Department of Computer Science,
October 1998.
[GoF95] Gamma, E., R. Helm, R. Johnson, J. Vlissides. Design Patterns:
Elements of Reusable Object Oriented Software. Addison-Wesley.
1995.
[HNS+10] Atzmon Hen-Tov, Lena Nikolaev, Lior Schachter, Joseph W. Yoder,
Rebecca Wirfs-Brock. Adaptive Object-Model Evolution Patterns,
SugarLoafPLoP 2010. To appear.
[HLPS09] Atzmon Hen-Tov, David H. Lorenz, Assaf Pinhasi, Lior Schachter:
ModelTalk: When Everything Is a Domain-Specific Language, IEEE
Software, vol. 26, no. 4, pp. 39-46, July/Aug. 2009.
[Jon99] Jones, S. A Framework Recipe. Building Application Frameworks:
Object-Oriented Foundations of Framework Design. Edited by
Fayed, M., Johnson, R., Schmidt,.D. John Wiley & Sons. 1999.
12
[JW98] Johnson, R., R. Wolf. Type Object. Pattern Languages of Program
Design 3. Addison-Wesley, 1998.
[KJ04] Kircher, M.; P. Jain. Pattern Oriented Software Architecture, Volume
3: Patterns for Resource Management. Wiley & Sons. 2004.
[KSS05] Krishna, A., D.C. Schmidt, M. Stal. Context Object: A Design
Pattern for Efficient Middleware Request Processing. 13th Pattern
Language of Programs Conference (PLoP 2005), Monticello, Illinois,
USA, 2005.
[Mar02] Martin, R. Agile Software Development: Principles, Patterns, and
Practices. Prentice Hall, 2002.
[RFBO01] Riehle, D., Fraleigh S., Bucka-Lassen D., Omorogbe N. The
Architecture of a UML Virtual Machine. Proceedings of the 2001
Conference on Object-Oriented Program Systems, Languages and
Applications (OOPSLA ’01), October 2001.
[RTJ05] Riehle D., M. Tilman, and R. Johnson. Dynamic Object Model.
Pattern Languages of Program Design 5. Edited by Dragos
Manolescu, Markus Völter, James Noble. Reading, MA: Addison-
Wesley, 2005.
[RY01] Revault, N, J. Yoder. Adaptive Object-Models and Metamodeling
Techniques Workshop Results. Proceedings of the 15th European
Conference on Object Oriented Programming (ECOOP 2001).
Budapest, Hungary. 2001.
[WYWJ07] Welicki, L.; J. Yoder; R. Wirfs-Brock; R. Johnson. Towards a
Pattern Language for Adaptive Object-Models. Companion of the
ACM SIGPLAN Conference on Object Oriented Programming,
Systems, Languages and Applications (OOPSLA 2007), Montreal,
Canada, 2007.
[WYW09] Welicki, L.; J. Yoder; R. Wirfs-Brock. Adaptive Object-Model
Builder. 16th Pattern Language of Programs Conference (PLoP 2009),
Chicago, Illinois, USA, 2009.
[WYW07] Welicki, L, J. Yoder, R. Wirfs-Brock. Rendering Patterns for
Adaptive Object Models. 14th Pattern Language of Programs
Conference (PLoP 2007), Monticello, Illinois, USA, 2007
[YBJ01] Yoder, J.; F. Balaguer; R. Johnson. Architecture and Design of
Adaptive Object-Models. Proceedings of the ACM SIGPLAN
Conference on Object Oriented Programming, Systems, Languages
and Applications (OOPSLA 2001), Tampa, Florida, USA, 2001.
[YJ02] Yoder, J.; R. Johnson. The Adaptive Object-Model Architectural
Style. IFIP 17th World Computer Congress - TC2 Stream / 3rd
IEEE/IFIP Conference on Software Architecture: System Design,
Development and Maintenance (WICSA 2002), Montréal, Québec,
Canada, 2002.
[YR00] Yoder, J.; R. Razavi. Metadata and Adaptive Object-Models.
ECOOP Workshops (ECOOP 2000), Cannes, France, 2000.