Beyond the simple Pipeline - managing processes over time
XML Prague , 17/06/2007
Geert Bormans, independent [email protected]
[email protected] realising the value of your content
Introduction
Who am I? Background
10 years team-lead and researcher researchgroup document architectures Catholic University Leuven
5 years VP professional services British/Canadian XML software vendor
Independent Consultant Mainly projects that surface the intrinsic value of (yet existing)
content Migrations, ‘legacy’ conversions, XML transformations Content Management, content reuse e-publishing
[email protected] realising the value of your content
Introduction
What to expect? A quick introduction of the problem space Introduction of the project Introduction of the solution A word about pipelines Challenges for building pipeline solutions The solutions Architecture Wider applicability and future plans A quick demo
So we all love XML pipelines by now…
Ever thought about managing them?
[email protected] realising the value of your content
Managing Pipelines - Problem space
[email protected] realising the value of your content
The project
The customer A financial institution working for the goverment Publishing yearly statements New legislation requires archiving the statements
Current Architecture Complex Monolithic Code Growing with yearly changes
[email protected] realising the value of your content
The project – Current Architecture
[email protected] realising the value of your content
Example Statement
[email protected] realising the value of your content
Next years Example Statement
Publisher company rebrand
ID added to statement
[email protected] realising the value of your content
Original Request for Proposal
[email protected] realising the value of your content
Managing XML instead of PDF ...
[email protected] realising the value of your content
... and version managing the pipeline
Each asset in the CMS is linked (through metadata) to a specific version of the pipeline, hence assuring the possibility to reconstruct a statement as it was printed before
[email protected] realising the value of your content
Pipeline benefits
Break up complexity Simpler to develop Easier to debug and maintain
Clear assignment of responsibilities Mix technologies easily Reuse components
[email protected] realising the value of your content
Pipeline solutions - challenges Execution
Fit components together and Compile Pipeline Execute with high performance in a robust way
Avoiding intermediate (de)-serialisation Allow for mixed technologies
Different technology per component possible Easy creation of pipeline Manageability
[email protected] realising the value of your content
Solutions Architecture
[email protected] realising the value of your content
Pipeline Component Managment System Processing steps are URI-addressable resources Steps pulled together at run time to create pipeline Execution by Pipeline Execution Engine Datamodel for simple pipeline
[email protected] realising the value of your content
Pipeline Execution Engine
Thin layer on top of NetKernel Resource oriented computing platform Microkernel that schedules resource requests Dynamic compilation of various scripting
languages Strong support for XML processing tasks Pipeline language: DPML
[email protected] realising the value of your content
DPML Example
[email protected] realising the value of your content
Meeting the challenges
Pipeline execution by NetKernel Unnecessary serialisation/deserialisation
avoided by notion of aspects and transreptors NetKernel supports pipeline components
developed in various technologies PCMS serves as abstraction layer between
pipeline development and execution PCMS takes care of pipeline version
managment
[email protected] realising the value of your content
Wider Applicability
Architecture provides version managment of pipeline components
Most important factor of complexity of XML processing is variation in time
Applications layered on this architecture will be better protected against damage by changes
A solution worthwhile considering for any complex XML processing application
[email protected] realising the value of your content
Future Plans
Non lineair pipelines (forking, looping) Dashboard for monitoring
Timings, breakpoints Import filters for
XProc Stylus Studio pipes
...
Questions and Discussion
Thanks for your attention