beyond the simple pipeline - managing processes over time xml prague, 17/06/2007 geert bormans,...

Beyond the simple Pipeline - managing processes over time

XML Prague , 17/06/2007

Geert Bormans, independent [email protected]

[email protected] realising the value of your content

Introduction

Who am I? Background

10 years team-lead and researcher researchgroup document architectures Catholic University Leuven

5 years VP professional services British/Canadian XML software vendor

Independent Consultant Mainly projects that surface the intrinsic value of (yet existing)

content Migrations, ‘legacy’ conversions, XML transformations Content Management, content reuse e-publishing


Introduction

What to expect? A quick introduction of the problem space Introduction of the project Introduction of the solution A word about pipelines Challenges for building pipeline solutions The solutions Architecture Wider applicability and future plans A quick demo

So we all love XML pipelines by now…

Ever thought about managing them?


Managing Pipelines - Problem space


The project

The customer A financial institution working for the goverment Publishing yearly statements New legislation requires archiving the statements

Current Architecture Complex Monolithic Code Growing with yearly changes


The project – Current Architecture


Example Statement


Next years Example Statement

Publisher company rebrand

ID added to statement


Original Request for Proposal


Managing XML instead of PDF ...


... and version managing the pipeline

Each asset in the CMS is linked (through metadata) to a specific version of the pipeline, hence assuring the possibility to reconstruct a statement as it was printed before


Pipeline benefits

Break up complexity Simpler to develop Easier to debug and maintain

Clear assignment of responsibilities Mix technologies easily Reuse components


Pipeline solutions - challenges Execution

Fit components together and Compile Pipeline Execute with high performance in a robust way

Avoiding intermediate (de)-serialisation Allow for mixed technologies

Different technology per component possible Easy creation of pipeline Manageability


Solutions Architecture


Pipeline Component Managment System Processing steps are URI-addressable resources Steps pulled together at run time to create pipeline Execution by Pipeline Execution Engine Datamodel for simple pipeline


Pipeline Execution Engine

Thin layer on top of NetKernel Resource oriented computing platform Microkernel that schedules resource requests Dynamic compilation of various scripting

languages Strong support for XML processing tasks Pipeline language: DPML


DPML Example


Meeting the challenges

Pipeline execution by NetKernel Unnecessary serialisation/deserialisation

avoided by notion of aspects and transreptors NetKernel supports pipeline components

developed in various technologies PCMS serves as abstraction layer between

pipeline development and execution PCMS takes care of pipeline version

managment


Wider Applicability

Architecture provides version managment of pipeline components

Most important factor of complexity of XML processing is variation in time

Applications layered on this architecture will be better protected against damage by changes

A solution worthwhile considering for any complex XML processing application


Future Plans

Non lineair pipelines (forking, looping) Dashboard for monitoring

Timings, breakpoints Import filters for

XProc Stylus Studio pipes

...

Questions and Discussion

Thanks for your attention

beyond the simple pipeline - managing processes over time xml prague, 17/06/2007 geert bormans,...

Documents