lotusphere 2006 ad212 introduction to dxl

AD212Introduction to DXL

Dick AnnicchiaricoSoftware EngineerIBM Lotus Domino

Agenda

What is DXL?

How do I use it?

What can it do for me?

What is DXL?

DXL is XML

DXL is the Domino XML Language

DXL is an XML representation of Domino data

To understand DXL, you need to understand the basics of XML

What is XML?

XML: eXtensible Markup Language

XML is a markup language

A markup language is a text format

has special syntax (“markup”) for the structure and meaning of the data

also represents the data itself

Markup languages are popular because they are plain text

they are easy to use

parsers and tools are usually widely available

Key XML Concepts

XML represents data and structure, not presentation/formatting HTML can define both

XML is not a single language, it provides for a family of languages XML spec. defines syntax and structure rules, but no specific markup

application designer defines specific markup

that is why XML is extensible

the number of XML vocabularies is unlimited

Markup Example (HTML)

<html> <body> <h2> This is a level 2 heading. </h2> <p> This is a paragraph with some <i>italic</i> text. </p> </body></html>

Some HTML markup specifies presentation details. Here, <i> specifies italic text.

The actual data is shown in color

The rest is markup HTML spec. defines this markup

Markup Example (XML)

<?xml version="1.0" encoding="utf-8"?><presentation> <slide title="Markup Example”> <bullet>Captivating point 1.</bullet> <bullet>Captivating point 2.</bullet> </slide></presentation>

The actual data is shown in color

The rest is markup XML spec. does not define this markup This is a hypothetical XML “vocabulary” that I made up

XML Basic Constructs

XML declaration

Elements

Attributes

Text Content

XML Declaration

Specifies the version and encoding of the XML document

Latest XML version is 1.1 but 1.0 is still the most widely used

utf-8 and utf-16 are the most popular character encodings

Elements

Begin with a start tag (<name>) and end with an end tag (</name>)

Have content between the start and end tags

Describe a hierarchical structure of the data

XML spec. does not define the element (tag) names; the developer of the XML vocabulary does

A DTD or schema formally defines the XML vocabulary – what it can and can’t contain

Root Element

The entire XML is called the XML document

An XML document has exactly one root element

The root element is also called the document element

Attributes

Have the syntaxattribute-name=“attribute-value”

Value contains additional data for the element

XML spec. does not define attribute names; the developer of the XML vocabulary does

Use of attributes versus elements is a design trade-off; could have done:<title> Markup Example</title>

Text Content

Text that is between an element’s start and end tags

Comprises most of the real data in typical XML documents

<bullet> element contains text

Does <slide> element contain text?

A Quick Note on White Space

Maybe.• Yes, if schema allows for mixed

content (elements and text) which is fairly rare

• No, if schema allows for element content only

• Something to be careful about!

Is this white space significant?

It is convenient to indent

your XML to reflect thestructure, but don’t do itwithin elements that

allowmixed content or textcontent only (like

<bullet>)

Tree Structure of XML

presentation

bullet

presentation

bullet

If <slide> does not allow mixed

content

If <slide> allows mixed content (text

and elements)

A Quick Note on DTDs and Schemas

• <presentation> is the root element• it contains a sequence of <slide> elements• a <slide> element has

– a title attribute that takes a text value– A sequence of <bullet> elements

• a <bullet> element contains text

Pseudo-code for a DTD/schemafor the above XML

Document Type Definitions (DTDs) and XML Schemas are ways to define an XML “vocabulary”

XML schemas are themselves XML documents and allow for more rigid definition than DTDs

Well-formed vs. Valid XML

<?xml version="1.0" encoding="utf-8"?><presentation> <slide title="Markup Example”> <bullet>Captivating point 1.</bullet> </slide></mess>

<?xml version="1.0" encoding="utf-8"?><presentation> <slide title="Markup Example”> <callout>Look here!</callout> </slide></presentation>

<?xml version="1.0" encoding="utf-8"?><presentation> <slide title="Markup Example”> <bullet>Captivating point 1.</bullet> </slide></presentation>

This XML is not well-formed. The <presentation> start tag

does not have a matching end tag.

This XML is well-formed butnot valid. Our hypothetical

schema does not allow a <slide> to contain a <callout>.

This XML is both well-formedand valid

You now know all of the important

things about XML. So… What is DXL?

You Already Know A Lot About DXL

Earlier, we learned that:

DXL is the Domino XML Language

DXL is an XML representation of Domino data

Now that you understand the fundamentals of XML:

You understand the fundamentals of DXL as well!

But you still need to learn what data can DXL represent? what are the tools for processing DXL and how do I use them? what could I use DXL for?

What Data Can DXL Represent?

Documents including rich text

Most design notes form, subform, page, frameset, view, folder, shared field, shared actions,

image resource, applet resource, agent, script library, database script, Help Using document, Help About document, data connection

Database properties

Database ACL

Special objects profile document

agent data

Notes catch-all for things that don’t yet have a DXL representation

About the Examples

Examples are in a database Baseball2004.nsf Major League Baseball batting statistics for the year 2004

See the Help About document for instructions

Available on DeveloperWorks about one month after Lotusphere http://www-10.lotus.com/ldd/sandbox.nsf/ViewByConferencesNonJava

Examples assume input files are in C:\ls06

Presentation focus is on concepts, not fine details

For more info, please visit the Meet the Developers lab Dolphin Asia 1

A First DXL Example

Open the baseball database in the Designer

Select a form and do Tools – DXL Utilities – Viewer

Look at the DXL for the form

Repeat the exercise for other design notes

How do I use DXL?

Tools for Processing XML and DXL

Use Tools

Export some data to XML format

Application-specific(e.g. DXL Exporter)

Store it Files and Databases

Send it Network protocols

Parse and process it DOM

Transform it XSLT

Import some data from XML format

Application-specific(e.g. DXL Importer)

XML DXL

How do I use XML?First of all:

DOM: Document Object Model

presentation(DOM Element)

slide(DOM Element)

bullet(DOM Element)

Captivating point 1.(DOM Text)

Captivating point 2.(DOM Text)

DOM Parser

Markup Example(DOM Attribute)

Application-Specificresults

SAX: Simple API for XML<?xml version="1.0" encoding="utf-8"?><presentation> <slide title="Markup Example”> <bullet>Captivating point 1.</bullet> </slide></presentation>

SAX Parser

Start Element: presentation

Start Element: slide(with attribute list)

End Element: slide

Text: Captivating point 1.

End Element: bullet

Start Element: bullet

End Element: presentation

Application-Specific

EventProcessing

XSLT: Extensible Stylesheet Language: Transformations

XSLT Processor

A set of instructions for transforming the data:• Match patterns in the input XML• Generate output data

XML document XSLT style sheet

Often another XML file, but can be HTML or any

How do I use DXL?Now back to:

Tools for Processing DXL

Designer Tools – DXL Utilities menu

Web server URL command: ?ReadViewEntries

XML APIs – DOM, SAX, XSLT

DXL APIs – Export, Import

Text processing tools, editors, search, diff, etc. depending on the character encoding of XML documents

utf-8 may be your best bet

Third-party XML tools (e.g. XML Spy)

Designer Tools – DXL Utilities menu

Exporter: exports the design note to a file in DXL format

Viewer: exports and displays the DXL

Transformer: exports and does an XSLT transformation

Example: Tools – DXL Utilities – Exporter

Export PlayerBasic form to file C:\ls06\PlayerBasic.xml

Open and view the file in your favorite text editor

Tips on Using DXL and Other XML Files

Name DXL files with .xml extension for XML-aware tools

Some editors may display “newline” characters strangely

Lines can get very long in XML check for editor “wrap lines” option

tends to make “diff” tools less effective

DXL Exporter inserts newlines to help alleviate this

IE and other tools automatically indent

Consider a real XML editor such as XML Spy

Windows Notepad can edit utf-16 (Unicode) files

Example: Tools – DXL Utilities – Transformer

Sample result on right

FormFieldSummary.xslt is the XSLT style sheet that is the “program” for the transform Only about 20 lines of code XSLT is a powerful tool

Select one or more forms

Select style sheet FormFieldSummary.xsl

Domino Web Server Command: ReadViewEntries

Request view entries in DXL format with URL command:http://server-name/db-name/view-name?ReadViewEntries

Append optional URL arguments for more control over the data:&argument-name=argument-value

starting and ending points

number of entries to return

sorting by column

restrict to a single category

Try these examples now in a browser:

http://server/Baseball2004.nsf/basicStats?ReadViewEntrieshttp://server/Baseball2004.nsf/basicStats?ReadViewEntries&ResortDescending=3

LotusScript APIs

Export, Import, XSLT, SAX, DOM

Java APIs - Domino

Export, Import, XSLT, DOM

DxlExporter DxlImporter DocumentgenerateXML

Session

ItemparseXML

transformXML

EmbeddedObjectparseXML

transformXML

MIMEEntityparseXML

transformXML

RichTextItemparseXML

transformXML

•Export via DxlExporter or Document.generateXML

•Parse with DOM using parseXML methods in conjunction with org.w3c.dom.Document classes

•Process with XSLT using transformXML methods

Any industry standard XML Java APIs can be used also!

C APIs

Export, Import, XSLT

Export Import XSLTDXLCreateExporterDXLDeleteExporterDXLExportACLDXLExportDatabaseDXLExportIDTableDXLExportNoteDXLGetExporterPropertyDXLSetExporterProperty

DXLCreateImporterDXLDeleteImporterDXLGetImporterPropertyDXLImportDXLSetImporterProperty

XSLTAddParameterXSLTCreateTransformXSLTGetTransformPropertyXSLTSetTransformPropertyXSLTTransformXSLTTransformDeleteTransform

Conceptual View of DXL Exporter

Exporter

Input Object

Output Object(contains DXL)

•database•document•document collection•note collection

Options•schema/DTD•bitmap to GIF•force note format

•string•richtext item•stream•other XML processor

Conceptual View of DXL Importer

Importer

Input Object(Contains DXL)

•database

Options•create/replace/update•create FT index•schema/DTD validation•replace db properties•replica req’d for replace/update

•string•richtext item•stream•other XML processor

List ofnote IDs

OutputObject

Tips on API Language Trade-offs

LotusScript has the most options for output objects

LotusScript supports pipelining output from one XML processor “pipes” directly into the input of another

LotusScript and Java DOM object models are different can currently do more with the Java API

C API input and output DXL is provided using callback functions

C API doesn’t support SAX or DOM

Generally, use the language that you are most accustomed to.

If you are “language-agnostic”, here are some considerations.

What Can DXL Do For Me?

Some Uses For DXL

Reporting

Integration with other apps

Archival and restoring

Replacing or updating notes

Web applications

Source control and comparison

Transforming DXL to other XMLs

Transforming DXL to other formats

Creating design and documents from scratch

DXL is an Alternative to Other Mechanisms

DXL is not necessarily the best choice LotusScript, Java, or C APIs may do the job easier

Some cases where DXL might be the way to go are when DXL can accomplish something the other APIs cannot

when you’re working with other XML formats

when you want to share data in a platform-independent format

for a document-centric rather than API-centric processing model

Let’s Look at Some API Examples

Examples are all agents in the Baseball2004 database

Available on the Actions menu as shown on the left

All of these agents use the LotusScript or Java APIs shown in earlier slides

Example: Archival and Restoring

This Java agent imports the DXL into a new database, DXLCopy.nsf

Open DXLCopy.nsf and verify contents

This Java agent exports the entire database to DXL

Results in file DXLCopy.xml

Delete DXLCopy.nsf if it exists

The Java DXL Exporter Code from Action 1

public class JavaAgent extends AgentBase{

public void NotesMain(){

Session session = getSession();AgentContext agentContext = session.getAgentContext();Database db = agentContext.getCurrentDatabase();

String filename = "C:\\ls06\\DXLCopy.xml";Stream stream = session.createStream();if (stream.open(filename)){

stream.truncate(); // Any existing file is erasedDxlExporter exporter = session.createDxlExporter();System.out.println("Exported " +

stream.writeText(exporter.exportDxl(db)) + " bytes to " + filename);

Tips on Archival and Restoring With DXL

Not the best solution if 100% full fidelity is required

DXL does not retain signatures

DXL does not decrypt/encrypt documents stores encrypted data in Base64

DXL does not represent some special database objects e.g. unread lists

DXL strives for 100% fidelity elsewhere

Replacing Notes With DXL

Currently, the PlayerBasic form looks like this

I’d like to improve the look: player name in blue space between table border and text silver background on heading row

But how do I know what to change in DXL? could become a DXL DTD/schema expert or, could just learn by example

Probably for the DXL power user

Figuring Out How to Specify Changes in DXL

See the view: 4. Rich Text Sandbox contains documents that use form Rich Text Example first document is similar to PlayerBasic form second document is what I want it to look like

Try this now: select the two documents and run

3. Export selected documents to file(s) use a diff utility to compare them determine the DXL changes that are

necessary for changes on left

Import Example: Replace Form

We determined that the changes needed are: add color=‘blue’ on <font>s preceding first and last name

add <table> columnspacing=‘.05in’

R7 only: change <table> refwidth from ‘3.0in’ to ‘3.3in’ (or remove)

add bgcolor=‘silver’ to <tablecell>s in the first <tablerow>

Try this now:

Make the above changes to PlayerBasic.xml

Run 4. Import: Replace form (PlayerBasic.xml)

See that the form has changed

Import Example: Create View

Suppose we want to programmatically create a view similar to 2. Basic Statistics with an additional column for At Bats

Try this now:

Tools – DXL Utilities – Exporter on view: 2. Basic Statistics save as BasicStatsAB.xml

Edit BasicStatsAB.xml in a text editor change <view> name to 3. Basic Statistics (w/ AB) and alias to

basicStatsAB copy and paste the Homerun <column> and its contents change new <column>’s itemname and title, and the <formula> to AtBats

Run 5. Import: Create view (BasicStatsAB.xml)

Use the new view in the Notes client

Tips For Importing Design Notes

DXL Importer ignores design notes by default use DesignImportOption to create or update

DXL Importer does not sign design notes sign programmatically or with Admin client

Importer provides imported note IDs

Use a REPLACE option to replace an existing design note match by unid attribute (Universal Note ID), or

match by name/alias + language + note type

May need ReplicaRequiredForReplaceOrUpdate = False

LotusScript Code for Import/Replace demoDim session As New NotesSession

Dim db As NotesDatabase

Dim importer As NotesDXLImporter

Dim stream As NotesStream

Dim filename As String

Set stream = session.CreateStream

filename = "C:\ls06\PlayerBasic.xml"

stream.Open(filename)

Set importer = session.CreateDXLImporter

Call importer.SetInput(stream)

Set db = session.CurrentDatabase

Call importer.SetOutput(db)

importer.DesignImportOption = DXLIMPORTOPTION_REPLACE_ELSE_IGNORE

Call importer.Process

Programmatic Examples of Modifying the DXL

We cheated on the last example we did the DXL Import programmatically

but we changed the DXL by hand

We could have changed the DXL programmatically using DOM

These are advanced examples for you to look at later Agent A1. Change and replace form with Java and DOM

Agent A2. Change and create view with LS and SAX

View 5. Additional info – C API XSLT utility

Updating Notes With DXL

Pretend David Ortiz really hit 50 homeruns

Import this DXL in UPDATE mode to perform the update

<document><noteinfo

unid='2FA75A266F437F58852570D7000BA960'/>

The DXL unid is used to locate an existing document

Only items specified in DXL are updated, others are unchanged

DocUpdate.xml

Update/Replace Document Demo

Select David Ortiz document from 2. Basic Statistics view

Run 3. Export selected documents to file(s) This writes a file named after the document’s note ID

copy this file to Doc.xml

Review Doc.xml and DocUpdate.xml if desired

Run 6. Import DocUpdate.xml (update mode) David Ortiz changes from 41 to 50 homeruns

Run 7. Import Doc.xml (replace/create mode) David Ortiz changes back from 50 to 41 homeruns

Tips For Importing Documents

DXL Importer creates new documents by default use DocumentImportOption for update or replace

update/replace matches on unid attribute

Richtext DXL has full support

learn by example

attachment/graphic data is in Base64 format, decoders readily available

use <attachmentref> path to import attachments directly from the filesystem

ConvertNotesBitmapsToGIF Exporter option

Learn DTD/schema if producing DXL from scratch

Transforming DXL to Other XMLs Select some documents

Run 8. Export selected documents (PlayersDxl.xml)

Look at the results in a text editor

“But I’d like to use all this data in a spreadsheet that supports XML”

XSLT Processor

XML formatreadable by

DXL containing multiple documents

Custom written XSLT style sheet

•This is a great place to use XSLT – XML to XML transforms

•But how do I know how to transform between the formats?

How To Transform DXL to Excel XML Format

Learn Excel’s XML format the easy way create a minimal spreadsheet by hand

save it as an XML spreadsheet

peruse the resulting .xml file

Map the DXL data to the Excel data see DXL data in PlayersDxl.xml

see simple Excel data in OnePlayerExcel.xml

DXL to Excel XML Mapping

DXL Excel XML

XSLT Style Sheet pseudo-code

Copy boilerplate content (from OnePlayerExcel.xml) that precedes and follows the actual workbook data

<Row> For each <document>, generate a <Row>

<item> <Cell> For <item>s of interest, generate a <Cell>

Generated any computed <Cell>s

Armed with this data, I created an XSLT style

sheet to convert a set of DXL documents to an

Excel XML spreadsheet.See

PlayersDxlToExcel.xslt

Transforming DXL to Excel Demo

Run 9. Export/Transform selected documents (PlayersExcel.xml)

Inputs are PlayersDxl.xml and PlayersDxlToExcel.xslt

Result is PlayersExcel.xml open this file in Excel

Tips On Transforming Between XMLs

Learn XSLT

Buy books to help you

Develop your XSLT style sheets in little baby steps

DOM or SAX can be used as well

Don’t implement your own XML parsing

Want More?

Meet the Speakers – Europe 8

Meet the Developers lab – Dolphin Asia 1

BP305: Combine XSL and DXL for Rich Web Apps, Wed. 1:30 – 2:30

BP313: Practical DXL for LotusScript, Wed. 4:15 – 5:15

Lotus Developer Domain online forumshttp://www-10.lotus.com/ldd/nd6forum.nsf/

This presentation available in one month athttp://www-10.lotus.com/ldd/sandbox.nsf/ViewByConferencesNonJava

Email Dick Annicchiaricodannicch@notesdev.ibm.com

Thank You For Attending!Please complete the evaluation form.

lotusphere 2006 ad212 introduction to dxl

italic text

Technology

prolifics at ibm lotusphere 2012

open dxl idea book - mcafee dxl idea book 1 open dxl idea...

lotusphere istanbul

lotusphere 2012: social business example - safebook

lotusphere 2012 ad105

lotusphere 2011 show104

lotusphere 2012 - the social business update

lotusphere comes to you 2011

lotusphere 2009 review 2009 01 26

lotusphere 2006 bp305 combine xsl and dxl for rich web...

lotusphere 2006: id107 - getting started with active...

dxl reference manual

lotusphere 2012 - mobility

lotusphere asean march 2011

2011 lotusphere presentation_signature_mortgage

lotusphere 2007 ad506 intermediate lotusscript

ibm alloy - lotusphere presentation

leverage dxl

ad104 - dxl re-invigorated

dxl reference manual v9.1