edi to xml using xml to take the headaches out of edi processing blair schneider mckay clareos, inc....

EDI to XMLUsing XML to take the headaches out of EDI

processing

Blair Schneider McKayBlair Schneider McKay

Clareos, Inc. Clareos, Inc. http://www.clareos.com

[email protected]

Blair Schneider McKay

Edi To Xml – Slide 2 Oct 21, 2004

Agenda

1.1. Project MotivationProject Motivation

2.2. EDI BasicsEDI Basics

3.3. Syntactic Conversion: EDI to XMLSyntactic Conversion: EDI to XML

4.4. Semantic Conversion: XML to XMLSemantic Conversion: XML to XML

5.5. Validating the Input DataValidating the Input Data

6.6. Storing and Using the ResultsStoring and Using the Results

7.7. System InstrumentationSystem Instrumentation

8.8. Q & A Q & A



Project Motivation

AT&T Custom User SolutionsAT&T Custom User Solutions (ACUS) (ACUS) niche telecomm biller for colleges, military bases, niche telecomm biller for colleges, military bases,

state governments, other AT&T units, etc.state governments, other AT&T units, etc.

small development group. XP and agile development.small development group. XP and agile development.

at issue was one of our VAS offerings: reconcile client at issue was one of our VAS offerings: reconcile client facility bills and help them address discrepancies.facility bills and help them address discrepancies.

The Issue: The Issue: Reconciliation was done by manual comparison Reconciliation was done by manual comparison of electronic usage records to paper invoices. This was costly of electronic usage records to paper invoices. This was costly and slow, and it created an extra quality-control burden.and slow, and it created an extra quality-control burden.



Agenda








8.8. Q & A Q & A



Electronic Data Interchange early alternative to facsimile and postal mailearly alternative to facsimile and postal mail standards-basedstandards-based

- ANSI / ASC X12 / DISA- ANSI / ASC X12 / DISA- UN/EDIFACT- UN/EDIFACT- - http://www.disa.orghttp://www.disa.org

hardware, data, software, and network specshardware, data, software, and network specs ““VANs”, mailboxes, wire-transfer protocols, etc.VANs”, mailboxes, wire-transfer protocols, etc.

widespreadwidespreadDISA estimates over 300,000 companies use it.DISA estimates over 300,000 companies use it.

becoming more integrated with XMLbecoming more integrated with XMLExample: “Electronic Business XML” (ebXML)Example: “Electronic Business XML” (ebXML)



Sample EDI DocumentISA~00~ ~00~ ~01~00001123

~ZZ~MICHIGANSTA ~040103~1950~U~00401~000001029~0~T~>GS~CI~006968523CRI~123456XYZ~20040407~1527~128~X~004010ST~811~000000001BIG~20040407~734555121299920040407~~~~~DIREF~EH~773REF~12~7345551212999REF~AP~7345551212999ITD~18~3~~~~20040421...snip...

SE~17921~10290002GE~2~128IEA~1~000001029

Modeled on samples in:Modeled on samples in:

http://www.sbc.com/Common/PDF/811v4010_CRIS-SW_CABS-SW+W_SBCT_IG.pdfhttp://www.sbc.com/Common/PDF/811v4010_CRIS-SW_CABS-SW+W_SBCT_IG.pdf



EDI Technical Summary

data element typesdata element typesnumeric, decimal, identifier, string, date, timenumeric, decimal, identifier, string, date, time

delimitersdelimitersdata-element-separatordata-element-separator

first character after “ISA”, often “~”first character after “ISA”, often “~”

sub-element-separatorsub-element-separator

first character of 16first character of 16thth ISA data element ISA data element

segment-terminatorsegment-terminator

character after the sub-element-separatorcharacter after the sub-element-separator

hierarchieshierarchiessegments nest in “loops”segments nest in “loops”

ISA

GS

ST

Details

SE

GE

IEA



Agenda








8.8. Q & A Q & A



Why XML?

• The 811 document is already hierarchical.The 811 document is already hierarchical.

• It’s It’s easiereasier. Many companies find EDI too . Many companies find EDI too expensive, too complex, and too inflexible.expensive, too complex, and too inflexible.

• Once something is in XML format, you can Once something is in XML format, you can “go anywhere” with it.“go anywhere” with it.

By converting EDI to XML you trade complex, By converting EDI to XML you trade complex, “single-use” tools for generic XML tools like DTD, “single-use” tools for generic XML tools like DTD,

W3C Schema, XSLT, SOAP, DOM, SAX, etc. W3C Schema, XSLT, SOAP, DOM, SAX, etc.



More Motivation…

USD SegmentIf USD08 is populated, then the amount in USD08 is the segment amount.If USD08 is not populated, then the following determination is made to determine thesegment amount:If USD05 = “M4”, then the amount in USD06 is the segment amount.If USD05 “M4”, then the product of USD03 and USD06 is the segment amount.If USD01 = “O”, then the segment amount is for information purposes only.The USD segment can be used to provide taper table information for WATS billing. Inthis use, USD01 would equal “O”.Note 1: When both USD03 and USD08 are populated, the amount in USD08 takesprecedence and becomes the segment amount.Note 2: In order to verify amount in USD08, it is necessary to multiply USD03 * USD06* USD07 * Percent of Interstate Usage. The Percent of Interstate Usage canbe found in an SI pair (DE 1000 = ‘PU’) within the SLN loop, when it is present.

From: http://www.sbc.com/Common/PDF/811v4010_CRIS-SW_CABS-SW+W_SBCT_IG.pdf, p6.2.8



Sample EDI Document - RevisitedISA~00~ ~00~ ~01~00001123

~ZZ~MICHIGANSTA ~040103~1950~U~00401~000001029~0~T~>GS~CI~006968523CRI~123456XYZ~20040407~1527~128~X~004010ST~811~000000001BIG~20040407~734555121299920040407~~~~~DIREF~EH~773REF~12~7345551212999REF~AP~7345551212999ITD~18~3~~~~20040421...snip...

SE~17921~10290002GE~2~128IEA~1~000001029

Modeled on samples in:Modeled on samples in:

http://www.sbc.com/Common/PDF/811v4010_CRIS-SW_CABS-SW+W_SBCT_IG.pdfhttp://www.sbc.com/Common/PDF/811v4010_CRIS-SW_CABS-SW+W_SBCT_IG.pdf



Raw XML<HL> <HL01>4</HL01> <HL02>3</HL02> <HL03>8</HL03> <HL04>1</HL04> <SLN> <SLN01>5.0</SLN01> <SLN02></SLN02> <SLN03>I</SLN03> <SLN04>M4</SLN04> <SLN05>11.85</SLN05> <SLN06></SLN06> <SLN07>I</SLN07> <SLN08>SV</SLN08> <SLN09>0705</SLN09>HL~4~3~8~1

SLN~5.0~~I~1~M4~11.85~~I~SV~0705

Edi2Xml.exe



The Edi2Xml Utility

Sample init file:Sample init file:

Nesting ISA 1

Nesting IEA 1

Nesting GS 2

Nesting GE 2

Nesting ST 3

Nesting SE 3

Nesting_FieldComp HL 4 3

Single *

class Segment {class Segment {

public:public:

enum Relationship { child, sibling, uncle };enum Relationship { child, sibling, uncle };

virtual Relationship comp( const Segment& other) const;virtual Relationship comp( const Segment& other) const;

// ...// ...

};};



Agenda








8.8. Q & A Q & A



XSLT Examples

<xsl:template match="ST[ST01='811']">

<Invoice>

<xsl:apply-templates/>

</Invoice>

</xsl:template>

<xsl:template name="conv_access_type">

<xsl:param name="cd"/>

<xsl:choose>

<xsl:when test="$cd='81'">Local</xsl:when> <xsl:when test="$cd='82'">IntraState/InterLata</xsl:when>

<xsl:when test="$cd='83'">IntraState/IntraLata</xsl:when>

...snip...



<HL> <HL01>4</HL01> <HL02>3</HL02> <HL03>8</HL03> <HL04>1</HL04> <SLN> <SLN01>5.0</SLN01> <SLN02></SLN02> <SLN03>I</SLN03> <SLN04>M4</SLN04> <SLN05>11.85</SLN05> <SLN06></SLN06> <SLN07>I</SLN07> <SLN08>SV</SLN08> <SLN09>0705</SLN09>

Translated XML – v.1

HL~4~3~8~1SLN~5.0~~I~1~M4~11.85~~I~SV~0705

Edi2Xml.exe

XSLT

<HL idx='4' parent='3' level='8' haschildren='yes'> <SLN id1='5.0' relcode='I' qty='1' type='money' included='yes' service-rendered='yes' svc_type='0705'> 11.85 </SLN> ...snip...



Translated XML – v.2<HL> <HL01>4</HL01> <HL02>3</HL02> <HL03>8</HL03> <HL04>1</HL04> <SLN> <SLN01>5.0</SLN01> <SLN02></SLN02> <SLN03>I</SLN03> <SLN04>M4</SLN04> <SLN05>11.85</SLN05> <SLN06></SLN06> <SLN07>I</SLN07> <SLN08>SV</SLN08> <SLN09>0705</SLN09>HL~4~3~8~1

SLN~5.0~~I~1~M4~11.85~~I~SV~0705

Edi2Xml.exe

XSLT

<GroupLevelCharge> <Charge serviceType='Measured Service‘> $11.85 </Charge> ...snip...



Agenda








8.8. Q & A Q & A



Validating the Input Data

EDI transaction sets contain checksums.EDI transaction sets contain checksums.• Control codes in matched segments (ISA/IEA, ST/SE, Control codes in matched segments (ISA/IEA, ST/SE,

GS/GE) must line up.GS/GE) must line up.

• Some segment-types (like HL) have global sequence Some segment-types (like HL) have global sequence numbers, parent-references, nesting indicators:numbers, parent-references, nesting indicators:

HLHL~~4545~~3737~~55~~00

HLHL~(~(my idmy id)~()~(my parent’s idmy parent’s id)~()~(my levelmy level)~()~(I have I have childrenchildren))

• Optional transaction totals:Optional transaction totals:

CTTCTT~~33

CTTCTT~(~(# of preceding # of preceding IT1IT1, or line-item, segments, or line-item, segments))



Validating the Input Data – With XML!

1. Hook into existing EDI validation rules using time-1. Hook into existing EDI validation rules using time-tested XML methods:tested XML methods:

• DTDDTD• W3C XML Schema LanguageW3C XML Schema Language• SchematronSchematron

2. Create sys-admin dashboards. 2. Create sys-admin dashboards. E.g.., express this in XSLT:E.g.., express this in XSLT:

display in bold-red typedisplay in bold-red type:: any loops whose instance-id is not ( any loops whose instance-id is not (previous-idprevious-id+1)+1)



Agenda








8.8. Q & A Q & A



Storing and Using the Results

EDI to XML makes great sense.EDI to XML makes great sense.

But EDI to XML to But EDI to XML to RDBMSRDBMS can be tricky. can be tricky.

If the goal is to produce a document (like a detailed If the goal is to produce a document (like a detailed invoice) just use XSLT to produce HTML, PDF, etc.invoice) just use XSLT to produce HTML, PDF, etc.

In many cases, you should just insert into the In many cases, you should just insert into the RDBMS only the values and labels you care about, RDBMS only the values and labels you care about, along with a unique id pointing to the (saved) XML.along with a unique id pointing to the (saved) XML.



Agenda







7.7. System InstrumentationSystem Instrumentation (Demo) (Demo)

8.8. Q & A Q & A

Q & A

Blair Schneider McKayBlair Schneider McKay

Clareos, Inc. Clareos, Inc. http://www.clareos.com

[email protected]

Source Code for the Edi2Xml tool, and bothSource Code for the Edi2Xml tool, and bothXSLT and document samples are available at: XSLT and document samples are available at: http://home.comcast.net/~blairschneidermckahttp://home.comcast.net/~blairschneidermckay/y/

edi to xml using xml to take the headaches out of edi processing blair schneider mckay clareos, inc....

Documents

xml example

blair schneider mckayedi

edi basics

input data

sample edi document

syntactic conversion

semantic conversion

agile development