initial data load extension module webinar february 4th, 2009

Initial Data Load

Extension Module Webinar

February 4th, 2009

Initial Data Load Module Webinar-Openbravo, Adrian Romero & Mikel Irurita

Agenda

Functionality and Demo 15 min.

Process and Tools 10 min.

Development Technique 5 min.

Q & A 30 min.


Agenda

Functionality and Demo

Process and Tools

Development Technique

Q & A


Goal:Provide a set of automated data import processes to quickly load core ERP

data into Openbravo.QuickStart customers: predefined templates (simplicity).Support full “new-install-to-production-ready” instance.

Functional HighlightsInitial Data Load

Solution Approach:Deliver as a extension module. Included in QuickStart.Spanish Translation.

Benefits: Enable end users to manage the process.Minimize considerably time of migration.Ensure fully-integrated set of validated ERP data.Complete and detailed documentation.



Supported entities:– Products– Price Lists– Bank Accounts– Business Partners– Open Payables

Data validation prior to import. Concise, easy-to-understand error log. Default value definition. REST Web service interface.

Simple and effective process

Define CSV ValidateConfiguredefault values

Import

– Open Receivables– Assets– Journal Entries– Standard Cost– On hand quantity / Stock


Functional Highlights

Fill several Openbravo entities from one CSV file

Business partners

Initial Data Load

Business partners

Addresses

Contacts

Templates: Simplicity.Intermediary stepFix structure.


DEMO

Initial Data Load Demo



Future plans.Initial Data Load

More entities to import:Balance sheetPayrolls.

More integration capabilitiesImport from other data formats. (XML, Excel, ...)Import from other third party applications.

Improvements for consultants: Flexibility: ability to customize templates adding new fields.


Agenda


Process and Tools


Q & A


Process and tools. Extract, Transform and Load.Initial Data Load

Extract Transform and Load (ETL) takes part part in:* Synchronization or replication of databases* Right-time or batch exchanges of data* Data warehousing* Data migration* Data quality

Extracting data from outside sources

Transforming it to fitoperational needs(which can includequality levels)

Loading it intothe end target (database or data warehouse)

Extract Transform and Load (ETL) is a process in database usage and especially in data warehousing that involves:


Process and tools. Load process structureInitial Data Load

Talend job* CSV file parsing* Data validation

Openbravo load process& REST web service* Data transformation* Data load using DAL

Extracting Transforming Loading

Validate button

Process button


Process and tools: Talend Open Studio overviewInitial Data Load

Graphical designer

Open source

Metadata-driven solution

Talend Open Studio is a project for data integration based on Eclipse RCP

Operates as a code generator

Large list of integration components


DEMO

Process and tools: Talend Open Studio demonstration

Quick Talend Open Studio review.

Initial Data Load



Process and Tools


Q & A

Agenda


Development technique. REST web services for Openbravo ERP.Initial Data Load

REST stands for Representational State Transfer. It relies on a stateless, client-server, cacheable communications protocoland in virtually all cases, the HTTP protocol is used.

Definition of a REST web service:* The base URI: for example http://example.com/resources* The MIME type of the data supported by the web service, XML, JSON, ...* The set of operations supported using HTTP methods. POST, GET, PUT, DELETE


Development technique. REST web services for Openbravo ERP.Initial Data Load

POST /openbravo/ws/org.openbravo.idl.process/Costing HTTP/1.1Host: localhost

<?xml version="1.0" encoding="UTF-8"?><ob:Openbravo xmlns:ob="http://www.openbravo.com"> <Costing> <Product>rma</Product> <Cost>5.5</Cost>(...)

response

request

<?xml version="1.0" encoding="UTF-8"?><ob:Openbravo xmlns:ob="http://www.openbravo.com" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <MaterialMgmtCosting id="FF808181266B867701266B8E6AE4008B" identifier="Raw material A Thu Jan 01 00:00:00 CET 2009"> <id>FF808181266B867701266B8E6AE4008B</id> <creationDate transient="true">2010-01-26T17:54:40.616Z</creationDate>(...)


Development technique. Code snipets, defining a new web serviceInitial Data Load

<?xml version="1.0" encoding="UTF-8" ?><provider> <bean> <name>process</name> <class>org.openbravo.idl.service.IdlWebService</class> <singleton>true</singleton> </bean></provider>

package org.openbravo.idl.service;

public class IdlWebService implements WebService {

@Override public void doPost(String path, HttpServletRequest request, HttpServletResponse response) throws Exception {

final String firstsegment = WebServiceUtil.getInstance().getFirstSegment(path); final VariablesSecureApp vars = new VariablesSecureApp(request); final ConnectionProvider conn = new DalConnectionProvider();


Development technique. Reading XML from request.Initial Data Load

final SAXReader reader = new SAXReader();final Document document = reader.read(request.getInputStream());

// check that the rootelement is the openbravo onefinal Element rootElement = document.getRootElement();if (!rootElement.getName().equals(XMLConstants.OB_ROOT_ELEMENT)) { throw new OBException("Root tag of the xml document should be: " + XMLConstants.OB_ROOT_ELEMENT + ", but it is " + rootElement.getName());}

// check that there exit one element to import.if (rootElement.elements() == null || rootElement.elements().size() != 1) { throw new OBException("Root tag of the xml document should have one child element.");}

Element entityelement = (Element) rootElement.elements().get(0);


Development technique. Writing XML to response.Initial Data Load

BaseOBObject obj = service.doInternalProcess(impent.getValues());

try { final StringWriter sw = new StringWriter(); final EntityXMLConverter exc = EntityXMLConverter.newInstance(); exc.setOptionEmbedChildren(true); exc.setOptionIncludeChildren(true); exc.setOptionIncludeReferenced(false); exc.setOptionExportClientOrganizationReferences(true); exc.setOutput(sw); exc.process(obj); String xml = sw.toString();

response.setContentType("text/xml"); response.setCharacterEncoding("UTF-8"); response.setHeader("Content-Encoding", "UTF-8"); final Writer w = response.getWriter(); w.write(xml); w.close();} catch (final Exception e) { throw new OBException(e);}


DEMO

Development technique. REST Web Service demonstration

REST Web Service execution

Initial Data Load



Wrap-Up

Process and Tools



Q & A

Interesting Links: http://forge.openbravo.com/projects/xidl

http://forge.openbravo.com/projects/idlspanish

Next Webinar:18th February Modularity Status Update

Achievements in modularity during 2009 and roadmap -in regards of modularity- for 2010

[email protected]

Initial Data Load

[email protected]

initial data load extension module webinar february 4th, 2009

Documents

initial data load extract

data warehousing

data migration

data quality extracting

data formats

load etl

data warehouse extract

core erp data