what you need to know about reporting & bi for ax2012 & d365

What you need to know about Reporting & BIfor AX2012 & D365

Gina PabalanEdgewater Fullscope, Data & Analytics PracticeManaging Director, Business Intelligencehttps://www.linkedin.com/in/ginapabalan/Mobile: 727-224-1245

https://www.linkedin.com/in/ginapabalan/

Edgewater Fullscope Overview

Strategy• BI Roadmap Strategy• Executive Facilitation• Data Requirements• Analytics “Stories”

• Agile Methodology• HPO Framework

Analytics• Data Visualization• Reporting • Dashboards• Training

• PowerBI

Data• ETL• Data Warehousing• Data Modeling• Data Migration

• ETL Automation tools• SSIS/SSAS

Edgewater Fullscope delivers innovative Microsoft ERP, CRM and BI solutions and services on

premise or in the cloud in North America and Europe. The award-winning company is one of the

largest resellers of Microsoft Dynamics 365 (formerly Dynamics AX and CRM).

Data & Analytics Practice

Reporting & BI Technology Stack

• PowerBI, Enterprise Portal/SharePoint, Mobile, Desktop, Social Collaboration, Alerts, Cloud vs on-prem

• Enterprise Dashboards, Metrics/KPIs, ScorecardsPerformance

Management

• Analytical, Interactive Data Visualization, Time Intelligence, Statistical, Predictive, AI (PowerBI, Excel, PowerView)

Analytics

• Governed Corporate, Ad-hoc, Operational (SSRS, Excel)Reporting

• SQL Relational DB, OLAP/SSAS, Tabular, cloud vs on-premData

• Data Marts, Perspectives, SSISDiscovery & Integration

• Network, Servers, Cloud vs On-Prem, SecurityInfrastructure

Delivery

FRO

NT

END

BA

CK

EN

D

Microsoft Tools Line Up

BACK END FRONT END

InteractiveVisualization

PowerBI (.pbix)

MobileReports

Mobile Report Publisher (.rsmobile)Formerly Datazen

PaginatedReportsSQL Server Reporting Services (SSRS)

AnalyticalReports

Excel (.xlsx)

Aggregated Data Models (OLAP, Tabular)

SQL Server Analysis Services (SSAS)

Data Consolidation & Transformation (ETL)SQL Server Integration Services (SSIS)

Data Warehouse

Transactional Data (Source)

SQL Database(Relational, Normalized)

SQL Database(Modeled, Transformed)

When you are considering reporting & analytics…. 2 questions…

How and where is the data sourced??

How will the information be authored and

delivered

• Transactional Application• Data Mart• EDW• SSAS multi dimensional• SSAS tabular

BI in Terms of Dynamics AX2012 and Microsoft….

Information within transactional ERP systems is highly normalized to make transactional processing more efficient and deliver necessary scalability to enterprise operations.

Reporting & Analytics Tool(FRONT END)

MR Data MartAX 2012

(6,400 tables)

Operational reports

SSAS20+ “OLAP

CUBES”

MDX

Management Reporter

2

Data Source(BACK END)

3

1

Ro

le C

ente

rs(S

har

ePo

int)

7

Expansion of Tables in AX

Transactional ERP Data vs. Prepared Data Set

Customer Information – Prepared Data >> ie, Data Mart or Data Warehouse

Customer Information - Transactional

Dynamics comes with out-of-box Reporting & Analytics. Do I need more??

fewOOB

many

AX 2009 or earlier

AX 2012D365 Finance &

Operations

OOB

Version of AX?

Reporting & Analytics

Requirements?

Multiple Systems?

No, or limited OOB

OOB

yes

Multiple Systems?

No

OOB

yesOOB

Upgrading Soon?

yesOOB

No

Prepare data to serve the organizations unique business reporting needs, not the needs of ERP transactions.

Authoring reports against the AX2012 transactional database can be complicated and inefficient, requiring a highly experienced AX resource with knowledge of the underlying database.

Once those reports have been authored, AND AS DATABASES GROW, running those reports can consume significant system resources, and have a negative performance impact on your production ERP system.

Its very easy – even for an experienced AX resource - to write a query against the AX2012 transactional database that is not optimized for system performance.

Merge/prepare legacy ERP data with current AX data for seamless reporting to the business (upgrades, acquisitions, new ERP).

Access for those who consume the data who do not have regular access to AX2012.

For AX2012, you will need at minimum a Data Mart (Simplified Data Model)

Fullscope’s Guiding Principles for delivering Reporting & BI with AX2012 clients…

Leverage as much of the Microsoft stack as possible. Customers have invested in Microsoft and Dynamics and the return on that investment should be fully exploited.

Stay away from anything proprietary – avoid a “black box” approach.

Deliver “baseline” capability that is not over-engineered, something the customer can leverage and easily modify and extend to make their own. “Out-of-the-box” BI solutions are conceptually appealing, however in reality, BI is quite unique to each customer.

Solution must enable the customer to be self-sufficient and not highly dependent on consultants for its long term care and feeding. It is expected that the customers’ reporting and BI solution will evolve over time, and they need to be able to drive that independently.

Understand the right front end reporting tool is one a specific user will use. If that is PowerBI, Excel or other – the right BI solution needs to support and enhance all.

Must be affordable. Most AX2012 customers are mid-market manufacturing organizations and do not have the budget for massive BI investments.

Fullscope’s Response…. Reporting and BI Accelerator

This is about managing the Back End data set, by delivering a SIMPLIFIED DATA MODEL of the AX database –accelerating ability to deliver vital information to the business… for reporting & analytics!

Simplified Data Model

Reporting & BI Accelerator

Unstructured Data

D365

AX 2009 AX 2012

Other SourceTransactional

Systems

BACK END FRONT END

Enterprise Data Warehouse

SQLRDW

SSAS

Baseline AX Data Warehouse

SSAS

SSAS

ETLAutomation

AX Adapter

D365 Adapter(ERP & CRM)

Non-proprietary – leverage

ANY front end

tool

Operational Reporting &

Analytics

D365 FO

D365fO - Data & Analytics Framework

A printed document, typically requiring transaction processing as a precursor (invoice, BOL)

Operational Data Visualization and Interaction

ManagementReporter

Financial Reporting

System of Intelligence

ManagementReporter

Data Mart Entity Store

Dynamics 365for Finance & Operations

Dynaway EDGE SKG

Data Entities

Dat

a En

titi

es


Dynaway EDGE SKG

D365Something

More

Low

Data Complexity

High

Single Source

Data Diversity

Multi Source

Reporting

Analytics/BI

Will we need something more??

Interacting with D365 Data (from outside the dotted line)


ManagementReporter


Data Entities

Dat

a En

titi

es


Dynaway EDGE SKG

OData (Small Data Sets)

DIXF Framework(large data sets)

System of Integration

Beware!System access to the DB is available in NON Production D365 environments

D365fO supports a system of integration with following capabilities:

OData v4 (Read/Write)

D365fO data are securely exposed via theODatav4 endpoint for interactive read/writeoperations that involve small batches. WhileODatav4 endpoints can be used for reportingwith PowerBI and other tools, for largevolume reporting BYOD is recommended.

JSON service framework

JSON is an efficient protocol to exchange data between interactive applications. While you would use the same endpoint for accessing Entities as you did with ODatav4 (small batch), you have the ability to invoke business events and actions in addition to accessing data.

Data Import/Export framework (DIXF)

You can use DIXF functionality via the Data Management workspace or programmatically using APIs for Large Batch import and export operations.

Bring your own Data Store (BYOD):

D365fO entities can be incrementally exported into your own Data Store/staging database as frequently scheduled batch transfers. BYOD is built using DIXF framework and supports Large Batch exports. Designed to support integration with data warehouses.

OData interface limits the number of records extracted at a time to 10k rows as of platform update7.

SMA

LL D

ATA

SET

SLA

RG

E D

ATA

SET

S

….The importance of preparing your data to support Reporting & Analytics

An important topic many overlook……

ManagementReporter


Data Entities

Dat

a En

titi

es


Dynaway EDGE SKG

Microsoft prepares data for reporting & analytics

…. so should you!

*** Avoid writing complex SSRS reports against the D365 OLTP ***

But Data Preparation typically involves much more than flattening out the data….

Basic transformations:

– De-normalizing

– Cleaning

– Deduplication

– Format revision

– Standardization

– Restructuring

– Conversions (ie, currency, timestamp, UoM)

Advanced transformations:

– Derivation

– Filtering

– Joining data from multiple sources.

– Splitting (explode BOM to see all components of Finished Goods)

– Data validation

– Summarization

– Aggregation

– Harmonization

o Advanced Data Transformations will be required if multiple systems are involved.o D365 Entities can only take “data prep” so far o Some of this “work” can be pushed to PBi – but there are limitations there as well

BYOD for ReportingThink of BYOD as your operational

data store for D365 data


ManagementReporter



Dynaway EDGE SKG

Data Entities

Dat

a En

titi

es

D365 BYOD

Allows SQL Queries

Note:• Microsoft Entities were developed for

data import, not reporting• Current product development is

focused on the Entity Store, not BYOD

System of IntegrationSystem of Intelligence

And if an Enterprise Data Warehouse is needed….

D365 BYOD

Data Entities

Enterprise Data Warehouse(Governed Global Data Source)

Other System of Record

Legacy ERP

MRA

llow

s SQ

L Q

uer

ies

Dynamics 365for Finance &

Operations

Data Entities

Entity Store

Data Mart

So, how do I know what I really need ?

..and avoid making short term decisions at the expense of long term?

….after all, we are implementing a new ERP system!

How do I avoid chasing butterflies?

How do I manage expectations and cost?….and avoid surprises

Start with…. Building LIST of reporting and analytics requirements to support your

D365 implementation For each requirement, identify the Business Owner and collect the following

information……

Reporting Requirements for Phase I ERP Implementation


(Visualization)


Does this report require the use of data sets over 10,000 records?

Does this report require complex calculations and/or significant use of cross functional data.


Is all the required data contained within D365?

Does the report require a mash up of data sourced from multiple sources?

Identify the additional sources


How frequently will the report be run:o Multiple times throughout dayo Dailyo Monthlyo Yearly


How “current” must the data be:o Up to the minuteo 15 minute lag acceptableo 4 hour lag acceptableo EoD currency acceptable

Now we can start answering these questions…

How will the information be Delivered ??

Where is the Data sourced?

Deciding where best to meet a reporting need….


D365 BYOD

Data Entities



Legacy ERP

ManagementReporter

Allo

ws

SQL

Qu

erie

s


Data Entities

Entity StoreData Mart

1

2

3 4

5

6

If a Customer requires a Data Warehouse, consider…..

Fullscope’s Reporting & BI Accelerator


D365 BYOD

Data Entities



Legacy ERP

ManagementReporter

Allo

ws

SQL

Qu

erie

s


Data Entities

Entity StoreData Mart

D365 Adapter(ERP & CRM)

ETL Automation

DataStageBYOD


BI Monitor

Data Dictionary

Features of the ETL Automation Tool

• Auto-generates consistent best practice SSIS and SQL code (non-proprietary)

• Pulls data from most any data source

• Data Lineage / Impact Analysis Diagrams

• Automated documentation

• Full-featured security

• Flexible load scheduler

• Analysis Services support - cube and tabular

• Reusable code library

• Dynamics AX and D365 Adapter

• Creates enum tables for easy description lookups

• View D365 table relationships

• View D365 field descriptions

• Team development environment

• Version Control

• Slowly changing dimensions

• Simplified DW Promotion (Dev/QA/Prod)

• Fields and Lookups are linked with metadata rather than fieldname/tablename

• Renaming objects has zero impact

• Embedded DW and OLAP best practices

• Tool is also being used for data migration during your D365 implementation

Leverage Fullscope’s Accelerator as Migration Tool to support the D365 Rollout

White Paper available on the topic

D365

D3

65

En

tityData Hub

LEGACYERP (s)

ExtractImport

ie, MicrosoftSQL ServerIntegrationServices(“SSIS”)

Transform

D3

65

Sta

gin

gD365 Entity Template

Fullscope’s Accelerator

Microsoft’s Data Management Framework

LoB System

ROI for Fullscope’s Accelerator

• ETL Automation tool will reduce data preparation time by 50-75%

• Baseline D365 enterprise data warehouse & BYOD saves over 1,200-1,600 hours from Day 1

• Beyond Reporting: ETL Automation tool will simplify and reduce time spent preparing legacy data for migration to D365 (50-75% more efficient).

Thank you!

Questions?

- Speak to your Fullscope Account team

- Contact us at [email protected]

Reporting and BI Workshop

Deliver D365 Data & Analytics Training

• What comes with D365 - overview

• Understanding benefits of Embedded and

where to use

• PowerBI licensing considerations – various

use cases

• Different reporting scenarios

• Understand when they will need something

more

Walk through deliver discovery process for

reporting (tool shared with Consulting)

Assist in the development of reporting

approach to include:

• Embedded PBI

• PowerBI

• Management reporter

• SSRS needs

• BYOD

• BYOD with EDW

what you need to know about reporting & bi for ax2012 & d365

Documents