cp-53: how to create effective data models to …...tuesday, 10:00-11:00 –gainesville 1-2 mehul...

CP-53:How to Create Effective Data Models to Support Your Dashboards and Reports

Keving Wang, Justin Sommer , Deltek

CONFIDENTIAL © Deltek, Inc. All Rights Reserved. 2

» Framework Manager Models - CQM vs. DQM – What it is and why it matters

» Data Module and Data Sets – Power your Dashboards without using Framework Manager

Session Overview

CER Role Capabilities


Capability Consumer CER

User

Advanced

CER User

CER

Developer

CER/BI

Admin

Web Admin

Run/Subscribe

ReportsX X X X X

Dashboard (View) X X X X

Dashboard (Create) X X X X

Report Authoring X X X

Data Module X X X

Event Studio X X X

Framework Manager X X

Admin Console X X


Relevant Roles for this Session» CER Developer & CER Admin

» IT

» Maybe ….. Advanced CER User

FM Models CQM vs DQMPerformance Improvement!

Compatible Query Mode (CQM)

»


CQM» Reports using the compatible query mode can benefit from new features implemented in

other components of the Cognos BI software stack, for example, RAVE visualizations. Do not

expect any enhancements to query generation or data processing for the compatible query

mode.

» CQM supports OLAP data sources such as Cognos Powerplay Cubes

»


Dynamic Query Mode (DQM)

DQM Details » Bursting reports: In CQM, when there is a master-detail relationship, CER will run the detail query for each row returned by the master query. This can result in the detail query being run a large number of time.

» Governor in FM named ‘(DQM) Master-Detail Optimization’: When this governor is set to ‘Cache Relational Detail Query’ then the DQM engine is able to process master-detail relationship more efficiently than they can be in CQM.

» 64 bit JDBC driver required (no OLE DB Driver needed)

» Java processes in CER 7.2.x (Cognos 11.0.13)

» Content manager & application server - the one java process you would see in Cognos 8.

» Query Service (XQE) to support Dynamic Query Mode - this was added in Cognos 10.

» Client container to support the new portal/dashboards/story boards/data modules/interactive


DQM Best Practices and Benefits• DQM is highly recommended when starting new projects (Deltek new

models are all DQM)

• Enabling DQM on existing projects is an optional migration (Deltek Legacy is still CQM) – Test all your reports!

• DQM performance may or may not be better

• Modifications may or may not be required for any report

• Complex reports might need modifications

• DQM enforces certain best practices that are optional in CQM

• DQM will not make certain assumptions that CQM does

• DQM may apply a different order of operations or join path resolution, which can potentially cause changes in a report output

• Deltek is getting different behaviors when switching existing models to DQM, make sure you do your testing before the switching.


DMR (Anyone? )


Data Module & Data Sets

Data Options

CONFIDENTIAL © Deltek, Inc. All Rights Reserved.13

Costpoint DB*

Packages

Data Module

Dashboards, Reports, Explorations (CER8)

Excel Upload*

Data Sets


Data SetsData sets can be used to gather a collection of items that you use frequently

Before We Get Started on the Data Module,Let’s Talk about…

» When defining a data set you

choose columns from a

package or data module…

» Apply filters to reduce the data

and summarize the data then…

» The data set is cached meaning

query performance is

improved

What Advantage Do Data Sets Give Me?



» Ctrl-Alt-M when

opening dataset

» Filter down data to

what you want to

analyze

» Schedule data set

refresh times

Modifying Datasets

Data Sets w/ Parquet

» Data sets use Parquet as columnar storage format

» Data set is stored in Content Store DB in compressed

and encoded to reduce the amount of storage; option to

store outside content store for growth management

» Parquet is one of the latest technologies used for

efficiency in data storage and scan/analytics, used

heavily in big data such as Hadoop and twitter; it is a

technology with improved read performance at the cost

of slower writes.

» It is a stable technology introduced in 2016 Oct.

» Excel upload in latest Cognos 11.1.x is loaded into

Parquet format as well

http://en.wikipedia.org/wiki/Column-oriented_DBMS

Data Sets Best Practice•Save frequently calculated expressions as columns.

This practice reduces the amount of expression evaluations at run time. Projecting, comparing, and

sorting simple column references and simple values (literals) is more efficient than evaluating

expressions.

•Avoid storing large numbers of columns that are never used by queries.

While data is both compressed and encoded to reduce the amount of storage, it’s still recommend to

avoid storing redundant or unnecessary columns.

•Use the sort option when creating a data set with column that is most frequently used in filters.

For large data sets, sorting the input can enhance the evaluation of predicates. Sorting the data on

the common column that is used in a filter, for example Country or Store, groups rows with the same

value. If a query includes predicates on that column, the query can determine more efficiently which

blocks of data it can ignore as it navigates the data.


Performance Testing with Data Sets

» Access via Browser (including Mobile)

» Easy to use for connecting different data sets together

» Map data from multiple sources

» Deltek Sources (Data Server currently on–premise only)

» Existing Framework Manager models

» Create calculations and more

» Uploaded Excel files (currently on premise only)

Data Module

» Combine data sets and Excel

sheets together with a data module

» Create the appropriate join

» Visualize the relations between

your different data objects

» And publish for others to use

Putting It All Together With Data Modules



» Data group on a numeric

column, by default ranges of

the groups will have equal

distribution

» Text column will allow you to

add in the data values to your

own groups

» Allows you to create buckets

that you can report on your

data like Quarters to Months

» Create rollups that do not exist

in Costpoint like Alt Proj

Data Groups in Data Modules


» Create your own drill

down path like PM to

Project

» Can leverage in reports

or dashboards

» Create multiple paths

across different data

fields

Navigation Paths

Navigation Path Setup

» Clean up blank values in your

data

» Create calculations so they can

be easily used on reports and

dashboards

» Note that Data Module features

will all be improved in CER 8

(CA 11.1)

Data Cleaning and Calculations in Data Modules


In CER 8- Let AI do its job!


Build an Awesome Dashboard in Seconds!


Data module and data sets» Future AI ability

» Better engine for query dataset, optional dedicated

query service

» The Query service advanced settings are used to

configure the Compute service that is used to process

data from uploaded files and data sets.

» The following Query service advanced settings can be

specified:

» qs.queryExecution.flintServer.maxHeap (Specifies the

maximum amount of memory that the Compute

service is allowed to use)

» qs.queryExecution.flintServer.sparkThreads


» Generate One Data set without security and set it to traverse only; Embed data module with

user security to filter dataset when user query the dataset; Schedule dataset refresh time by

trigger or time;

Data Module and Data Sets With Security

Notebook delivering a dataset

•Let users understand the underlying logic of “Code”. This is the opposite of how we usually think about

code. It is a series of “cells” containing executable code, or mark ups

•Notebooks can be saved and easily shared among users

•Underlying Python language has a standard library, and numerous third party libraries yielding a vast

array of existing code base and examples.


Less is Faster

Tuesday, 2:10 – 3:10 – Gainesville 1-2CP-52: Costpoint Business Intelligence: Upgrading from CER 7.1 to CER 7.2.3

Tuesday, 3:20-4:20 – Naples 1-2CP-53: Report Authoring Tips and Tricks in Costpoint Business Intelligence CER 7.2.3

Tuesday, 4:30-5:30 – Naples 1-2CP-55 Costpoint Business Intelligence CER 7.2.3- Executive Dashboards and Financial Reporting

Wednesday, 10:00-11;00 – Naples 3CP-56: As a Costpoint Business Intelligence CER End User, How Do I Leverage the New Features After We Upgrade?

Wednesday, 11:10-12:10 – Naples 3CP-53: Data Modeling-How to Create Effective Data Models to Support your Dashboards and Reports

Wednesday, 2:10 – 3:10 - Naples 3CP:54: Designing and Creating Information-Rich Interactive Dashboards in Costpoint Business Intelligence CER 7.2.3

Wednesday, 3:20 – 4:20 - Naples 1-2CP-50: Costpoint Business Intelligence Roadmap

Thursday 9:40-10:40 – Sarasota 1-2CP-59: Administration for Costpoint Business Intelligence CER 7.2.3 - Setting Up The New Security Model and How to Effectively Guard Your Data

Deltek BI Sessions

Tuesday, 10:00-11:00 – Gainesville 1-2 Mehul Desai, PCI

CP100: A Whole New World of Business Intelligence

Wednesday, 4:30-5:30 – Sarasota 1-2 – Hari Sury, Sury Analytics

CP-57:Analytics for GovCons - A Roadmap To Success

Thursday, 8:30-9:30 – Sarasota 1-2 – Rich Hardin, VSE

CP-51:Deltek Costpoint Business Intelligence (CER) Leveraging Analytics: The Old, The new And

Everything In Between

Customer/Partner BI Sessions

cp-53: how to create effective data models to …...tuesday, 10:00-11:00 –gainesville 1-2 mehul...

Documents