top 5 data architecture challenges with ron huizenga
DESCRIPTION
Watch the companion webinar at: http://embt.co/1r6IqZA Over the past 20 years, the role of data architects and modelers has changed significantly. Many initiatives are now business-driven as opposed to IT-driven, significantly changing the dynamics of solution delivery. This is compounded by complex environments consisting of a variety of solutions on disparate platforms. Corporate governance is also a growing concern, driving Data Governance, Data Quality and Master Data Management activities. Today's data architect must be prepared to address all these needs across business and IT. Join this session to learn about: - Team dynamics and changes in methodologies within IT - Defining an enterprise modeling strategy in a complex environment - Modeling techniques to help address these initiatives Learn more about ER/Studio at: http://embt.co/ER-StudioTRANSCRIPT
![Page 1: Top 5 Data Architecture Challenges with Ron Huizenga](https://reader034.vdocument.in/reader034/viewer/2022051818/549d2851ac7959c92a8b4972/html5/thumbnails/1.jpg)
EMBARCADERO TECHNOLOGIESEMBARCADERO TECHNOLOGIES
Top 5 Data Architecture
Challenges
Ron HuizengaProduct Manager – ER/[email protected]
![Page 2: Top 5 Data Architecture Challenges with Ron Huizenga](https://reader034.vdocument.in/reader034/viewer/2022051818/549d2851ac7959c92a8b4972/html5/thumbnails/2.jpg)
EMBARCADERO TECHNOLOGIES
5 Challenges
• Evolution of methodologies & culture
• Adapting to changing architecture
• Complex data environments
• Data quality
• Business focus
2
![Page 3: Top 5 Data Architecture Challenges with Ron Huizenga](https://reader034.vdocument.in/reader034/viewer/2022051818/549d2851ac7959c92a8b4972/html5/thumbnails/3.jpg)
EMBARCADERO TECHNOLOGIES
Evolution of Methodologies & Culture
3
• Traditional/Waterfall (up to 1999)– Big design up front– Rigid organizational structure
• Agile – The Rebellion? (1996 – present)– Incremental, time boxed delivery– Self organizing team (who needs roles?)
• The Hybrid (200x? – present)– Agile operational approach– Leverage individual roles/skill sets– Importance of architecture
![Page 4: Top 5 Data Architecture Challenges with Ron Huizenga](https://reader034.vdocument.in/reader034/viewer/2022051818/549d2851ac7959c92a8b4972/html5/thumbnails/4.jpg)
EMBARCADERO TECHNOLOGIES
Agile Process
4
![Page 5: Top 5 Data Architecture Challenges with Ron Huizenga](https://reader034.vdocument.in/reader034/viewer/2022051818/549d2851ac7959c92a8b4972/html5/thumbnails/5.jpg)
EMBARCADERO TECHNOLOGIES
ER/Studio – Agility Support• True multi-level sub-models• Repository
– Check-out/Check-in of sub-models– Individual data objects– Named versions– Annotation
• Reverse Engineering• Compare and Merge• Naming standards capabilities• Macros• Collaboration & publishing• Data interchange bridges
5
![Page 6: Top 5 Data Architecture Challenges with Ron Huizenga](https://reader034.vdocument.in/reader034/viewer/2022051818/549d2851ac7959c92a8b4972/html5/thumbnails/6.jpg)
EMBARCADERO TECHNOLOGIES
Adapting to Changing Architecture
• Relational database
• Object oriented
• Unstructured data
• Document management
• Service oriented architecture
• Big data
6
![Page 7: Top 5 Data Architecture Challenges with Ron Huizenga](https://reader034.vdocument.in/reader034/viewer/2022051818/549d2851ac7959c92a8b4972/html5/thumbnails/7.jpg)
EMBARCADERO TECHNOLOGIES
Service Oriented Architecture (SOA)
Publish & Subscribe
Federated Queries
Operational Data Store
Data Warehouse
Unstructured Data & DocumentsMetadata
Repository
Authentication& Security Services
Application ServerBusiness Process
ManagementBusiness Rule Management
Complex Event Processing Lifecycle Management Policy EnforcementBusiness Activity
MonitoringAdapters Communications Messaging
Metadata ManagementMaster Data Management
Transformation Canonical Model Canonical Messaging Data Integration
Data Federation Data Access Services Document ServicesContent Management
Services
Enterprise Service Bus
Mapping & Transformation
Business Objects
7
![Page 8: Top 5 Data Architecture Challenges with Ron Huizenga](https://reader034.vdocument.in/reader034/viewer/2022051818/549d2851ac7959c92a8b4972/html5/thumbnails/8.jpg)
EMBARCADERO TECHNOLOGIES
Big Data
• Volume
• Velocity
• Variety
• Veracity
8
![Page 9: Top 5 Data Architecture Challenges with Ron Huizenga](https://reader034.vdocument.in/reader034/viewer/2022051818/549d2851ac7959c92a8b4972/html5/thumbnails/9.jpg)
EMBARCADERO TECHNOLOGIES
ER/Studio – Native Big Data Support
• MongoDB– Versions 2.4 and 2.6
• Hadoop Hive– Versions 0.12 and 0.13
• Capabilities– Diagramming
– Reverse Engineering (JSON, BSON)
– DDL supported for Hive
9
![Page 10: Top 5 Data Architecture Challenges with Ron Huizenga](https://reader034.vdocument.in/reader034/viewer/2022051818/549d2851ac7959c92a8b4972/html5/thumbnails/10.jpg)
EMBARCADERO TECHNOLOGIES
ER/Studio – Big Data Notation Enhancement
• Physical Model
– Objects instead of Tables
• Nested Objects
– “Is Contained In” relationship type
10
![Page 11: Top 5 Data Architecture Challenges with Ron Huizenga](https://reader034.vdocument.in/reader034/viewer/2022051818/549d2851ac7959c92a8b4972/html5/thumbnails/11.jpg)
EMBARCADERO TECHNOLOGIES
Complex Data EnvironmentsEvolution:
• 38 years of construction
• 147 builders
• No Blueprints
• No Planning
Result:
• 7 stories
• 65 doors to blank walls
• 13 staircases abandoned
• 24 skylights in floors
• 160 rooms, 950 doors
• 47 fireplaces, 17 chimneys
• Miles of hallways
• Secret passages in walls
• 10,000 window panes (all bathrooms are fitted with windows)
11
![Page 12: Top 5 Data Architecture Challenges with Ron Huizenga](https://reader034.vdocument.in/reader034/viewer/2022051818/549d2851ac7959c92a8b4972/html5/thumbnails/12.jpg)
EMBARCADERO TECHNOLOGIES
Complex Data Landscape
12
• Proliferation of disparate systems
• ERP, mismatched departmental solutions
• SAAS (externally controlled and managed)
• Obsolete legacy systems
• Poor decommissioning strategy
• Point-to-point interfaces
• Data warehouse, data marts, ETL …
![Page 13: Top 5 Data Architecture Challenges with Ron Huizenga](https://reader034.vdocument.in/reader034/viewer/2022051818/549d2851ac7959c92a8b4972/html5/thumbnails/13.jpg)
EMBARCADERO TECHNOLOGIES
ER/Studio – Conquering Landscape Complexity• True multi-level sub-models (hierarchy)
• Reverse engineering (extensive list of platforms)
• Comprehensive metadata extensions (attachments)
• Naming standards capabilities
• Universal mappings
• Macros
• Data lineage
• Business Architect – data context (processes)
• Repository, collaboration & publishing
13
![Page 14: Top 5 Data Architecture Challenges with Ron Huizenga](https://reader034.vdocument.in/reader034/viewer/2022051818/549d2851ac7959c92a8b4972/html5/thumbnails/14.jpg)
EMBARCADERO TECHNOLOGIES
ER/Studio – Compare and Merge example
14
![Page 15: Top 5 Data Architecture Challenges with Ron Huizenga](https://reader034.vdocument.in/reader034/viewer/2022051818/549d2851ac7959c92a8b4972/html5/thumbnails/15.jpg)
EMBARCADERO TECHNOLOGIES
C&M cont.
15
![Page 16: Top 5 Data Architecture Challenges with Ron Huizenga](https://reader034.vdocument.in/reader034/viewer/2022051818/549d2851ac7959c92a8b4972/html5/thumbnails/16.jpg)
EMBARCADERO TECHNOLOGIES
16
C&M cont.
![Page 17: Top 5 Data Architecture Challenges with Ron Huizenga](https://reader034.vdocument.in/reader034/viewer/2022051818/549d2851ac7959c92a8b4972/html5/thumbnails/17.jpg)
EMBARCADERO TECHNOLOGIES
17
C&M cont.
![Page 18: Top 5 Data Architecture Challenges with Ron Huizenga](https://reader034.vdocument.in/reader034/viewer/2022051818/549d2851ac7959c92a8b4972/html5/thumbnails/18.jpg)
EMBARCADERO TECHNOLOGIES
18
C&M cont.
![Page 19: Top 5 Data Architecture Challenges with Ron Huizenga](https://reader034.vdocument.in/reader034/viewer/2022051818/549d2851ac7959c92a8b4972/html5/thumbnails/19.jpg)
EMBARCADERO TECHNOLOGIES
19
C&M cont.
![Page 20: Top 5 Data Architecture Challenges with Ron Huizenga](https://reader034.vdocument.in/reader034/viewer/2022051818/549d2851ac7959c92a8b4972/html5/thumbnails/20.jpg)
EMBARCADERO TECHNOLOGIES
20
C&M cont.
![Page 21: Top 5 Data Architecture Challenges with Ron Huizenga](https://reader034.vdocument.in/reader034/viewer/2022051818/549d2851ac7959c92a8b4972/html5/thumbnails/21.jpg)
EMBARCADERO TECHNOLOGIES
Data Quality
21
• Accuracy
• Timeliness
• Completeness
• Consistency
• Relevance
• Fitness For Use
![Page 22: Top 5 Data Architecture Challenges with Ron Huizenga](https://reader034.vdocument.in/reader034/viewer/2022051818/549d2851ac7959c92a8b4972/html5/thumbnails/22.jpg)
EMBARCADERO TECHNOLOGIES
Poor Data Quality Implications
22
• Costs a typical company the equivalent of 15% to 20% of revenue– Estimated by US Insurance Data Management Association
• Low Quality = Low Efficiency
• It is insidious – most data quality issues are hidden in day to day work
• From time to time, a small amount of bad data leads to a disaster of epic proportions
![Page 23: Top 5 Data Architecture Challenges with Ron Huizenga](https://reader034.vdocument.in/reader034/viewer/2022051818/549d2851ac7959c92a8b4972/html5/thumbnails/23.jpg)
EMBARCADERO TECHNOLOGIES
23
Space Shuttle
Challenger
QueCreek
Mine Flooding
When Data Flaws Happen…
![Page 24: Top 5 Data Architecture Challenges with Ron Huizenga](https://reader034.vdocument.in/reader034/viewer/2022051818/549d2851ac7959c92a8b4972/html5/thumbnails/24.jpg)
EMBARCADERO TECHNOLOGIES
Poor data quality isn’t a new problem
24
![Page 25: Top 5 Data Architecture Challenges with Ron Huizenga](https://reader034.vdocument.in/reader034/viewer/2022051818/549d2851ac7959c92a8b4972/html5/thumbnails/25.jpg)
EMBARCADERO TECHNOLOGIES
Mitigation Best Practice
25
• Adopt the philosophy of prevention
• Show thought leadership
• Be accountable at the points of data creation
• Measure, control, improve
• Establish data culture
![Page 26: Top 5 Data Architecture Challenges with Ron Huizenga](https://reader034.vdocument.in/reader034/viewer/2022051818/549d2851ac7959c92a8b4972/html5/thumbnails/26.jpg)
EMBARCADERO TECHNOLOGIES
Technique: Attachments (Metadata extensions)
26
![Page 27: Top 5 Data Architecture Challenges with Ron Huizenga](https://reader034.vdocument.in/reader034/viewer/2022051818/549d2851ac7959c92a8b4972/html5/thumbnails/27.jpg)
EMBARCADERO TECHNOLOGIES
Business Focus
• Governance
• Champion for data value, data quality
• Consultative approach – business acumen
• Establish a data culture
• Business collaboration
27
![Page 28: Top 5 Data Architecture Challenges with Ron Huizenga](https://reader034.vdocument.in/reader034/viewer/2022051818/549d2851ac7959c92a8b4972/html5/thumbnails/28.jpg)
EMBARCADERO TECHNOLOGIES
Addressing Governance through Models
28
Data Governance
Data Architecture Management
Data Development
Database Operations
Management
Data Security Management
Reference & Master Data Management
Data Warehousing
& Business Intelligence
Management
Document & Content
Management
Metadata Management
Data Quality Management
![Page 29: Top 5 Data Architecture Challenges with Ron Huizenga](https://reader034.vdocument.in/reader034/viewer/2022051818/549d2851ac7959c92a8b4972/html5/thumbnails/29.jpg)
EMBARCADERO TECHNOLOGIES
Data Management
29
• Shared responsibility and collaboration
– Data Management Professionals
– Business Data Stewards
• Cultural change leadership
– Adopting the principles and practices of data management requires leadership from change agents at all levels
![Page 30: Top 5 Data Architecture Challenges with Ron Huizenga](https://reader034.vdocument.in/reader034/viewer/2022051818/549d2851ac7959c92a8b4972/html5/thumbnails/30.jpg)
EMBARCADERO TECHNOLOGIES
• Powerful enterprise glossary, model & metadata collaboration
• Integrate key business terms and definitions with business systems
• View, store, and manage a single source of business definitions
• Attach business policies to daily workflows with contextual alerts and tips
Business Collaboration
30
![Page 31: Top 5 Data Architecture Challenges with Ron Huizenga](https://reader034.vdocument.in/reader034/viewer/2022051818/549d2851ac7959c92a8b4972/html5/thumbnails/31.jpg)
EMBARCADERO TECHNOLOGIES
Concluding Remarks
31
![Page 32: Top 5 Data Architecture Challenges with Ron Huizenga](https://reader034.vdocument.in/reader034/viewer/2022051818/549d2851ac7959c92a8b4972/html5/thumbnails/32.jpg)
EMBARCADERO TECHNOLOGIES
Thank you for attending!
• Learn more about the ER/Studio product family: http://www.embarcadero.com/data-modeling
• Trial Downloads: http://www.embarcadero.com/downloads
• To arrange a demo, please contact Embarcadero Sales: [email protected] 1 (888) 233-2224
32