rwdg slides: using agile to justify data governance

1

Copyright©2016RobertS.Seiner– KIKConsulting&EducationalServices/TDAN.com

Non-InvasiveDataGovernance™isatrademarkofRobertS.Seiner&KIKConsulting#RWDG@RSeiner

RealWorldDataGovernanceUsingAgiletoJustifyDataGovernance

MonthlyWebinarSeriesHostedbyDataversityRobertS.Seiner– KIKConsulting/TDAN.com

December15,2016– 2:00p.m.EST

2

2



• Real-WorldDataGovernance– MonthlyWebinarSeries– January19,2017– ThreeApproachestoDataStewardship– February16,2017– TheNewNon-InvasiveDataGovernanceFramework– March16,2017– ApplyDataGovernancetoAgileEfforts

– MonthlyontheThirdThursdayat2pmEST– RegisterOn-LineatDATAVERSITY.net or TDAN.com or KIKConsulting.com

Real-WorldDataGovernance2017UpcomingWebinars

3

3



• Non-Invasive Data Governance:The Path of Least Resistance and Greatest SuccessISBN 9781935504856 / Technics Publishing / Amazon.com

• KIKConsulting.comHome of Non-Invasive Data Governance™

• DATAVERSITY EventsEnterprise Data Governance On-LineVirtual Conference – January 25, 2017Kick Off Session – Selecting the Right Approach to Data Governance

Enterprise Data World 2017Atlanta, Georgia – April 2, 2017Tutorial – Comparing Approaches to Data Governance

Real-WorldDataGovernanceIntheNews

4

4



• The Data Administration Newsletter (TDAN.com)– Bi-Monthly Publication– December 2016 Issue Available– New Content 1st & 3rd Wednesdays– Articles, Columns, Features, Blogs– Published by Bob Seiner– Produced by DATAVERSITY– Subscribe On-Line for Updates

• Just Announced– DATAVERSITY Training Center– On-Line Course Plan – 7 Courses– Non-Invasive Data Governance– Produced by DATAVERSITY

Real-WorldDataGovernanceIntheNews

5

5



• The Agile development methodology is here to stay.

• Data Governance is not going away any time soon.

• These two discipline share some common ground but often compete when itcomes to the “right” thing to do when it comes to managing the data.

• The disciplines need to learn to play well together.

• The old mantra of “do unto others” applies here in a big way.

UsingAgiletoJustifyDataGovernanceAbstract

6

6



• This webinar will cover:– The audience of both Agile and Data Governance– The governance aspects of Agile– Why Data Governance Practitioners Should Embrace Agile– Agile considerations for Data Governance– How to Use Agile to Justify Data Governance

UsingAgiletoJustifyDataGovernanceAbstract

7

7



• DataGovernance– DataGovernanceistheExecutionandEnforcementofAuthority

OvertheManagementofDataandData-RelatedResources.

• DataStewardship– DataStewardshipistheFormalizationofAccountability

OvertheManagementofDataandData-RelatedResources.

UsingAgiletoJustifyDataGovernanceDataGovernanceDefinition

8

8



• Non-InvasiveDataGovernance™– Thepracticeof:

• applyingformalaccountability&behavior• throughnon-invasiveroles&responsibilities• toexistingand/ornewprocesses• toassurethatthedefinition,production&usageofdata• assuresregulatorycompliance,security,privacy,protection&quality.

– Non-Invasivedescribeshowgovernanceisappliedtoassurenon-threateningmanagementofvaluabledataassets.

– Thegoalistobe• Transparent• Supportive• Collaborative.

UsingAgiletoJustifyDataGovernanceDataGovernanceDefinition

9

9



• TheAgileManifesto– DevelopmentMethodologyLeadingtoGoodDesign– People/Interaction– Speed/Effectiveness– Software/WorkingProducts– Requirements/Collaboration– Response/Evolvement

UsingAgiletoJustifyDataGovernanceAgileDefinition

10

10



• The Waterfall Approach– A linear and sequential approach to software development, each waterfall stage is

assigned to a separate team to ensure greater project and deadline control,important for on-time project delivery. A linear approach means a stage by stageapproach for product building, e.g.1. Analyze and Prioritize Business Requirements / Needs2. Design Business Requirements Translated into IT Solution3. Processes Defined, Layouts Built, Code Implementation4. Evolves into Tested Solution for Implementation5. Testing for Evaluation by End-User6. Evaluation and Maintenance

UsingAgiletoJustifyDataGovernanceAgilevs.Waterfall

11

11



• What Does Senior Leadership Want from Data Governance?– Understood Data – Govern Data Definition– Quality Data – Govern Data Production– Protected Data – Govern Data Usage

UsingAgiletoJustifyDataGovernanceTheAudienceofBothAgileandDataGovernance

12

12



• What Does Senior Leadership Want from Agile?– Speedy Delivery– Incremental Delivery– High Quality Delivery


Mr.McFeelyofMisterRoger’sFame

“SpeedyDelivery”

13

13



• What Does Senior Leadership Think About Data Governance?– The reason why they are implementing Data Governance:

• Data must be managed as an asset.• The data must be protected.• The data drives analytics and decision-making so it must be high quality.• The auditors are telling them they must have formal governance.• Other organizations are governing their data.


14

14



• What Does Senior Leadership Think About Data Governance?– The reasons why they are not implementing Data Governance:

• They do not understand all of the approaches that are available.• They have the perception that …

– DG is very complex based on the organization.– DG moves very slowly.– DG can be quite expensive.– DG requires a lot of resources.– DG will change the culture.– DG is not a project.


15

15



• What Does Senior Leadership Think About Agile?– The reason why they are implementing Agile:

• Large projects take too long and they cost way too much.• Large projects must be managed to be delivered quickly and accurately.• The project must be delivered early and continuously.• The project communications must be face-to-face and constant.• High Quality and Timely Delivery are Critical.


16

16



• What Does Senior Leadership Think About Agile?– The reason why they are not implementing Agile:

• The fact is that they are attempting to deliver projects using Agile.• Sold on the virtues of Agile. Not sold on the virtues of Data Governance.• And … Agile is only being used for large and complex projects.


17

17



• What does Senior Leadership know about relating the two?– Very Little

• How much effort has been made to relate the two?– Very Little

• What effort has been made to get the Program and Projects aligned?– Very Little

• How well are Data Governance Programs acting Agile?– Very Little


18

18



• Agile is a Development Methodology– Who does what, when and how when it comes to the development effort.– Very structured.– Very disciplined.– Very timely.

UsingAgiletoJustifyDataGovernanceTheGovernanceAspectsofAgile

19

19



• Governance is Execution and Enforcement of Authority.– Who does what, when and how when it comes to the data discipline.– Very structured.– Very disciplined.– Very timely.


20

20



• Governance Bill of “Rights”.– Getting the “right” people– Involved at the “right” time– In the “right” process– Using the “right” data– To do the “right” thing– To get the “right” result

… at least most of the time.


21

21



• If Governance is what we say it is …--- Governance is theoretically built into …

– Develop Functionality– Integrate and Test– Demo Release– Client Feedback– Make Changes– System Testing– Next Iteration


22

22



• Development Bill of “Rights”– Getting the “right” development people– Involved at the “right” development time– In the “right” development process– Using the “right” development data– To do the “right” development thing– To get the “right” development result

… at least most of the time.


23

23



• The focus of Agile is …– People / Interaction– Speed / Effectiveness– Software / Working Products– Requirements / Collaboration– Response / Evolvement

UsingAgiletoJustifyDataGovernanceWhyDataGovernancePractitionersShouldEmbraceAgile

24

24



• The focus of Agile is …– People / Interaction – Data Governance Practitioners Want This!– Speed / Effectiveness– Software / Working Products– Requirements / Collaboration– Response / Evolvement


25

25



• The focus of Agile is …– People / Interaction – Data Governance Practitioners Want This!– Speed / Effectiveness – Data Governance Practitioners Want This!– Software / Working Products– Requirements / Collaboration– Response / Evolvement


26

26



• The focus of Agile is …– People / Interaction – Data Governance Practitioners Want This!– Speed / Effectiveness – Data Governance Practitioners Want This!– Software / Working Products – Data Governance Practitioners Want This!– Requirements / Collaboration– Response / Evolvement


27

27



• The focus of Agile is …– People / Interaction – Data Governance Practitioners Want This!– Speed / Effectiveness – Data Governance Practitioners Want This!– Software / Working Products – Data Governance Practitioners Want This!– Requirements / Collaboration – Data Governance Practitioners Want This!– Response / Evolvement


28

28



• The focus of Agile is …– People / Interaction – Data Governance Practitioners Want This!– Speed / Effectiveness – Data Governance Practitioners Want This!– Software / Working Products – Data Governance Practitioners Want This!– Requirements / Collaboration – Data Governance Practitioners Want This!– Response / Evolvement – Data Governance Practitioners Want This!


29

29



• ThefocusofDataGovernanceis:– ManageDataasanAsset– FormalizeAccountability– FollowtheRules– GovernConsistently

UsingAgiletoJustifyDataGovernanceAgileConsiderationsforDataGovernance

30

30



• Howdoyoubegintobringtogetherapproachesthataredifferent?

• Beginbylookingatwhattheyhaveincommon.

• Strengthenthat.

• Buildonthat.

• Rinse&Repeat.


31

31



• AgilePrinciples:– CompleteFeatureBeforeMoving– Testing– EarlyandOften– ActiveUserParticipation– Timing– Quickly,Incrementally– RequirementsataHighLevel– ManageTechnicalDebt– Lightweight&Visual

– RequirementsEvolve/TimescaleFixed– SoftwareasProgressMeasure– DevelopSmall,IncrementalReleases– GoodDesign,TechnicalExcellence– SimplicityisEssential– MaximizingtheAmountofWork


32

32



• AgilePrinciples(whereDataGovernancecanjustifiablyhaveanimpact):– CompleteFeatureBeforeMoving– Testing– EarlyandOften– ActiveUserParticipation– Timing– Quickly,Incrementally– RequirementsataHighLevel– ManageTechnicalDebt– Lightweight&Visual

– RequirementsEvolve/TimescaleFixed– SoftwareasProgressMeasure– DevelopSmall,IncrementalReleases– GoodDesign,TechnicalExcellence– SimplicityisEssential– MaximizingtheAmountofWork


33

33



• AgilePrinciplestobeTargetedtoJustifyGovernance1. ActiveUserParticipation2. RequirementsataHighLevel3. TechnicalDebttobePaid4. RequirementsEvolve/TimescaleFixed

UsingAgiletoJustifyDataGovernanceHowtoUseAgiletoJustifyDataGovernance

34

34



1. ActiveUserParticipation‒ DataGovernanceSpeaksthisSameLanguage‒ Essenceofbeingnon-invasiveisto:

‒ Usersfollowagovernedprocess‒ Formalizeuseraccountability‒ Usersescalateissuestoformalizedecisionmaking


35

35



2. RequirementsataHighLevel– ThisiswherethereisthegreatestpotentialtojustifyDataGovernanceforAgile.– DGpractitionerspridethemselvesoncapturingrequiredmetadata.– Agilepractitionerspridethemselvesonbeingstructuredandwithspeed.

• WoW:Bothpractitionersmustagreeonrequirementslevel.


36

36



• RequirementsataHighLevel– ThisiswherethereisthegreatestpotentialtojustifyDataGovernanceforAgile.– CanDataGovernancepractitionerslimitrequirements?

• VocabularyMetadata– BusinessTerminology• BusinessMetadata– DataStandards&Dictionaries• TechnicalMetadata– Database,FileandModel• LineageMetadata– DataMovement,ETL• LinkageMetadata– Standards,Policies,Procedures,BetweenLevels


37

37



• RequirementsataHighLevel– Logical&PhysicalModelingofData

• Modelsrequirequestions…aboutcardinality,retention,growth,etc.• Modelspromoteacommonunderstanding• ModelsrequiretheparticipationofaDataAnalyst/DataArchitect.

• WoW:Withoutdetailedrequirements– projectscreatetheirownunderstandingwhichinturncanresultinambiguity,replication,etc.


38

38



• RequirementsataHighLevel– QuestionsforDataPeople:

• Toomuchdocumentation?• Notenoughdocumentation?• Doweuseitall?• Howdoweuseit?• CanweworkoutanagreementwiththeAgilepeople?• Isn’tmakingnodecisiononrequirementadecisionitself?

• WoW:Considerthevalueyougetfromthemetadatayourequire.


39

39



• RequirementsataHighLevel– Summary• Limit Data Demand for Requirements• Achieve an Agreed Upon Level of Requirements• Provide Tools to Effectively Capture Essential Requirements• Provide Resource(s) Focused on Data Governance / Agile Alignment

• WoW: Incrementally Apply Agreed Level of Data Requirements to Agile


40

40



3. TechnicalDebttobePaid– Requirementsthatwilleventuallyneedtobeaddressed/ParkingLotList– Alsoreferredtoasdesigndebtorcodedebt– Metaphorreferringtotheeventualconsequencesofpoorsystemdesign

The debt can be thought of as work that needs to be done before a particular job canbe considered complete or proper. If the debt is not repaid, then it will keep onaccumulating interest, making it hard to implement changes later on. Wikipedia

– Debtcanbeaccumulatedineachstepofthemethodology– AgileorWaterfall

• WoW:ThisisthebiggestreasonwhyDataPeopleareskepticalaboutAgile.


41

41



• TechnicalDebttobePaid– ReasonswhyTechnicalDebtaccumulates

• Business pressures - where the business considers getting something releasedsooner before all of the necessary changes are complete, builds up technical debtcomprising those uncompleted changes.

• Lack of process or understanding - where businesses are blind to the concept oftechnical debt, and make decisions without considering the implications.

• Lack of test suite - which encourages quick and risky Band-Aids to fix bugs.• Lack of documentation - where code is created without necessary supporting

documentation. That work to create the supporting documentation represents adebt that must be paid.


42

42



• TechnicalDebttobePaid– ReasonswhyTechnicalDebtaccumulates

• Lack of collaboration - where knowledge isn't shared around the organization andbusiness efficiency suffers, or junior developers are not properly mentored

• Parallel development at the same time on two or more branches - can cause thebuildup of technical debt because of the work that will eventually be required tomerge the changes into a single source base. The more changes that are done inisolation, the more debt that is piled up.

• Delayed refactoring – As the requirements for a project evolve, it may becomeclear that parts of the code have become unwieldy and must be refactored inorder to support future requirements.

• Lack of knowledge - When developer simply doesn't know how to write code.


43

43



• TechnicalDebttobePaid– ApplyGovernanceSoThatTechnicalDebtDoesNotAccumulates

• Formal Accountability for Managing Technical Debt• Formal Escalation of Issues Associated with Technical Debt• Formal Reporting of Technical Debt Status


44

44



4. RequirementsEvolve/TimescaleFixed– ThisisthebiggestchallengewhenjustifyingDataGovernanceforAgile.– Fromboth perspectives

• EvolvingRequirements• FixedPeriodofTime.

– Shootingatamovingtarget.


45

45



• Requirements Evolve / Timescale Fixed– Requirements evolve means a gradual process in which the requirement (including

data requirements) change into a different usually more complex or better form.

– Let’s break this down:• Gradual process – not all at once – requires constant governance• Changes into something different – requires constant governance• Usually more complex or better form – requires constant governance


46

46



• Requirements Evolve / Timescale Fixed– Management of Requirements requires Governance of Requirements.– Provide 2 of 3 – Time, Resources, Effort– Agile is a form of Project Governance – Just not Data Governance.

• WoW: Bottom line – Governance is the Execution & Enforcement of Authority


47

47



• Requirements Evolve / Timescale Fixed – Summary– The challenge is to gather the:

• Most valuable requirements / documentation – and use them effectively• Most effective requirements – and make them available• Most available requirements – and make certain essentials are covered• Most essential requirements – in a short period of time.


48

48



• This webinar covered:– The audience of both Agile and Data Governance– The governance aspects of Agile– Why Data Governance Practitioners Should Embrace Agile– Agile considerations for Data Governance– How to Use Agile to Justify Data Governance

UsingAgiletoJustifyDataGovernanceSummary

49

49



• Real-WorldDataGovernance– MonthlyWebinarSeries– January19,2017– ThreeApproachestoDataStewardship– February16,2017– TheNewNon-InvasiveDataGovernanceFramework– March16,2017– ApplyDataGovernancetoAgileEfforts

• QuestionsandAnswers

Real-WorldDataGovernanceConclusion

50

50



• RobertS.SeinerKIKConsulting&EducationalServices– KIKconsulting.comTheDataAdministrationNewsletter– TDAN.comPostOfficeBox112571,UpperSt.Clair,Pennsylvania15241412.220.9643,412.220.9644(Fax)[email protected]@tdan.com@RSeiner#RWDG

Real-WorldDataGovernanceContactInformation