![Page 1: Tedd Hellmann / David Poncelow - Splunk › files › 2016 › slides › step-up-your...Best Prac2ces for Developing Splunk Apps and Add-ons (Jason Conger) How to Build a Solu2on](https://reader036.vdocument.in/reader036/viewer/2022070802/5f02ef217e708231d406bbc5/html5/thumbnails/1.jpg)
Copyright©2016SplunkInc.
TeddHellmann/DavidPoncelowProductManager/SeniorSoDwareEngineer,Splunk
STEPUpYourAppDevelopmentGame
![Page 2: Tedd Hellmann / David Poncelow - Splunk › files › 2016 › slides › step-up-your...Best Prac2ces for Developing Splunk Apps and Add-ons (Jason Conger) How to Build a Solu2on](https://reader036.vdocument.in/reader036/viewer/2022070802/5f02ef217e708231d406bbc5/html5/thumbnails/2.jpg)
Disclaimer
2
DuringthecourseofthispresentaMon,wemaymakeforwardlookingstatementsregardingfutureeventsortheexpectedperformanceofthecompany.WecauMonyouthatsuchstatementsreflectourcurrentexpectaMonsandesMmatesbasedonfactorscurrentlyknowntousandthatactualeventsorresultscoulddiffermaterially.Forimportantfactorsthatmaycauseactualresultstodifferfromthosecontainedinourforward-lookingstatements,pleasereviewourfilingswiththeSEC.Theforward-lookingstatementsmadeinthethispresentaMonarebeingmadeasoftheMmeanddateofitslivepresentaMon.IfreviewedaDeritslivepresentaMon,thispresentaMonmaynotcontaincurrentor
accurateinformaMon.WedonotassumeanyobligaMontoupdateanyforwardlookingstatementswemaymake.InaddiMon,anyinformaMonaboutourroadmapoutlinesourgeneralproductdirecMonandis
subjecttochangeatanyMmewithoutnoMce.ItisforinformaMonalpurposesonlyandshallnot,beincorporatedintoanycontractorothercommitment.SplunkundertakesnoobligaMoneithertodevelopthefeaturesorfuncMonalitydescribedortoincludeanysuchfeatureorfuncMonalityinafuturerelease.
![Page 3: Tedd Hellmann / David Poncelow - Splunk › files › 2016 › slides › step-up-your...Best Prac2ces for Developing Splunk Apps and Add-ons (Jason Conger) How to Build a Solu2on](https://reader036.vdocument.in/reader036/viewer/2022070802/5f02ef217e708231d406bbc5/html5/thumbnails/3.jpg)
HowshouldIbuildmyapp?
![Page 4: Tedd Hellmann / David Poncelow - Splunk › files › 2016 › slides › step-up-your...Best Prac2ces for Developing Splunk Apps and Add-ons (Jason Conger) How to Build a Solu2on](https://reader036.vdocument.in/reader036/viewer/2022070802/5f02ef217e708231d406bbc5/html5/thumbnails/4.jpg)
EVERYTHINGYOUNEEDTOBUILD
SplunkDeveloperGuidance
![Page 5: Tedd Hellmann / David Poncelow - Splunk › files › 2016 › slides › step-up-your...Best Prac2ces for Developing Splunk Apps and Add-ons (Jason Conger) How to Build a Solu2on](https://reader036.vdocument.in/reader036/viewer/2022070802/5f02ef217e708231d406bbc5/html5/thumbnails/5.jpg)
STEPupyourgame STEP-interacMvelearningenvironment ExploretopicsthroughTechniquesandRecipes Technique:explorethedetailsoffeaturesyoucanuseinapps(modularinputs,customvisualizaMons,customalertacMons,…)
Recipe:diveintothedetailsofbringingseveraltechniquestogethertoaddressabusinessgoal
![Page 6: Tedd Hellmann / David Poncelow - Splunk › files › 2016 › slides › step-up-your...Best Prac2ces for Developing Splunk Apps and Add-ons (Jason Conger) How to Build a Solu2on](https://reader036.vdocument.in/reader036/viewer/2022070802/5f02ef217e708231d406bbc5/html5/thumbnails/6.jpg)
STEPupyourgame
6
DataIngest
HEC
BasicDataInput
IndexerAck
…
ModularInput
CheckpointResults
InputValidaMon
…
……
VisualizaMons
…
Search
…
SimpleXML
CustomViz
...
CustomCommands
Workflows
...
![Page 7: Tedd Hellmann / David Poncelow - Splunk › files › 2016 › slides › step-up-your...Best Prac2ces for Developing Splunk Apps and Add-ons (Jason Conger) How to Build a Solu2on](https://reader036.vdocument.in/reader036/viewer/2022070802/5f02ef217e708231d406bbc5/html5/thumbnails/7.jpg)
STEPupyourgame
7
DataIngest
HEC
BasicDataInput
IndexerAck
…
ModularInput
CheckpointResults
InputValidaMon
…
……
VisualizaMons
…
Search
…
SimpleXML
CustomViz
...
CustomCommands
Workflows
...
![Page 8: Tedd Hellmann / David Poncelow - Splunk › files › 2016 › slides › step-up-your...Best Prac2ces for Developing Splunk Apps and Add-ons (Jason Conger) How to Build a Solu2on](https://reader036.vdocument.in/reader036/viewer/2022070802/5f02ef217e708231d406bbc5/html5/thumbnails/8.jpg)
STEPupyourgame
8
STEPPreview2Techniques1Recipe
![Page 9: Tedd Hellmann / David Poncelow - Splunk › files › 2016 › slides › step-up-your...Best Prac2ces for Developing Splunk Apps and Add-ons (Jason Conger) How to Build a Solu2on](https://reader036.vdocument.in/reader036/viewer/2022070802/5f02ef217e708231d406bbc5/html5/thumbnails/9.jpg)
Nextsteps
9
Telluswhattechniquesandrecipesyouneed!
![Page 10: Tedd Hellmann / David Poncelow - Splunk › files › 2016 › slides › step-up-your...Best Prac2ces for Developing Splunk Apps and Add-ons (Jason Conger) How to Build a Solu2on](https://reader036.vdocument.in/reader036/viewer/2022070802/5f02ef217e708231d406bbc5/html5/thumbnails/10.jpg)
STEPintoreal-worldexamples
![Page 11: Tedd Hellmann / David Poncelow - Splunk › files › 2016 › slides › step-up-your...Best Prac2ces for Developing Splunk Apps and Add-ons (Jason Conger) How to Build a Solu2on](https://reader036.vdocument.in/reader036/viewer/2022070802/5f02ef217e708231d406bbc5/html5/thumbnails/11.jpg)
Planningajourney
Pla'ormandtools:akitbagforourjourney
UIandvisualiza2ons:whattheappslooklike
Workingwithdata:whereitcomesfrom&howwemanageit
Addingcode:usingJavaScriptandSearchProcessingLanguage
Packaginganddeployment:reachingourfirstdesMnaMon
DealingwithOAuth
Aler2ng
Buildingintelemetrywithhigh-performancedatacollec2on
splk.it/devguide
![Page 12: Tedd Hellmann / David Poncelow - Splunk › files › 2016 › slides › step-up-your...Best Prac2ces for Developing Splunk Apps and Add-ons (Jason Conger) How to Build a Solu2on](https://reader036.vdocument.in/reader036/viewer/2022070802/5f02ef217e708231d406bbc5/html5/thumbnails/12.jpg)
1.StartwithaQuesMonsBacklog Architecture– WhatdoesatypicalSplunkapplicaMonreferencearchitecturelooklike?– WhatcommonparadigmsareapplicabletoSplunkappdevelopment?– Whatarethetypicaldeploymenttopologies?WhyshouldIchooseaspecificone?Whataretheconfoundingfactors
onthechoiceofmytopology?– HowdoIparMMonmySplunksoluMons?– Whatarethetradeoffsofvarioustypesofinputs?– HowdoIarchitectmySplunksoluMonanddeploymentforaverylargescale?– HowdoIarchitectmySplunksoluMonforthecloud?WhatarespecificconsideraMonsfordeployingtoAWSorAzure?– What’sthelandscapeofSplunkextensionpoints?– HowdoIintegratedatafromSplunkintoexisMngapplicaMonsandsystems?– HowdoIplananddesignarobustalerMngandmonitoringsubsystemontopofSplunk?– WhatshouldIconsiderformysizingrequirements?– WhatarerecommendedconfiguraMonsofSplunkdeploymenttomeetmysizingrequirements?– ShouldIarchitectmysoluMontoindexmydatainlocaldatacenter(zone)orcentrally?– WhatarethingswecanautomaMcallydegradesowecanmakesureourcoreexperienceisworking?– Whensomethinghappens,howdoIeffecMvelypropagatetheinfoandreacttoit?– HowareothersoluMonsonSplunkbuilt?Whatwerethechallenges?Howhavetheybeenaddressed?
PackagingandDeployment– HowdoIpiecetogethervariouspartsofaSplunkapp(customsearchcommands,modinputsetc.)?– HowdoIpackageaSplunksoluMonwithasingleinstallthatautomaMcallyrollsoutallthenecessarydependencies?– HowdoImanagemySplunksoluMonversioning,backwardandfuturecompat?– What'sthebestwaytosplitupcustomappsfordeployment?
Development– HowshouldIsetupmydevelopmentenvironmenttobeproducMvewithSplunk?– WhataredifferentwaysofhowIdevelopmySplunkapp?ProsandconsofusingspecificSDKvsRESTAPIs?
ProsandconsofusingSimpleXMLvsAdvancedXMLvsWebFramework…– HowdoIanalyzeadatasourceforaTA?– WhatarethedifferentwaysofenrichingthedatainSplunk?Whataretheirtradeoffs?– WhenshouldIuseeventtypesandtransacMonsfordataclassificaMon?– HowdoIextendSplunktodefineacustominputcapability?– WhenshouldIusemodularinputsvsscriptedinputsvs..?– Whatarestreamingvsnon-streamingoutputsconsideraMons?– HowdoIdealwithlong-runningscripts?Handlingshutdown/restartofSplunk?Concurrency?Statepersistenceetc.
– WhyshouldInotusetransacMons?– WhenshouldIusepivotvststats?– WhyshouldIusedatamodels?– Whenmydatasourcetouchesonmanydatamodels,shouldIassumecompleteseparaMonorheavyinheritance?– HowdoIextendanexisMngdatamodel?– WhatdoesCIMofferandwhyshouldIbuildCIM-compliantapps?– InthecontextofCIM,whatarethetradeoffsofusingmyprops.confandtransforms.confandrewriMngthemon
indexing,completelydiscardingthevendorsuppliedfieldnames?HowdoIreconciletheadvantagesofacleaninterface&normalisaMon,butatthecostoflosingalignmentwithpublishedvendordocumentaMon,andalearningcurveforexisMngusers?
– HowdoImanagemysoluMondeclaraMveconfiguraMon?HowdoIdetect/troubleshootbadconfig?– HowdoIlogandanalyzedatathatisnoteventdriven(certainwebfeeds,htmlparsing,imagemetadata)?– Compareandcontrastad-hocsearchingvsbackgroundsearching– HowdoIhandletransientfaults?– HowdoIeffecMvelymanagecredenMals?– What’stheeffectofsearchheadlocaMononmyappandtheoveralluserexperience?– HowdoIdevelopanintegratedmechanismtoletmeconnectSplunktomyMOM(messagingmiddleware)andindex
mymessages?– HowdoIhandletherequirementthatappconfigsmustbedifferentacrossdifferentservertypesinadistributed
environment(e.g.appsonsearchheadsshouldn'thaveinputsenabled)? Quality/Compliance
– WhatqualitygatesshouldIconsider?Whatkindofpara-funcMonalcharacterisMcsareimportanttoconsider?– WhatheurisMcsdoIusetobless/blockarelease?– HowdoItestadatamodel?– HowdoIprepareeventgeneraMonwhenbuilding/tesMnganapp?– WhatkindofperftesMngshouldIdoandhow?– HowdoItestUI?– HowdoIsecuritycerMfymysoluMon?– HowdoIdesigntosaMsfymyretenMonandcompliancepolicies?– HowdoIarchitecttodesignmyavailabilityrequirements?– HowdoIhandlegeographicdisasterrecovery/faulttolerance?– HowdoIproperlyinstrumentmysoluMonsothatIknowwhat’shappening?
SustainedEngineering– HowdoImaintain/service/supportSplunkapps?– HowdomycustomershandleupdaMngtheircustomizedconfigsoncenewversionsofmyappcomeout?
Business– WhyshouldIbuildonSplunk?– WhatkindofskilldoIneedmydevstohavetobuildaSplunksoluMon?– Whatisthecommunitybuilding?HowarecurrentdevscreaMnguniqueexperiencesusingSplunk–Itypicallywantto
seesomemarketplacesuccess – Costandpricingareveryimportanttomeasaentrepreneurdeveloper.IfIamcomingintobuildatoolthatwillbe
commercializedIneedtoknowthatthecoststructureofSplunkwon’tcausemyservicetobeeconomicallyunprofitable.
WhatdoesatypicalSplunkapplica2onarchitecturelooklike?
HowshouldIsetupmydevenvironmenttobeproduc2vewith
Splunk?
HowdoIintegrateSplunkintoexis2ngsystems?
HowdoIpreparemyeventgenera2onwhendeveloping&
tes2nganapp?
HowdoIpackageanapp?dealwithappversioningandupdates?
12
![Page 13: Tedd Hellmann / David Poncelow - Splunk › files › 2016 › slides › step-up-your...Best Prac2ces for Developing Splunk Apps and Add-ons (Jason Conger) How to Build a Solu2on](https://reader036.vdocument.in/reader036/viewer/2022070802/5f02ef217e708231d406bbc5/html5/thumbnails/13.jpg)
2.IdenMfyExtensibilitySurfaceArea§ Datainges2on&indexing
– Inputê Modularinputsê Custom(trained)sourcetypesê Customsourcesê HTTPEventCollecMon
– Datainges2onpipelineê FieldextracMonsê FieldtransformaMons
– Indexingê Customindexes
§ Searching– Searchauthoring
ê Customsearchcommandsê Macros(basic,parametrized)ê Savedsearches
– Dataclassifica2onê Eventtypesê TransacMons
– Dataenrichmentê Lookupsê KVstorecollecMonsê WorkflowacMons
– Datanormaliza2onê Tagsê Aliases
– Dataminingê cluster&dedupê anomalousvalueê kmeansê predictcommands…
§ Processing&repor2ng– Search-2memapping
ê Datamodels
– CIMextensions
– CustomVisualiza2ons
– CustomUI– Pages,views&dashboards
ê JS,CSSExtensionsê Customsetupscreens
– Scheduledprocessingê Scheduledreports
– Aler2ngê Scriptedalertsê CustomalertacMons
– Branding&naviga2onê CustomappnavigaMonê Appbranding
– Manageabilityê Customsplunkwebcontrollersê Customsplunkdendpoints
13
![Page 14: Tedd Hellmann / David Poncelow - Splunk › files › 2016 › slides › step-up-your...Best Prac2ces for Developing Splunk Apps and Add-ons (Jason Conger) How to Build a Solu2on](https://reader036.vdocument.in/reader036/viewer/2022070802/5f02ef217e708231d406bbc5/html5/thumbnails/14.jpg)
3.Minebusinessrequirements4.FormulatelearningobjecMves5.Designaround3and4
…
14
![Page 15: Tedd Hellmann / David Poncelow - Splunk › files › 2016 › slides › step-up-your...Best Prac2ces for Developing Splunk Apps and Add-ons (Jason Conger) How to Build a Solu2on](https://reader036.vdocument.in/reader036/viewer/2022070802/5f02ef217e708231d406bbc5/html5/thumbnails/15.jpg)
§ Data§ Searchlanguage§ AggregaMngsiloedmetricsinto
meaningfulKPIs§ DatamanipulaMon§ DatanormalizaMon§ Sub-searches§ Config-driven§ PersistencewithKVstore§ Macros
§ Viz:§ Dynamicscaling§ Customizingin-theboxviz
controls
§ Generalsearchpaserns§ SearchopMmizaMons§ UxPrototyping§ AdapMng3rdpartyvizlibrary§ CompositechartswithinteracMons§ Dealingwithhigh-volumedatasets§ TroubleshooMngperfissues§ Post-processornot-post-process–
deploymentimplicaMons§ AutomatedUItesMng(w.Selenium)
§ Setngthestage§ OverallSplunkappstructure§ UItechnologyselecMon:
SimpleXMLvsSplunkJS§ Modularity§ Dev&testenv§ Devworkflow§ Modularity§ Dataonboarding§ CIMcompliance§ Tools
§ Post-processing§ IntegraMngwith3rdparty
component§ UnittesMng(w.Mocha)§ PersisMngstate(peruser)
§ Datamodeling§ Usinglookups§ Buildingabaselinelookuptable§ WindowsofMme/CustomMmeranges§ OverlayingMmedata
§ Usingsub-searchestocorrelatedata§ TroubleshooMngsearches
§ Customnav§ UxacMviMespermeaMngalldev
§ Datamining:§ ExploraMon§ PreparaMon:filtering/deduping/
buckeMng§ UsingadvancedstaMsMcsfuncMons§ Threshold-basedanomalydetecMon§ EvaluaMnggoodness/accuracy
Plusnon-funcMonaltopics: § Appversioning§ PackagingInstallaMon§ Securityreview
§ Deployment§ Publishingtosplunkbase§ AppcerMficaMon
![Page 16: Tedd Hellmann / David Poncelow - Splunk › files › 2016 › slides › step-up-your...Best Prac2ces for Developing Splunk Apps and Add-ons (Jason Conger) How to Build a Solu2on](https://reader036.vdocument.in/reader036/viewer/2022070802/5f02ef217e708231d406bbc5/html5/thumbnails/16.jpg)
BuildingSoluMonsontheSplunkPlauorm1. SplunkReferenceApps
Complete,workingreal-worldSplunksoluMonsbuilttogetherwithpartners(Conducive,Auth0)
2. SplunkDeveloperGuide
Thisisunbelievable,itcoversmosteverythingIlearnedthehardway…–BernieMacias,TechnicalArchitect,Zillow
dev.splunk.com/goto/devguide
![Page 17: Tedd Hellmann / David Poncelow - Splunk › files › 2016 › slides › step-up-your...Best Prac2ces for Developing Splunk Apps and Add-ons (Jason Conger) How to Build a Solu2on](https://reader036.vdocument.in/reader036/viewer/2022070802/5f02ef217e708231d406bbc5/html5/thumbnails/17.jpg)
SplunkReferenceAppDemo
SplunkReferenceApp–PluggableAuditSystemsplunkbase.splunk.com/app/1934/ORsearchnamefromSplunkWebUI
![Page 18: Tedd Hellmann / David Poncelow - Splunk › files › 2016 › slides › step-up-your...Best Prac2ces for Developing Splunk Apps and Add-ons (Jason Conger) How to Build a Solu2on](https://reader036.vdocument.in/reader036/viewer/2022070802/5f02ef217e708231d406bbc5/html5/thumbnails/18.jpg)
Takeaways Appdevelopment!=rocketscience
STEPupyourgamewithtechniquesandrecipes
Getintouchwithus
– LeavefeedbackforSTEP
– Comebyourbooth,getsomeswag
![Page 19: Tedd Hellmann / David Poncelow - Splunk › files › 2016 › slides › step-up-your...Best Prac2ces for Developing Splunk Apps and Add-ons (Jason Conger) How to Build a Solu2on](https://reader036.vdocument.in/reader036/viewer/2022070802/5f02ef217e708231d406bbc5/html5/thumbnails/19.jpg)
Resources
19
![Page 20: Tedd Hellmann / David Poncelow - Splunk › files › 2016 › slides › step-up-your...Best Prac2ces for Developing Splunk Apps and Add-ons (Jason Conger) How to Build a Solu2on](https://reader036.vdocument.in/reader036/viewer/2022070802/5f02ef217e708231d406bbc5/html5/thumbnails/20.jpg)
Relatedbreakoutsessions&acMviMes
20
FasterSplunkAppCer2fica2onwithSplunkAppInspect(GrigoriMelnik/AndyNortrup)BestPrac2cesforWorkingwithSplunkCloud(DennisBourg/EricSix)HTTPEventCollectorinSplunk6.4-MoreSuperPowers!(GlennBlock/ItayNeeman)
BuildingSplunkVisualiza2onswiththeNewCustomVisualiza2onAPI(MarshallAgnew)DashboardWizardry(NicholasFilippi/SiegfriedPuchbauer)
BestPrac2cesforDevelopingSplunkAppsandAdd-ons
(JasonConger)
HowtoBuildaSolu2onfromScratch:ACaseStudyofPartnerEngagementandCo-Development
(VladimirMelnik/IgalVanier)
OnboardYourDataFasterUsingtheSplunkAdd-onBuilder
(EliasHaddad/GuodongWang)
![Page 21: Tedd Hellmann / David Poncelow - Splunk › files › 2016 › slides › step-up-your...Best Prac2ces for Developing Splunk Apps and Add-ons (Jason Conger) How to Build a Solu2on](https://reader036.vdocument.in/reader036/viewer/2022070802/5f02ef217e708231d406bbc5/html5/thumbnails/21.jpg)
THANKYOU