on-premise, in- cloud: reviewing integration options for
TRANSCRIPT
Reading Virtual UG – 27th April 2020Joe Griffin
ON-PREMISE, IN-CLOUD: REVIEWING INTEGRATION OPTIONS FOR DYNAMICS 365
AGENDA
• About Me
• Traditional Integration Challenges
• Why Integrate Now?
• Available Tools• Microsoft
• On-Premise Gateway• Azure Data Factory Self-Hosted Integration Runtime• Azure App Service Hybrid Connections
• The Rest• KingswaySoft• Tibco Scribe• Skyvia
• Closing Remarks
2
ABOUT ME
• Director @ SOLO Cloud Solutions, MS Gold Partner consultancy practice in the UK
• Hold various certifications (Translation: I like shiny badges! 🤩)
• Worked with Dynamics CRM/Dynamics 365 for around 5 years; previous background of SQL, C#, Office 365; lots of my daily work involves the Power Platform, including:
• Power Apps
• Power Automate
• Power BI
• Bespoke technical integrations involving these products.
• PRINCE2 PM/Agile Practitioner
• Certified Scrum Master (Scrum Alliance)
• CRMUG/D365UG NW Committee Member
3
THE DREADED “I” WORD…
4
WHY SUCH A BAD RAP?
• Integrations can be ridiculously expensive: Not just in £ terms, but also in upfront/ongoing effort. Consequently…
• Integrations can become complex: LLD, HLD’s, CAB approval…more acronyms than a passive aggressive email! To make sense of it all, therefore…
• Integrations require insane technical expertise to build/maintain: Chances are, you’ll need a developer, which ultimately means…
• Integrations can be ridiculously expensive: Déjà vu anyone? The cycle repeats…
5
THE VICIOUS CYCLE
Business
Commences
New
Integration
Project
Solution
Architecture
Project Team
Initiated
6-8 month
development
cycle
Solution No
Longer meets
original
requirements
6
THE FLIP SIDE
• When done correctly, integrations (particularly those involving on-premise / cloud business systems) can deliver a lot of benefits:• Streamlines cloud adoption
• Increases agility
• Can lead to enhanced customer/stakeholder experience.
• Key aspect with all this is to identify a cost-effective and simple to execute solution, that also adequately addresses any security concerns.
7
THE POWER PLATFORM: A NEW BEGINNING FOR INTEGRATIONS?• At the risk of sounding hyperbolic, the Power Platform is a game-changer
when it comes to eliminating the all too often costly integration project.
• Using the Power Platform, it is now possible to take your on-premise data and:• Continually synchronise it into a CDS database.• Trigger or perform actions using Power Automate or Azure Logic Apps.• Publish or import data into a Power BI Report.• Develop a low-code application using canvas Power Apps
• This can take less than 30 minutes to implement, all the while satisfying any security/access concerns.
• With such a wide array of tools to easily bring on-premise data into Dynamics 365 and the Power Platform, there is really no excuse for at least exploring what’s out there.
8
SESSION PURPOSE
• My mission statement for today:To inform and guide attendees on the currently available
tools to incorporate on-premise data sources into Dynamics 365 or the Power Platform.
• We will deep dive and evaluate each option, including its pros and cons.
• Brace yourself for demos…
• Only some of these tools can be directly leveraged from within Power BI, Power Apps and Power Automate.
• Some pictures shamelessly plagiarised lovingly selected from multiple online sources.
9
MICROSOFT TOOLS
10
ON-PREMISE GATEWAY
OVERVIEW
A gateway for those building solutions involving the Power Platform and who wish to integrate their on-premise data sources alongside Power Apps, Power BI, Power Automate and more.
+ Full supported and regularly updated connector from Microsoft
+ Once installed, fully manageable from Office 365.
+ Flexible deployment options – organisational or personal.
+ Can be used in conjunction with Azure Logic Apps
- Organisation gateway requires admin privileges.
- Some read, write and data response limitations
- Personal mode restricted for use in Power BI only.
VERDICT
If you’re not sure where to start, use this. Straightforward to deploy and manage, it gives you everything you need to work with your on-premise data in the Power Platform.
11
AZURE DATA FACTORY SELF-HOSTED INTEGRATION RUNTIME
OVERVIEW
For advanced data processing needs, Azure Data Factory provides a self-hosted Integration Runtime (IR) (translation: gateway) to surface your on-premise data within multiple online sources – including the Common Data Service.
+ Simple and easy to deploy
+ Ideal for integrations taking data both inside and outside the Power Platform
+ “Pay as you go” – run complex data activities for the price of a few cups of coffee each month.
- Import only – no support for use alongside Power Apps, Power Automate etc.
- Potential learning curve, if you don’t come from a SQL Server background
- Requires a cost / technical investment in Microsoft Azure.
- Microsoft advise Self hosted IR not to be installed on the same machine as an on-premise gateway.
VERDICT
Solutions that traditionally utilise SSIS for complex, multi-data source processing should now be use Azure Data Factory instead. The Self-Hosted IR supports this objective when working with on-premise data sources.
12
SUCH A GREAT(?) TOPIC, IT HAS ITS OWN SESSION!
13
https://www.youtube.com/watch?v=tQstFRLjshA
AZURE APP SERVICE HYBRID CONNECTIONSOVERVIEW
Tailored for high bespoke integrations involving Azure App Service, Hybrid Connections provide near-unlimited potential to work with your on-premise resources.
+ Powerful and highly tailorable to your integration.
+ Consumable from within Power Apps and Power BI via HTTP / Web API connector
- Requires significantly more technical know-how to implement
- Metered service - £7.29/month for first 5GB, £0.75/GB thereafter
- If not existing, necessitates a technical/infrastructure investment in Microsoft Azure.
VERDICT
If there is a need to consume on-premise data via Virtual Entities or to use an existing on-premise API, this option (with some additional technical development) potentially has merits; otherwise, technical overkill when compared with other options.
14
DEMO
15
3RD PARTY TOOLS
16
KINGSWAYSOFT SSIS ADAPTER
OVERVIEW
3rd party toolkit, allowing you to work directly with both on-premise / online Dynamics 365, using SQL Server Integration Services (SSIS) packages.
+ Tried and tested solution, with flexible import / extract options
+ Leverages familiar tools, such as SSIS, to build out “no-code” integrations.
- Potentially out of step with future technical direction of Microsoft
- Requires full blown SQL Server or developer know-how to execute.
- Additional licensing costs for your overall solution.
- Supports integrations with Dynamics 365 / CDS only; no support for Power Apps, Power Automate etc.
VERDICT
The mainstay and “reliable” integration option but starting to become outpaced by other available solutions.
17
TIBCO SCRIBE ONLINE
OVERVIEW
A tool designed for non-developers, Scribe Online provides a fully managed data integration option, with capability to extend this to on-premise resources.
+ Fully managed service, no infrastructure commitment / developer expertise required.
+ Supports FetchXML querying, Merge Block and Alternate Keys
+ Full control over deployment option – Microsoft Azure or AWS
- Most expensive option - £4,000/year minimum
- Uncertain future following Tibco acquisition?
- Supports integrations with Dynamics 365 / CDS only; no support for Power Apps, Power Automate etc.
VERDICT
Once a go-to solution, the needs this product addressed can now be achieved more easily – and cheaply – using other options.
18
SKYVIA
OVERVIEW
A solution that has existed for several years now, some recent changes and new features
+ Built in SQL editor when working with Dynamics 365 data.
+ Free to use (with some limitations)
+ Can expose a SQL / OData Endpoint, for use alongside Power Automate, Power BI etc…
- …but requires a paid plan.
- Additional cost for integrations that need to run frequently
- Poor support for free plans; you get what you don’t pay for!
- All data is hosted in US data centres; therefore, some latency/data residency concerns
VERDICT
A “Little Mac” challenger option, and potentially a cheap option for satisfying simplistic data integrations.
19
WRAPPING UP
20
SO, DO WE HAVE A WINNER?
• I remain firmly on the fence ☺
• Depending on where you are in your integration solution, a specific option discussed in the presentation may serve you well, not at all or in conjunction with others.
• Best Supporting Actor/Actress:
• Self-Hosted IR
• Lifetime Achievement Award:
• KingswaySoft
• Best Newcomer Award:
• Skyvia
• Best Picture:
• On-Premise Gateway
21
FINAL THOUGHTS
• If you are adopting some/all aspects of the Power Platform, having a roadmap that includes on-premise data integration is essential.
• On-Premise Gateway tool provides little excuse for at least considering this integration upfront as part of your solution design.
• Use the other solutions discussed in this presentation in conjunction or to enhance the capabilities of the on-premise gateway.
• Long-term, unless strict legal or data residency concerns prevent this, prepare for the eventual death of your on-premise environments.
22
Q&A
THANK YOU FOR
ATTENDING!
Any more questions or feedback? Please get in touch!
Joe Griffin
https://crmchap.co.uk
joejgriffin
joejgriffin
JJGriffin