develop’your’analysts,’and’they’ll’pay’for’themselves’’...today’s’agenda’[00:13]’!...
TRANSCRIPT
Develop Your Analysts, and They’ll Pay for Themselves
Develop Your Analysts, and They’ll Pay for Themselves [00:01] [Peter Monaco] Hello everyone and welcome to our webinar. So I’m here with Russ, he’s here, and going to help me answer some questions and then also fire me if I say something stupid.
Today’s Agenda [00:13] So to start, let’s go over the agenda. There’s a lot of ways obviously to cover this topic and we’ll be talking about five – culture, the processes that your analysts can be a part of, the different skillsets that your analysts need, the type of involvement that they need in your organization, and finally the barriers, basically getting in and out of the way for them.
Poll Question #1 If you could hire only one analyst, would you pick: a) A domain expert; b) A data wizard; or c) A visualization guru [00:46] But before we begin, I wanted to ask one poll question. If you could hire only one analyst, would you pick a domain expert, a data wizard, or a visualization guru? [Facilitator] Alright. Peter, we’ve got poll question up right now. So, if you could hire only one analyst, would you pick a domain expert, a data wizard, or a visualization guru? We’ll leave that open for a few moments to give everyone that’s on the line a chance to respond. Alright. Let’s go ahead and close that poll and show the results. Well Peter, it looks like we’ve got 21 percent responded a domain expert, 54 percent responded a data wizard, and 24 percent responded a visualization guru. [Peter Monaco] That’s actually really interesting. So we have the chance to ask the same question at the Healthcare Analytics Summit. And the results there were a little bit different and swung more in the direction of a domain expert. And so, I think this kind of gets that there is a lot of
different ways where we think we can stuff the perfect analyst. And I hope to cover some ways in which we can develop analysts across all three of these areas.
What do we mean by “analyst”? [02:11] But first, we should talk about what do we mean actually by an analyst. There are really multiple terms that encompass these same skill sets. We can talk about them in terms of being architects, business intelligence developers, analysts obviously, and the one I relate to most, being nerds. But they all revolve around the same basic skillsets, that is talking to and manipulating data using SQL, ETL, modeling, but then also analyzing and communicating those results through some visualization tool, Excel, or domain expertise. We will use the term “analyst” to describe all of these. So really think about more the skills versus the titles and what it means in your organization.
Culture An environment fostering improvement [03:09] So with that, let’s talk about how they fit in to your culture.
Motivating with value and purpose [03:14] One of the things that we realize is that you need to motivate them with value and purpose. There is this incredible talk on what motivates us that was given by Dan Pink in a TED talk, and he talks about what are the things that actually motivate someone. And money is what most of us think of, but not in the way that you think. It’s not pay us millions but pay us enough to take money off the table. Get them thinking about the actual work versus the pay. It’s a reflection of how much analysts are valued in your organization, it helps keep good analysts around. And we’re not talking again paying in the 99th percentile of the pay range but something closer to 60 to 75 percent. They are not trying to pay them as little as possible but just enough so that they are focused more on the work. With that, he also talks about these two motivators – the first one being autonomy, the desire to be felt directed; the next one mastery, the urge to get better at stuff; and finally purpose. We’ll talk more later on autonomy and mastery but now I want to focus on purpose and how it rolls into the organization.
Purpose [04:42] It’s this idea from Steve Jobs that gives us some insight into how we need to be thinking about an analyst’s role. “It doesn’t make sense to hire smart people and tell them what to do, but hiring smart people to tell us what to do.” Well I’m not saying the analysts should rule the organization. But if you think about where they sit in your org, analysts create analytics. They’re looking into the data that’s being generated from the organization. But they need to actually be leveraged by people who are ready, able, and eager to use it. It needs to be adopted.
Analytics – Best Practices – Adoption – Outcomes Improvement [05:25] And if you think about where these analysts sit in terms of outcomes improvement, this is where they can start to derive that purpose. The best analytics and the best best practices are really nothing without that adoption, without those people who are going to use those analytics on the frontlines to change behavior and therefore improve outcomes – because outcomes improvement will stall without this adoption.
The role of an analyst is outcomes improvement not “building a dashboard” [05:53] So surround your analysts with people who are going to use the value they provide because really the role of an analyst is outcomes improvement, not “building a dashboard”. I’ve worked with nursing directors who have sat down with me for hours to really explain their passion for the work, why it matters, and why they are so excited to use data to change. So you need to surround your analysts with people who are data-‐driven.
Processes Fail fast [06:20] So once you’ve established the culture, there are some processes that could really stunt or delay development of analysts.
Poll Question #2 If data surfaced showing a provider’s dismal performance, how comfortable would you be presenting the data in a meeting with that provider present? [06:30] But first, another poll question. So if data surfaced showing a provider’s dismal performance, how comfortable would you be presenting the data in a meeting with that provider present? [Facilitator] Alright. We’ve got that poll launched. If data surfaced showing provider’s poor performance, how comfortable would you be presenting the data in a meeting with that provider present? Would you be very comfortable, slightly comfortable, it depends on the provider, I’d hide the provider names, or I’d call in sick? So we’ll leave that open for a few moments and also remind everyone that if you have any questions or comments, please be sure to use the questions pane in your control panel. Alright. Let’s go ahead and close this poll and see how folks have responded.
Alright. It looks like 29 percent said they would be very comfortable, 18 percent said slightly comfortable, 34 percent said depends on the provider, 18 percent I’d hide the provider names, and 1 percent did say they’d call in sick.
Poll Question #2 [07:33] [Peter Monaco] That seems to really match up well with the same question we asked at the Healthcare Analytics Summit. So we asked that same question in terms of 2 percent of the person actually attending, HAS and the analyst. And what we saw is that those attending may not have been analysts. So I think it was a less heavy audience in terms of the analyst position, would be more comfortable than the actual analyst in providing this information in the meeting. And I think from this, you can see that it’s really kind of hitting this in terms of comfortability, whether a good chunk, almost a third, are very comfortable, but then there’s almost 50 percent where it really depends.
Analytics are still scary [08:23] And I think that means analytics are still scary, even in mature organizations. So there’s some things to remember. One, you need a baseline. You need to measure if you want to improve. So you need that starting point. The next is that analysts are only the intermediaries between the data and the people creating it. They’re the “messengers”. And the last one, which I’ll then emphasize most is that errors are a part of this process. So to kind of get at that poll question. It’s why we use factors of agile development and it’s something that we use as really a trouble detector. We want to have full scheme into trouble so that in the context of finding truth in the data, bad news is good news.
Failing Slow [09:15] If something is going to fail, you want it to fail as quickly as people, so that we can get closer and closer to not the right answer but the best answer. So one way to think about it is the opposite. So if we talk about failing slow, with tight boundaries of failure, with implied consequences, if you fail, it changes the process from a learning opportunity to a long slow progress, where really interest and momentum go to die. Do it right the first time sends the wrong message to your analyst and your organization and says we can’t experiment, so we can’t learn from our mistakes and we can’t deviate from some plan that was originated before we had data to support it. So what happens when we champion failing fast?
Failing Fast [10:14] When you give your analysts some flexibility to really hone their problem solving skills and use them and you describe failure as something else, then suddenly the processes shifted from getting the right answer to getting the best answer and that those involved have collaborated together from the start and actually achieved something as a team, rather on deciding logic or rules without first seeing data. And really you’re robbing yourself if you’re an analyst and you’re robbing your organization of opportunities to develop.
The answer you find through failure is usually different and more useful Than the one you find avoiding it [10:56] And what we’ve seen is really interesting is that the answer you find through failure is usually different and more useful than the ones you’ll find in avoiding it. So, some examples of these are showing a dashboard of data at the very first meeting, if you have data to support it. Giving stakeholders something to actually respond to versus thinking theoretically about how to solve the problem. Give them something to hate, something to criticize. Otherwise, you’re going to spend weeks of defining a cohort or the best way to calculate or measure and over time everyone will start to lose interest and probably stop showing up to the meeting. The shot in the dark helps everyone realize that it doesn’t have to be perfect to get started. And really if you are avoiding surfacing something, that’s a pretty good indicator that it means it’s the first thing that needs to be surfaced. If it’s someone that you’re afraid to invite to a meeting, it probably means they are the first person you should invite to the meeting.
Skillsets Cultivate the right skills [11:53] So, what are the skillsets that need to be developing in your analysts, now that you cannot define the culture and some processes that will enable them to develop more quickly.
Does this look familiar? [12:07] As I walk through this little animation, I want you to see if this resonates with any of you. You have an architect working on something, a BI developer waiting for something, an end user not providing feedback until the product is actually polished and finished and everyone is okay with it.
Starting knowledge by role… [12:29] Everyone is really waiting for this waterfall to reach them rather than iterating and communicating along the way. Not only that, but we found that oftentimes each silo may start thinking of another person as lazy or incompetent simply because they don’t understand what each other is doing. This might be how people are thought of in your organization in terms of their skillsets. We have a data architect, an outcomes analyst, a Business Intelligence. There might be more or less with different roles. But really they’re kind of siloed into their area and their specialty. And what we’ve recommended is that over the long term, you encourage skillsets across the spectrum. This can include areas like predictive analytics and statistics sheet where you start to actually build out and encourage being aware of those skills. Your members might have preferences and weaknesses but they should be able to handle and talk about each area with familiarity. And I’m not saying to build an expert across everything because that is impossible and it’s not scalable. But really to encourage understanding of each area. So that’s great, Peter, you want us to hire an expert. But let’s talk about really where to start with this – because it might not mean a single person but a team.
Questions to start… [13:59] So what are the skills of your analysts? Take stock of the skillsets and their proficiency. You can do this for individual analysts but also do it for your team. Does your team cover the basis in all of these areas that are going to make them a well-‐honed machine in terms of building out analytics. Just as important is where does this team “sit”? So, when I worked on a project with a coworker or I know other individuals in our organization, when we’re working together on something, we sit next to each other but the emphasis is on building a data model or a visualization. We get together multiple times during the day and are conferring with each other, building out the back and front end simultaneously, letting each partner inform the other. And at least weekly, sometimes even daily, we talk to stakeholders and users for feedback. This might be impossible for some team setups if they’re remote for this notion of “sit”. But really you want to make them function as much like a single person that’s possible. So having daily stand-‐ups, having daily calls, having more room sessions with pizza and energy drinks. These are all very good things.
Why a “generalist”? [15:24] So why do we encourage this “generalist”? There’s a few things. So, I know I get a lot of job satisfaction and the fact that I’m able to shift my work across the spectrum and in a self-‐directed way. If you remember, that autonomy, that ability to be self-‐directed is deeply satisfying. But maybe more importantly, that I can actually start to walk on data model, the functionality, and visualizations and match those to a user request. So users have their requirements, their ideas for what they need to solve the problem. And the more I’ve been able to build out skillsets across the spectrum, the easier it’s been to translate those into analytics or certain visualizations that would best facilitate that answer. And we all know that depending on the BI tool, the data model used for each of these could be drastically different. But knowing that being aware of that will help me realize what type of data model I need to build out visualizations, and then even down to the databases and schemas that I can get a feel for and an idea of the actual data I need to support both.
[16:49] We know that all of this can be at last in translation and that’s why we talk about making it as seamless as possible – because I think there’s a lot of examples across the industries where that complexity of simply a user requirement could be translated in so many different ways. And I think especially in healthcare, there’s the complexity to caring for patients that is not kind to hand off. And on a smaller scale, the same thing could be said for building an analytic solution. So you want to make this as seamless as possible.
Building Skillsets [17:25] So let’s talk more about the actual skillsets that we need to build out. So how do we build out areas of weakness? Well, one of the first things you can actually do, and this might come into hiring, but there should be soft skills that drive your culture. I think I can safely say I was hired purely for soft skills not because of any technical prowess or domain expertise. I had no healthcare experience, a few years of SQL in a much simpler environment, and I perhaps stressed the truth in terms of having visualization experience. But think of soft skills as a support services for more technical skillsets. At Health Catalyst, we kind of encompass these three of smart, hardworking, and humble, and they drive a lot of our hiring physicians to the point where that will turn down candidates who might be extremely strong in one area but lack these soft skills. In terms of the actual hard skillsets, inch your way towards usefulness. So if you’re the analyst, you need to define the resources for basic training, and if you’re not the analyst, build out those resources for them. So you need to inch them along. So having this basic training of people listings going towards dummy projects or exercises and training modules for them to engage in on the way, and finally times where they actually get to apply it to real projects. This could be something like having them rebuild the product that’s already there, having them walk
through the model and the visualization and go through that full spectrum, have them redo it, or have them take the first shot in the dark on visualization or building a metric for a dashboard. The thing you always need to remember is keeping that balance between shadowing and mentorship and best practices with actually letting them hone their skills into fire, letting them drive something potentially slightly before they’re comfortable doing it, so that they can start to really hone out those skills. And again, we’re talking here about learning opportunities down to failure. So you need to give them chances to fail and chances to learn.
Visual Story Telling [20:02] One area, probably back to getting a special emphasis for is especially due to its minimization in our poll at the Healthcare Analytics Summit and then also in the one we just went through, is visual story telling. So, I think one of the reasons why it’s minimized often is because it is really easy to make things hard to understand. This can make you think this visualization stuff is really overrated. To you that’s not helping. I would like you to think of this map, where every piece of information available is on the map. But really if you think about it, do you need to know the land’s features? Do you need to know these town names that have nothing to do with the actual railway stops? Do you need to
know, because the railways are drawn to the exact scale and exact curvature? No. So it was actually redesigned to help with its single purpose.
Visual story telling It is easy to make things hard to understand [21:06] Which stop do I get off? It’s clean and it’s not the scale, because the scale didn’t matter. Its points are equidistant because that’s you need to know how many stops there, not how far part they are. And just one river for reference, not all the rivers.
Visual Story Telling [21:27] And there’s a few things that in the vein of that, that we would recommend reading. And there are these two books by Edward Tufte and Stephen Few, that with these tools, you’ll be able to really design simple tools with profound insights.
The most important question Should I use a pie chart? [21:44] And perhaps answer the most important question of should you use a pie chart? To which I will borrow an answer from David Thorne who is a famous comedy writer. The answer is no, you should not. And I’m mostly kidding because it can make sense with some instances. But with those two books, you will actually be able to figure out when those instances will make sense and when you should utilize this chart. Everyone loves to hate on the pie chart.
Involvement Don’t minimize the analyst role [22:14] Another thing. So now that you have your analysts and you started to build out the skillset and surround them with the culture and the processes, how do you involve them? At what point in the process do you involve them in building out analytic solution?
Think of furnishing a room…[22:33] So, I’d like to have everyone on the call think about furnishing a room. Do you ever try and throw all of the furniture in the same room? By thinking, oh we need a couch and make a toilet and a sink and a fridge and a chair and a lamp and the list goes on. And then trying and make it all fit, as if it could accomplish the purpose simply by having all of the stuff in there. Or do you design a room that perhaps looks incredible but that no one uses because it’s completely dysfunctional? I think most people decide what the purposes of the room is fill it with things to accomplish that purpose. So that may be as simple as a powder room right for the kitchen where everyone entering it has the exact same purpose, or as complex as maybe a kitchen where there are multiple ways to solve this problem and it’s highly individualized.
But sometimes we ask the same thing of our analysts… [23:35] If you think about that analogy, we sometimes ask analysts a similar question. Here is this spreadsheet, here is the metric list, build us a dashboard. But really the development of your analysts depends on them flexing their problem solving skills and applying them to your problems, not a checklist of metrics that they are building out.
Awesome dashboard v1 [24:03] So what can happen is that I think all of us have been here at some point, where you come to the first meeting with your awesome dashboard v1 with everything on it. At the Healthcare Analytics Summit, we were able to kind of carry out this exercise and this was actually a strategy where maybe it’s some weird tetris problem. I know I’ve used this technique quite often. We tried to kill all the white space on a dashboard. It’s not recommended.
Awesome Dashboard v2 [24:37] Or if they come out from a different vein and think that they maybe need to make something that’s aesthetically pleasing or elegant or maybe just a feet of technical prowess but not necessarily useful.
Awesome Dashboard v3 [24:50] The sweet spot is when the act is tailored to the problem and it is designed to solve, not the metrics in it. So whether that’s something brutal in its simplicity, where it’s maybe a single one chart, or if they glorify spreadsheet. As long as you designed it with a purpose in mind and with the end users informing that purpose on how they use it, it’s going to be a lot better. And we’re not saying that failing is bad, but we want to minimize avoidable failures due to lack of a metrics filter and provide context and direction, not step-‐by-‐step instructions, so that the first failure is much closer. In fact, if you think about it, it would be much easier if the metrics were the only metrics you asked them to develop in the first place. So how do we decide what is truly important, even though everyone wants everything on the same sheet? You build filters into the process.
What metrics are you dealing with? [25:59] So what type of metrics are you dealing with? We are talking about perhaps accountability. The metrics have to be right. Think here of regulatory measures. For research, you might get into the – well it would be nice to know, or just in case, let’s bring in this metric. Something to explore. But for improvement, you only need just enough data. Decide on the purpose of your analytics application and give it that single track. In another sense, thinking about improvement. Don’t spend weeks on logic to capture that remaining 5 percent of patients. If 95 percent of the data directionally captures the process you are trying to improve, then move on. In fact, in most cases, you’ll find there’s probably less than 95 percent. So if you designed your dashboard around improving a process, don’t confuse accounting for 100 percent of patients with improving that process.
Involve, don’t minimize [27:08] The other filter, so we’ve talked about kind of filtering yourself in terms of what metrics you include. But the other filter needs to happen right up front and it’s with how analysts are involved in the process. So our first poll question, it has kind of highlighted that domain, and really understanding the data in that domain is key. So we need to not isolate analysts from it. Involve them in the very first meeting the metrics are being decided, when the problem is being hashed out because what that does is it builds curiosity, it builds passion, and that’s’ where value is really realized. So, one of the techniques – I’ll frame it this way. So writers use a framework developed by an author named Randy Ingermanson called the “Snowflake Method” and they start with a single sentence that describes the plot in the book. And from that same sentence, they break it down into three sentences that maybe cover with three main plot points of the book. And then from those three or four sentences, they build that to three or four paragraphs. They start to get into the characters. And those three or four paragraphs get into the three or four pages. And by the time we are done, you actually have seams that link all the way back up to the single purpose.
And so, we’d like to describe something similar, where you start with the goal of your application. Then you follow with the different aims that you are trying to accomplish, how are we going to accomplish this goal. And from there, metrics to support those aims. So going all the way from goal to metric. And finally, how do you use these metrics to accomplish the aims? With user stories that start from how someone is actually going to leverage these metrics to accomplish these goals and aims. So, not only will this context the way you build out this designer on metrics, not only will it filter out things but it will give the analyst an understanding of user needs as they use their dashboard.
Example sepsis “snowflake] [29:41] So let’s touch on a few examples. Let’s think in the area of sepsis. We have a goal of perhaps improving early recognition of sepsis in the emergency department. From there, we create a first aim of increasing compliance for routine sepsis screening protocol. And further down metrics, so talk about compliance by location and provider and maybe shift, a time from arrival to triage, and mortality rate of screened versus unscreened. The point of this is that it gives you something that’s limited in scope. The functionality that’s described here is not huge but it’s easily translated into a shot in the dark. The context and purpose are maintained and especially
the analyst was present to soak in the passion and experience of those providers who were present in the meeting.
Example Revenue Cycle User Story [30:43] Another transition to user story in a different area – so maybe your revenue cycle, where a typical layout for a user story is usually ‘As a’ ‘I want’ ‘So that’. So for example, as a patient access manager, ensuring patient registration, authorization and verification, so that there are no delays in billing, loss of revenue or patient dissatisfaction. You get the feeling for a user how a user will actually sit down and interact with the dashboard. The metrics are called out specifically but they are implied, as is the functionality of the dashboard and what metrics are important to see next to one another.
Do you want metrics or smart people using data to think about Your problems? [31:33] The really important point here is that the emphasis is taken off metric building and onto solving the problem – because in the end, you don’t want metrics. You want smart people using data to think about your problems. One example we’ve seen of this is if there is an organization who has decided to build a new wing. And once they’ve reframed the question of instead of building out this new wing, what is the problem, why are we even thinking about this in the first place. And giving the analyst that context versus validating some decision they have already made, they actually reframed the question and got a different answer where they needed to only keep their clinics open a little bit longer and that alone saved them millions of dollars and again the analysts paid for themselves just by allowing them to see the context of a problem and use their skills that have been developed to solve it.
Barriers Getting in and out of the way [32:40] The last thing that we’re going to talk about here is to really close the loop and talk about things that might get in the way of an analyst developer.
Barriers to data [32:52] So there are going to be barriers to data. So first, provide your analysts with full access to a testing environment. Sometimes they are limited in their effectiveness because there are some technical barrier restricting access to the EDW or to some source. So give your analysts the opportunity to actually build, break, and re-‐build data sets within your data warehouse. We need a sandbox to store anything and everything that we find useful and self-‐serve that curiosity and passion that we’re trying to build. Again, remember autonomy and mastery. We want to be self-‐directed in how we do things but then also get better at the things that we think are going to help us in our job. The next one, surprise, they need an EDW, and really with good reason. So, we had the opportunity again to ask a poll question to someone else to another group of people and the amount of time that is spent actually gathering data versus analyzing it, almost 80 percent of analysts spend 60 to 80 percent of their time hunting and gathering data. At that point, you might as well not call them analysts.
Barriers to data [34:24] So, how hard can it be to gather data for analytics? In a weak analytic system, very hard. So they are spending time hunting for data, gathering, compiling, running, distributing, all of these non-‐value-‐add tasks. Having a strong analytic system which hinges upon having an EDW shifts an analyst from non-‐value-‐add to value-‐add. And if you’re thinking about developing the analyst and getting the right skills cultivated, you help them to get really good at the value-‐add, not sending out PDS.
Other barriers [35:10] Here are some other barriers that might prohibit and prevent analysts from developing. One of them is access to people. So not only do they have access to the data they need, but the actual people they need to get the job done – are they in brainstorming meetings, are they friends, so to speak, with the frontlines, so will use what they build. You also would ask if you kill any “ghosties” that are haunting your analyst. So there may be some potentially mental barriers where they have reports that they have spent most of their time gathering, compiling and running. And that now if you think about that previous slide, their job is being shifted from that report running to actually interpreting data, and they may feel that their job is being replaced. But really if you think about it, we’re simply reframing their work into something that’s more value-‐add than ever before. The next idea, and this one is really important, is work prioritization. This is where the armor is dumb if you are someone who is an advocate of the analyst. And if you are the analyst, these need to be done for you, where you will start to become useful, especially to develop skills that are going to make you into a critical piece of outcomes improvement, and people will start to clamor for your services. So, you need to make sure that they are managing that claimer. Prioritize for them and be a barrier between them to people who might prevent them from
getting work done. Maintain an agreement backlog, whether that’s through some Cloud-‐based tool or spreadsheet, and provide that single source of truth for what they should be working on. The last barrier I’ll speak of is obviously a technology barrier. So, as we play in databases and more testing and running queries, this is going to utilize a lot of resources. But there’s been studies that have found that it only takes a few seconds of waiting before someone becomes disengaged and actually have to re-‐engage with that work, and it’s something in the area of three or four seconds. And they actually have to spend closer to seven seconds to get re-‐engaged. And that inefficiency could actually lead you to think, oops, I need more analysts, versus actually solving a technology problem. So you spend money on a faster server or money on four new analysts. But also in another sense, giving your analyst a voice into the tools that are going to facilitate their job. Let them test, let them play, and let their vote actually count in what gets deployed to them. One way we’ve seen this happen is through analyst actually getting new computers yearly that then got deprecated into roles that actually didn’t need a computational heavy computer. So don’t limit their effectiveness simply by the resource that you provided them with.
Lessons Learned [38:41] So here are some lessons that I hope you’ve taken away from this presentation today. One is that in your culture, the analyst’s role is not to build dashboards but to get outcomes improvement. The second is that the process is to encourage failing fast in that first meeting. The skillsets that really you want, perhaps your team is a generalist team that they can carry out the full spectrum of analytics through encouraging that in a single person but then also making sure your team is well rounded. The involvement of applications of analysts upfront so that your applications – we all know that they’re not just a technical problem. If they were, then we wouldn’t have this webinar. This is the hard thing. And involving your analysts in each of these processes is what really helps them develop. And lastly, eliminating barriers so that it’s easy for them to do their job.
Thank you [39:51] Thank you.
How interested are you in someone from Health Catalyst contacting you to schedule a demonstration? [39:53] [Facilitator] Alright. Thank you, Peter. Before we start with the Q&A session, we do want to say that our webinars are meant to be educational about various aspects of our giant industry, particularly from a data warehousing analytics perspective. We have had many requests, however, for more information about what Health Catalyst does and what our products are. If you are interested in having someone from Health Catalyst reach out to you to talk more about Health Catalyst, please answer this poll question. And while this poll question is up, we’re just going to pause this for a bit and then we’re going to shift around and make sure Russ is in here, so he can help to answer some questions. It’s getting closer here. Now, also while this is up, also we have had quite a few requests for the presentation slides. I would like to remind everyone that we will be providing the slides along with the recorded presentation and the results of the poll questions shortly after the event. So let’s go ahead and get to the first question. But first, we’ve got a nice comment, saying, “Excellent information. I look forward to applying this knowledge to my organization.”
QUESTIONS ANSWERS What presentation tool do you recommend for dashboards?
[Russ Staheli] I think really it depends on the goal of the application again. So if it’s potentially a more self-‐served environment, matching that request and functionality with the tool. We’ve seen a lot of utilization across a lot of tools from Excel to Tableau to QlikView. So really tailoring that to what the user request is and what the purpose of the application is. And as you become a data-‐driven organization, I think it’s wise to know that there is going to be multiple venues for these dashboards to present it back to them. And one of the things that is very important to us as an organization is identification and adoption. So trying to see how many users we can get using these tools and creating a culture of data in your system and bringing that to a single type of a tool, generally you can’t get the same level of adoption as you would if you were to enable end users to get their data to multiple schools.
Are there typical org charts used by healthcare systems for an analytics team?
You know, we put a couple of blogs out related to this same question and there’s generally two patterns which I think all of us would recognize that are being centralized and that are being decentralized. And as we had opportunities to interact with multiple health
systems and see both of those strategies at play, one thing that we don’t think works is if you go completely to one end of the spectrum or the other end of the spectrum. What we found to be most effective is centralizing certain skillsets that are very specialized and decentralizing more of the common skillsets that are more you pick with just across the organization. So it’s a little bit of both but it would be interesting hearing your feedback as well as to ways that you’ve seen to best organize these teams.
What questions do you ask to hire for soft skills?
[Peter Monaco] So, I know personally in a few interviews, I’ll take humility, for example. We’ll ask when they’ve been part of a project that failed, that didn’t go as planned, and how they actually responded to that and how they leveraged that learning into their next thing. Because I think one of the important things about humility, and I think it was emphasized if you were at the Healthcare Analytics Summit, is that humility depends on setting a goal that may be kind of they call it big hairy and audacious. And so, seeing times when and seeing repetitive examples of someone who set goals that might have been out of reach but always strove to payment. That’s’ a few ways that we’ve asked in our interviews to give that questions. [Russ Staheli] And we most definitely did not only interview based on soft skills. There was our filtering process and we have an opportunity to interview a lot of candidates as a growing organization for a time hiring about three analysts in this broad definition a week. We did have to come up with some structure and some specific questions that we would ask both to do technical assessment and soft skills assessment. I think there would be ways for us to share some of that if you want to reach out. I’m not sure what’s the best process for them to reach back out to us. [Facilitator] Well you can reach out through the webinars at HealthCatalyst.com in email address. That’s one of the best ways and all that will come through me and I can make sure that I could pass on to you, Russ, and you, Peter, to respond to that. I personally see the soft skills as being so important in terms of you talked about having access to people. If you don’t have, how do you build more and more access to people and if you’re given initial access, how do you keep that access to people and being able to
not only explain in a straightforward way what the data means but also to be able to keep those relationships, so that you can continue to explain those things. Those things are very important skillset.
As a domain expert who wants to migrate into a data analyst role, what would be the first three things I could do on my own to prepare for such a role?
[Russ Staheli] So, wow, I think Peter and I might have different opinions here and I’d love to take a little bit more time to think that through. Some of the things that come to mind are the most ubiquitous skillsets that we see across our analysts. One of those being SQL. You know, as much as we try to create and have tools or as many tools as there are in the industry that try to create a layer for action above that, it feels the core to the skillsets in all of our roles here at Catalyst, understanding how to manipulate the data. And those books that Peter suggested in his slide deck are another great resource that has helped individuals start to understand how do they tell a story with data, not just how do they interact with data, but what is the best way to represent data back to users. There are other, of course, user groups that you could join with individuals that have a similar passion and I’m not sure, Peter, if you want to jump in here. [Peter Monaco] No, I was reading these questions as they come in. But yeah, so I think you have a unique advantage potentially by having that domain expertise because you can then start to see what you would like out of the data. So with that domain expertise, start to self-‐serve those using books that’s kind of the Catalyst there. So, using the SQL training to get up the data you’d like and then the visualization books are agnostic to a visualization tool, and really that’s something that can translate across tools and tools are getting more and more self-‐served these days. So being able to understand what it takes to make a great visualization, I think those two books are a great starting point for that.
How likely is it that analysts want to develop as a generalist? I typically see them gravitate to a specific skillset.
There are two bands. There are two different camps here for sure. Of the individuals that we have hired over the past three or four years, which I think would represent probably about 70 to 80 different analysts. So I think we have a sizeable end to look at. I would say the vast majority or the majority, may not be the vast majority, but the majority have actually been very interested in learning and expanding their skillsets. I would maybe say 60 to 70 percent of those
individuals. There are definitely the groups that want to focus very deeply on a specific area, and those are very valuable individuals as well. And a balance of both generalists and specialists are very valuable. We have a team here at Catalyst that sort of is where the specialists all gather together, and it’s amazing to watch how the rest of the company leverages that, and they come down and ask the question, the deeper, deeper question, “Hey, can you help me with this data model? Hey, can you help me with this visualization? Can you help me with this complex SQL statement that we’re having performance issues on?” So shying away from having those types of individuals on the team is not what we’re trying to suggest. Just that there is a large percentage of these individuals that could be very effective as generalists, and that minimizing those hand-‐offs as you’re solving a user’s problem has shown great efficiencies and a way that have enabled our analysts to take in themselves.
Can you explain the difference between a data architect and a domain expert?
[Russ Staheli] Yeah. So, we start to get into some of this vocabulary stuff and in our industry, it’s still pretty mushy. I remember I worked at Intermountain Healthcare for about six years and my title was an analyst. Yet, daily I was doing what these people at Intermountain would call an architect role. I was doing that daily. I was down in the weeds data modeling, identifying data from source systems and understanding how I would merge that data together to tell a complete story. So for us, the way that we’ve usually used those terms internally, and again, these are defined in different ways and shared where you go, is that the architect is more of an individual that is doing data modeling, that is looking at very specific data tables and schemas and bringing them together and having those data elements interact with each other. Whereas a domain expert may not necessarily know the technical pieces. This is more of an individual that understands the content that would be found within those tables, say, yeah, it’s their discharge plan, or they might know a lot about the ADP records. Or is there, you know, a cardiovascular, they’ll know about their modules, it’s in their EMR system that supports that. When we talk about domain experts here at Catalyst, we’re not specifically identifying a technical skillset. More of an expertise and knowledge of the
specific content area. Anything you’d add there, Peter? [Peter Monaco] No. That like covered it.
What aspects of SQL are most important? DDL or a DML complex queries like Aggregation?
[Peter Monaco] You know what, it really depends on, I would say, the complexity of the problem you’re trying to solve and really how many sources, how close the source system matches the output you’re looking for. So, as you’re trying to pull stuff from multiple places that may be fragmented, you might have to get more complex with the queries you’re writing to normalize it and clean it versus something where data might be brought into a single EDW that has more common linkages, that that complexity might be able to be shifted. But we’ve seen cases where, as everyone talks about, there is a lot of different ways to define a cohort of patients. And so, that type of thing is something where that complexity won’t go away completely but it’s the best you can get in terms of getting source systems into an EDW will help in that respect. [Russ Staheli] Yeah, I would say as the maturity of health systems grows, we find that in that they start to use tools like Enterprise Data Warehouses in the way. We see the need for all of our analysts to have DDL knowledge going away. They’re slowing down. It’s not nearly as critical in the day-‐to-‐day life as DML or Data Manipulation Language. And so, we do see that the majority of our analysts, once there is a mature underlying structure readily available and great hard work and almost great things already set up, you’re able to focus more on the business problem and writing the code that would solve the business problem or address the business problem than you are at writing the technical pieces to ensure that those problems run in an efficient manner, etc., etc.
[Facilitator] Alright. Well we have a question about the titles of the two books that you mentioned, the two visualization books that you’ve mentioned. We do have those on the slides we’ll hand out. I actually have those up. One is the second edition, The Visual Display of Quantitative Information and the other is Information Dashboard Design: The Effective Visual Communication of Data. But the titles of both those along with their authors will be included in the slide that we will be distributing to everyone.
[Russ Staheli] Yeah. For those that enjoy the theory of visual design and want to understand the principles, the guiding principles that sit behind why you wouldn’t use a pie chart, for example, pick up the Edward Tufte book which is The Visualization Display of Quantitative Information. Or any of his books. He has some great other books as well. For those that are more just needing day in, day out practical guidance as to, okay, I’m showing this kind of data, I want to see what the best way to solve this problem and just have the answer for them, Stephen Few’s literature is more closely related to that. [Facilitator] Thanks Russ. We’ve got a few more questions. It looks like we’ve got time for at least two more questions.
QUESTIONS ANSWERS How would you go about teaching someone SQL for the strong background in requirement gathering and manipulating dashboards that is not as strong in SQL how they fit in the application tool?
[Russ Staheli] There are definitely an alignment with passion, where we don’t in our model force people to go down the generalist’s path and there’s generally an interest that kind of pushes people down, and we just enable that and make it easy for them to go after that passion. And what I had found, and I’m not sure if this is across the board, Peter, and I’d want you to get your thing, but what I had found with our team members is those that have the passion, we assign the project that kind of just pushes them into the fire more than anything else. They have the opportunity, they have support from their team members around that are stronger in SQL but there isn’t a specific book that we have them pick up. We pair them with someone and have them do a little paired programming for a part of it but also let them run on their own a little bit. Back to our concept of failing fast and allowing people to, you know, creating that culture of failure not being a problem, not being something bad. It’s generally how we see that happening and it’s been fairly natural and it’s been something that as people have been passionate, they’ve been able to grow into the skillsets for the most part. There are some that definitely have the passion but just will never have the technical jobs and there’s discussions that need to take place there but for the most part, we’ve seen them come up the speed through giving them real life projects and pairing them up with someone that has a strength in that area. [Peter Monaco] And really I think that matches well with my experience at Catalyst and it began, when I was first hired, I know a little bit of SQL but a different – it’s
actually mySQL, and so a different structure and a much more complex environment here. So I remember being inserted into a project that demanded domain expertise and visualization expertise, of which I had none. And that insertion of being surrounded by people who have strength in those skills, it’s really where I failed multiple times in a single day, where things I ran did not work, I didn’t get the right answer. And being able to bounce that across to someone rather than being scared about divulging information that I don’t know a lot and being able to have resources to help me out there, I think that goes well with more than anything but especially SQL.
Anything else in terms of a client experience that you can share where failing fast led to analyst success?
[Peter Monaco] Yeah. I was going to say, how many do they want? So I know more recently we worked with clients who didn’t actually want the results shown to providers until they were sure that everything was up to snap. And so, it’s actually a couple of months before that was shown. And so this is perhaps an example of not failing fast, where at the end of those couple months, they showed the results to the providers and it was way off, according to them. And what was realized through that experience is that if they had just included the providers from the very first meeting to hash out what their requirements were for those metrics, there would have been a lot simpler processes, something that maybe would delay two or three months because they were afraid of that failure could have been shortened to simply a month because everyone was in sync. And that’s really where the failing fast comes in for me – is whether I’m building a data model and I’m not sure how it’s going to fit in with the visualization, rather than conjecture, actually build the visualization on top of it. See how well it works. And if it doesn’t work, build back to the backend. Constantly reiterating between that continuum of building out a data structure to visualization, to actually having end users use it, and having each one inform the other.
[Facilitator] Alright. Well thank you so much. We’ve reached the top of the hour. So I would like to let everybody know, shortly after this webinar, you will receive an email with links to the recording of the webinar, the presentation slides, and the poll question results. Also, please look forward to the transcript notification we will send you once it is ready.
On behalf of Peter Monaco, Russ Staheli, as well as the rest of us here at Health Catalyst, thank you for joining us today. This webinar is now concluded.
[END OF TRANSCRIPT]