Brick and Caleb discuss the topic of Master Data Management (MDM) for analytical data. It's conceptually simpler than it might at first seem, but takes commitment to implement and maintain. No Snickers bars were harmed in the making of this episode.
Blue Margin helps private equity and mid-market companies organize their data into dashboards to support strategy execution and create a culture of accountability. We call it The Dashboard Effect, the title of our book and podcast.
Visit Blue Margin's library of additional BI resources here.
#masterdatamanagement #MDM #businessintelligence #reporting #BI #dashboard #datawarehouse #azure #powerbi
Welcome to the Dashboard Effect Podcast. I'm Brick Thompson.Caleb Ochs:
And I'm Caleb Ochs.Brick Thompson:
Hey, Caleb, how are you doing?Caleb Ochs:
Oh, not too bad Brick, How are you?Brick Thompson:
I'm doing well, thank you. Good. I think we're going to stay on this theme of doing shorter podcasts. I think we'll shoot for about 10 minutes on this one. Our topic today is a piece of data governance, known as Master Data Management. And we're just going to kind of talk about this at a high level not get into a lot of detail, the technical detail about how to implement, but just want to kind of introduce people to the concept and talk about some of the higher level considerations.Caleb Ochs:
Yeah, and I know for me, it's gonna be a struggle to stay out of the technical weeds. So you might have to keep me honest.Brick Thompson:
All right, fair enough. All right. So do you want to describe what Master Data Management is conceptually?Caleb Ochs:
Yeah, I can do that. So Master Data Management at a high level, is a way to, it's really a process, for creating and maintaining a proper, consistent view of your organization's data all throughout the organization. Yeah, I guess that that sums up pretty well, anything you wouldBrick Thompson:
Yeah, well, I mean, that's high level. But let add? me let me try to bring it down to earth a little bit, just with an example. And you know, there could be, we could come up with 100 examples, but let me give one from one of our clients. So we had a client who had a business that operated vending machines, 1000s of them. They had fleets of trucks out there and big warehouses. And so they had a lot of data around the products that they were selling in these vending machines. And they were private equity owned, and buying lots of smaller vending machine companies. And as they brought those companies together, they would bring data in from those newly purchased companies. But they wanted to be able to see data rolled up across them. So, a simple example of a problem they would run into is that they might be selling Snickers bars, in these various different companies. And in one company, a Snickers bar might be listed as "Snickers Bar". And in a second company, it might be listed as "Snickers." And in a third company, it might be "12 ounce Snickers." And in fourth company, it might be "Snickers bar, 12 ounces." And those were all the exact same product. But in order to be able to report in a rolled up way, how many Snickers bars, 12 ounce Snickers bars did we sell, they had to use master data management to be able to tell the analytics system that those four things were really the same thing.Caleb Ochs:
Yeah, yeah. With that company was ridiculous. I mean, they had so many products. I don't even remember the number it was hundreds of 1000s of different variations of products. It was crazy. There was like 10 different types of Snicker bars.Brick Thompson:
Right. Right. And probably, you know, 50 different descriptions for those 10 different types.Caleb Ochs:
So in this case, you know, there are different types of Master Data Management, operational versus analytical, and so on. We're just talking about analytical here. So just Master Data Management, or MDM is how it's abbreviated, that's related to analytical systems, BIi systems. So in order to make it possible to do reporting in a rolled up fashion, we had to have some way of getting that data, mapping that data so that a Snickers bar is a Snickers bar,Caleb Ochs:
Right. And that hits on a key point of operational Exactly. All right. So keeping you to our time limit, versus analytical, right? Operational would mean that you go to each of the systems, and you say, Snickers bar needs to be Snickers bar in all of the systems and you take on this giant process of doing that and make sure that it's all set up right. And then when you add a new product, you make sure that it's the same across all of the systems. So that's more operational master data management. But when you get to the analytical side, it's removed from the source. So the benefit of that is that those, let's call them business units, that have been running on their respective systems, they get to keep doing what they're doing. They don't get bogged down by red tape of, if you put in a new product, you have to now go into the system and say that this is a new product, and then whatever that would entail. Essentially, it's just doing it in the reporting. So you're taking what's in the operational systems and putting them and grouping them into what they should be. conceptually, how do you do that? So conceptually, this is, that's a key word here. There's a big upfront effort, right. You've got... And it really depends on the size of your data... But this example, and I think it's helpful to stay in that context of the vending machines, is they had so many products. So it was a huge upfront effort to go through all of those products and say, this product is that product, this product is that product and so on. It takes a long time. Once you get through that, then it's more of a maintenance thing. But essentially, what that looks like in your data is you have the original name, because you always want to map back to the original system, at some point. You never want to get rid of that ability. So you'd have your original name, but then you'd also have a cleansed name right alongside of it. So you may have those four different iterations of Snickers. But next to each of those records is the cleansed name, that should be 100% consistent,Brick Thompson:
Okay, so that when you get to your reporting, you're using that cleansed name column.Caleb Ochs:
And the way that you make that happen is... actually it takes some work up front. You need a subject matter expert. You need some kind of software tool. And hopefully that software tool has good fuzzy logic that will then present to that subject matter expert, "Hey, I think these two things, or these three things may actually be the same thing." And then that subject matter expert will go through and validate and say, yeah, these are the same thing. And we're going to call them all Snickers bar. And so that upfront load can be significant. Now it can be small; you maybe get it done in a week. But it may also be two months to get it done. The good news is, once you've done that upfront lift, generally it becomes just a much smaller maintenance task going forward. So you still need a subject matter expert, but they're now operating on an exceptions basis. So they might sit down once a day or once a week, sit down with their master data management system and be fed a group of new records where the system says, "Hey, I don't know how to map these. I haven't seen these before, I think they might map to this thing." And then that subject matter expert can say, "Yep, it does or no, I'm going to create a new one for it." That's very conceptual. Different systems handle it differently, but that's basically how it goes.Caleb Ochs:
Yep. Yeah, that's right. And the interesting part about that is, if you think about it, again, I'm trying to stay out of the technical weeds, but as soon as you've done that mapping, the side benefit that you get is that a Snickers bar, let's say it does have the weight on it, but only one system has the weight, then you can apply that weight to all of the other records as well. So you can enrich your reporting out of the other systems as well. If you do that mapping correctly,Brick Thompson:
That's right. And you can go back and edit what that common name is going to be at some point, and the system will automatically apply those to all of those things that have been linked to that description.Caleb Ochs:
Right, right. And I think you also hit on a key thing, is that you do need a piece of software. You're not going to, I mean, you technically you can do some fuzzy matching and build something in the database itself. But you ultimately need to present something to a user, and there has to be some manual intervention.Brick Thompson:
Yeah, and there are tons of these systems out there. We don't have time to cover those here. We've used a few of them. They all sort of do the same thing, some better than others. It's a very specialized area of data governance. And so there's a lot of resources out there to do it. I think one of the points I want to make, though, is that ongoing, you do need a human resource. And hopefully, you know, after you've done the upfront lift, it's a it's a very small need, but it doesn't go away. You have to keep your data clean as part of the governance process.Caleb Ochs:
Yeah, that's right. I'm actually just reading something that says that Gartner estimates that 85% of master data management projects fail.Brick Thompson:
Does it say why?Caleb Ochs:
It lists out some reasons, but you know, I don't think we're going to fit that in our timeline here. Maybe we'll go into that another time. But you know, that's a big, we've seen it. I mean, it's a complicated thing to do, you know, and it is it's very intensive. So if you're going to do it, you have to be intentional about solving it.Brick Thompson:
I think I can only think of one example where I saw it, I wouldn't say fail exactly, but kind of limp along. And it was really that that ongoing cleanup was not made part of somebody's process, a responsibility of some SMEs position. So in this particular case I'm thinking of, they did the initial cleanup fine. But then ongoing, they just didn't have the commitment to keeping it there, at least while we were working on it. It's possible when they took over the system that they saw the need for that and got it, but I think that is one of the bigger challenges, is somebody has to own that.Caleb Ochs:
Yeah, you're right. You're right. I mean you let it go for too long, and then it's another huge effort to clean it up again. You've got to stay on top of it.Brick Thompson:
I was thinking about, oh, we're at 10 minutes. Well, really quickly, I was thinking about the different types of data that this typically shows up on. Product data is a big one. The other big one is customer data. So especially in private equity on companies that are doing roll ups, you'll find that two companies that are being brought together actually serve the same customers, but they almost never have that customer listed the same way in their systems. So that's the second big area that it's really useful to do this Master Data Management clean up and maintenance so that you can do good rolled up reporting.Caleb Ochs:
That's right. Yep.Brick Thompson:
Okay. I think that covers it. Caleb, thank you.Caleb Ochs:
40 seconds over. Or something like that.Brick Thompson:
All right. Talk to you soon.