One of the most clichéd blogging tricks is to declare something popular as dead. These click bait, desperate posts are popular among click-focused bloggers, but not for me. Yet here I am, writing an “is dead” post. Today, this is about sharing my responses on-going social media posts. They go something like this:
OP: No one loves my data models any more.
Responses: Data modeling is dead. Or…data models aren’t agile. Or…data models died with the waterfalls. Or…only I know how to do data models and all of you are doing it wrong, which is why they just look dead.
I bet I’ve read that sort of conversation at least a hundred times, first on mailing lists, then on forums, now on social media. It has been an ongoing battle for modelers since data models and dirt were discovered…invented…developed.
I think our issues around the love for data modeling, and logical data models specifically, is that we try to make these different types of models be different tasks. They aren’t. In fact, there are many types, many goals, and many points of view about data modeling. So as good modelers, we should first seek to understand what everyone in the discussion means by that term. And what do you know, even this fact is contentious. More on that in another post.
I do logical data modeling when I’m physical modeling. I don’t draw a whole lot of attention to it – it’s just how modeling is done on my projects.
Data Modeling is Dead Discussion
One current example of this discussion is taking place right now over on LinkedIn. Abhilash Gandhi posted:
During one of my project, when I raised some red flags for not having Logical Data Model, I was bombarded with comments – “Why do we need LDM”? “Are you kidding”? “What a waste of time!". The project was Data Warehouse with number of subject areas; possibility of number of data marts.
I have put myself into trouble by trying to enforce best practices for Data Modeling, Data Definitions, Naming Standards, etc. My question, am I asking or trying to do what may be obsolete or not necessary? Appreciate your comments.
There are responses that primarily back up the original poster’s feelings of being unneeded on modern development projects. Then I added another view point:
I’ll play Devil’s advocate here and say that we Data Architects have also lost touch with the primary way the products of our data modeling efforts will be used. There are indeed all kinds of uses, but producing physical models is the next step in most. And we have lost the physical skills to work on the physical side. Because we let this happen, we also have failed to make physical models useful for teams who need them.
We just keep telling the builders how much they should love our logical models, but have failed to make the results of logical modeling useful to them.
I’ve talked about this in many of my presentations, webinars (sorry about the autoplay, it’s a sin, I know) and data modeling blog posts. It’s difficult to keep up with what’s happening in the modern data platform world. So most of us just haven’t. It’s not that we need to be DBAs or developers. We should, though, have a literacy level of the features and approaches to implementing our data models for production use. Why? I addressed that as well. Below is an edited version of my response:
We Don’t All Have to Love Logical Data Modeling
First of all, the majority of IT professionals do not need to love an LDM. They don’t even need to need them. The focus of the LDM is the business steward/owner (and if i had my way, the customer, too). But we’ve screwed up how we think of data models as artefacts that are "something done on an IT project". Sure, that’s how almost all funding gets done for modeling, and it’s broken. But it’s also the fact of life for the relatively immature world of data modeling.
We literally beat developers and project managers with our logical data modeling, then ask them “why don’t you want us to produce data models?” We use extortion to get our beautiful logical data models done, then sit back an wonder why everyone sits at another lunch table.
I don’t waste time or resources trying to get devs, DBAs or network admins to love the LDMs. When was the last time you loved the enterprise-wide AD architecture? The network topology? The data centre blueprints and HVAC diagrams?
Data Models form the infrastructure of the data architecture, as do conceptual models and all the models made that would fill the upper rows of the Zachman Framework. We don’t force the HVAC guys to wait to plan out their systems until a single IT application project comes along to fund that work. We do it when we need a full plan for a data centre. Or a network. Or a security framework.
But here we are, trying to whip together an application with no models. So we tell everyone to stop everything while we build an LDM. That’s what’s killing us. Yes, we need to do it. But we don’t have to do it in a complete waterfall method. I tell people I’m doing a data model. then I work on both an LDM and the PDM at the same time. The LDM I use to drive data requirements from business owners, the PDM to start to make it actually work in the target infrastructure. Yes, I LDM more at first, but I’m still doing both at the same time. Yes, the PDM looks an awful lot like the LDM at first.
Stop Yelling at the Clouds
The real risks we take is sounding like old men yelling at the clouds when we insist on working and talking like it is 1980 all over again. I do iterative data modeling. I’m agile. I know it’s more work for me. I’d love to have the luxury of spending six months embedded with the end users coming up with a perfect and lovely logical data model. But that’s not the project I’ve been assigned to. It’s not the team I’m on. To work against the team is a demand that no data modeling be done and that database and data integration be done by non-data professionals. You can stand on your side of the cubicle wall, screaming about how LDMs are more important, or you can work with the data-driving modeling skills you have to make it work.
When I’m modeling, I’m working with the business team drawing out more clarity of their business rules and requirements. I am on #TeamData and #TeamBusiness. When the business sees you representing their interests, often to a hostile third party implementer, they will move mountains for you. This is the secret to getting CDMs, LDMs, and PDMs done on modern development projects. Just do them as part of your toolkit. I would prefer to data model completely separately from everyone else. I don’t see that happening on most projects.
The #TeamData Sweet Spot
My sweet spot is to get to the point where the DBAs, Devs, QA analysts and Project Managers are saying "hey, do you have those database printouts ready to go with DDL we just delivered? And do you have the user ones, as well?" I don’t care what they call them. I just want them to call them. At that point, I know I’m also on #TeamIT.
The key to getting people to at least appreciate logical data models is to just do them as part of whatever modeling effort you are working on. Don’t say “stop”. Just model on. Demonstrate, don’t tell your teams where the business requirements are written down, where they live. Then demonstrate how that leads to beautiful physical models as well.
Logical Data Modeling isn’t dead. But we modelers need to stop treating it like it’s a weapon. Long Live Logical!
I had so many more questions I wanted to talk about during my recent State of the Union of Data Modeling 2016, but one hour goes by quickly when you have tools, industry, professionals, standards and user groups to cover. I’m interested in your observations and comments about these questions:
Has data modeling accomplished all it needs to? Are we just in the maintenance phase of data modeling as a practice and profession?
What industry trends (tools, processes, methods, economics, whatever) are impacting (positive or negative) data modeling the most today?
How has the cost of data modeling changed since 1980s?
How has the return on data modeling changed since the 1980s?
How has risk changed in data modeling since the 1980s?
Data Modeling tools have so much maturity of features in them today. But along with that prices have reflected those changes. How have the prices of enterprise data modeling tools impacted data modeling on enterprise projects?
Have you worked with any non-IDEF1x/IE data modeling notation recently?
Have you worked with any open source data modeling tools?
What new features/enhancements/changes would you like to see in data modeling tools? Processes? Notations?
Why haven’t we solved the “no one loves me or my models” problem more widely?
I’ll add my thoughts on these in the comments, but I’d like to hear your responses as well.
Subscribe via E-mail
- September 2016
- August 2016
- June 2016
- May 2016
- April 2016
- March 2016
- February 2016
- January 2016
- December 2015
- November 2015
- September 2015
- July 2015
- June 2015
- May 2015
- April 2015
- March 2015
- February 2015
- January 2015
- December 2014
- November 2014
- October 2014
- August 2014
- July 2014
- June 2014
- May 2014
- April 2014
- March 2014
- February 2014
- January 2014
- December 2013
- November 2013
- October 2013
- September 2013
- August 2013
- July 2013
- June 2013
- May 2013
- April 2013
- March 2013
- February 2013
- January 2013
- December 2012
- November 2012
- October 2012
- September 2012
- August 2012
- July 2012
- June 2012
- May 2012
- April 2012
- March 2012
- February 2012
- January 2012
- December 2011
- November 2011
- October 2011
- September 2011
- August 2011
- July 2011
- June 2011
- May 2011
- April 2011
- March 2011
- February 2011
- January 2011
- December 2010
- November 2010
- September 2010
- August 2010
- July 2010
- February 2009