Browsing articles tagged with " data modeling"

What about XP, Agile, SCRUM, Lean…? #EDW2014

Mar 31, 2014   //   by Karen Lopez   //   Blog, Data Modeling, Events, Speaking  //  No Comments

At the upcoming Enterprise Data World 2014, I’ll be doing a half-day presentation on Driving Development Projects with Enterprise Data Models

Here are a few teases for what we will be talking about:

image

image

image

 

The Abstract:

Monday, April 28, 2014
08:30 AM – 11:45 AM

Level: 
Intermediate

Join this session to see how data fits in real-world enterprise development projects. We’ll answer such questions as:

  • "Who does what?"
  • "Why are we doing this?"
  • "Will it slow things down?”
  • “Will it work with agile development?”
  • "Will I have to actually talk to a data architect?"
  • “What about the Cloud?”
  • "What are the biggest mistakes teams make?"
  • "Will I still have a job?"

The session will feature demos of common data modeling-to-database processes, including reverse engineering, forward engineering, generating DDL, alter scripts, and more. You will leave with 10 tips for making model-driven database development successful in your organization’s culture and environment.

5 Classic Data Modeling Mistakes…And How to Avoid Them

Mar 12, 2014   //   by Karen Lopez   //   Blog, Data, Data Modeling, Speaking  //  No Comments

Slides from my frequent DAMA / Enterprise Data World presentation on Data Modeling mistakes.  You can click on the stopwatch in the player to auto-advance the slides.

There’s no sound; these are just the slides. If you’d like attend a presentation on this topic, ask your local user group (DAMA, ERwin, PASS, etc.) to invite me.

Data Modeling is Iterative. It’s not Waterfall

Mar 7, 2014   //   by Karen Lopez   //   Blog, Data, Data Governance, Data Modeling, Database Design, DLBlog  //  7 Comments

Sure, data modeling is taught in many training classes as a linear process for building software.  It usually goes something like this:

  1. Build a Conceptual Data Model.
  2. Review that with users
  3. Build a Logical Data Model
  4. Review that with users
  5. Build a Physical Data Model
  6. Give it to the DBA
  7. GOTO step one on another project.

And most team members think it looks like this:

image

Training classes work this way because it’s a good way to learn notations, tools and methods.  But that’s not how data modeling works when the professionals do it on a real project.

Data modeling is an iterative effort. Those integrations can be sprints (typical for my projects) or have longer intervals. Sometimes the iterations exist just between efforts to complete the data models, prior to generating a database.  But it’s highly iterative, just like the software development part of the project. 

In reality, data modeling looks more like this:

Data Model Driven Development - Karen Lopez

This is Data Model-Driven Development.  The high-level steps work like:

  1. Discuss requirements.
  2. Develop data models (all of them, some of them, one of them).
  3. Generate Databases, XML schemas, file structures, whatever you might want to physically build. Or nothing physical, if that’s not what the team is ready for. 
  4. Refine
  5. Repeat.

These, again, are small intervals, not the waterfall steps of an entire project.  In fact, I might do this several times even in the same sprint. Not all modeling efforts lead to databases or physical implementations.  That’s okay.  We still follow an iterative approach.  And while the steps here look like the same waterfall list, they aren’t the same.

  • There isn’t really a first step.  For instance, I could start with an in-production database and move around the circle from there.
  • We could start with existing data models. In fact, that’s the ideal starting point in a well-managed data model-driven development shop.
  • The data models add value because they are kept in sync with what’s happening elsewhere – as a natural part of the process, not as a separate deliverable.
  • The modeling doesn’t stop.  We don’t do a logical model, then derive a physical model, throwing away the logical model.
  • Data  modelers are involved in the the project throughout its lifecycle, not just some arbitrary phase. 
  • Modeling responsibilities may be shared among more roles.  In a strong data model-driven process, it is easier for DBAs and BAs to be hands-on with the data models.  Sometimes even users.  Really.

By the way, this iterative modeling approach isn’t unique to data models.  All the models we might work on for a project should follow this project.  Class diagrams, sequence diagrams, use cases, flow charts, etc. should all follow this process to deliver the value that has been invested in them.  That’s what Agile means in “the right amount of [modeling] documentation”. Data model driven development means that models are “alive”.  

If you are a modeler and re-enforcing the wrong perceptions of needing a waterfall-like approach to data modeling, you are doing it wrong.  You might be causing more pain for yourself than anyone else on your project.

Data Models aren’t just documentation checklist items.  They model the reality of the living, breathing systems at all points in its life.  They deliver value because they are accurate, not because they are “done”.

Data Modeling…Data, Numbers and Bikinis

image

My Twitter friend Geoff Crane ( blog | @papercutpm ) made this for me today.  So I’ll just leave it here for you.

Your Christmas Cookies Can Teach You About Data: Sugar Cookies

iStock_000014842218XSmall-Xmascookies

I don’t do a lot of baking. My kitchen is mostly the place where I blend my breakfast and enable my caffeine addiction. But my family has a tradition of making dozens and dozens of cookies every holiday season. Sugar cookies, No Bake Cookies, Snickerdoodles…the list just goes on and on.

As I was looking in my pantry for ingredients this year, I started thinking about how the process of producing cookies was a lot like data architectures. I may have been drinking. I’m pretty sure of it, actually. A lot. I mean I’m a lot sure I might have been drinking. A lot.

This week I bring to you a short series about Christmas Cookies and data.

Sugar Cookies

Yum! Probably the most common version of Christmas cookie is the decorated, cut out sugar cookies. Recipe books, blogs and food network shows make them look so easy. They contain just a few simple ingredients (butter, sugar, flour, salt, vanilla, eggs) that form the basis of almost all other higher forms of cookies.

What makes these special is what you do with that dough. The most exciting versions have you to roll out the dough, cut it out with cute cookie cutters, bake, cool, then decorate them. It’s just cutters and icing, right?

The Big Lie

I’m here to tell you that it’s all a lie. First, unless you have a lot of practice, the dough never rolls out cleanly because a whole lot of things have to go right first. Then you cut them out and they fall apart or tear. You’ll end up burning the first few batches until you know how your oven heats and how your baking pans work.  Maybe you need at Silpat liner. Or parchment paper.  Or an actual baker.

But no amount of equipment prepares you for the disaster of decorating them. They NEVER come out like the pictures. Those cookies on blogs and in recipe books are probably made by specialist magical cookie elves who spent their 10,000 hours learning to make cookies from Betty Crocker herself.  With Photoshop. I’m pretty sure every decorated cookie recipe is shopped worse than a Ralph Lauren model.

There are all kinds of warnings in the recipes: let the cookies cool on a rack. But who has time for that? Be agile and decorate them while the cookies are still in the cooling sprint. Oh. Crap. What the heck happened? If you haven’t spent a lot of time doing some test and training baking, your first set of cookies are going to be an embarrassment.

Burnt Cookie by Flare http://www.flickr.com/photos/75898532@N00/Cookie - http://joshuafennessy.com/

Silver Balls, Silver Balls…

And did you know that those little silver and gold balls that are the key part of the most beautiful cookies ARE NOT SUPPOSED TO BE EATEN? It says so right there on the label, “To be used as a decoration, not as a confection”. I bet you didn’t even RTFML. You’ve been unintentionally poisoning your kids and grandpa for decades. Or maybe intentionally. I won’t ask.

Silver Balls with warning (c) Karen Lopez

Lessons Learned

What does this teach us about data?

  1. Recipes make everything look easy. A lot of people see the recipe books and assume that making these cookies is very easy. And yet it’s difficult to get them right. The dough needs to be the right temperature and have the right ratio of ingredients to make the dough the right consistency. This requires not just a recipe, but a lot of practice.  It also requires good technique, the right tools and the right environmental factors.

    The same thing applies to data architecture. Sure, one can watch a 45 minute presentation on what all those boxes and lines are, but until they have applied the principles then lived with the results of their practice designs, they won’t really understand why one cannot just use melted butter or leave out the baking soda because it’s easier. It takes a lot of experience to be a good architect. Just like it takes a lot of experience to make beautiful decorated cookies.

  2. Demos of data modeling and design tools make everything look a lot easier than they are in real life. Part of this is because demos take time to give and they have to deal with the easy case. Sure you can migrate a database from Oracle to SQL Server by running a wizard. But you might not like the database or the data that comes out the other end. In fact, I can guarantee it you won’t. Migrating from one infrastructure to another always requires analysis, design, and implementation expertise. Decisions, even. Tools are never a substitute for design.
  3. If you are an amateur, you’re going to make a lot of mistakes. Heck, even professionals will make mistakes. But amateurs are going to make more.  It’s how it works.  You make mistakes, learn from them, get better. You’re going to burn a lot of data, and therefore users and ultimately customers.  You can read all the recipes in the world and watch all the episodes of Iron Chef, but living with the results of your design decisions is what helps you learn. It’s okay to make a lot of mistakes if you are learning in a class. Or are working on a development project iteration.

    Production, though, is like learning to cook your first meal for Christmas dinner for a close family of 20-30 people. It doesn’t scale well and you’ll just end up disappointing everyone in a big way. Heck, you might even kill some people with your bad design.  You might have some letters after your name, but until you get to the professional level, don’t call yourself a chef.  Well, you can, but your customers aren’t going to trust you after the second batch.

  4. You need to read and learn. Warning labels are a good start. The great think about most data principles is that they haven’t changed a lot. The technologies have, but not the foundations.  If you don’t read and learn, you won’t be in a position to deal with change that is coming whether you want it or not.
  5. Some ingredients for data actually don’t really help the data. Comma delimited data in a column is fast. It allows people to go around the whole data governance process. Stuffing internal-only customer data in to AddressLineFour is fine, right? Until someone prints that on the envelope and mails it to the customer. Sure, these cute workarounds are shiny and happy. You need to be able to see when people are proposing the equivalent of shiny silver balls. They are pretty, but not for use in real life. You can quote me on that.

There are probably a lot more lessons to be learned from Sugar Cookies, but I just wanted to cover the basics. Just like the ingredients for Sugar Cookies.

Big Challenges in Data Modeling–Data Model Governance

May 21, 2013   //   by Karen Lopez   //   Blog, Data, Data Governance, Data Modeling, Data Stewardship, Events, Speaking  //  2 Comments

Join me and an excellent panel of industry experts in discussing the state of Data Modeling Governance.  No, not Data Governance, but Data Modeling Governance.  This free webinar is Thursday,  23 May at 2 PM EDT.  While the formal part starts at 2 PM, you are free to join us 15 minutes before as we prepare for the event.  You can even post your Data Modeling Governance questions then so that we can answer them

We data architects spend a great deal of time advocating for organizations to treat their data as an asset.  We champion the efforts to set up stewardship programs and data governance councils.  We insist that data conform to enterprise naming and modeling standards.  We enforce data policies, measure data quality, report deficiencies and track anomalies.  But do we follow our own advice when it comes to managing “our” data – metadata and data models?

In this webinar, we’ll be tackling the questions of:

  • Do you have budget (money and time) to govern the data modeling environment?
  • How can we get the resources we need to properly govern our data models?
  • Who sets permissions and manages them?
  • When does data modeling by email work?
  • Are data modeling artefacts part of the production systems operations?
  • Are there multiple data modeling tools in your environment? For the right reasons?
  • Are we loving our data models as much as our data?
  • …plus more.

My panellists this week have a great deal of experience in working on a variety of enterprise environments:

  • Anne Marie Smith, Ph.D. is an Information Management professional and consultant with broad experience across industries. She has exceptional, demonstrated skills in business requirements gathering and analysis, data governance and stewardship, data architecture, data and process modeling, strategic data management, meta data management, data quality management, master data management, data warehouse planning and design, project management, and information systems methodology development.
  • David Loshin is the President of Knowledge Integrity, Inc, (www.knowledge-integrity.com), a consulting company focusing on customized information management solutions including information quality consulting and training, business intelligence, metadata, and data standards management. David is among Knowledge Integrity’s recognized experts in information management.
  • Pete Stiglich is a Senior Healthcare Data Architect at Perficient. He has over 25 years IT experience having worked in Enterprise Data Architecture, Data Management, Data Modeling (Conceptual, Logical, Physical, Dimensional, Data Vault, EDW), Data Quality, DW/BI, MDM, Metadata Management, Data Quality, and Database Administration (DBA)
  • You! I always consider the audience as the first panellist in a webinar.   We have a open, engaging webinar configuration where attendees can chat with each other and ask questions to the panel.

You’ll need to pre-register, but it’s fast and free.  Bring your questions, comments, snark and observations.  See you Thursday.

Oh, and if you are late reading this and the webinar has already happened: no worries. We record every event and post it to the Dataversity website. You’ll miss all the great chatter in the chat room, though.

Data Modeling is Dead? Join Us 24 Jan 2PM EST to Discuss

Jan 21, 2013   //   by Karen Lopez   //   Blog, Data, Data Modeling, Events, Speaking  //  1 Comment

Big Challenges logo

During this month’s Big Challenges in Data Modeling we’ll be talking about the state of data modeling. 

Last month Tom LaRock (@sqlrockstar | blog) wrote a post Data Modeling is Dead, Long Live Data Modeling

Data modeling is dead. It is a product of an era that has passed; that of corporate silos that created their own versions of software to suit their own needs.

That is no longer the world in which we live. That era was one that had high costs associated with building and maintaining a database of customers.

Today’s era is one where you can subscribe to Salesforce.com for just a few dollars a day. You can decide for yourself to run a new report. How much did that same report cost in the old era? How long would it take for IT to deliver that report? That’s why businesses today are using such services, because it reduces time and costs.

You need to read the whole post to get his position, but I find that his take on the state of data modeling is common in the IT world.  I posted a link to his blog post to a LinkedIn group and there was an extensive discussion.

I’ve invited Tom to join a real-world data architect and me to talk the current state of data modeling and what the future holds for data architects. 

This Month’s Panelists

Thomas LaRock

  • Thomas LaRock is a seasoned IT professional with over a decade of technical and management experience. Currently serving as a Technical Evangelist for Confio Software, Thomas has progressed through several roles in his career including programmer, analyst, and DBA. Thomas holds a MS degree in Mathematics from Washington State University and is a member of the Usability Professional’s Association. Thomas currently serves on the Board of Directors for the Professional Association for SQL Server (PASS) and is a SQL Server MCM as well as MVP. Thomas can also be found blogging at http://thomaslarock.com and is the author of DBA Survivor: Become a Rock Star DBA (http://dbasurvivor.com).

Gabriel Tanase

  • Gabriel Tanase is a Data Architect currently with a mid-size business consulting organization. He has accumulated some tough years of ordinary real-life data modeling experience and the not-unheard-of distinction of having taught it academically before really practicing. He is currently specializing in keeping everybody else in the project happy while quietly enforcing information meaning and integrity.

This webinar is free to attend, but you must register.  It officially starts at 2PM EST, but you can join us at 1:45 when we start our prep. We’ll be taking questions via the Q&A and we offer a real time chat so that you can be part of the conversation, too. 

Pages:«123456»

Subscribe via E-mail

Use the link below to receive posts via e-mail. Unsubscribe at any time. Subscribe to www.datamodel.com by Email


Categories

Archive

UA-52726617-1