Browsing articles tagged with " data modeling"

Speaking: DAMA MN (Minneapolis) 17 October – Career Success in Turbulent Times + More

Oct 4, 2011   //   by Karen Lopez   //   Blog, Data Modeling, Speaking  //  1 Comment

BarbieEight

The next few weeks are going to be busy for me: I’m speaking at SQLSaturday Oregon (Portland) and then at the PASS Summit in Seattle.  Right after that I’m headed on DAMA chapter speaking tour.  First up on that tour is DAMA MN on Monday, 17 October 8:30 AM – 11:30 AM where I’ll be presenting:

Career Success in Data Management in Turbulent Times

A workshop on issues and ideas that today’s data architects and modelers can do to build their careers and networking skills with other data management professionals.

Workshop topics will include:
• Demonstrating your expertise
• Building a portfolio of your success stories
• Getting others to sell your skills and business value
• Building & extending your data management skill set
• 10 Steps to highlighting you and your work

Bring your thoughts, ideas, and experiences.

You’ve Just Inherited a Data Model:  Now What?

The good news is that someone else has done the hard work of architecting a data model and you just have to take on minor maintenance…or is that the bad news? Or have you been tasked with implementing a pattern or industry standard data model? Perhaps a team member has sent the world’s best resignation letter and won’t be helping you with the model. Learn the 5 steps you MUST take before working with a new data model.

Attendees will also receive a detailed checklist for the 5 steps.

Details about the event are provided on the DAMA MN webpage.  I hope to see you there.

What the Heck is an Entity Instance Diagram?

Jul 27, 2011   //   by Karen Lopez   //   Blog, Data, Data Modeling, Social Networking  //  4 Comments

I was recently contacted by a friend who is taking a database design course for some help in understanding an assignment.  His first task was to create a conceptual data model and then prepare entity instance diagrams for that data model.  He was wondering what an entity instance diagram was.  So was I, as I had not heard that term before.  So I fired off a search and came up with only one hit:

Google Search for Entity Instance Diagram: 1 hit.

 

I’m pretty sure that’s the first time I’ve every unintentionally found only one returned result for a search term before.  Unfortunately that one webpage has broken images, so I couldn’t see what one looked like.  So I told my friend:

Facebook: Karen Lopez quote

 

I was thinking of examples such as the ones in Simsion & Witt’s  Data Modeling Essentials [aff link]:

 

Sample data table from Data Modeling Essentials.

 

I did some more looking around and using Bing, found one more result which pointed to a PowerPoint slide deck by Ellis Cohen from his course on Theory, Practice & Methodology of Relational Database Design and Programming. In this deck I found an example of what he calls an entity instance diagram, which pretty much is what I thought it was.  I’ve been creating an using these sorts of things to explain how a data model should be used but I never had a name for them.  I called them sample data, data prototypes, data validation , worked examples, or just examples.  Now we have a name and a TLA!

In Cohen’s slide, he’s using an Entity Instance Diagram (EID) to to demonstrate weak entities:

 

Weak Entity Instance Diagram

I usually use Excel to prepare these, as I can reuse the data for each one.  I even have an ER/Studio macro to generate the tabs in a spreadsheet (one for each entity/table selected in a submodel). This makes preparing the sample data go much faster. 

So it looks like we have an answer now for what the heck is an entity instance diagram….and I have a name for a technique that I use all the time.  If you have other examples of this term, I’d love to see them.

 

Checklist from 16 June DArchVC Presentation

Jun 16, 2011   //   by Karen Lopez   //   Blog, Data, Data Modeling  //  1 Comment

I’ve uploaded a much more detailed checklist for what to do when you inherit a data model from someone else.

You’ve Just Inherited a Data Model Checklist

What would you add to it?

Half Day Seminar 7 April #EDW11 – 10 Physical Data Modeling Blunders – Discount Coupon

Mar 30, 2011   //   by Karen Lopez   //   Blog, Data, Data Modeling, Database, Speaking  //  No Comments

imageFor 24 Hours of PASS Virtual conference,  I gave a short, 40 minute overview of 5 Physical Design blunders.  At Enterprise Data World in Chicago on 7 April, I’m giving a half day seminar on 10 Blunders (NOW with 5 BONUS BLUNDERS!).  Unlike the virtual conference, this workshop will have demos and labs – PLUS 5 BONUS BLUNDERS!

If you act quickly, I have a special deal for those of you who want to join me for this event: A discount coupon.  The normal one day admission is $795.  But if you register now, you can register for the Thursday full day, including my seminar, for only $195.  Yes, that’s a $600 savings to be part of the biggest blunderfest best seminar of the conference.

Coupon code DATACHICK for $600 off ($195 net total)

Use this registration link to get yourself set up.  And don’t forget to use the coupon code DATACHICK to save that $600.  There are still great hotel rates, too.

Join me at the world’s largest vendor-neutral data management conference.  See you there!

1980 Called and it Wants Its Deliverables Back

Feb 13, 2011   //   by Karen Lopez   //   Blog, Data, Data Modeling  //  7 Comments

 

imageI’ve been doing this data modeling stuff for a really long time.  So long that I just put "20+ years" on my slides…and I’m well past that 20.  However, having done this for a long time does not qualify me as an expert.  What qualifies me is that I know how to adapt my tools, methods and approaches as development tools and methods change.  What worked for projects in 1986 doesn’t necessarily work now. Not only do we have radically more advanced features in our tools, we have many more platforms to support.  We have a greater variety of development approaches.  Our architectures are much more distributed and much more complex than they were in the nineties.

Aerobic Excerise http://en.wikipedia.org/wiki/File:Aerobic_exercise_-_public_demonstration03.jpgBack in the mid-eighties I worked in the US Defense consulting business.  The somewhat serious joke was that consultants were paid $1 million for each inch of documentation they delivered.  It didn’t matter what was in the models or whether they were correct; it only mattered how many reams of paper were delivered.  That sort of "all documentation is great" mindset still exists well into 1999, long past its usefulness.   The world has changed and we data architects need to find a replacement for the publication fashion equivalent shoulder pads, thongs, leggings, skinny ties, and lace fingerless gloves.

Those differences mean that the deliverables we produce need to be radically different from what they were in 1999.  Our team members are no longer going to open up a 175-page prose document to find out how to use and understand the data model or database.  They don’t want all that great metadata trapped in a MS Word document or, worse, buried in an image.  The can’t easily search those and they can’t import that knowledge into their own models and tools. 

As much as I don’t want this to be true, no one wants to read or use massive narratives any longer.  Sure, if they are really, really stuck a quick write up might be helpful, but sometimes a quick video or screencast would be faster and easier to produce. If you are spending days or weeks writing big wordy documents, the sad truth is there is a high likelihood that absolutely no one except your mother is going to read or appreciate it…and she isn’t actually going to read it.

I’ve significantly changed what I produce when I release a data model.  I constantly monitor how these deliverables are used.  I look at server logs or general usage reports to continually verify that the time I’m spending on data-model related deliverables is adding value to the project and organization.  The main way I gauge usefulness of deliverables is by how urgently my team members start bugging me to get them up on the intranet where they are published. 

Here are my top 10 recommendations for approaching your data model deliverables:

  1. Get formal, lab-based hands-on training for staff.  Or use staff that are already trained in the tools and version of the tools they are using. You may be missing out on features in the tools that make publishing data models much easier that the methods you are currently using.  I had a client who was struggling to support an elaborate custom-developed application that they didn’t really know how to use or maintain.  It used a deprecated data model format to build an HTML-based report of the data model.  Sound familiar? Almost all tools provide a feature to generate such reports in seconds. 
  2. Move away from very large, manual documentation.   Think in terms of publishing data and models, not documents. Prose documents are costly to produce and maintain.  They do more harm than no documentation at all when they are not maintained.  The are difficult to search, share, and use.  This is not how the vast majority of IT staff want to consume information.  Team members want their data model data (metadata) in a format that is consumable, that can be used on a huge variety of platforms and that is interactive, not locked only in a PDF.
  3. Know how long it takes to produce every deliverable.  Having this information makes it easier for you and your team to prioritize each deliverable.  I once had a project manager ask if cutting back the automatically generated reports could save time for getting data modeling completed. I could show her that the total time to put the documents on the intranet was only about 5 minutes.   My document production data also helps other modelers estimate how long a release will take to produce.
  4. Stop exporting data for features that can be done right in the tool.  Move data model content that is locked in MS Word documents into the models or stop producing it.  Exporting diagrams as images and marking them up with more images means all that mark-up is unsearchable.  It also means that every change to the data model, even a trivial one, triggers a new requirement to recreate all those images.  Modern tools have drawing and mark-up features in them. Cost/benefit of exporting and annotating outside the modeling tool means you’ll always be spending more than your "earn".  You’re creating a data model deficit.
  5. Stop producing deliverables that require a complete manual re-write every time there is a new release.  Unless, of course, these sorts of things are HIGHLY valued by your team and you have evidence that they are used.  I’m betting that while people will say that they love them, they don’t love them as much as getting half as much data architecture work done.
  6. Focus on deliverables that are 100% generated from the tools.  Leave manually developed deliverables to short overviews and references to collaborative, user-driven content.   Wikis, knowledgebases, and FAQs are for capturing user-generated or user-initiated content.
  7. Focus on delivering documentation that can be used by implementers of the model.  That would be Data Architects, DBAs, and Developers.  Probably in reverse order of that list in priority.  Yes, you want to ensure that there are good definitions and good material so that any data architect in your company can work with the model even after you’ve won the lottery, the largest number of people who will work with the models are business users, developers, and DBAs.  Think of them as your target audience.
  8. Automate the generation of those deliverables via features right in the tools – APIs, macros, etc. Challenge every deliverable that will cost more to produce once than the value it will deliver to ever single member.
  9. Move supporting content that is locked into MS Word documents (naming standards, modeling standards and such) to a collaborative environment like a wiki or knowledgebase.  Don’t delay releases just to deliver those items.  Theses formal deliverables are important, but from a relative value point of view, they should not hold up delivering data models.
  10. Focus now on making a process that is more agile and less expensive to execute while also meeting the needs of the data model users.   If it is taking you more time to publish your data architectures than actually architecting them, you are doing it wrong.

 

While data modeling standards have stayed relatively stable over the last 10-20 years, our methods and tools haven’t.  If you are still modeling like it’s 1980, you stopped delivering value sometime around 1999.

Bad Data Models / Database Designs Kill – Under Modeling

Jan 6, 2011   //   by Karen Lopez   //   Blog, Data, Data Modeling, Database, DLBlog  //  3 Comments

imageDr. Danielle Ofri has a frightening post about how data design can cause serious consequences.  She talks about running up against a 1000-character limit when trying to create a patient history for someone about to undergo a risky surgery:

I panic for a moment, fearful that the computer has frozen and that I’ve lost all my work — something that happens all too frequently. But I soon realize that this is not the case. Instead, I’ve come up against a word limit.

It turns out that in our electronic medical record system there is a 1,000-character maximum in the “assessment” field. While I’ve been typing, the character number has been counting backward from 1,000, and now I’ve hit zero. The computer will not permit me to say anything more about my patient.

If you’d done any database design, you know that even if you design a good, business-driven design, others who use the database might apply their own rules to the data on the way in or out of the database.

I remember designing a Point of Sale system database for an appliance retailer.  Our data model needed to support the selling of high-end appliances as well as bulk purchases for high-end appliances.  So our transaction amount columns were significantly large.  A developer on the application team thought our monetary field lengths were insanely large, so he enforced  a limit on a transaction of $9,999.99 for the total transaction.   To make matters worse, this system went all the way into production with that limit.  So on day one of the roll out, sales people couldn’t sell the most highest margin items such as a professional quality stove ($14,000) or sell to developers who were buying 100 low end stoves in one transaction.  Their orders had to be chunked up into tiny $9,000 mini-transactions.  Order completion was taking hours instead of minutes.  Credit cards were being rejected for too many large amount transactions back to back.  In other words, a nightmare deployment because organizations trying to spend tens of thousands of dollars were walking out and going to other retailers with “real systems” to make their purchase.

However, no lives were being lost (that I know of).  Some people may have gone longer without heat (furnaces) or with rotten food (freezers and fridges), but in the overall scheme of things the impact on customers was not life and death consequences.

If we get back to Dr. Ofri’s situation, though, she was faced with a terrible data dilemma: how to describe a complex patient history in 1000 characters.  This number probably sounded like a huge number when the project team was adding that “feature” to the system.  I’d even bet their own test data only went as high as 200 characters or so.

I’m also guessing that since this system is a fairly high-risk project that some expert user (or many) was responsible for approving the design of field lengths.  Perhaps he or she also thought that 1000 characters was enough.

In desperation, I call the help desk and voice my concerns. “Well, we can’t have the doctors rambling on forever,” the tech replies.

That response from IT (even if it a help desk tech who has no clue as to why there is a limit) makes me mad and afraid at the same time.  You’ve all heard it in your design reviews, haven’t you?

  • That’s way too long of a field.
  • It won’t fit on one window without scrolling
  • No one has an e-mail address THAT long
  • You are over modeling it
  • That’s ridiculous.  No one is going to sit there and type that long
  • 255 was good enough for the last 10 years, it’s good enough for the next 10 years
  • The indexes on that column will be too large
  • Go ahead and make the column that long; we’ll just truncate it in the application

The business users are frightened by the negative comments and agree that 25 is sufficient for e-mail address, not even realizing that some of their own e-mail addresses are longer than that.

As I blogged recently about Over Modeling, it’s only over modeling if it doesn’t meet the business need.  Sure, we might make concessions to technical feasibility (“make every column 2000 characters, just in case”), but our designs should be business driven.

Dr. Ofri ends her story by saying:

I’ve finally condensed my patient’s complicated medical conditions to exactly 1,000 characters. I quickly hit “save” before I lose everything. I wish him good luck on his operation, wondering if his surgeons will have to condense the entire operative report to 1,000 characters as well. What happens if there are complications?

For my next medical evaluation, I think I will use haiku.

I don’t know about you, but I wouldn’t want to read that about my own patient record.

Next up: What could we have done differently on our project

No Man is a Failure Who Has Friends: 5 Architecture Tips from It’s a Wonderful Life.

Dec 17, 2010   //   by Karen Lopez   //   Blog, Data Modeling, DLBlog, Fun, Social Networking  //  4 Comments

This post is a rerun from 2009, but the truth is still there.

image It’s a Wonderful Life is a part of many a holiday tradition.  While most of you probably know this as a Christmas film, the links to the Christmas holiday are fairly weak.  There’s an angel, some snow, and a background Christmas tree with a bell.  Other than that the story itself could take place at any time of the year.  So if you’d put off seeing this film because you think it is all happy, shiny, Christmas cheer-ish, you need to think again.

George Bailey is a protagonist in a small town, Bedford Falls, located in upstate New York. He finds himself on the brink of financial ruin due to no fault of his own.  He goes to visit the town big wig and resident bully, Henry Potter, to get a loan.  Potter tells him that with no equity in his life insurance policy, he’s “worth more dead than alive.”  In a fit of despair, George decides to sacrifice his life so that his family and company (The Bailey Building and Loan) can survive. However, his plans are side-tracked by Clarence, his guardian angel.  George rambles that he wished he’d never been born.  Sounds pretty dark, doesn’t it?  Not that Christmas-y at all. Clarence sees this as a great opportunity to show George just what a great impact he has had on not just his community, but on the world.

So George is shown what his family, friends, town and world would be like if he’d never been born.  The new town is called “Pottersville”, after the antagonist Mr. Potter.  It’s not a good thing.  Clarence tells him:

“Each man’s life touches so many other lives. When he isn’t around he leaves an awful hole, doesn’t he?”

and

“No man is a failure who has friends.”

The “feel good” part of the film is when George gets to return to the world he wished he’d never been born into.  I’ll leave the assignment up to you to see how this all happens and how George and his family do once he returns.

I’ve seen this film at least 50 times in my life, perhaps even more.  In fact, most people would probably think that I’m a bit obsessed with this whole story.  To which I say: everyone needs a hobby.  This just happens to be one of mine.

So what does this have to do with architecture and data management?  I think plenty.

  1. Enterprise-class projects can’t be done by one person.  No one is a failure who has good team mates who collaborate well.  You don’t have to be friends with them, or even like them that much.  But you do have to find a way to collaborate with them.
  2. Architecture done well can have all kinds of impacts elsewhere.  Each architect’s work touches so many other parts of the architecture.  When it isn’t there, it leaves an awful hole…to be filled by a non-architect to do. One small great architecture component can have huge impacts on solution quality for a long time in the future.  When George is a kid, he saves his brother Harry’s life.  But in the Pottersville world, Harry dies.  Clarence tells George: Every man on that transport died! Harry wasn’t there to save them, because you weren’t there to save Harry. 

    If you aren’t there to create the architecture, or if the architecture you create isn’t used, then the good stuff it could deliver won’t be there when it is needed.

  3. Fastest isn’t always “bestest”.  In running George and Clarence out of his Pottersville bar, Nick the Bartender explains what works in the bad world: Hey look, mister – we code fast here for people who want to get lots of stuff done fast, and we don’t need any characters around to give the joint "atmosphere". Is that clear, or do I have to slip you my left for a convincer? Well, I paraphrased that quote.  But you get the point. Perhaps that was Nick’s collaboration method – using bouncers to run people out.  Some people feel that data and other architectures are just there for some sort of religious checklist nirvana.  It’s our jobs, as architects, to show them why architecture plays a key role project success.  A left convincer might work in the short run, but the best way to get support for an architecture is have one that works.  Good architectures need to be designed.  Hacking away on a pseudo architecture is more Pottersville than Bedford Falls.
  4. Architecture is more than drawing boxes and lines. George was fabulous at motivating people to do the right thing.  The first way he did this was by living by his own principles.  When put in a tight spot, he did the right thing.  We architects need to do the same thing.  We can’t tell development teams that they must treat data with respect and then treat our own meta data as if it weren’t important.  This means ensuring that our architectures are managed with real tools, backed up, disaster-proofed, and generally treated as production data – which they are.
  5. Sharing a vision is important. George was also great with expressing vision. He could get people to rally ‘round a cause by getting others to see what was in it for them.  His monologue to Potter about why the Bailey Building and Loan should carry on after his father’s death is a classic.  But his best was saved for Mary, in the moonlight:

George: What is it you want, Mary? What do you want? You want the moon? Just say the word and I’ll throw a lasso around it and pull it down. Hey. That’s a pretty good idea. I’ll give you the moon, Mary.

Mary: I’ll take it. Then what?

George:  Well, then you can swallow it, and it’ll all dissolve, see… and the moonbeams would shoot out of your fingers and your toes and the ends of your hair… am I talking too much?

I’m still not sure how I’m going to work that line in with one my vision/architectural reviews, but I’m still thinking about it.  Look for it on call soon.

For those of you who have seen IAWL and might appreciate some derivative works, I leave you with:

Carolyn’s Sill’s video and song “George Bailey”.  This is one of my favourite holiday running songs for the tempo and overall good feelings it leaves me with.  Her songs are available on iTunes.  While you are watching this video, head over to iTunes and buy it.  As an independent artist, she deserves the 99 cents for putting this together.

Next up is Angry Alien’s It’s A Wonderful Life in 30 Seconds with Bunnies.  A 30 second overview of the film.  Be sure to click on the bunny outlines at the end to see some clips that couldn’t be part of the 30 second summary.

Finally, dear reader, I want you to know that you personally touch many lives by being part of our communities, both here and on Twitter.

Happy Wonderful Life, everyone.

Pages:«123456»

Subscribe via E-mail

Use the link below to receive posts via e-mail. Unsubscribe at any time. Subscribe to www.datamodel.com by Email


Categories

Archive

UA-52726617-1