Browsing articles in "Blog"

Bad Data Models / Database Designs Kill – Under Modeling

Jan 6, 2011   //   by Karen Lopez   //   Blog, Data, Data Modeling, Database, DLBlog  //  3 Comments

imageDr. Danielle Ofri has a frightening post about how data design can cause serious consequences.  She talks about running up against a 1000-character limit when trying to create a patient history for someone about to undergo a risky surgery:

I panic for a moment, fearful that the computer has frozen and that I’ve lost all my work — something that happens all too frequently. But I soon realize that this is not the case. Instead, I’ve come up against a word limit.

It turns out that in our electronic medical record system there is a 1,000-character maximum in the “assessment” field. While I’ve been typing, the character number has been counting backward from 1,000, and now I’ve hit zero. The computer will not permit me to say anything more about my patient.

If you’d done any database design, you know that even if you design a good, business-driven design, others who use the database might apply their own rules to the data on the way in or out of the database.

I remember designing a Point of Sale system database for an appliance retailer.  Our data model needed to support the selling of high-end appliances as well as bulk purchases for high-end appliances.  So our transaction amount columns were significantly large.  A developer on the application team thought our monetary field lengths were insanely large, so he enforced  a limit on a transaction of $9,999.99 for the total transaction.   To make matters worse, this system went all the way into production with that limit.  So on day one of the roll out, sales people couldn’t sell the most highest margin items such as a professional quality stove ($14,000) or sell to developers who were buying 100 low end stoves in one transaction.  Their orders had to be chunked up into tiny $9,000 mini-transactions.  Order completion was taking hours instead of minutes.  Credit cards were being rejected for too many large amount transactions back to back.  In other words, a nightmare deployment because organizations trying to spend tens of thousands of dollars were walking out and going to other retailers with “real systems” to make their purchase.

However, no lives were being lost (that I know of).  Some people may have gone longer without heat (furnaces) or with rotten food (freezers and fridges), but in the overall scheme of things the impact on customers was not life and death consequences.

If we get back to Dr. Ofri’s situation, though, she was faced with a terrible data dilemma: how to describe a complex patient history in 1000 characters.  This number probably sounded like a huge number when the project team was adding that “feature” to the system.  I’d even bet their own test data only went as high as 200 characters or so.

I’m also guessing that since this system is a fairly high-risk project that some expert user (or many) was responsible for approving the design of field lengths.  Perhaps he or she also thought that 1000 characters was enough.

In desperation, I call the help desk and voice my concerns. “Well, we can’t have the doctors rambling on forever,” the tech replies.

That response from IT (even if it a help desk tech who has no clue as to why there is a limit) makes me mad and afraid at the same time.  You’ve all heard it in your design reviews, haven’t you?

  • That’s way too long of a field.
  • It won’t fit on one window without scrolling
  • No one has an e-mail address THAT long
  • You are over modeling it
  • That’s ridiculous.  No one is going to sit there and type that long
  • 255 was good enough for the last 10 years, it’s good enough for the next 10 years
  • The indexes on that column will be too large
  • Go ahead and make the column that long; we’ll just truncate it in the application

The business users are frightened by the negative comments and agree that 25 is sufficient for e-mail address, not even realizing that some of their own e-mail addresses are longer than that.

As I blogged recently about Over Modeling, it’s only over modeling if it doesn’t meet the business need.  Sure, we might make concessions to technical feasibility (“make every column 2000 characters, just in case”), but our designs should be business driven.

Dr. Ofri ends her story by saying:

I’ve finally condensed my patient’s complicated medical conditions to exactly 1,000 characters. I quickly hit “save” before I lose everything. I wish him good luck on his operation, wondering if his surgeons will have to condense the entire operative report to 1,000 characters as well. What happens if there are complications?

For my next medical evaluation, I think I will use haiku.

I don’t know about you, but I wouldn’t want to read that about my own patient record.

Next up: What could we have done differently on our project

Thanks Everyone Who Voted: I Won a SCOTTEVEST XAE

Jan 4, 2011   //   by Karen Lopez   //   Blog, Fun  //  No Comments

imageI’m happy to share that my video won the SCOTTEVEST XAE Unboxing Contest.  That means that Rob is going to be wearing the 20-pocket beautiful jacket.

Watch for a new unboxing video by the both of us when it gets here.

I appreciate everyone who took the time to view my video, which is the first one I’ve ever done that included me – all the others I’d done were for webinars and featured only PowerPoint slides and software demos.

Once I got over the whole “I hate having my picture taken” part, it was fun to do.

I’ve had a couple people ask me what I used in producing the video:

 

  • Camera: My iPhone 3GS (my video camera had not yet arrived).
  • Video production: TechSmith’s Camtasia (love it)
  • Intro segments: Standard library components that come with Camtasia
  • Music: Hallelujah Chorus from Project Gutenberg
  • Winking Barbie effect: Camtasia using the callout feature to mimic a wink.

I took Erin Stellato’s (blog | Twitter) advice from her winning SQL Cruise video and did it all in one take.  It’s funny how if you plan to have only one take you don’t spend too much time trying to get every single line/point perfect, so it flows better and is more fun to do.

Enhanced by Zemanta

Travel Vest for Women Unboxing Video + Need Your Vote

Dec 31, 2010   //   by Karen Lopez   //   Blog, Fun  //  3 Comments

I’ve uploaded a video to YouTube of an unboxing video I made for a contest.  As is my usual style, it is full of snark and happy things…at the same time.

. You can help me win the contest by:

  1. Watching the short video I made
    A DataChick Christmas

  2. “Liking” the video on YouTube (You may need to double click on the video above to view it on YouTube)
    To LIKE the video you have to be on the YouTube page and click on the thumbs up button on the lower right of the video:

image

 

What’s the contest and prize? Scott Jordan, Founder of SCOTTEVEST, is running a video contest that ends on New Year’s Eve (that’s TODAY).  The prize is a beautiful SCOTTEVEST Leather XAE jacket that I want to win for my husband and business partner, Rob.

image

Isn’t it beautiful? Like my Travel Vest for Women, it has mucho pockets (20 in this jacket, 22 pockets in mine).  It even has a pocket for an iPad.  Look at that price, though…I would really like to win one for Rob.

 

I really appreciate your help in viewing and voting on the video.  I’m sure Rob would love to have this jacket…and when they come out with one for women, he’ll have to win one for me, too.

You Can Be … a Data Diva

Dec 30, 2010   //   by Karen Lopez   //   Blog, Data, Fun, WIT  //  2 Comments

I have mixed feelings about the whole Barbie phenomenon.  I had a talking Barbie growing up and I loved her and her friend Stacey.  But I don’t remember much about the way she dressed…perhaps because my Grandmother sewed much of the clothes I had for her.  Maybe I was shielded from the party-girl outfits that were available. 

I recently found this as an online game at Mattel.com:

image 

This is to promote Mattel’s recent Computer Engineer Barbie product.  What I find interesting is that they’ve focused on the data aspects of computing.  Most likely this is done only for the alliteration, but I do like getting early to girls about the importance of data.

I’m hoping that my friend @datadeva sees this, too.

Love Your Data.  Get your kids to love their data, too.

The Perfect Data Model, Gone to Hell (MI) Due to Bad Web Form Design

Dec 29, 2010   //   by Karen Lopez   //   Blog, Data, Data Modeling  //  7 Comments
Sample form rendered by Mozilla Firefox. (Clic...

Image via Wikipedia

I don’t normally work in the UX/UI design world, but I know enough from constantly filling out web forms that too many designs out there are destined for a special ring of data Hell.  If you’ve followed any of my web form rants on Twitter, you may have heard this before…but it should be repeated.  Seth Godin recently blogged about his frustrations with annoying web forms for data collection:

The problem with letting your web forms become annoying is that in terms of time spent interacting with your brand, they’re way up on the list. If someone is spending a minute or two or three or four cursing you out from their desk, it’s not going to be easily fixed with some clever advertising.

I realize that many web designers live in the US and hate the fact that they have to complicate their beautifully simple designs with all these weird non-US things like regions, postal codes, country lists, etc.  But if their organization does business with non-US customers or data, they need to realize that the design must support these wacky international requirements.  

One of my favourite resources for good web form design is a FREE eBook by Graham Rhind on names and addresses in web forms  Graham specializes in internationalization and address formats, so he is the go-to guy for these sorts of things.

It’s not just internationalization, though, that causes web form design to go all to Hell.

My pet peeve is referenced in Seth’s post: using drop downs to force a user to choose from a list of hundreds or thousands of values.  These are annoying because drop downs usually require acute mouse skills as well as waste time.  Developers love drop downs because they don’t have to do much data validation – if it’s in the list, it’s supposed to be good data.  However, optimizing a developer’s task isn’t always the the best for customers who have to use the form.   In fact, I’ve come to realize that the more we optimize development, the more we have to take from the end user.  It should not be that way, but I see it over and over again.

A typical frustration I’ll face is a form that collect address information.  It will have fields in the same order that we’d typically see a mailing label, something like:

  • Name
  • Address line one
  • Address line two
  • Apartment#
  • City
  • State
  • Other
  • ZIP
  • Country

…with State and Country being a drop down of all the US States and Country being a list from somewhere on the web of a list of countries.  There might be some magic in the country list that then causes the list of states to change based on the country select.  The problem is that as one fills out the fields from top to bottom, he hits the State field before the country field.  He has to jump down a few fields to find the right country, then jump back to the drop down.  If he is very fortunate, this change in country does not require a complete refresh of the form so his data might still be there…or it might not.

Or, the web designer might think that we foreigners should use the Other field to fill our foreign state or province.  They might also beef up their data quality be requiring a State in the drop down, even if it only contains US states.  We users won’t know until we try to submit the form.

When I use forms that require me to pick a US State, I usually go with OH or OR since they are “close” to my Province Code of ON.  Sometimes I pick Hell, Michigan because it’s just as good of a place as any if web form constraints force me to enter bad data.  I’ve always wondered, though, how that impacts analytics for that data.

My other peeve is when Birth Date must be filled in via a drop down.  First one must pick from a list of months, then a list of dates 1-31, then a list of Years going back to the ice age….or somewhere near my birth year.  There are much better ways for web designers to collect and validate data.  I’d love to see business management sit down and enter a couple hundred addresses into their web forms to decide whether the forms are “good enough”.

When users find annoying forms, they are more likely to enter bad data.  Don’t ask me how I know this…let’s just say that there are many records of me out there, happily describing my nice home in Michigan, where I celebrate my 13th birthday ever year, in my apartment number 0000, which also shares a ZIP code with a popular TV series from the 1990s

Love your data; don’t torment it in the hands of end users before it even gets to you.

 

 

 

Enhanced by Zemanta

What Are We Doing? Why Are We Doing It? Will It Feel Better When We Stop?

Dec 29, 2010   //   by Rob Drysdale   //   Blog, Professional Development  //  9 Comments

These three little questions were on the board one day when we walked into our Signals and Systems class in our second year of university.  The class was all about applied mathematics related to control signals and systems and included some complex methods to solve the equations.  The concepts were tough and all of us struggled with it to the point that the average grade on the mid-term exam was 27%.  After the mid-term’s were graded, the professor started the next class with the three questions shown above.  His point was to get us to look at the big picture of what the class was about and why we were doing it and how it fit in with our courses overall.

imageOver the last little while I’ve seen a number of people talking about what they are working on right now, how busy they are, how much time they spend online, getting to Inbox Zero (yeah, as if I’ll ever get near that…) and a whole host of other things.  There are lots of theories and books including Getting Things Done, Priority Management, and First Things First.

These are all great for helping us get where we are going, but do we know where we are going?  In First Things First one of the key things is to actually come up with a plan and goals.  But I think we all slip up on this by doing the “Urgent – Important” and “Urgent – Not Important” things without putting aside any time for the big picture planning.  I know that it is too easy to let time slip away and continue to work on the daily/weekly/monthly things and go through life without a vision of where we want to be or the goals we want to achieve.

So here we are at the end of 2010 and as we think about the year and decade that is just ending, let’s look back at what we’ve done and achieved and how far we’ve come.  But don’t forget to look at what we want out of the next year, 5 years, 10 years and so on.  Then we can set the milestones that go with that and make sure that what we are doing is helping us to get where we want to be.  Otherwise we’ll be sitting here on December 29, 2011 wondering where the last year went….like I am on December 29, 2010.

Better Mobile, Social and Cloud Computing Skills = More Money

Dec 28, 2010   //   by Karen Lopez   //   Blog, Database, Professional Development, Social Networking  //  5 Comments

One of my resolutions for this past year was to work on my non-traditional database skills such as NoSQL and cloud computing/databases.  I am still at the research and discovery phase, but after reading this article, Paying top dollar for tech talent pays off,  I think I need to continue to commit time in these areas.

Google CEO Eric Schmidt explained why every one of his 16,000-plus employees just got a 10 percent raise. It was a salvo in the " war for talent," said one of the world’s most successful CEOs. And Google is still hiring hundreds of employees every week.

and

"Consumer-facing companies like Procter and Gamble, Ford and Coca-Cola have started hiring as well," noted Cournoyer, co-founder of venture capital firms Montreal Start Up and Real Ventures. He pointed out that just as these mega-enterprises need warehouse, sales and customer-service pros, they also need Web engineering talent to thrive in today’s market. That rings true in any industry these days.

I’m still waiting for my $3.5 million stock option to not leave a company. 

image

I’ve spent some time with Microsoft Azure and MongoDB, but I want to learn more about Hadoop and some other non-relational data stores.  Learning about these also means brushing up on my web development skills.  I’m hoping that my use of social networking and the addition of social features to our online presences will bring a better understanding to business problems that involve these solutions.

What I do understand is that if you are data architect working in the relational world you’ll still have a role to play for a very long time, no matter what recent articles are saying. However, if you want to be at the top of your game, a most relevant team member, a true enterprise data architect you won’t be able to rest on what you know about SQL Server, DB2, and Oracle alone.

The biggest issue I’ve had lately working with these emerging technologies have been that traditional data modeling tools aren’t yet supporting many newer data technologies. In fact, most don’t even support XML that well even though it has been around for a long time.  I want to tie conceptual and logical models to physical models of non-relational solutions without having to worry about importing or exporting meta data from tool to tool.

Do you think that the existing data modeling market will start supporting non-relational technologies? O r will we have to look for new tools for these? 

Enhanced by Zemanta

Blog Categories

Subscribe via E-mail

Use the link below to receive posts via e-mail. Unsubscribe at any time. Subscribe to www.datamodel.com by Email


Categories

Archive

UA-52726617-1