Browsing articles in "Blog"

The Perfect Data Model, Gone to Hell (MI) Due to Bad Web Form Design

Dec 29, 2010   //   by Karen Lopez   //   Blog, Data, Data Modeling  //  7 Comments
Sample form rendered by Mozilla Firefox. (Clic...

Image via Wikipedia

I don’t normally work in the UX/UI design world, but I know enough from constantly filling out web forms that too many designs out there are destined for a special ring of data Hell.  If you’ve followed any of my web form rants on Twitter, you may have heard this before…but it should be repeated.  Seth Godin recently blogged about his frustrations with annoying web forms for data collection:

The problem with letting your web forms become annoying is that in terms of time spent interacting with your brand, they’re way up on the list. If someone is spending a minute or two or three or four cursing you out from their desk, it’s not going to be easily fixed with some clever advertising.

I realize that many web designers live in the US and hate the fact that they have to complicate their beautifully simple designs with all these weird non-US things like regions, postal codes, country lists, etc.  But if their organization does business with non-US customers or data, they need to realize that the design must support these wacky international requirements.  

One of my favourite resources for good web form design is a FREE eBook by Graham Rhind on names and addresses in web forms  Graham specializes in internationalization and address formats, so he is the go-to guy for these sorts of things.

It’s not just internationalization, though, that causes web form design to go all to Hell.

My pet peeve is referenced in Seth’s post: using drop downs to force a user to choose from a list of hundreds or thousands of values.  These are annoying because drop downs usually require acute mouse skills as well as waste time.  Developers love drop downs because they don’t have to do much data validation – if it’s in the list, it’s supposed to be good data.  However, optimizing a developer’s task isn’t always the the best for customers who have to use the form.   In fact, I’ve come to realize that the more we optimize development, the more we have to take from the end user.  It should not be that way, but I see it over and over again.

A typical frustration I’ll face is a form that collect address information.  It will have fields in the same order that we’d typically see a mailing label, something like:

  • Name
  • Address line one
  • Address line two
  • Apartment#
  • City
  • State
  • Other
  • ZIP
  • Country

…with State and Country being a drop down of all the US States and Country being a list from somewhere on the web of a list of countries.  There might be some magic in the country list that then causes the list of states to change based on the country select.  The problem is that as one fills out the fields from top to bottom, he hits the State field before the country field.  He has to jump down a few fields to find the right country, then jump back to the drop down.  If he is very fortunate, this change in country does not require a complete refresh of the form so his data might still be there…or it might not.

Or, the web designer might think that we foreigners should use the Other field to fill our foreign state or province.  They might also beef up their data quality be requiring a State in the drop down, even if it only contains US states.  We users won’t know until we try to submit the form.

When I use forms that require me to pick a US State, I usually go with OH or OR since they are “close” to my Province Code of ON.  Sometimes I pick Hell, Michigan because it’s just as good of a place as any if web form constraints force me to enter bad data.  I’ve always wondered, though, how that impacts analytics for that data.

My other peeve is when Birth Date must be filled in via a drop down.  First one must pick from a list of months, then a list of dates 1-31, then a list of Years going back to the ice age….or somewhere near my birth year.  There are much better ways for web designers to collect and validate data.  I’d love to see business management sit down and enter a couple hundred addresses into their web forms to decide whether the forms are “good enough”.

When users find annoying forms, they are more likely to enter bad data.  Don’t ask me how I know this…let’s just say that there are many records of me out there, happily describing my nice home in Michigan, where I celebrate my 13th birthday ever year, in my apartment number 0000, which also shares a ZIP code with a popular TV series from the 1990s

Love your data; don’t torment it in the hands of end users before it even gets to you.




Enhanced by Zemanta

What Are We Doing? Why Are We Doing It? Will It Feel Better When We Stop?

Dec 29, 2010   //   by Rob Drysdale   //   Blog, Professional Development  //  9 Comments

These three little questions were on the board one day when we walked into our Signals and Systems class in our second year of university.  The class was all about applied mathematics related to control signals and systems and included some complex methods to solve the equations.  The concepts were tough and all of us struggled with it to the point that the average grade on the mid-term exam was 27%.  After the mid-term’s were graded, the professor started the next class with the three questions shown above.  His point was to get us to look at the big picture of what the class was about and why we were doing it and how it fit in with our courses overall.

imageOver the last little while I’ve seen a number of people talking about what they are working on right now, how busy they are, how much time they spend online, getting to Inbox Zero (yeah, as if I’ll ever get near that…) and a whole host of other things.  There are lots of theories and books including Getting Things Done, Priority Management, and First Things First.

These are all great for helping us get where we are going, but do we know where we are going?  In First Things First one of the key things is to actually come up with a plan and goals.  But I think we all slip up on this by doing the “Urgent – Important” and “Urgent – Not Important” things without putting aside any time for the big picture planning.  I know that it is too easy to let time slip away and continue to work on the daily/weekly/monthly things and go through life without a vision of where we want to be or the goals we want to achieve.

So here we are at the end of 2010 and as we think about the year and decade that is just ending, let’s look back at what we’ve done and achieved and how far we’ve come.  But don’t forget to look at what we want out of the next year, 5 years, 10 years and so on.  Then we can set the milestones that go with that and make sure that what we are doing is helping us to get where we want to be.  Otherwise we’ll be sitting here on December 29, 2011 wondering where the last year went….like I am on December 29, 2010.

Better Mobile, Social and Cloud Computing Skills = More Money

Dec 28, 2010   //   by Karen Lopez   //   Blog, Database, Professional Development, Social Networking  //  5 Comments

One of my resolutions for this past year was to work on my non-traditional database skills such as NoSQL and cloud computing/databases.  I am still at the research and discovery phase, but after reading this article, Paying top dollar for tech talent pays off,  I think I need to continue to commit time in these areas.

Google CEO Eric Schmidt explained why every one of his 16,000-plus employees just got a 10 percent raise. It was a salvo in the " war for talent," said one of the world’s most successful CEOs. And Google is still hiring hundreds of employees every week.


"Consumer-facing companies like Procter and Gamble, Ford and Coca-Cola have started hiring as well," noted Cournoyer, co-founder of venture capital firms Montreal Start Up and Real Ventures. He pointed out that just as these mega-enterprises need warehouse, sales and customer-service pros, they also need Web engineering talent to thrive in today’s market. That rings true in any industry these days.

I’m still waiting for my $3.5 million stock option to not leave a company. 


I’ve spent some time with Microsoft Azure and MongoDB, but I want to learn more about Hadoop and some other non-relational data stores.  Learning about these also means brushing up on my web development skills.  I’m hoping that my use of social networking and the addition of social features to our online presences will bring a better understanding to business problems that involve these solutions.

What I do understand is that if you are data architect working in the relational world you’ll still have a role to play for a very long time, no matter what recent articles are saying. However, if you want to be at the top of your game, a most relevant team member, a true enterprise data architect you won’t be able to rest on what you know about SQL Server, DB2, and Oracle alone.

The biggest issue I’ve had lately working with these emerging technologies have been that traditional data modeling tools aren’t yet supporting many newer data technologies. In fact, most don’t even support XML that well even though it has been around for a long time.  I want to tie conceptual and logical models to physical models of non-relational solutions without having to worry about importing or exporting meta data from tool to tool.

Do you think that the existing data modeling market will start supporting non-relational technologies? O r will we have to look for new tools for these? 

Enhanced by Zemanta

Uploading Large Documents to SharePoint

Dec 22, 2010   //   by Karen Lopez   //   Blog  //  2 Comments

These are my recommend actions for uploading large-ish documents to SharePoint, since I get many support requests about this, usually late in the night when I’m not expected to be working.  I’m hoping that Dr. Google and Nurse Bing can help those people struggling with how to get their midnight-oil-generated documents uploaded.

SharePoint Document Library Upload Document If you are getting upload failures or timeouts when using a Document Library’s menu to upload a document, there are alternatives that may work out better for you.  However, whether or not you are able to use them is based on a few factors. First, the exact instructions are dependent on the version of SharePoint you are using.  The features I outline below are available in versions of SharePoint starting with 2003 (or WSS 3.0).  Second, they are dependent on how your installation of SharePoint is configured by the administrators.   So while there are no guarantees that these will work, they are certain worth trying.

The default maximum file size for SharePoint is 50MB.  If your document or file is larger than that, your administrator will have configure SharePoint to accept larger sized files.  You could, though, try these workarounds to get is uploaded

  1. Make multiple smaller files out of the larger file in order to get it uploaded while you wait on your administrator to adjust the configuration. 
  2. Zip the file, which depending on the format of the file, could significantly reduce the size of the file.

But you do have other options, again dependent on how your document library and SharePoint server are configured. 

Email the document to the Document Library

If the document library has been configured to receive mail, you can upload the document by just sending an e-mail to that library if that has been configured.  Note that each library will have a separate e-mail address so you’ll need to look it up.  Do so by clicking on Settings on the library toolbar, then choose Document Library Settings:


Document Library Settings

From there you’ll see basic information about the library:

Document Library Settings image

I’ve blurred out some of my information, but the e-mail address list will be a full e-mail address.  If you copy that from the list and put it in the TO: field of an e-mail, you can attach the file and send it to the document library.  Make sure that the file and the subject line of your e-mail are meaningful as they may be used in populating some of the fields in the library.

There are some gotchas with this method:

  1. Your e-mail server may be configured to restrict sending of documents over a certain size, so this e-mail method may not work.
  2. Just like blocked files in SharePoint, your e-mail services may block attachments with certain extensions.

Use the File Explorer Method

SharePoint also offers another method for getting files into a document library: using a file explorer interface.

To use this method, go to the document library and choose ACTIONS, then Open with Windows Explorer:

Actions Open with Windows Explorer image

You’ll need to use Internet Explorer for this option to show up. If you are using any other browser, you won’t see all those options, including the Open with Windows Explorer.

A normal explorer window will open and you can drag and drop your file into the library, just like you were copying a file between any other two folders.  You may first be promoted to supply your login again.

Don’t place the document in the Forms folder.


Drag and drop

Once that has completed, you’ll need to go back to the library and update the fields for that document.

Document Library

Special Note: You may also be having problems uploading a file if it has an extension that is blocked from being uploaded by the SharePoint Administrator.  It is common for certain file types to be restricted in order to reduce the risk to other SharePoint users.  None of the above techniques will work if the extension of the document has been blocked.

Connect to Outlook

There is another method of linking the SharePoint document library to Outlook, but I personally don’t like that method because every time I open Outlook I am promoted to log in.  This isn’t just for uploading, but for working with SharePoint items from within Outlook. You can set this up by going again to ACTIONS, then Connect to Outlook:



You will receive several prompts and warnings which you will need to agree to before finally being prompted by Outlook to configure the synchronization.  Since this blog post is only about uploading a document and not synching files between Outlook and SharePoint, I’ll point you to Microsoft’s information about Connect to Outlook.  It might be worth a try if the other methods still don’t work for you.

I hope one of these methods worked for you.  If you’ve found other methods, please leave them in the comments.

Finally, if you still aren’t able to get the file uploaded, it’s time to contact your SharePoint administrator.  She or he will need the following information:

  1. The size of the document.
  2. The format/file extension of the document
  3. The browser and version you are using
  4. The Operating System you are using (XP, Vista, Windows 7, Linux, etc.)
  5. The link to the document library you are trying to upload to
  6. The exact error message you are receiving (Screen shots are best)
  7. Description of your internet connection (wireless, cable, dial-up)
  8. User ID you used to log in to SharePoint
  9. The approximate date and time (with time zone) you were experiencing the issue

The more information you can provide, the faster someone can help you get the fi

SQLPASS Sessions Recordings are Here

Dec 20, 2010   //   by Karen Lopez   //   Blog, Database, Professional Development, Speaking  //  No Comments



I know what  I’ll be listening to over the next few weeks.  An early Christmas present, too.

If you weren’t able to attend the PASS Summit (or even if you did) you can order the DVD set of recordings, too.  You get PDF handouts of the slides and audio recordings of all the regular sessions.  I’m listening right now as I blog. 

I plan on first tackling all the database design-related sessions, then ones on performance tuning.  I also want to listen to the professional development session because there are some very successful professionals out there who took the time to share their thoughts on the non-technical side of being a data professional.

12 Days Of SQL – What is your Over / Under?

Dec 20, 2010   //   by Karen Lopez   //   Blog, Data, Data Modeling, Database  //  9 Comments

This post is Day 8 of the 12 Days of SQL, a series that was the idea of Brent Ozar (blog | Twitter) to have members of an elite group of foodie friends bloggers and writers who get together in person and virtually to improve our writing, branding, blogging and business skills.  This series has us picking our favourite SQL Community blog entry from outside the group and sharing with our readers.

On Day 7, Yanni Robel (blog|Twitter), DBA extraordinaire and superwoman at, wrote about Jonathan Kehayias’ post on No such thing as Small Change to Production database and the Change Management Process.  I first met Yanni during SQLCruise.  She also took us to a Disney Character Breakfast at Disneyland last week. This is significant because being kid-free, we don’t get out much when it comes to kid stuff.  We had a blast – a future blog post to follow.

George Bailey in Football Uniform For Day 8 I suppose that I should write a post that has to do with “Eight Maids a Milking”…but that’s just a bit too weird for a part-time vegan hippie woman in IT to pull off.  So I’m going to go with a football theme.  That’s my George Bailey in a football uniform ornament on the left. Let’s call this day’s present “Eight Tight Ends”.  Or “Eight Ends a Tightening?” Either one works.

I have chosen Louis Davidson’s Can you over-normalize? post.  Louis is a frequent speaker, book author, and blogger who focuses primarily on database design issues.  I met him in person recently at SQLPASS and volunteer with him on the PASS Data Architecture Virtual Chapater (DArcVC).

In his post, Louis asks an important question, “Can you over normalize?” His answer is that no, you really can’t.  But before you get your tinsel in a tangle, read what he said:

My rule of thumb is that:

1. The requirements dictate the database design

2. The requirements and the relational engine dictate the implementation

So unless you understand the requirements, you can’t design the optimal database, and if you don’t understand SQL Server, you are not going to end up with an optimal implementation.  Over-normalize? No.  Over-engineer? Definitely.

You know that I love contentious issues and levels of normalization is one of the major ones.  What I find most people miss in discussions about normalization is that it’s relevant to inserts, updates and deletes.  We normalize to avoid data anomalies that would decrease data integrity.  But when people complain about “over normalization”, they are usually looking only at retrieving data, not maintaining it.   All normalization looks excessive when you aren’t concerned with data integrity.  This leads to people wanting to trade off data integrity for performance gains, which is something we might do based on costs, benefits, and risks for the entire lifecycle of data. That’s why we denormalize data.  Sometimes the trade off is acceptable and sometimes it isn’t.

If you read them comments on Louis’ post, you’ll see just how many people still don’t understand the basics of normalization – why it is done, why it might be undone, why there isn’t a single number for the “right” level of normalization…which in a way was his point.

As you may have heard me say in a presentation:

There is no one right answer for all projects, all designs, all organizations, all environments.

George Football Mary in HydrangeaBushThis is why sports books have an Over/Under for a specific game.  They can’t have a single Over/Under for all teams, all locations, all days, all weather conditions.  There is no one right number.  Your normalization number should be based a on a whole sleigh ride full of inputs, including performance, data quality levels, trust, data architecture, etc. One project’s normalization design choices might be over done and a similar approach for another project might be undernormalized.

In fact, even George Bailey was calculating his Over / Under for Mary Hatch as she hid naked in the hydrangea bush.  That football uniform was serving him well in his calculations.

Take a couple of minutes and Buffalo Gals your way over to Louis’s post and let me know what you think.

Next up is Kendra Little (blog|Twitter) who is going to write about Nine Ladies Dancing for Day 9, I hope.  Kendra is an amazing presenter and DBA who works in the Pacific Northwest. I first met her at SQL Saturday Miami.  If you ever get an opportunity to attend one of her presentations or webinars, Charleston right to it.  She has fabulous slides and she shares a ton of practical information about SQL Server.

Brent O’s 12 Days of SQL post
Day1: Jeremiah Peschka
Day 2: Grant Fritchey
Day 3: Dave Stein
Day 4: Andy Leonard
Day 5: Erin Stellato
Day 6: Tim Ford
Day 7: Yanni Robel
Day 8: [This post] Karen Lopez
Day 9: Kendra Little
Day 10: Crys Manson

No Man is a Failure Who Has Friends: 5 Architecture Tips from It’s a Wonderful Life.

Dec 17, 2010   //   by Karen Lopez   //   Blog, Data Modeling, DLBlog, Fun, Social Networking  //  4 Comments

This post is a rerun from 2009, but the truth is still there.

image It’s a Wonderful Life is a part of many a holiday tradition.  While most of you probably know this as a Christmas film, the links to the Christmas holiday are fairly weak.  There’s an angel, some snow, and a background Christmas tree with a bell.  Other than that the story itself could take place at any time of the year.  So if you’d put off seeing this film because you think it is all happy, shiny, Christmas cheer-ish, you need to think again.

George Bailey is a protagonist in a small town, Bedford Falls, located in upstate New York. He finds himself on the brink of financial ruin due to no fault of his own.  He goes to visit the town big wig and resident bully, Henry Potter, to get a loan.  Potter tells him that with no equity in his life insurance policy, he’s “worth more dead than alive.”  In a fit of despair, George decides to sacrifice his life so that his family and company (The Bailey Building and Loan) can survive. However, his plans are side-tracked by Clarence, his guardian angel.  George rambles that he wished he’d never been born.  Sounds pretty dark, doesn’t it?  Not that Christmas-y at all. Clarence sees this as a great opportunity to show George just what a great impact he has had on not just his community, but on the world.

So George is shown what his family, friends, town and world would be like if he’d never been born.  The new town is called “Pottersville”, after the antagonist Mr. Potter.  It’s not a good thing.  Clarence tells him:

“Each man’s life touches so many other lives. When he isn’t around he leaves an awful hole, doesn’t he?”


“No man is a failure who has friends.”

The “feel good” part of the film is when George gets to return to the world he wished he’d never been born into.  I’ll leave the assignment up to you to see how this all happens and how George and his family do once he returns.

I’ve seen this film at least 50 times in my life, perhaps even more.  In fact, most people would probably think that I’m a bit obsessed with this whole story.  To which I say: everyone needs a hobby.  This just happens to be one of mine.

So what does this have to do with architecture and data management?  I think plenty.

  1. Enterprise-class projects can’t be done by one person.  No one is a failure who has good team mates who collaborate well.  You don’t have to be friends with them, or even like them that much.  But you do have to find a way to collaborate with them.
  2. Architecture done well can have all kinds of impacts elsewhere.  Each architect’s work touches so many other parts of the architecture.  When it isn’t there, it leaves an awful hole…to be filled by a non-architect to do. One small great architecture component can have huge impacts on solution quality for a long time in the future.  When George is a kid, he saves his brother Harry’s life.  But in the Pottersville world, Harry dies.  Clarence tells George: Every man on that transport died! Harry wasn’t there to save them, because you weren’t there to save Harry. 

    If you aren’t there to create the architecture, or if the architecture you create isn’t used, then the good stuff it could deliver won’t be there when it is needed.

  3. Fastest isn’t always “bestest”.  In running George and Clarence out of his Pottersville bar, Nick the Bartender explains what works in the bad world: Hey look, mister – we code fast here for people who want to get lots of stuff done fast, and we don’t need any characters around to give the joint "atmosphere". Is that clear, or do I have to slip you my left for a convincer? Well, I paraphrased that quote.  But you get the point. Perhaps that was Nick’s collaboration method – using bouncers to run people out.  Some people feel that data and other architectures are just there for some sort of religious checklist nirvana.  It’s our jobs, as architects, to show them why architecture plays a key role project success.  A left convincer might work in the short run, but the best way to get support for an architecture is have one that works.  Good architectures need to be designed.  Hacking away on a pseudo architecture is more Pottersville than Bedford Falls.
  4. Architecture is more than drawing boxes and lines. George was fabulous at motivating people to do the right thing.  The first way he did this was by living by his own principles.  When put in a tight spot, he did the right thing.  We architects need to do the same thing.  We can’t tell development teams that they must treat data with respect and then treat our own meta data as if it weren’t important.  This means ensuring that our architectures are managed with real tools, backed up, disaster-proofed, and generally treated as production data – which they are.
  5. Sharing a vision is important. George was also great with expressing vision. He could get people to rally ‘round a cause by getting others to see what was in it for them.  His monologue to Potter about why the Bailey Building and Loan should carry on after his father’s death is a classic.  But his best was saved for Mary, in the moonlight:

George: What is it you want, Mary? What do you want? You want the moon? Just say the word and I’ll throw a lasso around it and pull it down. Hey. That’s a pretty good idea. I’ll give you the moon, Mary.

Mary: I’ll take it. Then what?

George:  Well, then you can swallow it, and it’ll all dissolve, see… and the moonbeams would shoot out of your fingers and your toes and the ends of your hair… am I talking too much?

I’m still not sure how I’m going to work that line in with one my vision/architectural reviews, but I’m still thinking about it.  Look for it on call soon.

For those of you who have seen IAWL and might appreciate some derivative works, I leave you with:

Carolyn’s Sill’s video and song “George Bailey”.  This is one of my favourite holiday running songs for the tempo and overall good feelings it leaves me with.  Her songs are available on iTunes.  While you are watching this video, head over to iTunes and buy it.  As an independent artist, she deserves the 99 cents for putting this together.

Next up is Angry Alien’s It’s A Wonderful Life in 30 Seconds with Bunnies.  A 30 second overview of the film.  Be sure to click on the bunny outlines at the end to see some clips that couldn’t be part of the 30 second summary.

Finally, dear reader, I want you to know that you personally touch many lives by being part of our communities, both here and on Twitter.

Happy Wonderful Life, everyone.

Blog Categories

Subscribe via E-mail

Use the link below to receive posts via e-mail. Unsubscribe at any time. Subscribe to by Email



UA-52726617-1 Secured By miniOrange