Big Data Should Not Be Just Big

There seems to be a growing need to hoard every “bit” of Data so that a project can be considered a Big Data Project.  Collecting Data that is not relevant, or has no use can potentially be dangerous and provide faulty Data Sets.

There truly is a Data Deluge happening and sifting through the relevancy of the Terabits of Data that are being saved is time consuming and never completely free.  I located a fitting quote from a contributing writer at Forbes.com:

“With too little data, you won’t be able to make any conclusions that you trust.  With loads of data you will find relationships that aren’t real… Big data isn’t about bits, it’s about talent”–Douglas Merrill

So from that, and looking to what is meant by the final statement that Big Data isn’t about bits, but about Talent…what does that really lead us to?  I think that skimming through potentially relevant Data does a disservice to the concept of Analytics but at the same time, trying to connect the dots too many ways can have very negative effects too.

So where is it that we draw the line and make the right decisions about what is used for comparison and filtering?  Well that is exactly where the talent comes from.  The use of solid Data Specialists who in this day and age are not just DBA’s they are Business Analysts.  They can focus on eliciting the real needs of the business to identify and construct the underlying Data Base that will then feed the Applications and Dashboards that will be most relevant to the business.

But as companies grow, what one person needs may vary from what another needs…that is a struggle with Big Data.  It is all about the Volume, Variety and Velocity of the needs of the business and when the 3 V’s come together…well that is where the Talent comes into play.

I would say that there should actually be a 4th V in this term and that would be the Vector…where is the Data going to be used, who is going to use it and what is the potential outcome.  Can the end result be manipulated reliably so that conclusions in as close to real time be made…at the lowest level of the company hierarchy?  That is what will make a Big Data project truly successful.

There are few things that irk me more than the creation of a project and the power play that happens for the end result.  When it comes to data filtering and data analysis, I truly hope that Business Analysts out there really start to focus not just on what the Managers and VPs want…but also on the true needs of what the line staff needs.  I am a Sales Guy…pure and simple, I don’t need to have views and access to the activities of my colleagues…it is distracting.  What I need is a deeper insight into what my Customers and Potential Customers want and need.  Managers may need to see the comparisons so that they can build coaching and training…I need to reach my customers.  I need to be proactive and forward thinking so that I can work smart and hard and continue to be successful.

Today, marks the beginning of a New Year.  2013 will be filled with Ups and Downs.  I have made my list, I have checked it twice and I really do know who has been naughty and nice.  2013 is going to be the year of Krispy…12 months of complete focus on ME, My Family and My Job (in that order)…sure there will be spill over but I will be building some interesting tools over the next year to assist me in keeping that on track.

Cheers and Happy New Year!

Chris J Powell

Posted on January 1, 2013 in Big Data

Share the Story

About the Author

Back to Top