« Data quality monitoring and reporting | Small data in the age of Big Data »
Exact isn’t everything- Surf your data!
Sometimes an analyst needs to take off the accountants hat, forget the urge to chase down every last penny, and instead put on their surfing gear, grab the data surf board (i.e. their set of prefered data tools), and just surf some data.
There are some cases were “Exact” is the only acceptable level of data quality. When we’re sending invoices to our customers, not only does the amount need to be right to the penny, the invoice needs to make it to the right person, on time.
But sending accurate invoices is not exactly the cutting edge in terms of data.
The challenges for todays data driven organizations are to be able to make sense out of the ocean of data available, and to do it faster than the competition.
And the ocean does have the highest quality water all the time. In fact, some of the data is downright dirty.
But there is a lot of it, and it can tell us things.
Just like ocean surfing, when you are data surfing you can sometimes ride the wrong wave and end up underwater, you can spend hours on your board in flat water, hoping the surf will come up but instead getting no where.
But when the wave comes, and you ride it, letting yourself go with the flow- while it won’t give you any exact answers, it will give you a “feel” and sense of where the wave is going, and if you are fast to grab it, and your board is good, you might just find yourself getting some awesome, gnarly insight that will score you some competitive advantage- and you just can’t surf that wave if you are worried about only looking at Exact, perfect, “we’ve checked it three times” data.
Data Surf is up- and by the look of the ocean it’s going to be a rocking rolling ride- good luck out there!
« Data quality monitoring and reporting | Small data in the age of Big Data »
I believe you wanted to say “doesn’t” in this sentence: “…And the ocean does have the highest quality water all the time….”
I do agree with your suggestion that sometime one needs to surf the data to discover something unexpected and exciting that is unplanned. Can you suggest any tools to surf the data – say with a million rows and a dozen tables or so?