Over the last few years, I have been slowly getting introduced to the Star Trek universe. It comes with the territory when you are married to a tech geek; references to Star Trek abound, and if you want any hope of understanding those references you need to watch the show. As a history nerd who loves her historical dramas like The Tudors, The White Princess and venturing into the Fantasy realm with things like Game of Thrones, I was hesitant to move a show about the future, rather than the past or a fantasy version of the past. It didn’t help that previous attempts to get me into Star Trek started with binge-watching the original series from the 70’s with Captain Kirk. At the time I found the series hopelessly campy and stupid, quickly lost interest and wrote off the entire Star Trek universe.
But once I was exposed to later versions of the show, and the campiness was replaced with good writing, better storyline development and more realistic characters, my interest picked back up. In particular, I found myself intrigued with the character of Lt. Commander Data from The Next Generation. Data, an android equipped with AI, struggles again and again throughout the series with how to understand humans, and eventually with how to develop his own humanity. To Data, the things that we humans do don’t make any sense. We make decisions based on gut instinct rather than quantifiable statistics and data points, and that is a source of endless confusion for the walking computer with incredible AI that is Data.
Watching Captain Picard and the rest of the crew work with Data over so many seasons to try and explain humanity, and in turn watching them learn in some fashion to trust Data and his recommendations, was fascinating to me. It demonstrated a truth that seemed to be so simple. Relying too heavily on Data, or too heavily on human instinct, would inevitably result in missing something big, and thus put the entire ship and crew at risk. But combining the two tends to produce not only a clearer vision, but it tends to result in better decisions. It is not an easy relationship to maintain; each side tends to distrust the impulses and motives of the other, and often for very understandable reasons. But when the Data and the Human Instinct work together, they create a force to be reckoned with.
We have something similar happening now in the business world. Businesses are obsessed with the quantifiable. With the measurable. With calculations like ROI and EVM and other three or more letter acronyms. In short, they are obsessed with data. If it can’t be quantified and measured, it is deemed next to useless. This is the obsession with Big Data that leads essentially to technologists and businesses to worship at the alter of things like Watson and ERP systems.
The obsession is understandable if you think about it. Businesses are operating in a VUCA world, a Volatile, Uncertain, Complex and Ambiguous environment. A world of gray. When your world is gray, it is very comforting to pretend that the gray is really just Black and White, and you can be guided to the right answers and right decisions based on the Black and White insights you get from something like Big Data. You don’t have to look much farther for a good example of this than the job descriptions attached to job announcements. These job announcements and list of desired characteristics for an employee read like they came from the mind of a fantasy writer like George R.R. Martin rather than from any realistic place. But why is this the case? Because the hiring process has been taken over by Big Data in the form of Applicant Tracking Software, who needs ever-increasing data points to measure to determine the suitability of a candidate. It’s essentially Data running your hiring process.
But it’s important to never forget the human element. Worshiping at the alter of big data leads you to miss big events on the horizon. Kodak missed the development of digital photography, insisting that they had no data to indicate it was anything more than a fad. Nokia missed the development of smartphones, because again their data didn’t show them that anyone in the developing world was going to want them. If you lose your sight and your business focus in the pursuit of the quantifiable, the measurable, the Big Data, you are simply ensuring that you will miss something that will take your business down.
Not everything that is important can be quantified and measured. But not everything can be identified based on the human instinct side either. A balance must be struck for the clearest vision and best guidance. We don’t need Big Data. We need Thick Data.