Our Thinking

The Three V’s Of Big Data: Volume, Velocity and Variety

By Ed Higdon. Posted in Uncategorized

April 08, 2015

When it comes to big data, there are three significant, defining properties: volume, velocity and variety. Volume refers to the amount of data generated. Variety refers to the type of data generated. Velocity refers to the speed at which that data comes and how fast it is processed. As the “Three V’s” continue to expand at an exponential rate, companies must be able to adapt quickly in order to make meaningful use of that information.

We Create HOW MUCH Data?

Like The Blob of B Movie fame, the sheer volume of data generated over the last several decades has grown at an exponential rate, with no signs of slowing down. Consider this: Jeremy Waite, the head of digital strategy at Salesforce, recently stumbled upon a vintage advertisement from Time Magazine. In that ad, the company said it published 50 million words in 1965. Today, according to Waite’s calculations, it takes Twitter just over eight minutes to generate the same number of words. Taking it one step further, he postulates that by 2050, Twitter could generate as many as 50 million words per second.

It’s not just social media that’s generating extreme volumes of data. Eric Schmidt of Google recently calculated that every two days, the Internet generates approximately 5 exabytes (5 quintillion bytes) of data. According to Schmidt, it took us from the dawn of civilization until the year 2003 to generate that same amount of data. The volume of data we generate is enough to boggle the mind, and as more aspects of our lives become digitized, our data points will reach numbers previously only heard of in science fiction.

Variety Is The Spice of Data

Before the digital age, there was very little variety in data generation, since businesses and government agencies were the only groups creating data points. Now, think about all the ways in which a single human being generates data: social media, text messaging, streaming video, credit card swipes, loyalty card memberships, online gaming, smartphone apps, GPS, and so on. User generated content is one of the fastest growing and largest producers of data. Almost every action a person takes from the time they turn off their phone alarm clock in the morning, through every keystroke at work, until they watch their last movie on Netflix in the evening, generates a wide variety of data points.

The Internet of Things (IoT) , the networking of objects that have electronic components and Internet connectivity, is rapidly expanding. Everything from watches to household appliances to toothbrushes will be networked, expanding not only the volume of data humans generate but the variety of it, as well. As the IoT takes off, it won’t just be the work we do each day, the videos we post to YouTube or the iMessages we send to our friends and family that leave an imprint. It will be the items in our refrigerators, the capacity of our HVAC systems, the wattages of our light bulbs, and even some items we wear on our body that will add to the variety of data we turn out each day.

Processing At The Speed Of The Human Race

Back in “the old days,” data was processed in batches. An employee would gather a batch of data, run it through the server, and then wait for the result to be delivered. That process worked well when data generation occurred slowly. For example, quarterly sales numbers would be collected and processed, with the results delivered to executives as the next batch of quarterly sales data was ready to go. Organizations were used to delays and lags, and there was very little, if any, ability to make data-driven decisions in real time.

Now that we generate exabytes of data every 48 hours, and that variety of data is almost endless, processing speed is essential. Some of today’s computer chips are designed to process as many computations as the brain of a mouse. It’s estimated by 2023 that a single chip will be able to process at the speed of a human brain. Further extrapolations indicate that by 2045, chips will process at the same speed of the entire human race. As processing speed continues to grow, machine learning will improve and the “thinking power” of our devices will improve. Google’s self-driving car is already a reality, so what will come next? True artificial intelligence may not be a far cry.

Big data is transforming not only the way we do business, but the way the entire world functions. As the volume, variety and velocity of data continue to grow, there is no limit to the possibilities of the future.