BIG DATA vs TRADITIONAL DATA

To have a better understanding the difference between traditional and big data, where to find the data and what techniques you can use to process it?

‘Data’ is a broad term that can refer to ‘raw facts’, ‘processed data’ or ‘information’. To make sure we’re on the same page, let’s separate them before we get into the details.

We gather raw data, then we process it to get meaningful information.

Well, separating them was easy!

Now, let’s get into the details!

What are ‘Traditional’ and ‘big’ raw data?

We can look at data as being traditional or big data. If you are new to this idea, you could imagine traditional data in the form of tables containing categorical and numerical data. This data is structured and stored in databases which can be managed from one computer. A way to collect traditional data is to survey people. Ask them to rate how much they like a product or experience on a scale of 1 to 10.

Traditional data is data most people are accustomed to. For instance, ‘order management’ helps you keep track of sales, purchases, e-commerce, and work orders.

BIGdata

‘Big data’ is a term reserved for extremely large data.

You will also often see it characterised by the letter ‘V’.

As in “the 3Vs of ‘big data”. Sometimes we can have 5, 7 or even 11 ‘V’s of big data.

They may include –

• the Vision you have about big data,

• the Value big data carries,

• the Visualisation tools you use or

• the Variability in the consistency of big data. And so on…

However, the following are the most important criteria you must remember:

VOLUME

Big data needs a whopping amount of memory space, typically distributed between many computers. Its size is measured in terabytes, petabytes, and even exabytes

VARIETY

Here we are not talking only about numbers and text; big data often implies with images, audio files, mobile data, and others.

VELOCITY

When working with big data, one’s goal is to make extracting patterns from it as quick as possible. Where do we encounter big data?

The answer is: in increasingly more industries and companies. Here are a few notable examples.

As one of the largest online communities, ‘Facebook’ keeps track of its users’ names, personal data, photos, videos, recorded messages and so on. This means their data has a lot of variety. And with over 2 billion users worldwide, the volume of data stored on their servers is tremendous.

Let’s take ‘financial trading data’ for an extra example.

What happens when we record the stock price at every 5 seconds? Or every single second? We get a dataset that is voluminous, requiring significantly more memory, disc space and various techniques to extract meaningful information from it.

Both traditional and big data will give you a solid foundation to improve customer satisfaction. But this data will have problems, so before anything else, you must process it.

How to process RAW DATA?

The first thing to do, after gathering enough raw data, is what we call ‘data preprocessing’. This is a group of operations that will convert your raw data into a format that is more understandable and useful for further processing.

I suppose this step would squeeze in between raw data and processing! Maybe, we should add a section here…

DATA PROCESSING

So, what does ‘data preprocessing’ aim to do?

It attempts to fix the problems that can occur with data gathering.

For example, within some customer data you collected, you may have a person registered as 932 years old or ‘United Kingdom’ as their name. Before proceeding with any analysis, you need to mark this data as invalid or correct it. That’s what data pre-processing is all about!