What is Big Data?
I want to learn Big Data. I have no clue where and how to start learning about it.
Does Big Data really means data is big?
What are the tools and software I need to know to learn Big Data?
I often receive questions which I mentioned above. They are good questions and honestly when we search online, it is hard to find authoritative and authentic answers. I have been working with Big Data and NoSQL for a while and I have decided that I will attempt to discuss this subject over here in the blog.
In the next 21 days we will understand what is so big about Big Data.
Big Data – Big Thing!
Big Data is becoming one of the most talked about technology trends nowadays. The real challenge with the big organization is to get maximum out of the data already available and predict what kind of data to collect in the future. How to take the existing data and make it meaningful that it provides us accurate insight in the past data is one of the key discussion points in many of the executive meetings in organizations. With the explosion of the data the challenge has gone to the next level and now a Big Data is becoming the reality in many organizations.
Big Data – A Rubik’s Cube
I like to compare big data with the Rubik’s cube. I believe they have many similarities. Just like a Rubik’s cube it has many different solutions. Let us visualize a Rubik’s cube solving challenge where there are many experts participating. If you take five Rubik’s cube and mix up the same way and give it to five different expert to solve it. It is quite possible that all the five people will solve the Rubik’s cube in fractions of the seconds but if you pay attention to the same closely, you will notice that even though the final outcome is the same, the route taken to solve the Rubik’s cube is not the same. Every expert will start at a different place and will try to resolve it with different methods. Some will solve one color first and others will solve another color first. Even though they follow the same kind of algorithm to solve the puzzle they will start and end at a different place and their moves will be different at many occasions. It is nearly impossible to have a exact same route taken by two experts.
Big Market and Multiple Solutions
Big Data is exactly like a Rubik’s cube – even though the goal of every organization and expert is same to get maximum out of the data, the route and the starting point are different for each organization and expert. As organizations are evaluating and architecting big data solutions they are also learning the ways and opportunities which are related to Big Data. There is not a single solution to big data as well there is not a single vendor which can claim to know all about Big Data. Honestly, Big Data is too big a concept and there are many players – different architectures, different vendors and different technology.
What is Next?
In this 31 days series we will be exploring many essential topics related to big data. I do not claim that you will be master of the subject after 31 days but I claim that I will be covering following topics in easy to understand language.
- Architecture of Big Data
- Big Data a Management and Implementation
- Different Technologies – Hadoop, Mapreduce
- Real World Conversations
- Best Practices
Tomorrow
In tomorrow’s blog post we will try to answer one of the very essential questions – What is Big Data?
Reference: Pinal Dave (https://blog.sqlauthority.com)
32 Comments. Leave new
I cant wait to hear all about it, I have heard so many different things and your rubrics cube analogy seems to hit the nail on the head.
Nice Start !!!!!!!!!!!
Nice start of big data topic… awaiting next articles….!!!!!
Thank you very much for that series.
Thanks Pinal It’s a wonderful time to have discussion about BIgData…
Hi Pinal,
Very interesting topic. I will follow it for sure. Just one question: is it 21 or 31 days long? In the main title you say 21, while inside you wrote 31 a couple of times.
Thanks!
TY.Waiting for your Post
Thanks Pinal,
For this initiative. I will follow it for complete 31 days….
Regards,
Rajesh H.
It is not so much about data, but what you do with the data. To me “Big Data” is just a term to describe ALL of the data. It could be in the data warehouse, CRM database, text files, web logs, the list goes on.
How we use the data to make better decisions, improve processes and reduce risks is what makes “Big Data” valuable.
Just like the various definitions for Big Data there are numerous tools to process that data. A Hadoop cluster won’t work to alert me of a potential server failure, but it can crunch millions of rows from various sources very efficiently to identify a fraudulent claim. A Python script can alert me to a potential server failure, but would not be efficient at identifying a fraudulent claim.
Looking forward to following along on this journey as Big Data progresses into better decisions.
Michael Heindel
@BeachBum
exciting! let see what happens next 31 days
Hi Pinal, great initiative.
Thanks Pinal, Today I started to look for it and I got your email … my good luck :)
Interesting First day. Thanks!
Looking forward to this series. Thanks Pinal, appreciate your time like always.
I know its too early to ask this question, but whats your recommendation as to when an organization should decide they should move to NOSQL from RDBMS, like when data size reaches 500 GB/ 1 TB ?
Thanks Pinal..I searched about big data a lot but did not get any good article on that..hope this 31 days series worked out… Please incude some real world examples as well. Thanks.
Pretty excited for next 21 days to learn something big…:)
Hope to have a good learning ahead…
Looking forward to the future pieces. By the way, the post title says 21 days but I think you mean 31?
Hi Pinal,
Looking forward to this. The url says Day 1 of 21 but the article says in the next 31 days. Is this an error or will there be gaps causing the actual time to span 31 days?
Nice Series Pinal