Have you ever wondered about how Google, Facebook or Instagram handle such large volume of data with much ease? The answer is Big Data.

We all live in data world. We need to store data and process the data. If data reaches beyond the storage capacity and beyond the processing power. That data we are calling as ‘Big Data’.

Big Data is a term for data sets that are so large or complex that traditional data processing applications are inadequate. Big Data generates value from the storage and processing of very large quantities of digital information that cannot be analyzed with traditional computing techniques.

The first organizations to embrace it were online and startup firms. Firms like Google, eBay, LinkedIn, and Facebook were built around big data from the beginning. Like many new information technologies, big data can bring about dramatic cost reductions, substantial improvements in the time required to perform a computing task and service offerings.

It is used to handle high speed data such as High-frequency stock trading algorithms reflect market changes within microseconds and Machine to machine processes exchange data between billions of devices. Big Data isn’t just numbers, dates, and strings but also geospatial data, 3D data, audio, video and unstructured text, including log files and social media. Whereas Traditional database systems were designed to address smaller volumes of structured data.

Real-time big data isn’t just a process for storing petabytes or exabytes of data in a data warehouse, it’s about the ability to make better decisions and take meaningful actions at the right time. Fast forward to the present technologies like Hadoop give you the scale and flexibility to store data before you know how you are going to process it. Technologies such as MapReduce, Hive and Impala enable you to run queries without changing the data structures underneath.

Big Data is already an important part of the $64 billion database and data analytics. Facebook handles 40 billion photos from its user base. Walmart handles more than 1 million customer transactions every hour. Boeing 737 will generate 240 terabytes of flight data during a single flight across the US. Without the help of Big Data these would have remain a dream…