[MUSIC] In the previous videos, you have heard some of our speakers talk about data sources like social media, emails, and documents. All things you may not typically think of as data sources. In this video, we will define big data and big data and analytics. Look at the differences between structured, unstructured, and semi-structured data. And talk about the applications of big data. For example, a large bank was looking at improving their customer satisfaction ratings. The bank traditionally measured CSAT, or Customer Satisfaction, and NPS, or Net Promoter Score, as the two primary indicators. Both these were traditionally obtained from periodic surveys. We worked with the bank to obtain and link a large variety of different sources of data. First, we used the surveys and analyzed the natural language text in the open comments field. This gave us a good view of the topic areas of concern to their customers. Second, we recorded all the calls coming to the customer service call center. Based on the speech analytics, speech to text, and text analytics, we we're able to identify the key reasons for the calls and the emotional state of the callers and the responders. Finally, we analyzed the social media channels to assess what their customers were saying about the bank and their competitors. This allowed us to build a complete view of the customer's interactions and sentiments about the bank, which helped them make better decisions. The amount of information available is exploding as digitarization, and the Internet of things, has increased the number of data sources and the value and complexity of data. Now, we use big data as a term to describe a collection of data sets so large and complex that it becomes difficult to process using basic database management tools or traditional data processing applications. The large data sets involved can consist of numerous data formats in either a structured, a semi-structured, or an unstructured form. Let's take a look at what we mean by structured, unstructured, and semi-structured data. Think about the list of names, addresses, and phone numbers found in a phone book. This is an example of structured data. It is well defined data, like customer names, ages, identifiers, etcetera, that you can collect formally. The most popular platforms for structured data include, Oracle, Microsoft SQL Server, Microsoft Access, and so on. Big data can be associated with structured data sources, but not exclusively. Now, lets look at unstructured data. Unstructured data is not broken down into individual components. The data is a bunch of sentences that you need to make sense of, like in a Word document. It is a collection of videos or audio recordings on YouTube. It is millions of e-mails or pictures or social media posts. It can be a recorded conversation. The challenge is, how do you take this unstructured data and do something meaningful with it? To understand semi-structured data, take that Word document that represents unstructured data and add metadata. Tags to keywords so that it is easily searchable. Now you have semi-structured data. Semi-structured data does not conform to a structural format like relational or other standard formats. Semi-structured data includes tags and other markers to separate data elements. Big data is not just about the data. It is about the interconnectedness of the data. Big data sets can be linked together, and insights can be derived from those linkages. Today, organizations capture and store an ever-increasing amount of data. Internet availability, interconnectedness, rapid connection speeds, and mobility contribute to the torrent of data points being generated daily. Organizations want to realize the potential value of these extreme size data sets, and discard less and less information. Whether it is customer data or internal data. However, the existing means to process and analyze data cannot scale to extreme sizes economically. As far back as 2001, industry analyst Doug Laney, currently with Gartner, articulated a now mainstream definition of big data as four Vs. Volume, velocity, variety, and veracity. First, let's look at volume. Volume reflects the size of a dataset. New information is generated daily, and in some cases hourly, creating datasets that are measured in terabytes and petabytes. Many factors contribute to this increase in data volume. Transaction-based data stored through the years. Unstructured data streaming in from social media. Increasing amounts of sensor and machine-to-machine data being collected. In the past, excessive data volume was a storage issue. But with decreasing storage costs, other issues have emerged. Including how to determine relevance within the large data volumes, and how to use analytics to create value from the relevant data. The second V we want to look at is velocity. This reflects the speed at which data is generated and used. New data is being created every second. In some cases, it may need to be analyzed just as quickly. Radio Frequency Identification, or RFID tags, sensors, and smart metering, are driving the need to deal with torrents of data in near real time. Reacting rapidly enough to deal with data velocity is a challenge for most organizations. Variety is the third V, and it represents the diversity of the data. Data sets will vary by time. Social networking, media, text, and so on. And they will vary how well they are structured. Data today comes in all types of formats. Structured, numeric data, and traditional databases. Information created from line of business applications. Unstructured data in the form of text documents, email, video, audio, stock ticker data and financial transactions. Managing, merging, and governing different varieties of data is something many organizations still grapple with. Next, we have veracity. Data veracity refers to the biases, noise and abnormality in data. Is the data that is being stored and mined meaningful to the problem being analyzed? Veracity in data analysis is the biggest challenge when compared to things like volume and velocity. In scoping out your data and analytic strategy, you need to have your team and partners work to help keep your data clean, and create processes to prevent dirty data from accumulating in your systems. Even more important than the definition of data is what data promises to achieve. Effectively used, data can be transformed into insights and intelligence. Delivered where and when they are needed to make and implement strategic and operational decisions. There is one more V to take into account when looking at data and analytics. And that is value. Having access to data creates value only when you have the right data to clean strategic insights. Companies can generate significant value from your data. An online retailer for example, was planning to enhance their recommendation engine. The current software relied on a static set of rules to determine one of five different paths through their website. They wanted to modify this to make recommendations based on the individual profile of the customer, the amount of time they spend on a page, the keywords they enter, and what other customers like them have done in the past. So just how big is big data? Think about these facts. More than half of new data created is in video and audio formats. And by the year 2020, total global Internet traffic will exceed 200 exabytes per month. An exabyte is equal to 1 billion gigabytes. Global mobile traffic will increase to 30 exabytes per month, and will increase by 50% combined annual growth rate. The total number of users with Internet access will exceed 3.5 billion. The total number of mobile devices will exceed 10 billion. Think about that for a minute. What does all that data mean for organizations? How will organizations use this data? Big data is a game changer in making business decisions. Let's look at how organizations are currently using social media. In traditional use, businesses use the convening power of social media to boost their image and better anticipate consumer trends. Few organizations have harnessed the potential power of social media for applications beyond marketing and public relations. Let's take an example. A large investment bank is worried about its compliance risk. Regulators are monitoring customer complaints made directly to them as well as the social media channels of the financial institutions. The investment bank built a social media dashboard that monitored customer complaints on a regular basis made to their social media site as well as other public forums. The dashboard captured the rate of change of the number of messages on a particular topic, as well as the rate of change of sentiment with respect to the same topic. This allowed them to react and respond fast whenever there was a change in volume or sentiment related to themselves or their competitors. Let's recap what we just covered. Big data is made up of structured, unstructured, and semi-structured data. The amount of data that is being produced is growing at an astounding rate. And the key to this data is the interconnectedness of it all. We covered a lot of information here. You can use the interactive PDF to review the big data concepts. In the next video, you will hear from some of our PWC professionals about how we have used big data to solve client issues. [MUSIC]
