Advanced Predictive Analytics: Predicting the Outcome!

Predictive analytics encompasses a variety of statistical techniques from modeling, machine learning, data mining and others that analyze current and historical facts to make predictions about future events. In business, predictive models exploit patterns found in historical and transactional data to identify risks and opportunities. Models capture relationships among many factors to allow assessment of risk or potential associated with a particular set of conditions, guiding decision making for candidate transactions.

Till late 2010, most of the enterprise business intelligence focus and analytics was around structured and some was on semi-structured data (emails, logs and call records). The buzz words were “data-driven” business decisions or actions. With the era of big data and evolution of technology, organizations are now venturing into unstructured data like sensor data from RFID tags and online web content, and they are seeing the need for better analytical capabilities. The new buzz words that are rapidly gaining popularity from meetings to conferences is “advanced analytics” or “analytics-driven” or “big insights.”

With big data technology, the power of predictive analytics is getting a lot of coverage, and software vendors are touting the latest and greatest in technology and algorithms. Once deemed as boring and dull jobs for data geeks, the demand for statisticians and data scientists is high and rising. One of the recent articles from Harvard Business Review (HBR – Oct 2012) talks about “Data Scientist: The Sexiest Job of the 21st Century.”

As noted in the HBR article, “Data are essential, but performance improvements and competitive advantage arise from analytics models that allow managers to predict and optimize outcomes.” More important, the most effective approach to building a model rarely starts with the data; instead it originates with identifying the business opportunity and determining how the model can improve performance. According to research by Andrew McAfee and Erik Brynjolfsson, of MIT, companies that inject big data and analytics into their operations show productivity rates and profitability that are 5% to 6% higher than those of their peers. Often organizations look for help to start with their advanced predictive analytics project. There are very limited production processes that leverage the power of predictive analytics that is embedded in BPM or decision making processes.

In my recent experience from an Analytics Strategy and Assessment workshop for a client, the issue identified was not the tool or architecture, but how to get insights from myriads of data that exist. The questions posed were:

• Are we collecting and storing the right data?
• What insights can be generated from this data?

What organizations want to know is not what kind of technology to buy first or what techniques and training they need, but what kind of problem to go looking for. What kind of problem will show the greatest return on an investment in predictive analytics? Where can they apply predictive analytics and get a clear and compelling “win”?

7 Key Steps To Success

Based on my experience, here are the key 7 steps to get started with advanced predictive analytics

1. Define the problem or pain or opportunity

2. Identify the key metrics

3. Identify the right data that support metrics #2 above

4. Analyze and enrich the data

5. Build models for advanced predictive analysis

6. Experiment the model with test subject/group

7. Embed and implement the analytics as part of business process or application

Most of the organizations are lost in the process of identifying or collecting data before documenting step #1. As mentioned above from my recent experience, the organization was collecting and storing all possible data without defining what insights they wanted to generate to drive business at the tactical and strategic level. Operational decisions align well with predictive analytics model. Most of the operational and tactical decisions are made by front-line, field-level staff from call centers to customer service to sales reps. Predictive analysis that can be embedded into business applications as part of workflow processes can add a tremendous value to providing an excellent service to customers as well as significantly increasing results and outcome from cross-sell or up-sell opportunities.

Many predictive analytic tools support access to a wide range of data sources, including those typically branded “big data,” such as unstructured text, or semi-structured web logs and sensor data. The problem is that organizations are trying to apply these technologies to the wrong problem. With the urge to prove ROI from investments in big data and analytics, organizations focus on large and wider problems than focusing on operational level or tactical problems that can give an opportunity to implement and prove the solution.

Predictive models analyze past performance to assess how likely a customer is to exhibit a specific behavior in the future in order to improve marketing effectiveness. This category also encompasses models that seek out subtle data patterns to answer questions about customer performance, such as fraud detection models. Predictive models often perform calculations during live transactions, for example, to evaluate the risk or opportunity of a given customer or transaction, in order to guide a decision. Big data and predictive analytics can be combined together in the operational environment. By focusing on operational decisions, we can put big data to work, using it to drive predictions that improve our ability to make good decisions at the operational level. With advancement in computing speed, individual agent modeling systems can simulate human behavior or reaction to given stimuli or scenarios. The new term for animating data specifically linked to an individual in a simulated environment is Avatar Analytics.

From my recent analytics on social media marketing for a specific brand, we found out some interesting statistics, such as 23% of customers repurchase on the same day, and most of the repurchases happen within 5 days of initial purchase. These are key insights for marketing campaign. Healthcare and financial organizations have a huge potential to leverage predictive analytics for fraud detection, cross-sell and up-sell and interventions. In my experience at one of the leading healthcare companies where I managed the BI and Analytics team, analytics played a significant role on how members and patients were stratified by risk scores using statistical models.

Organizations are gaining momentum in leveraging big data for insights and advanced analytics using statistical model and predictive modeling across many industry domains. Whether it is using analytics to predict customer behavior, set pricing strategy, optimize ad spending or manage risk, analytics is moving to the top of the management agenda.