Prepare for the challenges of deploying machine learning for analytics

A lire sur: http://www.techrepublic.com/blog/big-data-analytics/prepare-for-the-challenges-of-deploying-machine-learning-for-analytics/#ftag=RSS56d97e7

By Scott Robinson in Big Data Analytics, January 4, 2014, 1:16 AM PST

Machine learning is the holy grail of analytics, but getting it in place includes some serious challenges. 

The pressure to exploit analytics as a strategic advantage is increasing for an obvious reason: The economic future is becoming more uncertain, and the entire point of analytics is to manage uncertainty. But just as the uptake of business intelligence (BI) over the past decade had its learning curve and practical barriers, analytics also presents a set of formidable challenges to an enterprise that hasn't yet invested.

Conventional analytics -- which standard BI tools and many easy-to-use third-party packages enable -- extend familiar measures and indicators along extrapolative lines that allow decision makers to look ahead or tighten the focus of well-known metrics to eliminate waste. That's all useful, and readily within the grasp of most organizations, without much outside assistance.

Advanced analytics (i.e., data-driven predictive power based on models that don't come from a human expert's head) is a capacity that goes in different directions altogether, to the point of requiring a new way of thinking among those who want to master it.

The high cost of uncertainty

Conventional analytics are often linear and easily visualized. Those who have done a deep dive into Microsoft's SQL Server BI suite are acquainted with the ease-of-use the stack offers, its endpoint more often than not being Excel, where casual BI users can crunch carefully culled datasets to their hearts' content, combining what-if with what they already know very well.

Advanced analytics perform a different function: Bodies of data are scanned for hidden patterns, often with no foreknowledge of what those patterns might be; this requires specialized software (Hadoop and its many cousins) and often calls for dedicated hardware. The critical distinction between conventional and advanced analytics is the former is guided by a human mind and intent on a particular result, whereas the latter is open ended with nobody at the wheel.

Staging data for driving

Advanced analytics occur when a dedicated computer scans large bodies of data for hidden patterns, relationships undetected that can yield critical business insights. This is far beyond looking to historical data for verification of what someone has already gleaned or intuited, or to trim an existing relationship for efficiency's sake.

Diving into an ocean of data in search of undefined treasure is an exercise in self-discipline and asks the user to surrender even the construction of the algorithm that does the exploring. When machine learning is employed, the algorithm doing the pattern detection refines itself as it progresses, free of human oversight. This is a serious shift in mindset regarding the process at work -- the people digging for results are making a conscious decision to trust the machine.

Matters of interpretation

Another huge shift in mindset when employing machine learning is understanding that you're relying on the machine to tell you what's important. In conventional analytics, you know what you're going for when you start your search; when the machine learns, you don't have an outcome in mind (though you may be hoping for one), and you take whatever the machine gives you.

That's hard enough to do if you're new to this style of BI, but it gets really challenging when you turn discovery over to the machine learning process, and you're presented with results that take you places you weren't expecting to go.

The ultimate challenge of machine learning as an analytics platform is the revelation of patterns within data that bear no obvious connection to what was previously known. For instance, you've studied client portfolios and discovered hard truths about why former clients defected; or, you're looking at your markets and finding that sales drop off consistently at certain times for reasons that are your fault. The machine produces patterns that demonstrate realities you never would have thought to seek.

Where it gets painful

Studies have consistently shown that data-driven analytics yield decisions that are more effective than human expert opinion can deliver. This is humbling, even humiliating, but solidly established. Numbers don't lie, but human beings -- even executives with the best intentions and the most successful records -- lie to themselves.

Committing to machine learning, and driving business decisions from the data upward, demands resolve, humility, and a true sense of discovery.