How to learn to stop worrying and love machine learning
By implementing algorithms that are able to learn from the data that they explore, machine learning technologies already outperform traditional analytics by far. (No wonder high-flying companies like Google, LinkedIn, Amazon and Pandora have built their businesses around it.)
The key is the ability of machines to independently assess patterns and outcomes across far wider data sets than traditional analytics tools ever could. This obviates the need for time-intensive manual processing, and allows companies to fully exploit data collection techniques, employ cheaper storage, computing power and distributed database technologies — all of which are vital in an era where data doubles every two years.
In spite of such advantages, the very notion of machine learning still tends to trigger primal fear and mistrust among those who see the removal of human analysis from the equation as the first step to their irrelevance. So when exploring the potential of implementing machine learning in a business, tread lightly. Here’s how:
Analyze your environment
Corporate culture can affect machine learning success. Is your company a first mover with general technology trends, or does it tend to lag behind? Competition also serves as a good motivator. Is there a company in your industry already making strides with machine learning? If they are already monetizing machine learning technology, that’s a pressure point that your company must heed.
As with any new initiative, it always helps if there’s senior leadership onboard. Don’t obsess about the title — it can be the CTO, a domain expert, you name it. They just need to have a vision and a problem to solve.
And finally, examine whether or not your company has the infrastructure to even support machine learning. At this stage of maturity, data scientists play a key role translating domain experts’ needs into machine learning algorithms. Your company will need good technical building blocks and juicy projects to attract them. Competition for these resources is also at an all-time high, so you will need to consider cost as well.
Show that machine learning doesn’t eliminate jobs
It’s natural for users to worry that machine-learning technologies will eliminate jobs — theirs or someone else’s. But that’s wrong! Like adding a new employee, machine learning helps to increase productivity but may force some process changes. Spending time with those who will be most affected will help them to envision how they will become empowered, not obsolete. For example, we worked with one database marketing professional who was worried that machine learning would obviate the need for his job—not the case. In fact, he became one of the domain experts vital to the project and is now seen as a machine learning expert within his organization.
Work with the machines
Initially, users will be reluctant to allow the machines to do the analysis, make a decision and implement it. So as you work through the early projects, integrate these trust-building techniques:
Start small to prove value. Find a specific, solvable business problem that responds well to a data-driven approach, and continually improve the algorithms through testing and error analysis. As the successes snowball, so will trust in the technology.
Implement manual checkpoints to make the process interactive and validate the machine’s conclusions. Use the level of confidence provided in machine learning algorithms as a barometer. A 90 percent confidence level may not need human intervention, while an algorithm with a lower level of confidence may benefit from such a checkpoint.
Create a validation feedback loop. One of machine learning’s big values is the unforeseen insights it produces, but it can be hard to accept counter-intuitive findings. Going back to the inventory management system that wants to order 400 shirts–if your intuition says no, don’t order the shirts. But be sure to check to see whether the demand was there. If so, then the machine was right, and you will be more apt to trust future recommendations.
Define key performance indicators (KPIs) to measure success and use them to test your machine learning strategies rigorously. This will not only give you numbers to back up your success, but also the data necessary to continually improve.
As you apply your experiences to the next project, you will start with a higher level of trust, and your organization will be well positioned for machine learning success!
Srjdan Kovacevic, Baynote VP of engineering, and Nikki Hariri, Baynote data scientist, also contributed to this article.
Robin Morris is senior data scientist at Baynote, a provider of personalization solutions for online retailers, and associate adjunct professor in the Department of Applied Math and Statistics at the University of California, Santa Cruz.
