Chapter 10-11-12. Find AI Opportunities - 4 Stages

合集 - the business case for ai(10)

1.Chapter 6 Optimize decision making with AI - Simple versus Intelligent Data Analytics2024-12-252.Chapter 1: The Promise of AI2024-12-253.Chapter 7 ML development life cycle2024-12-254.Chapter 8, 9 B-CIDS: 5 pillars of AI preparation → Jumpstart Approach2024-12-25

5.Chapter 10-11-12. Find AI Opportunities - 4 Stages2024-12-25

6.Chapter 3-4. 5 tips for AI success & 5 myths of AI2024-12-257.Chapter 5. How AI can improve Business Process2024-12-258.Chapter 13 Build or Buy2024-12-269.Chapter 14. Measure Success2024-12-2610.Book notes: The Business Case for AI: A Leader's Guide to AI Strategies, Best Practices & Real-World Applications2024-12-26

收起

Whose Job is AI

It’s common for management teams to assume that data scientists inherently know which problems to solve for the company. However, this bottom-up approach to AI rarely leads to meaningful results. While data scientists and ML engineers can identify AI opportunities and initiate exciting projects, many of these initiatives are more suited for research publication than for generating business value. Relying solely on this bottom-up method for discovering AI opportunities is risky.

• Limited Business Insight: Data scientists new to a company may not fully understand the business challenges. Their data exploration might not reveal inefficiencies in daily processes and workflows, making it difficult to propose impactful AI solutions.
• Need for Extended Business Exposure: Without significant exposure to the business, data scientists are less likely to identify and solve real problems that the company faces.
• Difficulty in Gaining Management Buy-In: Data scientists, who are often focused on technical aspects, may **struggle with company politics **and lack the authority or inclination to engage with upper management to get support for AI projects.
• Budgetary Oversight: Unaware of the company's financial constraints, data scientists might initiate AI pilots that are too costly or risky, leading to potential cancellation or postponement by management due to inadequate cost justification.
• Risk of Poor Outcomes: A bottom-up approach can result in the organization not recognizing the value of AI, leading to the possibility of abandoning AI pursuits and reassigning or releasing data scientists.

‼️A Better Approach: Leadership-Driven AI Strategy

• Leaders as AI Opportunity Identifiers: Instead of leaving AI to technical teams, leaders and domain experts can develop the skills to spot promising AI opportunities that align with business goals.
• Technical Teams as Enablers: Technical teams should inform leaders about AI feasibility and assist in executing the vision, ensuring that AI initiatives are grounded in reality and business needs.
• Maximizing Results: By adopting a leadership-driven approach rather than a bottom-up method, organizations can focus on AI projects that are purposeful and have a high impact.

two ways you can find AI opportunities that are aligned with the business:

• “organic discovery” 原汁原味的AI
  The first starts from a new business problem you’re looking to address. The breakdown of the problem reveals that parts of it can benefit from AI. As this happens organically, I’ll refer to this as the “organic discovery” of AI opportunities.
  [新问题剖析以挖掘AI应用] the need for AI often surfaces as projects are fleshed out and broken down into subproblems. While, for some problems, it could be immediately apparent that they could benefit from AI, for others, it can be more obscure.
• “proactive discovery” 老问题的AI赋能
  The second approach is to actively investigate existing processes, customer pain points, and legacy systems in the organization with the goal of finding opportunities that would benefit from AI. I’ll refer to this as the “proactive discovery” of AI opportunities.

Most beneficial scenarios to apply "proactive discovery":

• When you’re thinking about replacing legacy systems and business processes with modern solutions
• 🎉When you’re planning your company- or department-wide AI strategy
• When you’re trying to start a pilot project to gain AI experience

两种场景下考虑发掘AI机遇 → 接下来的具体方法（框架）深挖识别真正高价值的AI基于
Whichever the method, once you’ve found several AI opportunities,
you must dig deeper to ensure that these opportunities are truly promising.
That’s what we’re going to learn in the next two chapters—a framework for
discovering high-impact AI initiatives：how to recognize and frame
potential AI initiatives, collaborate with experts, and surface the most impactful
initiatives.

How to find AI Opportunities

The HI-AI Discovery Framework requires an organization to take four steps:

• Step 1: **Identify **Potential AI Initiatives (to ensure that an opportunity is truly AI-worthy)
• Step 2: **Frame **Potential AI Initiatives (for clarity on benefits and measurability)
• Step 3: Use Experts to **Verify **(to determine feasibility and implementation readiness of initiatives)
• Step 4: Score Potential AI Initiatives (to **prioritize **and surface the HI-AIs)
  

具体来说，第 1 步有助于发现具有 AI 价值的机会，这些机会具有粗略的 商业意义并具有基础构建块。该过程从确定您当前的解决方案开始，然后回答几个问题。无论您是希望 AI 来取代当前的手动流程，还是遇到全新的问题，第 1 步都将帮助您区分值得 AI 的问题和不需要 AI 的问题。

为了清晰和可衡量，在第 2 步中，我们讨论了如何构建 PAI。框架就是用细节来记录 PAI，写文档，这些细节可以揭示痛点、好处和感兴趣的指标来衡量 ROAI。框架有多种用途，从更轻松地与 AI 专家协作到促进评分和优先级排序。

第 3 步：使用专家进行验证和第 4 步：对 PAI 进行评分。这些步骤相结合，使您能够评估计划的实施潜力，并找到要追求的最佳选择（即 HI-AI）。正如我们所讨论的，PAI 虽然很有前途，但由于数据、复杂性和时间等各种问题，可能无法实施，这就是第 3 步至关重要的原因。此外，鉴于一个组织可能有一系列的 PAI，第 4 步帮助他们战略性地追求最有前途和可行的 PAI。

总结：
通过识别、构建和评分 PAI 以查找 HI-AI，为毛可以以不同的方式使用这些知识。您可以将其与公司或部⻔级别的 AI 战略结合使用，也可以将其与一次性项目结合使用。最终，目标是淘汰失败者并增加每项举措的成功机会。

Step 1. Identify Potential AI Initiatives (PAI, π)

• First, you must determine your current problem’s starting point.
• From there, you must ask three (or four) questions.

📌If you get a yes to most questions, then you have a PAI.

Starting Points:

• Starting point A. Old Problem with a Current Manual Solution
• Starting point B. Starting Point B: Old Problem with a Current Software Automation
• Starting point C. Starting Point C: New Problem with No Current Solution

Questions:

1. Does the problem require complex decision-making?
2. Is this a high-workload problem?
3. Do you know what **data **is needed and is the data available?
4. Does the existing software automation have accuracy or manageability issues? (starting point B, only)

The table below further summarizes the questions to ask for the different starting
points to determine if you’ve found a PAI.

Step 2. Frame Potential AI Initiatives (有点像写立项书)

Framing PAIs is about documenting them so that the benefits and impact of each
initiative are clear and measurable. To frame your PAIs, first, create a
spreadsheet. This spreadsheet should have the following information at a
minimum:

1. The Pain Point
2. Project Description
3. Potential Benefits
4. Expected Return on AI Investment (ROAI)
5. Data and Feasibility Notes

Pain Point

• The current approach
• The problem with the current approach
• The workload in **quantitative **terms
  By framing an AI initiative with such specifics, it becomes easier to determine
  where you are and where you’d like to go. It’ll help you better articulate the AI
  benefits and expected ROAI, which we’ll get to shortly.

The project description

is essential to have this when talking to your AI experts
in Step 3. Although the project description is always general, you can, of course,
get into the specifics, such as a list of AI problems to solve. For example, is this
a text classification problem, an information extraction problem, or both? This is
nice to have, but not a necessity as your AI experts will help identify the specific
models to develop in reaching your goals.

Example:

Pain Point: In a single day, each analyst can comfortably analyze 200
reviews manually. However, the team gets about 2,000 reviews a day for
analysis. Even though the work is split between five analysts, each analyst
has to regularly put in four additional hours a day to handle the backlog.
The analysts have also been complaining about a poor work-life balance due
to regular overtime work.
Project Description: Automatic complaints extraction system: this system
would automatically detect and extract complaints from user reviews with
minimal human involvement. Humans should only have to analyze reviews
that are hard to parse and verify extracted complaints with low confidence.

Potential Benefits

documenting the benefits of addressing the pain point defined earlier.

• Increase the number of reviews analyzed per day
• Reduce analyst workload
• Improve analyst work-life balance
• Improve overall customer experience with a faster turnaround
• Increase revenues by taking on more customers without increasing headcount
• Create new revenue streams by exploring other verticals with similar automation

Return on AI Investment (ROAI)

How do you measure the business impact of AI? When companies try to assess
the success of AI initiatives, they’re often looking for an ROI (❌) - the financial
gain or loss from an investment relative to its cost. However, that’s not the only
way, or the right way, to evaluate AI initiatives.

【ROI是整个solution/product/service需要考虑的。有了这个前提，针对AI我们要考虑的是AI在这方案当中所带来的产出】
AI initiatives are meant to solve a problem, not necessarily generate more
revenues. While some initiatives can provide an immediate revenue boost, many
take years to observe a financial impact. In fact, in the near term, you may
experience a decrease in revenues as you invest in the foundations for AI.
For that reason, instead of focusing on the ROI, we’re going to focus on the
ROAI (✔️) - the return on AI investment or the return on automation investment.
ROAI is not a metric but a concept; it’s a way to know if AI is having an impact
on your business processes, products, and services.

ROAI tracks improvement over a baseline measurement. To compute the ROAI, you’d first have to define

1. the metrics of interest. These metrics should tie in closely with
   the potential benefits and pain point we discussed earlier.
2. Along with ROAI also comes expected ROAI, which is an improvement goal.
   It can be a loose target or a minimum acceptable value that you can revise with time.

Example:
Metric: review analysis time
Baseline Measurement: twelve hours per day per analyst
Expected ROAI: 50% reduction in time to analyze reviews

Data and Feasibility Notes (数据的大概可行性说明)

These notes should articulate:

• If you have the right data and in the right volume
• Known issues about the data
• Any feasibility information about the initiative
  When you identified a PAI, you did some basic data assessment. That can become your initial documentation. However, the final notes here should be an expansion of that information verified by your experts.

Step 3: Use Experts to Verify

在AI和机器学习领域，尤其是在当前大型语言模型（LLM）迅速发展的背景下，很多人容易产生一种误解，认为AI的应用变得简单了。这种“简单化”的思维背后隐藏着几个重要的层面，值得深入探讨。
首先，理解复杂性：虽然现有的工具和平台使得某些AI应用的实现变得更为直观，但真正的挑战往往在于问题的复杂性。专家们之所以能成为专家，正是因为他们理解了数据的复杂性、模型的局限性以及业务需求与技术解决方案之间的匹配。简单的“Happy Path”往往忽略了在实际部署过程中可能遇到的各种问题，例如数据质量、模型过拟合、业务场景的变化等。
其次，风险评估：专家通常具备丰富的经验，能够识别潜在的风险和挑战。他们不仅关注如何实现一个技术解决方案，更重要的是评估这个解决方案是否能有效解决实际问题，是否符合业务目标。对于一个新技术的应用，专家会考虑其可行性、经济性和伦理性等多方面的因素，而这些往往是初学者或非专业人员所忽视的。
第三，战略思维：专家的视角更为全面，他们不仅关注技术实现，还会考虑技术与业务战略的结合。AI和ML的应用并非单纯的技术问题，而是需要与公司的整体战略相结合。专家会引导团队思考：这个AI项目是否真的能为公司带来价值？是否能提升客户体验、优化流程或降低成本？在这个过程中，专家的意见可以帮助团队避免盲目追逐技术潮流，而是关注于实现实际的业务价值。
最后，持续学习与适应：AI和ML领域快速变化，专家们通常具备持续学习的能力，能够适应新技术和新方法的变化。他们会关注行业趋势和技术发展的动态，确保自己的知识和技能始终处于前沿。这种能力使他们在面对新问题时，能够提供更具深度和广度的见解。
综上所述，专家的意见不仅仅是技术上的指导，更是战略层面的考量。他们帮助团队从更深层次理解AI的潜力与局限，确保技术的应用能切实服务于业务目标。对于那些希望在AI领域取得成功的团队来说，重视并倾听专家的意见，将是他们避免误入歧途、实现有效应用的关键。
Industry problems are not always straightforward. There are many factors that contribute to the success of AI initiatives, including data, metrics, infrastructure, and even proper framing of the problem. That’s why expert verification is critical. Experts can help surface red flags, address technology or data gaps, and help you avoid potential pitfalls.

Explain Your Objectives

【把上面的文档给专家看】
Go over the pain point, project description, and potential benefits of each initiative. Communicate any known facts about the data or existing accuracy measures. Also, describe how you plan to use the envisioned solution in your workflow.

Get Experts’ Take on Each Initiative。

With the information you’ve provided to your technical experts, they’ll be
able to ask follow-up questions, bring up red flags, and assess feasibility.
Point out obvious red flags. An expert can tell you if there are serious
concerns with your idea, your data, or how you’re attempting to use the AI
solution. Say you’re looking to deploy your AI solution as a cloud service,
but what you truly need is a solution with no network latency. An expert
can certainly detect such issues and bring it to your attention.【发现问题】

Reframe and refine ideas. Certain AI initiatives are framed poorly. Suppose
an initiative is made up of several independent initiatives. Some are AI
related, and some are not. In this case, your technical expert can help break
the problem down and determine which parts would benefit from AI.【定位AI】

Provide alternative solutions. Some problems can be solved with off-the shelf
solutions or alternate methods that do not require AI or large amounts
of data. It may not be something you’re aware of, but your technical expert
should be. It can be offered to you as an option. This will save you from
going through implementation only to realize that there’s a simpler solution. 【更好的方案】

Provide an estimated implementation timeline. Certain AI projects can take
weeks or months to develop, which can be a nonstarter for some teams. So,
it’s important to get an estimate of the implementation timeline. You can
then pad additional time for testing, iteration, and full deployment.【合理的时间线】

Perform a feasibility analysis. If there are no obvious red flags and the
project is well-framed, experts should provide a feasibility assessment. Is
this project doable or not? A feasibility assessment can be in the following
form:

• Type-a: A close review of your framed PAI
• Type-b: The above and exploratory data analysis
• Type-c 📌 : All of the above and a prototype of the AI solution

Through feasibility analysis, experts will get to dig deeper into your AI
initiatives. At this point, they may find gaps that you need to address or other red
flags that were not initially obvious. For example, if there is an issue with data
quality, experts can bring that to your attention and guide you on improving data
quality before implementation. Additionally, through a type-c feasibility
analysis, they’ll also be able to provide an accurate assessment of whether it can
be done.
The type of feasibility analysis you would perform rests on the complexity of the
project. The more complex or fuzzy the project, the more critical it is to perform
a type-c feasibility analysis.

Quick Tip: A safe way to approach AI initiatives is for every project to go
through a type-c feasibility analysis, which involves a prototype. Unlike physical
product prototypes, software prototypes are reusable. If well developed, you can
expand these prototypes into your minimum viable product (MVP).

Go or No Go?

【专家意见之后我们能决定这个AI机会做还是不做】
Now the moment of truth. In the end, what you’re trying to determine in Step 3 is:

1. If you should start the implementation of an MVP
2. If you should consider an off-the-shelf AI solution
3. If you should reframe or refine your problem definition
4. If you should collect data, collect better data, or “generate” data
5. If you should use an alternate or simpler non-AI solution

If the answer to the first or second question is a yes and everything else is a no,
then congratulations, you’ve found an implementation-ready PAI. If there are
known gaps, address those gaps before answering these questions again. As
we’ve seen throughout this book, not all problems benefit from AI, and even if
they do, they must fit within your company’s timeline, budget, and
infrastructure.

Step 4: Score Potential AI Initiatives

everything is about priority. 【量化】
By using an agreed-upon scoring method, it becomes easier to prioritize initiatives and surface the HI-AIs repeatedly, which is the premise for the following I2R2 scoring method.
【潜力大且失败风险小 ≈ 业务价值 x 技术难度】I2R2 is a simple scoring method that favors initiatives with high potential and impact, while minimizing risk on failure.
1-3-5 scoring for the questions below:

1. Is the initiative ready for implementation? (I1)
   • data readiness
   • is there a simpler solution rather tha AI?
   • technical challange?
2. What is the estimated impact size? (I2)
   • Helps a large group of users, customers, or employees
   • Positively impacts multiple aspects of your business
   • Provides a sizable boost to revenues
   • Results in significant cost savings
3. Is the ROAI clear? (R1)
   • metric
   • baseline exists?
4. What’s the risk if this initiative fails? (R2)
   • The solution has unacceptable performance numbers
   • The solution is too complex to be put into production
   • The implementation team bailed out at the last minute
   • But what happens if your AI initiative fails?

Score, Rank, Prioritize, and Surface HI-AIs

• Initiatives with a score of 4 and above after expert verification are HI-AIs. As a
  rule of thumb, you should try to go for projects with a score of 4 or greater.
• If you’re pursuing a project with a score between 1 and 3, you should take a closer
  look to see if it makes sense to continue.

Often, closing specific gaps can improve initiative scores, turning weak initiatives into HI-AIs.