The 5 Crucial AI Questions Every Leader Needs to Ask
Before making Klarna's $40 million mistake… (With Guest Andrea Chiarelli)
Let’s stay connected—find me on LinkedIn and Medium to further the conversation.
Most leaders I talk with, friends I interact with, aren’t asking “Should we use AI?” They’re asking, “How do we use it without losing ourselves? How do we use it well?”
That’s why I invited
—writer, strategist, and author of The Art of Asking Questions—to share this with you. His post reads like a caution sign for thoughtful leaders, managers, entrepreneurs, and creators navigating tech decisions under pressure.In it, Andrea breaks down Klarna’s costly mistake and offers five deeply practical questions every leader should ask before saying yes to the next AI rollout. If you're leading anything right now, these five questions are worth printing out and keeping close.
Let’s get into it. Here’s Andrea…
In February 2024, Klarna CEO Sebastian Siemiatkowski was riding high on AI success. His company's chatbot was handling 2.3 million customer conversations, doing the work of 700 agents, and promising $40 million in savings. "AI can already do all of the jobs that we, as humans, do," he declared to Bloomberg.
By May 2025, Klarna was quietly hiring human customer service agents again. The CEO's explanation? Their AI-first approach had gone "too far" and resulted in "lower quality" service. "Cost unfortunately seems to have been a too predominant evaluation factor," Siemiatkowski admitted.
Klarna isn't alone in this expensive lesson. Air Canada's chatbot invented a refund policy during a customer conversation, costing the airline C$800 when they tried to argue they weren't responsible for their bot's promises. McDonald's spent three years testing AI-powered drive-thrus before ending the program after memorable failures like trying to add bacon to ice cream orders and accidentally processing an order for 260 Chicken McNuggets.
The pattern is unmistakable. According to IBM's survey of 2,000 CEOs, only 1 in 4 AI projects delivers the promised return on investment, and just 16% are scaled across the enterprise. Meanwhile, 64% of CEOs admit "the risk of falling behind drives them to invest in some technologies before they have a clear understanding of the value they bring."
🎯 These failures share a common thread: leaders are treating AI as a matter of technology adoption when it's actually a deeply strategic challenge mixed with change management.
Why Most AI Projects Fail Before They Begin
Here’s a framework to help understand how complex change actually takes root in organisations (inspired by the work of the Center for Open Science).
The pyramid has five progressive levels, where successful implementation of higher levels depends on successful implementation of lower levels:
Infrastructure - Make it possible
User Interface/Experience - Make it easy
Stakeholder Buy-in - Make it normative
Incentives - Make it rewarding
Governance - Make it required
Klarna's failure becomes crystal clear through this lens. They had the infrastructure (AI integration worked) and made it easy to deploy, but they did not have stakeholder buy-in and acceptance. In addition, their incentives were misaligned, focusing on cost reduction over quality.
In sum, they jumped to implementation without building the foundation for sustainable adoption.
A Framework To Get AI Implementation Right
Here are the five critical questions every leader should ask, mapped to the change levels that actually determine success.
🖥️ Question 1: Do we have the infrastructure to support this?
As a starting point, every AI project will need the AI technology itself - whether an API, platform or something else. But, beyond that:
Do you have systems for monitoring the quality of AI outputs and iterating based on feedback?
Do you have the appropriate safeguards in place in terms of data security and privacy?
Is your data strategy robust and scalable, to support evolving AI needs?
Most organisations focus on deployment infrastructure and almost forget that there’s a lot more than a shiny new API.
🚩 Red flag: You're planning AI deployment without planning for integration, monitoring and improvement.
💁🏽♀️ Question 2: How will people actually interact with this?
User experience determines adoption more than technological capabilities:
How will AI integrate with existing workflows?
What happens when AI fails?
How intuitive are the handoffs between AI and human intervention?
🚩 Red flag: You're designing AI functionality without understanding current user behaviors and pain points.
🧗♀️ Question 3: Who needs to buy into this change, and why would they?
Successful AI adoption requires acceptance from customers, employees and partners:
How will you collect user feedback and address emerging concerns?
How will you address resistance and build trust?
Do the intended users of the AI tools have the right skillsets, or might reskilling and upskilling be required?
🚩 Red flag:You haven't talked to the people who will actually be affected by AI implementation.
🏅 Question 4: How do our incentives support the outcomes we want?
Adoption is deeply affected by incentives, particularly for internally facing AI projects. You’ll have to think about various aspects of organisational culture:
What currently gets measured and rewarded in your organisation?
How might these existing incentives undermine or support AI success?
How can your organisation reward innovation and continuous improvement in AI adoption?
Where do people choose between efficiency and quality?
🚩 Red flag: Your success metrics are all about savings and efficiency without measuring value to internal and external stakeholders.
🧑🏻⚖️ Question 5: What governance will sustain this over time?
AI implementations need formal structures to survive leadership changes, budget pressures and shifting priorities:
How will you formalise AI standards, escalation procedures and quality requirements?
How will you ensure that AI is used ethically and transparently?
What policies need to change?
Who owns ongoing oversight and continuous improvement?
🚩 Red flag: You're treating AI as a project rather than an ongoing capability that needs institutional support.
A Different Conversation About AI
The next time someone proposes an “AI solution” in your organisation, resist the urge to jump immediately to implementation planning. Instead, work through the pyramid systematically and make sure you consider all these elements:
Start with infrastructure
Design the experience
Build stakeholder buy-in
Align incentives
Establish governance
Klarna's expensive lesson teaches us that success with AI implementation is all about building the organisational foundation that allows that technology to create genuine value.
The companies that get this right are those that manage to achieve transformative and long-lasting change. And it’s not that hard - the top priority is shedding the temptation to embrace the hype, so you can have a grounded conversation that is tailored to your organisation and its goals.
The questions you ask before implementing AI will determine whether you're building something sustainable or setting yourself up for an expensive reversal. Choose wisely.
Andrea.
Thank you, Andrea! If his work resonated, consider subscribing to follow more of his work.
If you’re a founder, entrepreneur, creator, pastor, or nonprofit director, you’re not just making tech decisions; you’re shaping culture. And culture doesn’t change just because you add AI. It changes when your questions change.
Andrea’s post helps us ask the right ones.
If You Only Remember This:
Tech adoption is easy. Leadership adoption is not.
AI won’t fix broken culture or unclear incentives
The best tools fail when the people using them don’t feel heard
Ask different questions and you’ll build different outcomes
Slow, strategic clarity always beats fast, reactive pressure
If this was helpful, pass it along to someone else navigating AI and leadership. And if we haven’t connected yet, subscribe!
 | A guest post by
|
It was a great pleasure to work on this with you, Joel!
AI won’t fix a broken culture, or unclear priorities. This post is a masterclass in slowing down to ask better questions before chasing shiny solutions.