In the rush to adopt AI, the greatest risk is not a flawed algorithm. More often than not, it is a leader who assumed technical accuracy meant ethical soundness, and never thought to ask the hard questions until it was too late.
Digital transformation has made artificial intelligence a priority for many companies. Yet, the most consequential AI failures of recent years share a common thread. At their core, they were not engineering failures. Instead, they were leadership failures. Someone approved a deployment without asking what the model was trained on, someone saw a warning and stayed silent, and someone trusted a vendor's assurance and called it due diligence.
When regulators come knocking, "the algorithm decided" is not a defence. Neither is "we used a vendor." AI leadership begins with understanding how AI goes wrong, and ends with owning the answer to a single question: who is accountable?
Drawn from a session with Kay Pang, CEO and Managing Director of Kay Pang Law Practice LLC, who is also an Associate Faculty at SMU Executive Development at our flagship EXCEL Leadership Programme, below are five questions every leader must ask before deploying, trusting or scaling any AI system.
1. What was the AI trained on, and whose reality does it reflect?
AI learns from historical data. However, bias lives in data, and history was often unfair. Amazon built a hiring tool to vet applicants by observing patterns in resumes submitted to the company over a period of 10 years. Due to male dominance across the tech industry, most came from men.
Inadvertently, the tool taught itself to penalise the word "women's." No one audited the model before deployment.
2. Who is accountable when this goes wrong?
If you cannot name a specific person—not a team, not a system, not a vendor—accountability does not exist. When a regulator calls, "the vendor built it, IT integrated it, the committee approved the business case" is merely a chorus of everyone pointing elsewhere.
Name an individual with the authority to monitor and suspend the system, before you sign off.
3. How would we know if the AI was failing?
AI accuracy varies by task. A radiology AI dashboard showed 93% average accuracy in trials across 23 hospital sites. But what the dashboard fails to mention that accuracy varies from 71% to 97% across
sites because no one defined what a failure signal looked like, or who would receive it.
The dashboard measured efficiency; nobody measured accuracy by site. If a system started quietly getting things wrong today, how many days before someone would know?
4. What decisions should AI never be allowed to make alone?
Define the floor of human judgment before deployment, not after a crisis. When an AI triage system classified a patient as low-priority with no clinician review, the patient deteriorated over four hours.
Map every output the AI produces, and for each one define the minimum human review required before it becomes final.
5. Is your organisation ready to be responsible for this?
Technology readiness is not governance readiness. A global firm rolled out an AI HR tool across 34 countries flawlessly, but faced works council complaints in Germany, mass overrides in Japan, and data-privacy fears in Brazil.
It deployed into a governance vacuum and discovered the vacuum the hard way. Culture and processes must precede the tool.
The Leadership Difference
AI can process vast data, find hidden patterns and operate without fatigue. But only leaders can carry moral accountability, hold ethical lines when results are commercially attractive, and make judgment calls with incomplete information.
The risk is not that AI replaces leaders. The risk is that leaders hide behind AI.
The executives who preside over tomorrow's AI failures will rarely be careless or malicious. They will simply not have known which questions to ask, and will have assumed accountability did not apply to them.
You now know the questions; and you now know the accountability is yours. What you do with that is leadership.