Why Product Managers Hold the Key to Ethical AI Success

Artificial intelligence (AI) is transforming regulated industries like healthcare, finance and legal services, but navigating these changes requires a careful balance between innovation and compliance.

In healthcare, for example, AI-powered diagnostic tools are enhancing outcomes by improving breast cancer detection rates by 9.4% compared to human radiologists, as highlighted in a study published in JAMA. Meanwhile, financial institutions such as the Commonwealth Bank of Australia are using AI to reduce scam-related losses by 50%, demonstrating the financial impact of AI. Even in the traditionally conservative legal field, AI is revolutionizing document review and case prediction, enabling legal teams to work faster and more efficiently, according to a Thomson Reuters report.

However, introducing AI into regulated sectors comes with significant challenges. For product managers leading AI development, the stakes are high: Success requires a strategic focus on compliance, risk management and ethical innovation.

Related: Balancing AI Innovation with Ethical Oversight

Why compliance is non-negotiable

Regulated industries operate within stringent legal frameworks designed to protect consumer data, ensure fairness and promote transparency. Whether dealing with the Health Insurance Portability and Accountability Act (HIPAA) in healthcare, the General Data Protection Regulation (GDPR) in Europe or the oversight of the Securities and Exchange Commission (SEC) in finance, companies must integrate compliance into their product development processes.

This is especially true for AI systems. Regulations like HIPAA and GDPR not only restrict how data can be collected and used but also require explainability — meaning AI systems must be transparent and their decision-making processes understandable. These requirements are particularly challenging in industries where AI models rely on complex algorithms. Updates to HIPAA, including provisions addressing AI in healthcare, now set specific compliance deadlines, such as the one scheduled for December 23, 2024.

International regulations add another layer of complexity. The European Union’s Artificial Intelligence Act, effective August 2024, classifies AI applications by risk levels, imposing stricter requirements on high-risk systems like those used in critical infrastructure, finance and healthcare. Product managers must adopt a global perspective, ensuring compliance with local laws while anticipating changes in international regulatory landscapes.

The ethical dilemma: Transparency and bias

For AI to thrive in regulated sectors, ethical concerns must also be addressed. AI models, particularly those trained on large datasets, are vulnerable to bias. As the American Bar Association notes, unchecked bias can lead to discriminatory outcomes, such as denying loans to specific demographics or misdiagnosing patients based on flawed data patterns.

Another critical issue is explainability. AI systems often function as “black boxes,” producing results that are difficult to interpret. While this may suffice in less regulated industries, it’s unacceptable in sectors like healthcare and finance, where understanding how decisions are made is critical. Transparency isn’t just an ethical consideration — it’s also a regulatory mandate.

Failure to address these issues can result in severe consequences. Under GDPR, for example, non-compliance can lead to fines of up to €20 million or 4% of global annual revenue. Companies like Apple have already faced scrutiny for algorithmic bias. A Bloomberg investigation revealed that the Apple Card’s credit decision-making process unfairly disadvantaged women, leading to public backlash and regulatory investigations.

Related: AI Isn’t Evil — But Entrepreneurs Need to Keep Ethics in Mind As They Implement It

How product managers can lead the charge

In this complex environment, product managers are uniquely positioned to ensure AI systems are not only innovative but also compliant and ethical. Here’s how they can achieve this:

1. Make compliance a priority from day one

Engage legal, compliance and risk management teams early in the product lifecycle. Collaborating with regulatory experts ensures that AI development aligns with local and international laws from the outset. Product managers can also work with organizations like the National Institute of Standards and Technology (NIST) to adopt frameworks that prioritize compliance without stifling innovation.

2. Design for transparency

Building explainability into AI systems should be non-negotiable. Techniques such as simplified algorithmic design, model-agnostic explanations and user-friendly reporting tools can make AI outputs more interpretable. In sectors like healthcare, these features can directly improve trust and adoption rates.

3. Anticipate and mitigate risks

Use risk management tools to proactively identify vulnerabilities, whether they stem from biased training data, inadequate testing or compliance gaps. Regular audits and ongoing performance reviews can help detect issues early, minimizing the risk of regulatory penalties.

4. Foster cross-functional collaboration

AI development in regulated industries demands input from diverse stakeholders. Cross-functional teams, including engineers, legal advisors and ethical oversight committees, can provide the expertise needed to address challenges comprehensively.

5. Stay ahead of regulatory trends

As global regulations evolve, product managers must stay informed. Subscribing to updates from regulatory bodies, attending industry conferences and fostering relationships with policymakers can help teams anticipate changes and prepare accordingly.

Lessons from the field

Success stories and cautionary tales alike underscore the importance of integrating compliance into AI development. At JPMorgan Chase, the deployment of its AI-powered Contract Intelligence (COIN) platform highlights how compliance-first strategies can deliver significant results. By involving legal teams at every stage and building explainable AI systems, the company improved operational efficiency without sacrificing compliance, as detailed in a Business Insider report.

In contrast, the Apple Card controversy demonstrates the risks of neglecting ethical considerations. The backlash against its gender-biased algorithms not only damaged Apple’s reputation but also attracted regulatory scrutiny, as reported by Bloomberg.

These cases illustrate the dual role of product managers — driving innovation while safeguarding compliance and trust.

Related: Avoid AI Disasters and Earn Trust — 8 Strategies for Ethical and Responsible AI

The road ahead

As the regulatory landscape for AI continues to evolve, product managers must be prepared to adapt. Recent legislative developments, like the EU AI Act and updates to HIPAA, highlight the growing complexity of compliance requirements. But with the right strategies — early stakeholder engagement, transparency-focused design and proactive risk management — AI solutions can thrive even in the most tightly regulated environments.

AI’s potential in industries like healthcare, finance and legal services is vast. By balancing innovation with compliance, product managers can ensure that AI not only meets technical and business objectives but also sets a standard for ethical and responsible development. In doing so, they’re not just creating better products — they’re shaping the future of regulated industries.