When AI meets insurance: Fairness, climate and the future of cover

Insurance touches almost every part of daily life, yet it remains one of the least examined corners of the economy when it comes to artificial intelligence. Premiums are rising, insurers are pulling out of high-risk areas, and algorithms are increasingly deciding who gets cover and at what price, often without those affected understanding how those decisions were made.

The Wharton School and UNSW Business School recently brought together regulators, academics and industry practitioners in Philadelphia to discuss these issues at the Responsible AI & Analytics for Insurance Workshop. The event opened with a keynote panel titled “When AI Meets Insurance: Regulation, Accountability, Climate, and Affordability.” The session was moderated by Fei Huang, Associate Professor at UNSW Business School's School of Risk and Actuarial Studies and creator of the Fair Pricing Playbook, an open-source framework for fair insurance pricing. She brought together four panellists with different vantage points on the same problem.

Philip Barlow is Associate Commissioner at the DC Department of Insurance, Securities and Banking, and has led work examining unintentional bias in car insurance pricing. Ben Keys, Professor of Real Estate and Professor of Finance at the Wharton School, whose research covers credit scoring, climate disaster insurance and the mortgage market. John Johansen is a Senior Principal at Oliver Wyman, working directly with insurers on AI implementation. Professor Kevin Werbach, Professor of Legal Studies and Business Ethics and Faculty Lead of the Wharton Accountable AI Lab, has spent three decades exploring major trends in emerging technology, including legal and ethical aspects of artificial intelligence.

Together, they covered ground that extends well beyond the United States. Rising premiums, AI-driven pricing decisions, questions of fairness and the growing gap between insurable and uninsurable property are issues Australian business leaders will recognise, whether they sit in financial services, property or risk management.

Following is an edited transcript of the discussion, condensed for length and clarity while preserving the substance and intent of each contributor's remarks.

A/Prof. Huang: In one sentence, what is the most surprising thing you have learned about insurance?

Mr Barlow: People seem to insure things that appear to carry low risk, and not insure things where their literacy about the risk is poor.

Mr Johansen: Insurance is almost organic in societies. It is innate that people say, "If your barn burns down, I will help you rebuild it, and if mine burns down, you will help me." That origin story, communities coming together to share risk, is worth remembering.

Prof. Keys: I think about insurance as essential financial infrastructure. If you cannot get an insurance policy, you cannot qualify for a mortgage, and most home buyers need a mortgage to pay for a house. The United States has a $55 trillion housing market and a $15 trillion mortgage market, and neither can function without a well-functioning insurance market.

Prof. Werbach: Insurance is a microcosm of the issues we deal with across AI policy and governance, including both operational and technical questions and some genuinely hard ethical ones. Insurers have to make decisions about coverage, underwriting and pricing, which forces everyone to be explicit about the choices being made.

A/Prof. Huang: When I mention insurance to ordinary people, it tends to carry negative feelings, even though the word itself does not. The negative news is usually about coverage being unaffordable or unavailable, or about insurers not behaving well.

Learn more: A four-step framework: improving fairness in insurance pricing

An audience poll was then run on the panel's central theme, asking attendees what concerned them most about the use of AI in insurance.

A/Prof. Huang: The biggest concern is a lack of transparency and explainability, followed by bias and discrimination, then privacy and a lack of human insight. John, is this consistent with what you are hearing from clients?

Mr Johansen: It is. Explainability is a key area. We cannot tell someone the price was the price because the AI said so. We have to be able to leave a trail: to go back and show exactly how a price was arrived at. Human oversight is also a big issue. As organisations chase automation, they still want a human in the loop. Privacy and data use are a real struggle because achieving the best results with these models often means sharing as much data as possible, and the models can be leaky in ways that are not always apparent.

A/Prof. Huang: What does the current landscape of AI use look like among insurers?

Mr Johansen: Insurance entities are already advanced users of data analytics. Many have used machine learning models for the better part of a decade. With generative AI, we are seeing tiered approaches, from giving everyone access to tools such as Microsoft Copilot, through to dedicated experimentation teams. What is making the real difference is pairing people who understand the business process with people who are fluent in AI tools, sometimes with someone from governance involved to keep things on track.

A/Prof. Huang: Larger firms presumably have stronger AI capabilities than smaller ones. What does that imply?

Prof. Keys: We are seeing a story of the haves and the have-nots in modelling capability, and AI is likely to widen that gap. Research on the California wildfire market shows that more sophisticated insurers, with better modelling tools, can identify safer risks within what appears to be a high-risk area and write policies only for those customers. That leaves everyone else in a worse position because the average risk in the remaining pool rises, meaning people who cannot obtain cover from the most sophisticated insurer end up paying more. AI could enhance the capabilities of weaker teams, but it could also exacerbate adverse selection.

A/Prof. Huang: Philip, as a regulator, do you have concerns about how firms use AI?

Mr Barlow: My concerns about AI are the same ones I have about people. Poor training and poor oversight lead to poor decisions, and that oversight is often unclear to the people using it. One issue I often see is companies with less expertise copying what more capable companies are doing without fully understanding what they are copying, and that is how insurance companies get into trouble. I do not want a claim to be denied or underpaid without a human review. That feels like a minimum standard.

A/Prof. Huang: Do you see a way to mitigate that competition problem from a regulatory perspective?

"AI and advanced analytics are helping insurers think more granularly about risk, and helping property owners make better decisions about hardening their properties against damage"

JOHN JOHANSEN

Mr Barlow: Regulators struggle with this constantly, because we cannot set one standard for companies that do things well and a different standard for everyone else. Rules have to be targeted at the worst performers, not the best, even if that means extra work for the companies already doing things properly.

A/Prof. Huang: Kevin, what is the future of ethical AI governance?

Prof. Werbach: I started teaching AI law and ethics in 2016, focused on concerns such as facial recognition systems performing differently by race and gender. Cases such as the Northpoint system for predicting recidivism in parole decisions made clear that hard choices have to be made. That system was accused of racial bias when researchers examined outcomes in aggregate, but its developers argued that their calibration analysis, which checked the accuracy of each risk score by race, showed no bias. Both were valid measures of fairness, but they could not be satisfied simultaneously because of differences in base rates across the population. That forced governance conversations toward formal structures, culminating in the European Union's AI Act, which requires risk management processes, testing, disclosure and bias mitigation. The problem is that a lot of time and money can go into the process without addressing what bias mitigation actually means in practice. We are now in an uncertain period, partly due to political pushback against regulation slowing innovation, so there is no clear legal and regulatory path forward at present.

A/Prof. Huang: Any quick tips for practitioners wanting to get their AI governance right?

Prof. Werbach: Start with the NIST AI Risk Management Framework. It sets out a general set of questions to work through as a starting point, though with the caveats already discussed.

A/Prof. Huang: Philip, your department released a report on bias in car insurance. What did you find?

Mr Barlow: We looked at whether there was unintentional bias in car insurance in the District of Columbia. Insurance companies argued that looking at premiums alone was not enough, that claims data also needed to be examined, so we obtained that information. We found that premiums for black and Hispanic drivers were higher than for other drivers, and the difference in claims was higher again. We are now trying to understand why claims differ by race, in the hope of narrowing that gap. Even once that is understood, we still want to look closely at premiums, because a factor that predicts risk well may also correlate closely with race.

A/Prof. Huang: Ben, your research has looked at credit scores as a potential proxy here. What have you found?

Prof. Keys: We have looked at what determines property insurance pricing for households, beyond disaster risk alone. For most of the distribution of risk, credit score matters more than disaster risk, and most homeowners are not aware that their premiums are so strongly shaped by it. We lack claims data to properly validate this pricing against actual risk, so it is unclear whether it reflects accurate risk assessment or other factors, such as some households shopping around less. Recent data shows the average African American household has roughly a 100-point lower credit score than the average white household, meaning those households pay more on their mortgage rate and substantially more for insurance: a compounding affordability problem.

A/Prof. Huang: John, do insurers themselves worry about discrimination and bias in their models?

Mr Johansen: Emphatically yes, partly because they want to run ethical businesses, and partly because of the consequences when it does happen, with regulators and lawyers taking notice. I worked with one insurer moving from a deterministic pricing engine toward something more AI-driven, with a focus on transparency: keeping a record of what was asked of the AI and what it returned, so outcomes can be reviewed later. Running the new approach in parallel with the old one for a period allows differences in outcomes to be measured and explained before fully switching over.

A/Prof. Huang: Philip, do you agree, based on what you see as a regulator?

Mr Barlow: There is another group with all the relevant data needed to do this kind of analysis: the insurance companies themselves. I have never seen an insurance company do this kind of work and make it public. If they wanted to, they could do more of it on their own.

Learn more: How insurers can mitigate the discrimination risks posed by AI

The panel then turned to climate risk and affordability, following an audience poll on the most appropriate response to insurance becoming unaffordable due to climate risk. The options were risk-based pricing borne by homeowners, government subsidies, or a shared public-private solution, with the last two proving most popular.

A/Prof. Huang: Ben, does this result surprise you?

Prof. Keys: I am a little surprised at the appetite for more government involvement, though the idea that the market alone decides property insurance pricing is already something of a myth. These are heavily regulated markets, and dependence on state-run insurers of last resort, known as fair plans, has grown enormously. California now has over 600,000 households on its state fair plan, more than four times the number in 2019. My concern with these schemes is the temptation to underprice risk to keep constituents happy, a lesson learned from the National Flood Insurance Program, which has run large deficits due to underpriced policies for decades and has encouraged people to keep rebuilding in high-risk areas. That is part of the reasoning behind a proposal I have worked on for a federal reinsurance entity, which would move pricing decisions one step away from the states and one step away from the immediate political pressure of individual voters.

A/Prof. Huang: Is there an interaction between AI and climate risk here, John?

Mr Johansen: There is. AI and advanced analytics are helping insurers think more granularly about risk, and helping property owners make better decisions about hardening their properties against damage. We are seeing new providers focused specifically on advising customers on steps to reduce the likelihood of a claim.

A/Prof. Huang: Philip, is there a regulatory view on the tension between better risk prediction and pricing some customers out of the market altogether?

Mr Barlow: Some people live in risky areas because they have no other choice, and others choose to live there for other reasons, and those two situations need to be treated differently. I do not think the answer is to stop underwriting properly and charge everyone the same. That balance is ultimately a policy question for the community to decide, not something regulators or actuaries alone should determine.

A/Prof. Huang: A related concern for regulators and consumers is why premiums rise the way they do. Does AI make that harder to explain?

Prof. Werbach: We need to be precise about what explainability means. One level is simply knowing whether AI made the decision at all. Another is understanding the data and the model's characteristics, which are difficult for most consumers to interpret, even when disclosed. The deepest level is interpretability: actually being told why a decision was made, and there is often a trade-off there between accuracy and fairness. With generative AI, which is non-deterministic, providing that kind of explanation becomes harder again.

Prof. Keys: Other countries are ahead of us here. The United Kingdom requires a two-page, plain-English description of what a property policy covers and why it is priced as it is. The insurance industry in the United States has been reluctant to adopt anything similar voluntarily, and no state has mandated it.

"Beyond the tension between public and private approaches, there is a catastrophic risk we do not deal with well: tail risk"

KEVIN WERBACH

A/Prof. Huang: To close, what is the single biggest challenge facing the insurance industry over the next five to ten years?

Mr Barlow: Things are becoming more complex, and ensuring new approaches still adhere to the principles we all want insurance to follow is only going to get harder.

Mr Johansen: The environment favours organisations that can adapt quickly. Those already effective at using AI tools are becoming more effective, while others risk falling further behind, and the pace of change will test every organisation's ability to keep up.

Prof. Keys: Property insurance does not pool risk over a lifetime the way health insurance does, since people are not going to relocate en masse to lower-risk areas and back again. That will create growing political tension, both within and between states, about who bears the cost of rapidly rising risk.

Prof. Werbach: Beyond the tension between public and private approaches, there is a catastrophic risk we do not deal with well: tail risk. Climate is one part of that, but the emerging cyber and safety risks tied to advanced AI systems are firmly on the agenda too, and we need to find ways to insure against them even as they remain difficult to fully understand.

The panel closed with a discussion of where practitioners, regulators and academics might usefully work together, from data sharing on climate risk models to greater collaboration between insurance research and mortgage market research. For an industry often seen as slow-moving, the message from this panel was clear: AI is already reshaping pricing, claims handling and risk assessment, and the decisions being made now about transparency, fairness and mitigation will shape who can access cover, and at what price, for years to come.

To access the full recording and slides for all talks presented at the workshop, please visit this website.