They Are Coming and You Need to Be Prepared – Regulations and the Use of AI and Machine Learning in Business Decision Making

Background

Historically, albeit not that long a history, regulators have been concerned about how businesses collect and safeguard individual, personal non-public information. Now, that focus has markedly shifted as companies of all sorts welcome next-generation technology and data mining tools into daily business functions. The current intensive spotlight is on consumer-facing commercial enterprises that use consumer data sets and artificial intelligence (AI)- and machine learning (ML)-enabled analytical models to make business decisions. It is now clear that regulators in the United States, European Union and around the globe believe that well-defined guardrails are needed to ensure the ethical use of AI- and ML-powered data and analytics in decision making. As those regulations are enacted, businesses need to conform their operations and functions and comply with those legal guardrails – starting now.

Current Law and Other Initiatives

Colorado’s Division of Insurance initially planted the flag in 2021 (§§10-1-109, C.R.S. and 10-3-1104.9, C.R.S.) and then recently issued draft supporting regulations. The law goes into effect this year and, at this point, regulates life insurers’ use of external personal data and information sources (e.g., electronic chart display and information system (ECDIS)) or employment of algorithms and models that use ECDIS, where the resulting impact of such use is deemed to result in unfair discrimination against consumers on the basis of race, color, national or ethnic origin, religion, sex, sexual orientation, disability, gender identity or gender expression. The draft regulation, that the industry largely views as overbroad, takes the statute one step further by encompassing any data or information source that is used to supplement or supplant traditional underwriting factors, including credit scores, social media habits, purchasing habits, home ownership, educational attainment, licensures, civil judgments, court records, occupations that do not have a direct relationship to mortality, morbidity or longevity risk, and any insurance risk scores derived by the insurer or third party from a similar information source. The regulation may be subject to change, but nonetheless it is indicative of the increased regulatory focus on AI/ML data-based decision making that, no doubt, will migrate to environments beyond insurance, such as credit-scoring and lending and academic admissions criteria, labor and employment decisions, and employee benefits assessments.

California, Connecticut, Louisiana, New York, Rhode Island and Washington, D.C. all have weighed in by adopting similar legal governance guidance and protocols, as has the White House with its “Blueprint for an AI Bill of Rights” released in October 2022, followed by the U.S. Department of Commerce’s National Institute of Standards and Technology’s “AI Risk Management Framework” in January 2023. United Kingdom and EU regulators have their own sets of policies and legal frameworks to address potential, perceived AI-driven bias (intentional or not) and to enable transparency of model-driven consumer-related decisions. It is noteworthy that almost all of the state, federal and international legal guardrails not only require companies to comply with a robust set of rules, but also attribute direct responsibility for any failures to boards of directors and senior management. The regulators are coming, and businesses need to step ahead now.

Measuring Bias

Thus far, there is no standard measurement for bias (intentional or not). Instead, current laws and other requirements often require the company to determine if the AI/ML-driven decision making results in “disproportionate negative outcomes” that have a detrimental impact on a protected group (including race, color, national or ethnic origin, religion, sex, sexual orientation, disability, gender identity, or gender expression). Companies may include some combination of “neutral” factors in making business decisions, as long as they are considered for all other similarly situated individuals. But aggressive regulatory enforcement will follow once the prohibited factors are breached and bias (or perceived bias) results, and it will be vital for companies to have compliant, standard procedures and protocols firmly entrenched throughout the enterprise.

What to Do Now

Below are a few suggestions for companies and their data technology teams to address and stay ahead of the evolving AI/ML requirements. The new rules will require more resources.

1. Stay aware of ongoing legislation, laws, and guidance frameworks in order to stay compliant.
2. Build a surveillance monitoring system with the capability to assess AI outputs to ascertain whether unfair bias could occur.
3. Evaluate your AI systems and, to the extent there is third-party vendor involvement, their AI systems as well. Evaluations should be continuous.
4. Respond to consumer complaints, including those that will inevitably be made to regulators, about any perceived or real potential bias with transparent communications and the ability to easily demonstrate data processes and decision making.
5. Build in systemic flexibility to mitigate and address bias (real or perceived) when it is identified and ensure your vendors are on the ready.
6. Train, train, train on AI use, not just for those in the technology teams but throughout the company.
7. Ensure that the chief risk officer has the full support of and direct access to the board and C-suite decision makers.
8. Ensure that all business segments are aligned on an ongoing basis with the guiding principles of applicable law, NIST and the White House.
9. Be prepared to provide confidential access to regulators to address concerns. This process should be carefully considered in advance so the company can safeguard its AI/ML software and algorithms.
10. There is no substitute for a robust oversight and communication system, including for your vendors. Among other things, ensure those vendors are committed to responding to regulators about their predictive models and AI/ML use and decision-making outputs.
11. Establish a regularly updated company checklist to respond to inevitable regulatory examinations, securing robust AI and ML compliance that includes regular review of all of the above.

Now is the time to create and implement new internal systems to ensure responsible use of data and AI/ML-driven decision making and minimize legal risks for the company, the board and senior management.

Thompson Hine Business Law Update - May 2023 (Download)

This article may be reproduced, in whole or in part, with the prior permission of Thompson Hine LLP and acknowledgement of its source and copyright. It is intended to inform clients about legal matters of current interest and is not intended as legal advice. Readers should not act upon the information contained in it without professional counsel. This document may be considered attorney advertising in some jurisdictions. © 2023 THOMPSON HINE LLP. ALL RIGHTS RESERVED.